BT's Undertakings to Ofcom, accepted in September 2005, require BT to publish relevant key performance indicators (KPIs).
These play an important role in assuring all the stakeholders concerned with BT's performance that the Undertakings are being delivered.
The information is in two parts:
Further KPI reports are available on the BT Wholesale and the Office of the Telecommunications Adjudicator sites.
We are interested in your views, particularly those of other communications providers. Please e-mail: tsrundertakings@bt.com, heading your message "KPI Reporting".
New this time
We are pleased to draw your attention to improvements in the presentation of the KPIs for Partial Private Circuits (PPCs). We had been preparing these figures manually on a quarterly basis. Now we have automated the process and produce monthly figures (still published quarterly), allowing a clearer view of any movements. At the same time we publish z-test results in respect of PPCs to give greater transparency.
In this edition, we publish the results of relevant responses from our annual employee survey. As part of this we track employee perception of the company's commitment to compliance. The results are shown in the Training and Culture section.
Performance Review
We have been publishing for over two years, and have built a useful base of information. In general, we have found little evidence of difference between the product performance BT receives and that provided to its communications provider customers. Where there have been minor differences these have either been to BT’s relative disadvantage; converging over time (as in the Flexible Jumpering issue discussed below) or the result of known factors that we have addressed. KPI performance is an indicator of equivalence but not, alone, evidence that equivalence is being delivered or not. It is also worth remembering that we do not serve all of our end-users on an equivalent basis – the migration of the installed base is underway but scheduled to continue to 2010. The results so far, though, are encouraging and offer evidence that BT does not systematically favour its own businesses over those of its wholesale customers.
LLU and Flexible Jumpering
BT's equivalence testing has identified that BT is achieving materially better results than other communications providers (CPs) in the delivery of SMPF Basic Provide. This is largely due to BT's use of "flexible jumpering". This facility is available to all CPs through EMP on an EoI basis. If you are interested in using it, please contact your Openreach customer business manager.
Z-Tests
Several of our KPI sections contain equivalence tests which use a statistical tool called a "z-test". A z-test allows us to understand whether or not any difference in performance between the service BT provides to itself and to others is likely to result from random variation. This is clearer than looking at raw performance data, differences are not tested for significance. On the charts you will see a zone denoted by two blue lines. If the results fall within this zone, it is likely they are random variations. If they fall outside the zone consistently, we investigate the difference. This, for example, helped us to understand the benefits of flexible jumpering (above).
Technically, the z-test uses the normal distribution of probabilities to define the zone where differences are most likely to be due to random factors. Actual results are then plotted against this. The z-tests are only reliable where there is adequate data for the normal distribution to be reliably calculated. For that reason, we do not perform the test for every product. There is further description of z-tests at: http://en.wikipedia.org/wiki/Z-test/ (note:external link)
The z-test is, therefore, a useful tool for identifying areas where there may be equivalence issues. It does not, in itself, prove whether BT is, or is not, providing service on an equivalent basis. That can only be established by looking at the detail of how BT is providing its services.
Individual CP Results
We have received comments that some individual CPs are interested in understanding how their individual results compare with BT's. This is not something that BT publishes as part of its equivalence activity, which generally looks at performance in aggregate. At an individual level many factors such as geographical distribution and product mix can skew the results. If however there are particular service concerns that mean you need better to understand the performance you are receiving please contact your Customer Business Manager.
Just what is "BBRD1" anyway?
In some of the product KPIs you will see specific abbreviations of the measurement we are using. We include them so that people with an interest can identify the statistics accurately when cross referring to other KPI publications, such as those supplied to Ofcom or the Office of the Telecoms Adjudicator.
