Ofcom Benchmark

In 2008 SamKnows partnered with Ofcom to carry out a comprehensive study of broadband provider performance, a follow-up from our own study completed in August of that year. In both cases SamKnows defined and funded the technical solution with the objective of creating a data set which would enable consumers to have the best possible understanding of the broadband performance delivered by the UK's leading ISPs . In a nutshell, SamKnows was the first to recognise that there was no robust methodology or technology for identifying the actual broadband performance delivered by ISPs, so we created it.

As well as the FAQs below you can also read about it in the SamKnows Labs section of the site here.

How did you start working with Ofcom?

When they heard of our own project, Ofcom, the UK communications regulator, invited us in for a cup of tea and a chat about our ‘solution’. A team at Ofcom had been scoping a project to examine actual broadband performance, and there was a clear fit with SamKnows’ own objectives. To cut a long story short, Ofcom selected SamKnows as its technical partner in the project, and SamKnows provided the methodology and funded the deployment in order to make the project happen. This also led to us working with market research company GfK NOP, who Ofcom commissioned to recruit and manage a representative panel of UK broadband users.

When did it start?

SamKnows began assembling its own panel in early 2008. Then in the summer of 2008 we partnered with Ofcom in order to build a much larger panel of demographically-representative broadband consumers. The panel was recruited by GfK NOP and we began collecting data from this panel in October 2008.

What tests are performed?

We are currently performing the following tests:

  1. Latency (ICMP based)
  2. Packet loss (ICMP based)
  3. DNS query resolution time
  4. DNS query failure rate
  5. Web page loading time
  6. Web page failure rate
  7. VoIP call quality (MOS)
  8. VoIP call jitter, delay, packet loss
  9. SMTP relaying speed
  10. Web based download speed test (HTTP)
  11. Web based upload speed test (HTTP)
  12. Non web-based download speed test
  13. Non web-based upload speed test
  14. Single-stream download speed test
  15. Multi-stream download speed test
  16. Single-stream upload speed test
  17. Multi-stream upload speed test

Tests 1-6 above are performed against common UK-based internet destinations. The speed tests, VoIP tests and SMTP relaying tests are performed against a cluster of our own test servers which we change periodically and load balance to ensure sufficient capacity.

We are always developing new tests and can upgrade the existing units with new testing profiles remotely once they are ready for production use.

Can panellists see their own individual results?

Panellists recruited by SamKnows have online access to their broadband performance data. Members of the Ofcom panel are not currently given access to their own performance data.