Prof. Nick Feamster Argues For Better Internet Speed Tests in CACM

November 20, 2020

Many consumers use online speed tests to track the download and upload speeds of their home Internet connections. But these tools offer more than just a measurement for how fast one’s network can download files or stream video. The data they gather are also used by policymakers — from city governments to the federal agencies like Federal Communications Commission (FCC) — to assess the current state of broadband speeds around the United States and the world, and to make decisions about where (and how) to make investments to improve access to high-speed Internet connectivity. 

Although these tests have long been useful for identifying gaps and shortcomings in Internet connectivity, the COVID-19 pandemic has made these gaps even more apparent, as citizens around the world have come to depend on high-quality internet access for everything from education to work to telehealth. Mitigating the “digital divide” is critical for reducing inequities, and a big part of that problem involves gathering reliable, accurate data about the speed, reliability, and quality of residential broadband Internet connectivity.

In the December 2020 issue of Communications of the ACM — the flagship publication of the Association for Computing Machinery — UChicago CS Professor Nick Feamster and co-author Jason Livingood of Comcast provide an overview of how existing speed tests operate and explain why the future Internet will need entirely new types Internet speed tests that can do more than simply measure the speed of the ISP.

Feamster and Livingood explain that today’s common speed tests today were designed for an earlier era of the Internet where the ISP was typically the “bottleneck” — resulting in unreliable or misleading data. New tools must be developed that capture the structure of the modern web, with higher speeds, multiple connected devices running simultaneously in most homes, modern home network infrastructure (such as home wireless networks), and new applications from video conferencing to gaming where user experience ultimately depends on a much broader array of factors than conventional notions of “speed.”

A decade ago, the throughput for transferring files was the most important metric for speed tests, but today, interactive applications such as gaming and video conferencing require low latency as well as raw speed. The bottlenecks in modern networks have also shifted from ISP last mile access networks to home devices such as home wi-fi networks and network interconnects, the authors write.

In an accompanying video interview, Feamster explains that future speed tests can no longer just summarize their measurement with a single number.

“A speed test needs to measure more metrics than simply just downstream throughput,” Feamster said. “We really need to start taking a much more holistic view into network performance.”

Feamster suggests that speed tests should also measure additional factors, such as the performance of common consumer applications and the experience of the people actually using those apps.

“The challenge that we’ll have going forward is how to represent that information both concisely and intuitively for consumers, regulators, Internet service providers and really anyone who wants to know if the Internet is performing in a way that provides a good quality of experience for its users,” Feamster said.

Looking ahead, Feamster is setting his sights on a broader set of problems concerning the digital divide, beyond speed and application performance. With collaborators and other groups at the University of Chicago, including the Center for Data and Computing, the School of Social Services Administration, K12 outreach office, and the Consortium on School Research, the group is working with civic organizations in Chicago to better understand these connectivity gaps from technical, policy, economic, and social angles. The collaboration hopes to help groups from civic organizations to local and federal governments better understand and remedy problems underlying the “urban digital divide”.