Demystifying Internet Measurement: A Profile of Doug Madory
Summary
Dubbed “The man who can see the internet” by the Washington Post, Doug Madory has made significant contributions to the field of internet measurement. In this post, we explore how internet measurement works and what secrets it can uncover.
Internet measurement is all about understanding the intricacies of internet activity and performance on a global scale. It involves analyzing data from various sources, which often means detecting outages, identifying security threats, and understanding how geo-political forces impact the internet.
The importance of internet measurement
Because the internet is the defining technology of our time—impacting businesses, individuals, and society at large—understanding its performance and reliability is crucial. Industry experts have advanced the field significantly, and though often operating behind the scenes, they have provided invaluable insight into what’s happening with the internet at scale.
This is important for improving availability, performance, and security, but it’s also been critical for business leaders and policy-makers to understand the relationship between the internet and society itself.
Doug Madory, the director of internet analysis at Kentik, is a key figure in the field of internet measurement. His educational background is in electrical engineering and network engineering, and he’s dedicated his career to understanding and analyzing global internet connectivity for well over a decade. His journey began as an officer in the United States Air Force, where he worked in a telecommunications role. After his military service, Doug started his civilian career at Renesys, specializing in BGP analysis for service providers and telecom companies.
A collaborative effort
Over recent decades, collaborative efforts between industry experts and academia have advanced the field significantly. Organizations like the Center for Applied Internet Data Analysis, or CAIDA, at the University of California, San Diego; the Georgia Institute of Technology; and the Internet Initiative Japan are at the forefront of this research, contributing to better tools and methodologies for internet measurement.
At Kentik, Doug works with these and other organizations to analyze global internet connectivity, outages, and security issues. This has led to notable contributions such as detecting and analyzing major incidents such as the 2021 AWS US-East-1 outage. In December 2014, he discovered North Korea’s first social media site, which briefly allowed global access before being shut down. This incident underscored the importance of vigilance and awareness in internet measurement while providing a glimpse into the sometimes surprising aspects of the field.
Doug’s work often intersects with geopolitical issues because today’s societies and governments rely on the internet. In that light, he’s written extensively about internet shutdowns in regions such as Northern Africa and Southeast Asia, offering crucial insights into how these events impact connectivity and access to information. His analysis of internet connectivity in Ukraine during the ongoing conflict has been particularly noteworthy, highlighting how geopolitical events affect the global internet landscape.
Finding answers with the data
Internet measurement relies primarily on BGP and NetFlow data. The Border Gateway Protocol, or BGP, is the routing protocol the internet runs on, making it an important source of information about how traffic is moving globally. Analyzing publicly available BGP data provides insights into how internet traffic moves from nation to nation among public clouds. It highlights the efforts of countries to suppress information, re-route traffic, or otherwise manipulate internet traffic in their geographic region.
Synthetic testing measures the internet’s performance by sending artificial test traffic between test agents deployed in strategic locations around the world. Metrics from these tests provide information like packet loss and network latency along the path among service providers, along submarine telecommunication cables, and so on.
Flow data, such as NetFlow, provides information about traffic flowing through network devices. In this way, engineers working in internet measurement can gain insight into specific traffic volume, traffic type, and traffic patterns. This is crucial for understanding how application traffic is delivered over the internet; it plays a role in understanding security threats against publicly accessible networks and helps us understand accessibility to information worldwide.
For example, using flow data and BGP information, Doug could detect and track a significant drop in traffic in and out of Kenya immediately after recent violent protests. With this data, he determined that the cause of the significant drop in traffic was likely a deliberate attempt by the Kenyan government to stifle internet access for its citizens.
The man who can see the internet
Dubbed “The man who can see the internet” by the Washington Post, Doug’s expertise and dedication have made significant contributions to the field of internet measurement. His work, along with others in academia and industry, ensures that the internet remains a reliable and secure tool for communication and commerce. By understanding and analyzing the complex web of global internet connectivity, Doug and his colleagues in the field help maintain the integrity of the global internet infrastructure.
For more insights from Doug Madory, follow his blog posts on the Kentik website and listen to his appearances on the Telemetry Now podcast. Also, make sure to watch LinkedIn Live, where Doug was interviewed about his experiences working in internet measurement.
