"[T]he Internet is not something that you just dump something on. It's not a big truck. It's a series of tubes."
I presently reside in a condominium building. My building "offers" precisely one option for accessing the internet: entering into a contract with Comcast in exchange for access to Comcast's latest stab at a chic re-branding, XFINITYTM—y'know, "The future of awesome."
I pay Comcast approximately six-hundred dollars per year to provide download speeds of ten megabits per second ("mbps").1
Yesterday, I tried to watch a six-minute YouTube video. The video is offered in the following resolutions: 1080p; 720p; 480p; 360p; and 240p.2
I first tried watching the video in 1080p, reasoning that Google has officially certified "The Future of Awesome" as "HD Verified" in my geographic location.
See for yourself (click to enlarge):
Please note the following:
- The text left of the graph, which reads: "Users on YouTube HD Verified networks should expect smooth playback most of the time when watching high-definition YouTube videos (720p and above)" (emphasis added); and
- Below the graph, "The Future of Awesome" is listed as YouTube HD Verified.
Needless to say, hoping to watch in 1080p was wishful thinking. So I downgraded to 720p; the video continued to buffer in perpetuity. Becoming increasingly incensed, I proceeded to try watching in 480p, 360p, and finally 240p.
At 360p, after buffering for several minutes, the video began to stream for the first time. But not for long: it froze and resumed buffering less than thirty seconds into the video.
Even at 240p, the lowest possible resolution, the resolution that's been obsolete since DVDs became available in the United States in March 1996—over eighteen-and-a-half years ago—the video continued to buffer for several minutes.
Fully twenty minutes after I initially pressed play, the video began streaming. But I soon abandoned the effort outright, because the video's resolution was so poor that a dispassionate observer looking over my shoulder could have realistically concluded I was watching this:
Thankfully—despite Comcast's and its peer corporations' efforts on the contrary—there is an
easy and free way to find out just exactly what you're getting in exchange for forking over $600 to Comcast.
Recall "The Future of
Awesome" contractually promises me 10 mbps. What was I actually getting?
That's right. My download speed averaged .80 mbps, eight-tenths of a single megabit per second. For those without ready access to an abacus, that's eight percent of the service I purchase.
While this anecdote is particularly egregious, I'm not alone. Inexcusably poor internet service speed is a rampant problem in this country.
But why? How is this possible? How can Comcast and its peers get away with this?
Here's the short answer:
- Extremely effective rent-seeking behavior; and
- A concerted, highly successful campaign of intentionally obfuscating how consumers access the internet.
A comprehensive, plain English explanation follows below.
—————————— ◊ ◊ ◊ ——————————
To get to the true root of the problem, I'm going to back up and start at the beginning. Bear with me.
There are several distinct types of internet networks, and each type is designated by a numbered "Tier." Tiers are not mutually exclusive, so many networks operate as a Tier [#X] in certain respects but also as a Tier [#Y] in other respects.
At the top of the pyramid—colloquially known as the internet's "backbone"—are Tier 1 networks, which consist of approximately a dozen gigantic data centers dispersed across the globe. These privately-owned networks receive Tier 1 designation because each can reach every part of the internet.
But a single Tier 1 network cannot reach every aspect of the internet alone. Instead, inside those data centers, network ABC physically connects to network XYZ (and vice versa). This physical connection—known as "peering"—enables internet traffic on ABC's network to seamlessly cross onto XYZ's network.
Aggregating this practice creates the internet's "backbone," and it explains the definition of what constitutes a Tier 1 network: a network receives Tier 1 designation because it can reach every corner of the internet through its various peering agreements with other networks.
This may strike some ardent capitalists as counter-intuitive, but the generally-accepted definition of "peering" are traffic exchanges agreements where no money is exchanged. Why not? Because peering is considered mutually beneficial: "I take your traffic, you take my traffic, and together we both reach corners of the internet that are otherwise inaccessible without a peering agreement."
To be precise, these types of arrangements—ones where no money is exchanged—are called "settlement-free peering."
Alas, settlement-free peering arrangements break down when the free exchange of internet traffic isn't mutually beneficial, or, perhaps more accurately, when Network A accepts a significantly greater volume of internet traffic than it sends to Network B.
One step down the pyramid—below these free exchanging peering Tier 1 networks—are Tier 2 networks. Tier 2 networks do engage in some settlement-free peering with other networks, but they also pay other networks to take their traffic. Paying a network to take traffic is a practice known as "buying transit." Buying transit enables the purchaser-network to have its traffic accepted by its counterparty, and thus distributed to all networks connected to the internet.
At the bottom of the pyramid are the corporations that most individuals pay for internet access—such as Comcast, Verizon, and AT&T. In common parlance, the consumer-facing aspects of these corporations are known as Internet Service Providers ("ISPs").
But calling them ISPs isn't terribly precise and can be confusing. In legal-slash-telecommunications jargon, the companies you pay for internet access are known as "last-mile ISPs." This is admittedly a misnomer, but the nomenclature is rooted in the idea that last-mile ISPs carry internet traffic over "the last mile"—from their data center(s) into your home.
While consumers need last-mile ISPs to access content on the internet, the inverse is true too: Internet content providers ("edge providers") also need access to the internet so that their content is successfully delivered into a consumer's house.
The consumer-provider internet access situation is not, however, perfectly symmetrical. Because last-mile ISPs are effectively capable of playing gatekeeper to consumers, edge providers expend a great deal of time and effort attempting to circumvent last-mile ISPs' gatekeeping abilities.
But in the interest of keeping things straightforward, here's a simplified hypothetical.
Assume there are four total actors. First, there is a pair of parties who want/need access to the internet:
- Bob, a regular dude, resides in Washington, D.C. and he's a Netflix subscriber;
- Netflix, an edge provider, needs access to the internet to provide its internet content.
And then there's second a pair of parties, the companies that provide internet access to Netflix and Bob:
- Comcast, Bob's last-mile ISP;
- Cogent, Netflix's ISP, which Netflix pays in exchange for Cogent to take Netflix's internet traffic. Cogent, however, must buy transit in order to enable Netflix's traffic to reach individual consumers.
Bob comes home one evening and decides to watch a two-hour HD movie available on Netflix. As Bob navigates through Netflix's various menus en route to finding the movie he wants to watch, Bob sends some internet traffic "upstream"—albeit a very low volume of traffic. Put another way, Bob is sending internet traffic from his house, across Comcast's "last-mile" to a Comcast data center. When Bob's movie begins streaming, Bob's now having internet traffic (i.e., the HD Netflix movie) sent "downstream": from Comcast's data center, across the "last-mile," and into Bob's house.
There is obviously a massive disparity in the amount of traffic that Bob sends across Comcast's "last mile" and the amount of traffic that Netflix sends across Comcast's "last mile." Bob sends a handful of bits across Comcast's network when navigating Netflix's menus; Netflix sends approximately three gigabytes across Comcast's network, in the form of a two-hour HD film.
Comcast isn't terribly pleased about the disparity in traffic exchanged between it and Netflix's ISP, Cogent, but Cogent paid Comcast to take Netflix's traffic, so, grumbling aside, Comcast doesn't do anything—yet.
But later, Comcast learns that Cogent recently entered into an agreement to have Verizon—a "competitor" last-mile ISP—take Netflix's traffic at a much higher price than Comcast currently charges Cogent to accept Netflix's traffic.
In short, Netflix (through its ISP, Cogent) pays Comcast X dollars to take Netflix's traffic, but Netflix (through Cogent) pays Verizon 10X dollars to take the exact same Netflix traffic.
So what does Comcast do? It makes life as difficult as possible for Netflix and Cogent until Netflix pays Comcast the same (or more than) Netflix pays Verizon.
But how can/does Comcast make Netflix's life difficult enough to induce a renegotiated increased price? By intentionally allowing congestion (and refusing to remedy it) at the physical locations where Cogent's servers physically connect to Comcast's servers. These physical locations are known as Interconnection Points ("IXP").
As more as more dual Netflix-Comcast subscribers in a given geographic location attempt to stream Netflix content, the IXP where Netflix's traffic crosses onto Comcast's network becomes congested.
Former Senator Ted Stevens was famously ridiculed for his description of the internet as "a series of tubes." Yet the metaphor isn't terribly inaccurate.3 Only so much internet traffic (like HD video) can fit through the physical port connecting one ISP's network to another's. This traffic-backup is aptly named congestion.
And make no mistake: Comcast is perfectly aware of the IXP congestion, and they can easily remedy it by installing additional ports—which costs a pittance—that would alleviate the congestion.
But what reason does Comcast have to alleviate the IXP congestion? Netflix revealed that smooth traffic transit to Verizon customers was worth $10X to Netflix. So Comcast will install those new ports and alleviate the congestion—just as soon as Netflix forks over another $9X (or more).
That's pretty odious for a bunch of reasons—most of which I won't get into here—but there are two worth noting:
First, last-mile ISPs' complaints about the volume of data they receive from content/edge providers rests on a flawed premise.
Last-mile ISPs proclaim: "We send Cogent a handful of bits and in return Cogent/Netflix sends us gigabytes upon gigabytes of data!"
So who really causes last-mile ISPs to accept large volumes of data? The last-mile ISPs would have you believe that the content providers, like Netflix, cause them to take troves of data—and thus the content providers should pony up.
But there's an equally persuasive argument that last-mile ISPs—not the content/edge providers—cause the traffic. Lest we forget what, exactly, we consumers are paying last-mile ISPs for: access to the internet, the whole internet—including streaming video.
So last-mile ISPs spend millions upon millions of dollars on advertising themselves as God's gift to the universe for providing consumers access to every corner of the internet. ("The Future of Awesome.") But behind closed doors, last-mile ISPs are perfectly content providing their subscribers only those corners of the internet that fork over enough cash.
Second, large disparities in the volume of traffic accepted versus the volume sent is an asinine reason to demand more expensive traffic exchange agreements. Internet traffic is simply data. The direction it flows has no bearing on cost/expense.
The only factors that determine the cost of internet traffic transmission are (a) the volume of data; and (b) the distance the data physically travels.
So last-mile ISPs' complaint that Netflix sends them a huge volume of data is certainty legitimate. But the disparity of traffic exchanged between Comcast and Cogent doesn't cost Comcast anything; rather, the volume of traffic Comcast accepts from Netflix is the primary driver of cost; the direction the data flows is completely irrelevant.
The other internet-traffic-cost-driver is physical distance. The greater the distance data travels, the more expensive traffic transmission becomes.
(As I briefly alluded to earlier, content providers are hard at work finding ways to circumvent last-mile ISPs' consumer gatekeeping power. Presumably concerned that last-mile ISPs will soon try using transmission distance to further inflate prices, content/edge providers have started building their own data centers as close as possible to last-mile ISPs' data centers.)
So if we're going get serious about pricing internet traffic transmission rationally, then it's time to stop complaining about traffic exchange disparities, and instead focus on pricing transmission as a function of volume and distance.
This is all pretty abhorrent. But I've saved the worst for last.
First, the ongoing "Net Neutrality" proceedings at the FCC do not address any of this. Instead, the FCC's latest proposed rules only address the ability/legality of last-mile ISPs choking/slowing/degrading content over the "last mile."4
Meaning that IXP congestion—which, from a consumer's standpoint, is functionally indistinguishable from degrading content in the last-mile—isn't going anywhere.
Finally, it's critical to consider the full implications of last-mile ISPs' behavior—especially who is ultimately footing the bill.
Hint: it's you, the consumer, and you're paying your ISP three times for functional Netflix service:
- First, you pay your ISP directly each month;
- Second, your ISP charges Cogent (Netflix's ISP) to accept any of Netflix's traffic—a cost passed onto Netflix and then passed from Netflix onto you; and
- Third, the real kicker, if you want to actually watch Netflix content with minimal buffering, then every time an IXP becomes congested, Cogent and Netflix has no choice but to hand over even more cash to your ISP to alleviate the congestion—yet another cost ultimately passed down to the consumer.
Oh, don't forget that you're making a fourth payment for functional service: the Netflix subscription itself.
So if the title of this post seemed hyperbolic at first blush, just remember: You're the one paying your ISP three times—and for what? 8% of what you thought you were paying for?
It's a racket that would make Tony Soprano blush.
1. "Broadband" internet service is defined as download speeds of five mbps or greater.
2. The resolution of a Blu-ray video is 1080p. The resolution of a DVD video is 720p. Both are considered "high definition" ("HD") resolutions. By contrast, 360p is considered "standard definition" and 480p is considered "enhanced standard definition" ("SD"). Television programs broadcast in SD now use 480p over 360p. Finally, 240p, known as "Low Definition" ("LD"), was originally the resolution of a VHS magnetic tape recording.
3. The same cannot be said for his accompanying comments, expressing frustration that "an Internet was sent by my staff at 10 o'clock in the morning on Friday. I got it [on Tuesday] Why? Because it got tangled up with all these things going on [in the series of tubes]."
4. Here's another kicker: the current proposed net neutrality regulations would, in fact, allow for the creation of internet "fast lanes." They just call it "paid prioritization" instead, and poof! the FCC claims with a straight face they oppose internet fast lanes.
