Alexis C. Madrigal

Google and Verizon proposed a new regulatory framework that they say could calm the heated network neutrality debate.

The idea that the network of networks we call the Internet should be open, so that data can flow unfettered from any node to any other node is part of the mythology and operation of the current system. But companies build and own those networks. Their corporate missions -- capturing maximum profit from investment in infrastructure -- don't include openness.

As you might expect, people who use the network (big Internet companies) have tended to favor net neutrality, while the people who build and operate the network (broadband providers) have opposed it.

That's what makes the Google-Verizon proposal interesting. It's a compromise between traditional adversaries, but from what we can tell from its brief two pages, it doesn't seem likely to make net neutrality advocates happy.

First, Google and Verizon separate wireless from wireline networks. On the former, net neutrality would not have to apply. Wireless bandwidth providers would only have to provide "transparency," so that we can all see if AT&T has cut a deal with a particular content provider.

"Because of the unique technical and operational characteristics of wireless networks, and the competitive and still-developing nature of wireless broadband services, only the transparency principle would apply to wireless broadband at this time," the companies wrote.

It's hard to see how that squares exactly with the opening statement of principle that began Google's blog post on the subject. "The original architects of the Internet got the big things right. By making the network open, they enabled the greatest exchange of ideas in history," the companies wrote in a post on Google's blog. "By making the Internet scalable, they enabled explosive innovation in the infrastructure."

If open networks are good, why should wireless be different? They don't make the case in these documents for why the "unique technical and operational characteristics" should change the fundamental underlying principle of the network. That's not to say there isn't a good argument, but it's certainly not in either the blog post or the policy document.

More troubling is that the language of the wireline net neutrality is squirrely. The companies suggest that they would be maintaining "net neutrality" on wireline services, but they'd allow "additional or differentiated services" over their networks.

"Such other services would have to be distinguishable in scope and purpose from broadband Internet access service, but could make use of or access Internet content, applications or services and could include traffic prioritization," they wrote.

Again, this is just a policy paper, but this seems like a slippery definition of what is and is not Internet traffic. Why not carry these "additional services" over the Internet, where they would be subject to the net neutrality rules that these companies claim to think is a good idea?

As one commenter on Google's blog wrote, "If you can't redefine the word 'neutrality', redefine the word 'Internet' instead."

Om Malik summed it up nicely:

The temptation to accept this compromise as good for everyone may force a version of network neutrality that leaves mobile, one of the fastest areas of innovation on the web, out of the new rules. It also enables an alternative version of the public Internet that could lead to the creation of a first-class and a second-class system of packet delivery.

Do those things matter? Probably, but maybe not as much as net neutrality advocates would contend. What I'm left wondering is whether this kind of proposal -- which exudes the sickly sweet smell of political horsetrading -- is what's needed to break the net neutrality stalemate.

A German design student launched a web surfing competition called Trailblazers earlier this month in Stuttgart. Starting and ending websites are specified; the winner is the one who travels from one to the other with the fewest number of clicks. No search engines, no holds barred! The event was live and held in meatspace, as documented in the video below. The next edition of the competition will be held at ARS Electronica in Austria.

The introductory instruction is somehow deeply satisfying to my inner nerd: "Surf the classic way from amazon to pirate bay." (The classic way!)

 

Via Cara Rose de Fabio / Today and Tomorrow

Happy birthday, Walden.

Today in 1854, Henry David Thoreau released his nuanced and readable account of two years that he spent largely alone in a cabin near Concord, Massachusetts.

"His distillation of two years living in relative seclusion offers deep insights not just into the natural world and humanity's place in it, but how that relationship was being impacted -- and degraded -- by the Industrial Revolution," Wired's Randy Alfred reminds us. "It remains to this day a trenchant criticism of the excesses of technology,"

Walden is a fantastically good book, and Thoreau's unadorned style feels shockingly contemporary, even if his analysis of networks differs from our own. "We are in great haste to construct a magnetic telegraph from Maine to Texas; but Maine and Texas, it may be, have nothing important to communicate," he wrote. And in one of the most famous and beautiful passages from the book, we read:

We do not ride on the railroad; it rides upon us. Did you ever think what those sleepers are that underlie the railroad? Each one is a man.... The rails are laid on them, and they are covered with sand, and the cars run smoothly over them. They are sound sleepers, I assure you. And every few years a new lot is laid down and run over; so that, if some have the pleasure of riding on a rail, others have the misfortune to be ridden upon.

Looking back at Thoreau, though, it's important to realize that he was as out of sync with his own times as he sometimes seems with ours. He's part of a long-standing American counterculture, the one that wonders whether all of our irritable striving to build and buy things is worth the bother.

The prominent journal, The North American Review declared as early as 1832 that "the general sentiment is decidedly, so far as we have been able to ascertain it, in favor of machinery. A few apostles of the opposite doctrine have arisen here and there; but their converts have not been numerous." The American love for machinery was widespread, and as historian Hugo Meier noted, "perplexed European observers."

In a country where so many gamely adopt the latest new gadget, we need our Thoreaus, not to stop the profusion of technology, but simply to remind us to use them well. There are spaces shot through our massively complex society to find "Simplicity! Simplicity! Simplicity!" by simply deciding to look for it.

Take another grave and important personality of the time, Abraham Lincoln. His views on technology, delivered in a series of speeches on "Discoveries and Inventions" in the years directly after Thoreau's Walden, were more positive. For Lincoln, technology did not debase humanity, as Thoreau would have contended, but it also wasn't a magical staircase leading to a better world under the label of Progress.

"Although convinced that 'discoveries and inventions' had rescued humankind from savage beginnings, produced abundance, and put genuine democracy within reach, Lincoln recognized that advancing technology alone would not guarantee freedom, but might bring new forms of mastery," the historian Eugene Miller summarized in a 2001 article for The Review of Politics. "Lincolnian statecraft seeks to moderate or limit this advance not through stringent controls, but by a moral teaching that builds on the natural to oneself and includes a doctrine of labor."

Skype, the Internet communications company once owned by eBay, is going public, according to documents documents filed today with the Securities and Exchange Commission. A company's prospectus is a rich source of small datapoints about how a place operates. Here are five interesting tidbits we noted combing through the filing:

• Skype has 560 million total users, 124 million monthly log-ins, and 8.1 million monthly paying users. (All numbers substantially up over 2009.)
• In the first six months of 2010, Skype users made 95 billion minutes of voice and video calls. Over the same period, the company generated $406.2 million in revenue. Even though not everyone is paying, that still works out to about 2.3 cents per minute averaged over the entire user base.
  
• 40 percent of Skype calls were video-to-video.
  
• The average paying Skype user spends $96 a year.
• Skype has to pay credit card companies back when users make fraudulent purchases with stolen credit cards. In 2009, that cost them $5.8 million. 

And though it's not really a factoid, I liked this confirmation that Europeans really do take the summer off:

Our net revenues exhibits seasonality because many of our users reduce their use of our products with the onset of good weather during the Northern Hemisphere's summer months and our users tend to use our products more in the fourth quarter during the holiday season resulting in weaker net revenue growth during the second and third quarter of the year. Furthermore, we experience significant spikes in the use of our products during significant world events, such as Christmas and the Chinese New Year, or regional events, such as the recent volcanic eruption in Iceland.

Twitter became substantially more useful over the past couple of months, courtesy of two small changes to the way the service's real-time search works. Neither tweak was announced, which is a shame because they might be the best things the service has done since it gained traction a few years ago. 

First, now, when you search for something like "Inception" or "Elena Kagan," the top three search results are calculated by the number of retweets that comment has received. Functionally, this means that the funniest, most relevant, or interesting tweets get surfaced. (See above.) Before, the real time search function used to bring up just the most recent tweets, which were often random or boring.

Second, because of the character limitations on Twitter, nearly everyone uses URL shorteners to post links. Some time in May, Twitter started peeking into those links. What that means is if you search "theatlantic.com," anybody linking to this site will appear, even if they use a URL shortener. For media websites, that's an incredibly useful bit of functionality because we can capture a larger percentage of the discussion around our posts. (See below.)

Though both changes were rolled out weeks ago, you may not have noticed them if you mostly use Twitter through a client like Tweetdeck and don't find yourself at Twitter.com very often. Perhaps, after years of playing catch up with user growth, Twitter's engineers are finally getting around to implementing useful and new features. (Or so we can hope -- when I went to tweet this story, I (of course!) got a fail whale.)

In an interview with Vibe Magazine, famed producer and rapper Dr. Dre dropped the most exciting tidbit the space hip hop world has ever heard.

Dr. Dre is working on an instrumental album about what the various planets sound like!

An instrumental album is something I've been wanting to do for a long time. I have the ideas for it. I want to call it The Planets. I don't even know if I should be saying this, but fuck it. [Laughs.] It's just my interpretation of what each planet sounds like. I'm gonna go off on that. Just all instrumental. I've been studying the planets and learning the personalities of each planet. I've been doing this for about two years now just in my spare time so to speak. I wanna do it in surround sound. It'll have to be in surround sound for Saturn to work.

Quick, somebody tell NASA! Ok, I'll do it.

Update: Accidentally left a 'p' out of one of Dre's titles up there. Now added.

[Via Rap Radar]

Mysterious and possibly nefarious trading algorithms are operating every minute of every day in the nation's stock exchanges.

What they do doesn't show up in Google Finance, let alone in the pages of the Wall Street Journal. No one really knows how they operate or why. But over the past few weeks, Nanex, a data services firm has dragged some of the odder algorithm specimens into the light.

The trading bots visualized in the stock charts in this story aren't doing anything that could be construed to help the market. Unknown entities for unknown reasons are sending thousands of orders a second through the electronic stock exchanges with no intent to actually trade. Often, the buy or sell prices that they are offering are so far from the market price that there's no way they'd ever be part of a trade. The bots sketch out odd patterns with their orders, leaving patterns in the data that are largely invisible to market participants.

In fact, it's hard to figure out exactly what they're up to or gauge their impact. Are they doing something illicit? If so, what? Or do the patterns emerge spontaneously, a kind of mechanical accident? If so, why? No matter what the answers to these questions turn out to be, we're witnessing a market phenomenon that is not easily explained. And it's really bizarre.

It's thanks to Nanex, the data services firm, that we know what their handiwork looks like at all. In the aftermath of the May 6 "flash crash," which saw the Dow plunge nearly 1,000 points in just a few minutes, the company spent weeks digging into their market recordings, replaying the day's trades and trying to understand what happened. Most stock charts show, at best, detail down to the one-minute scale, but Nanex's data shows much finer slices of time. The company's software engineer Jeffrey Donovan stared and stared at the data. He began to think that he could see odd patterns emerge from the numbers. He had a hunch that if he plotted the action around a stock sequentially at the millisecond range, he'd find something. When he tried it, he was blown away by the pattern. He called it "The Knife." This is what he saw: 

"When I pulled up that first chart, we saw 'the knife,' we said, that's certainly algorithmic and that is weird. We continued to refine our software, honing the algorithms we use to find this stuff," Donovan told me. Now that he knows where and how to look, he could spend all day for weeks just picking out these patterns in the market data. The examples that he posts online are just the ones that look the most interesting, but at any given moment, some kind of bot is making moves like this in the stock exchange.

"We probably get 10 stocks in any 10 minutes where we see something like this," Donovan said. "It's happening all the time."

These odd bots don't really make sense within the normal parameters of the high-frequency trading business. High-frequency traders do employ algorithms to look for patterns in the market and exploit them, but their goal is making winning trades, not simply sending quotes into the financial ether.

Here's the way a stock trade is supposed to work: a buyer says they'll pay some amount for 100 shares of a company, a seller makes an ask for slightly more money, and the two of them usually meet in the middle. Perhaps a middle man (no joke intended) helps match buyer and seller and takes a cut. That's the role that a lot of high-frequency traders play: they help make markets work. Regulatory changes over the past several years have extended their usefulness and provided a nice business model for those that can move quickly to provide options for buyers and sellers.

"Under the maker-taker model, market participants that offer to provide, or make, liquidity by posting an order to buy or sell a certain number of shares at a particular price receive a rebate," explained Michael Peltz in a June feature for Institutional Investor. "Those that execute against that order -- that is, take the liquidity -- have to pay a fee. Exchanges earn the difference between the rebate they pay and the fee they charge. The SEC limits taker fees to 0.30 cents a share; rebates tend to be lower for economic reasons, but for high frequency firms trading millions of shares a day, they can make for a pretty good living."

In a sense, they take nickel-and-diming down an order of magnitude or two. The advantage is that their trades are low-risk: they rarely hold positions for very long and any individual stock, future, or currency can't really sink the boat. High-frequency traders have become a target for all kinds of people, but most of them appear to make their money being a little faster and little smarter than their competitors. And if they are playing by the rules, they improve the quality of markets by minuscule amounts trade after trade after trade.

But the algorithms we see at work here are different. They don't serve any function in the market. University of Pennsylvania finance professor, Michael Kearns, a specialist in algorithmic trading, called the patterns "curious," and noted that it wasn't immediately apparent what such order placement strategies might do.

Donovan thinks that the odd algorithms are just a way of introducing noise into the works. Other firms have to deal with that noise, but the originating entity can easily filter it out because they know what they did. Perhaps that gives them an advantage of some milliseconds. In the highly competitive and fast HFT world, where even one's physical proximity to a stock exchange matters, market players could be looking for any advantage. 

"They are moving the high-frequency services as close to the exchanges as possible because even the speed of light matters," in such a competitive market, said Stanford finance professor Peter Hansen.

Given Nanex's data, let's say that these algorithms are being run each and every day, just about every minute. Are they really a big deal? Donovan said that quote stuffing or market spoofing played a role in the Flash Crash, but that event appears to have had so many causes and failures that it's nearly impossible to apportion blame. (It is worth noting that European markets are largely protected from a similar event by volatility interruption auctions.)

But already since the May event, Nanex's monitoring turned up another potentially disastrous situation. On July 16 in a quiet hour before the market opened, suddenly they saw a huge spike in bandwidth. When they looked at the data, they found that 84,000 quotes for each of 300 stocks had been made in under 20 seconds.

"This all happened pre-market when volume is low, but if this kind of burst had come in at a time when we were getting hit hardest, I guarantee it would have caused delays in the [central quotation system]," Donovan said. That, in turn, could have become one of those dominoes that always seem to present themselves whenever there is a catastrophic failure of a complex system.

There are ways to prevent quote stuffing, of course, and at least one of the members of the Commodity Futures Trading Commission's Technology Advisory Committee thinks it should be outlawed.

"Algorithms that might be spoofing the market are something that should be made illegal," said John Bates, a former Cambridge professor and the CTO of Progress Software. But he didn't want this presumably negative practice to color the more mundane competitive practices of high-frequency traders.

"There is algorithmic terrorism and then there is reverse engineering, which is probably just part of good business practice," Bates said.

For now, Donovan plans to keep putting out the charts, which he calls "crop circles," of the odd trading algorithms at work. That's an apt name for the visualizations we see of this alien world of bot trading. And it certainly gets at a central mystery surrounding them: if trading firms aren't sending out these orders, how are they getting into the market?

On the quantitative trading forum, Nuclear Phynance, the consensus on the patterns seemed to be that they simply just emerged. They were the result of "a dynamical system that can enter oscillatory/unstable modes of behaviour," as one member put it. If so, what you see here really is just the afterscent of robot traders gliding through the green-on-black darkness of the financial system on their way from one real trade to another.

No matter why the bots end up executing these behaviors, the Nanex charts offer a window onto a kind of market behavior that's fascinating and oddly beautiful. And we may never have seen them, if not for the mildly obsessive behavior of one dedicated nerd.

"Who looks at millisecond charts?" Donovan said. "You'd never see those patterns in any other fashion. The SEC and CFTC certainly weren't."

Here are a few more bots at work with explanations of what's going on.

Here we see a "flag repeater" being executed on the BATS Exchange, the third-largest equity market after the NYSE and NASDAQ. 15,000 quote requests were made in 11 seconds in a repeating pattern. Each iteration upped the quote a penny until $9.36, and then the algorithm went down the same way, a penny at a time.

This is an extreme closeup of just one second of trading of the stock SHG, the Shinhan Financial Group. This is 760 quotes from a total of 10,000 made in 12 seconds.

This chart shows a different kind of strategy. It represents 56,000 quotes in one second all at the same price (the top chart) but with the size of the order increasing by one (i.e. 100 shares) all the way up to 40,000.

I used to think that you couldn't do anything in the humanities in just one week, but maybe times are changing.

Last week, twelve scholars came together at the Center for History and New Media at George Mason University to participate in the inaugural One Week, One Tool program. Supported by the National Endowment for the Humanities, their mandate was to build something useful: they called it a barn raising, and the structure they built is Anthologize.org.

Anthologize is a Wordpress plugin that allows scholars, conference organizers, and bloggers to create eBooks out of websites. Its creators imagine it could be used by researchers to "sketch ideas, collaborate with co-authors, edit and develop research notes into arguments, publish conference proceedings, and engage in public scholarly communication without the typical barriers." Or perhaps teachers will turn their class blogs into custom publications.

So, what separates Anthologize from commercial blog-to-book services like Blurb or Lulu? (Both fantastic services, IMHO.) First, it's a Wordpress plugin, so if you're familiar with that tool (as many are), it should be easy to manipulate.

"Because it's open source, third-party developers can create translators and importers for other formats as well, and contribute them back," added Doug Knox, director of publication and digital initiatives at the Newberry Library, and part of the One Week, One Tool team. "Lulu and Blurb -- and others like FastPencil -- are focused on commercial blog-to-book publishing. They don't have as much flexibility in importing existing content, and they aren't as open to extending the range of output formats."

Anthologize doesn't currently support turning out actual printed books, but its PDFs could be uploaded to a Lulu.

I'm most impressed with the whole approach of the program. It's not just that they go a small team to do something (anything!) in seven days, but that they focused on scholarly infrastructure. They didn't just create an anthology, they created an anthologizer, and that seems worth applauding.