Alexis Madrigal is a senior editor at The Atlantic, where he oversees the Technology channel. He's the author of Powering the Dream: The History and Promise of Green Technology. More
The New York Observer calls Madrigal "for all intents and purposes, the perfect modern reporter." He co-founded Longshot magazine, a high-speed media experiment that garnered attention from The New York Times, The Wall Street Journal, and the BBC. While at Wired.com, he built Wired Science into one of the most popular blogs in the world. The site was nominated for best magazine blog by the MPA and best science Web site in the 2009 Webby Awards. He also co-founded Haiti ReWired, a groundbreaking community dedicated to the discussion of technology, infrastructure, and the future of Haiti.
He's spoken at Stanford, CalTech, Berkeley, SXSW, E3, and the National Renewable Energy Laboratory, and his writing was anthologized in Best Technology Writing 2010 (Yale University Press).
Madrigal is a visiting scholar at the University of California at Berkeley's Office for the History of Science and Technology. Born in Mexico City, he grew up in the exurbs north of Portland, Oregon, and now lives in Oakland.
Steve Chu has a new set of advisors, the Department of Energy announced today. The twelve-member board is surprisingly old-guard for a DOE chief from outside the Beltway.
These are insiders who know how to fight in Washington's trenches. They are not Silicon Valley. They are not really innovators, aside from Art Rosenfeld and Nick Donofrio. Energy may be an avocation for many on the list, but few have dedicated their lives to the field.A couple of examples to make the point: John Deutch advised Jimmy Carter on energy, then went on to helm the CIA under Bill Clinton. William Perry served as Secretary of Defense in the 1990s and is affiliated with the Hoover Institution. Norm Augustine was the CEO of Lockheed Martin, an under secretary for the Army, and chaired a controversial panel on NASA's future last year. If Augustine were a Pulp Fiction character, he would be The Wolf (Harvey Keitel), the mob fixer brought in to clean up messes.
If open networks are good, why should wireless be different? They don't make the case in these documents for why the "unique technical and operational characteristics" should change the fundamental underlying principle of the network. That's not to say there isn't a good argument, but it's certainly not in either the blog post or the policy document.
More troubling is that the language of the wireline net neutrality is squirrely. The companies suggest that they would be maintaining "net neutrality" on wireline services, but they'd allow "additional or differentiated services" over their networks.
"Such other services would have to be distinguishable in scope and purpose from broadband Internet access service, but could make use of or access Internet content, applications or services and could include traffic prioritization," they wrote.
Again, this is just a policy paper, but this seems like a slippery definition of what is and is not Internet traffic. Why not carry these "additional services" over the Internet, where they would be subject to the net neutrality rules that these companies claim to think is a good idea?
As one commenter on Google's blog wrote, "If you can't redefine the word 'neutrality', redefine the word 'Internet' instead."
The temptation to accept this compromise as good for everyone may force a version of network neutrality that leaves mobile, one of the fastest areas of innovation on the web, out of the new rules. It also enables an alternative version of the public Internet that could lead to the creation of a first-class and a second-class system of packet delivery.Do those things matter? Probably, but maybe not as much as net neutrality advocates would contend. What I'm left wondering is whether this kind of proposal -- which exudes the sickly sweet smell of political horsetrading -- is what's needed to break the net neutrality stalemate.
We do not ride on the railroad; it rides upon us. Did you ever think what those sleepers are that underlie the railroad? Each one is a man.... The rails are laid on them, and they are covered with sand, and the cars run smoothly over them. They are sound sleepers, I assure you. And every few years a new lot is laid down and run over; so that, if some have the pleasure of riding on a rail, others have the misfortune to be ridden upon.
Our net revenues exhibits seasonality because many of our users reduce their use of our products with the onset of good weather during the Northern Hemisphere's summer months and our users tend to use our products more in the fourth quarter during the holiday season resulting in weaker net revenue growth during the second and third quarter of the year. Furthermore, we experience significant spikes in the use of our products during significant world events, such as Christmas and the Chinese New Year, or regional events, such as the recent volcanic eruption in Iceland.
Technically speaking, there just isn't a simple "ignore my own phony orders" button that a trading firm could press.
"What a firm has is nine real time data feeds from the exchanges [e.g. NASDAQ] that are telling them what the quotes are from those exchanges in real time. Let's say I'm flooding some exchange, how do I know which orders to ignore?" Kearns asked. "I at least have to have my code pick up each incoming order to inspect it just enough to know it's my order, but then I haven't ignored it at all. These orders are very simple. You can look up the raw data. And each one is like a line of text. The expensive thing is not doing something complicated to that line of text, it's inspecting it in the first place."
The second reason that quote-stuffing is unlikely is slightly more difficult to understand. The basic idea is that we can only see the algorithms working in stocks on exchanges that are illiquid. There aren't a lot of buyers and sellers around. In fact, there aren't any. If there were, we wouldn't be able to see the patterns with such clarity because other people's bids and asks would mess them up. "That creates a problem with the argument that it's being done to slow down competitors," Kearns concluded. Essentially, on these specific stocks on these specific exchanges at these specific times, there aren't competitors to slow down.
So, if it's not quote-stuffing, why would a firm engage in this behavior? Lo and Kearns offered a few theories of their own about what could be happening.
"To be honest, we can't come up with a good reason," Kearns said. What's particularly difficult to explain is how diverse and prevalent the patterns are. If algorithmic traders are simply testing new bots out -- which isn't a bad explanation -- it doesn't seem plausible that they'd do it so often. Alternatively, one could imagine the patterns are generated by some set of systemic information processing mistakes, but then it might be difficult to explain the variety of the patterns.
Kearns does have a leading explanation, though, which he emailed to me after we spoke.
"It's possible that the observed patterns are not malicious, in error, or for testing, but for information-gathering," Kearns observed. "One could easily imagine a HFT shop wanting to regularly examine (e.g.) the latency they experienced from the different exchanges under different conditions, including conditions involving high order volume, rapid changes in prices and volumes, etc. And one might want such information not just when getting started, but on a regular basis, since latency and other exchange properties might well be expected to change over time, exhibit seasonality of various kind, etc. The super-HFT groups might even make co-location decisions based on such benchmarks."
MIT's Andrew Lo, who is the director of the school's Financial Engineering laboratory, offered a variation on that thesis. He contends that the algorithms are being used not to test latency but to probe the actual market conditions.
"What I think is going on is that there are algorithms that have been designed to monitor the markets and essentially create a kind of trolling function to try to identify orders that might be executed and to do that in a regular and relatively systematic way," he said.
He likened the algorithms to "financial radar."
"I think this is not random nor is it hard to understand what the motive is," Lo contended. "If you think about the way modern radar works, if you didn't know anything about radar, what you would see is pattern of electromagnetic radiation shot out at regular interviews and then you'd see patterns of the reflections of the objects out there.This is financial radar that we're seeing."
Traders want to put out tens of thousands of orders in a really short period of time precisely because they are probing for the split second when a buyer or seller shows up.
"Suppose that you would like to identify to the nearest millisecond when an order is placed and at what price. If you want to detect the trade to the nearest millisecond, you are going to have to submit orders faster than that," Lo said. "The pattern gives you a sense of the fineness of the mesh that's being constructed to try to capture the first trade that occurs."
That first trade acts as a forecast for where the price of the stock is going.
"If you see an order that's being placed at $1.05 at time T, and $1.06 at time T+1 then you start betting on that for the next few milliseconds," Lo explained. "The faster you can detect the trend, the more likely you are to make money."
Lo even offered a way of testing the algorithms to see if he's right about what they're doing. He figures that if you could squeeze into the patterned trading and take them up on an offer, they might switch into a different phase of operation.
"What would be interesting but potentially expensive to do is when you would detect patterns like this would be to trigger an order to hit the bid on the offer on one of these regular sweeps and see what happens to the pattern," Lo said.
Kearns argued, though, that the kinds of wild ordering strategies that we see aren't necessary to probe the market. "What's weird about these patterns, the sawtooth patterns, say," he said, "where you're alternating [prices up and down] with no hope of a trade is that If I were going to explore the idea of a large number of patterns to see what works, there's just no need to place orders that far from the market, especially given how quickly they are being removed."
The only people who know for sure what's going on in the market are the traders themselves and the exchanges on which they work.
At the highest level, though, the robot traders provide a unique lens on exactly how fast and complex our financial system has become. Upon the discovery of this new and apparently pervasive behavior, it is not immediately clear how to explain it, even for the brightest minds in the field.
Our regulators have tools built for assessing a market measured in seconds, but technology has pushed the markets down to the millisecond level.
"The observation to make is that this isn't as innocuous as it might seem, not so much because there is anything wrong with high-frequency trolling but rather because the regulatory infrastructure that monitors these markets are not designed to deal with this kind of latency and high-frequency," Lo pointed out. "That can create significant problems, not the least of which is the Flash Crash. There are fairness issues. There are transparency issues. There are stability issues. We need to resynchronize the regulatory infrastructure with the technology of our time."
"We're seeing innovations that dramatically increase the speed and throughput of the market, and that works great until it doesn't," Lo concluded. "And when you have some problem, like the flash crash, then you'll have version 2.0 and people will fix it. We're still in version one-dot-something and there are certainly improvements that have to be made in the regulatory infrastructure."
Images: All images courtesy of Nanex. Full explanations of the patterns at their site.
An instrumental album is something I've been wanting to do for a long time. I have the ideas for it. I want to call it The Planets. I don't even know if I should be saying this, but fuck it. [Laughs.] It's just my interpretation of what each planet sounds like. I'm gonna go off on that. Just all instrumental. I've been studying the planets and learning the personalities of each planet. I've been doing this for about two years now just in my spare time so to speak. I wanna do it in surround sound. It'll have to be in surround sound for Saturn to work.Quick, somebody tell NASA! Ok, I'll do it.
A new tool from the Sunlight Foundation embeds campaign finance information right into online news stories. Poligraft automatically exposes financial relationships between people and organizations, a function that would have required deep journalistic digging just a few years ago.
I ran a few of our political stories through Poligraft and not every one pulled up much interesting. But sometimes, the tool does reveal an important piece of context.
Take our recent story about John McCain's battle to find out more information about a satellite program that's currently administered by Lockheed Martin. Run the story through Poligraft and it highlights John McCain and Lockheed Martin, then calls up a campaign finance database to tell you how much money ($131,475) the company has given to the politician.
Campaign finance information has long been publicly accessible online, but it's rarely right at your fingertips when you might want it. Sunlight has effectively shrunk the distance between the data and the news -- and we love it.
There's even a browser bookmarklet that lets you follow the money with a single click.
Sign up to receive our free newsletters