Alexis C. Madrigal

Close of Business is a new video series that we're trying out. The idea is simple: at 5 p.m. (or thereabouts), we post a quick video summarizing the top three news stories of the day. Some of them we'll have written about; others will just be what people were talking about on the Internet.

Links to stories mentioned in this video:

Google Instant Starts to Hit iOS [9 to 5 Mac]

Kinect Pushes Users Into Sweaty New Dimension [The New York Times]

4% of Americans Use Location-Based Services [Pew]

While here in the U.S. of A., SunChips' noisy (but biodegradable!) bag got tossed out by Frito-Lay, the Canadians get to keep theirs. Here, we see Frito-Lay Canada's sustainability chief telling you all about that decision.

Bonus: if you still think the bag is too loud, they'll send you a free pair of ear-plugs. As you'll all recall, the SunChips bag sent me into a paroxysm of moralizing and despair. This video will inspire no such outburst.

Hat tip to Mark Karayan.

Close of Business is a new video series that we're trying out. The idea is simple: at 5 p.m. (or thereabouts), we post a quick video summarizing the top three news stories of the day. Some of them we'll have written about; others will just be what people were talking about on the Internet.

Links to stories mentioned in this video:

No GOP wave for Silicon Valley's Meg Whitman and Carly Fiorina [San Jose Mercury News]

Why Facebook Will Be Better than Foursquare [The Atlantic]

Meet Soylent, the Crowdsourced Copy Editor [Nieman Journalism Lab]

Facebook held a press conference today to discuss the latest in their mobile features. They talked about a streamlined log-in process and the addition of "deals" to their recently launched location-based Places application. They've also created a Places search API to enable mobile, social, local applications.

But it was while Dave Fetterman, an engineering manager, was talking that I was struck by a realization about why Facebook's Places platform is going to be better than its competitors.

"Places are ranked by how important they are to that user," Fetterman said.

Just like Facebook filters your News Feed based on what's most relevant to you and your friends, it's going to filter real-life locations in precisely the same way. It will be seamless and easy and ridiculously useful.

So, if all my friends check into Beretta in San Francisco, and I hit the "Nearby" button in Places, it will show Beretta at the top of a list of restaurants in the Mission. That's powerful. Think about it as a travel tool, too. You go to New York, where you probably have some friends. Even if you've never been there and have no idea where to go, your friends checking in at their favorite places will provide you with a custom list of places to go in the city.

All Facebook needs is everyone to start checking in and generating data. And that's where these new "deals" come into play. They enable businesses to give what amounts to a virtual coupon to proximate consumers. Facebook doesn't take a cut of that transaction or anything because they don't care about it. What they want is the data that you're generating so they can use it to fine-tune their algorithms. And as a bonus, local businesses market Facebook's product, Places.

One big example? Some day soon, Gap is going to give away 10,000 pairs of blue jeans to the first 10,000 people who check-in at their stores. And they'll be giving a 40% discount to those who check-in after that. Suddenly, tens of thousands of people will have used Places and hundreds of thousands (or millions) more will have heard about the promotion. Gap will drive foot traffic, 10,000 people will get free pants, and Facebook will silently collect their data and use it to best their location-based competition.

Could Foursquare and other competitors get better at optimizing results based on your connections? Sure, but they just don't have the scale of 200 million mobile users, the preexisting social network, nor the experience developing social algorithms that people like.

The drama at the intersection of Big Data and Big Money continues. The hoary London Stock Exchange is trying to keep pace with alternative trading platforms by rolling out a new system for high-frequency traders. Unfortunately, it's having trouble. Yesterday, trading was disrupted for two hours when the platform, known as Turquoise, encountered problems.

A preliminary investigation revealed that the issues were caused by "a human error" that "may have occurred in suspicious circumstances." No more details are available, and that statement is impossible to parse, but the upshot is that further planned whizbang upgrades to serve the needs of data-driven traders have been postponed.

The strange circumstances highlight an undercovered trend in the financial markets: the proliferation of alternative exchanges that cater to high-frequency traders. For example, pop quiz: what's the third largest equity market? The answer is BATS, which didn't exist in 2004.

The old-boys -- the LSE and NYSE -- have been steadily losing market share to these upstarts. To defend their competitive positions, they've been trying to keep up technologically, but it's not easy to deliver ultra-low latency and high-throughput trading, particularly when the incentives for disrupting the system are so large.

Via IEEE Spectrum.

Election Day is finally upon us, and most people are focusing on the minute-to-minute results streaming out of precincts across the country. Twenty or thirty years from now the specifics of elections won't mean much, but I do think that we'll remember this election because it heralded the reemergence of southern radical rightism.

A few weeks ago, Sean Wilentz provided a modern historical take in The New Yorker on the connection between the radical right groups of the past and the Tea Party of today. It's a fascinating treatment, but I actually think Betty Chmaj's 1962 article for The Atlantic, Paranoid Patriotism, may provide us with a better idea about how we might remember the Tea Party decades hence. Here are just a few excerpts and insights from her investigation.

"The emerging power of the radical right-wing groups in the South is a source of concern in national politics. The combination of two fears -- fear of Communism and fear of the Negro -- forms a particularly explosive threat to politics, education, and calm thinking. BETTY E. CHMAJ, who teaches at the University Center for Adult Education in Detroit, analyzes the types, traditional background, and possible effect on this fall's elections of the Southern radical right."

Chmaj's key insight is that, though the far right groups nominally organized to combat domestic Communists, there were none where they lived. "The enemy was a phantom," she wrote. And because the enemy was a phantom, it could be anywhere or anything. Communism, in the hands of the far right, became a kind of glue that bound together all the various things they didn't like under one big heading. She quotes the Alabaman Admiral John G. Crommelin tying together the various strands that wove their way through the fanatical side of the southern far right.

Crommelin announced his candidacy for governor in 1958 by declaring that the state of Alabama had been "selected by the Communist-Jewish Conspirators as the proving ground," to test means for carrying out their "satanic plot to mix the blood of the White Christian people of the South with negroes," in order to achieve their ultimate objectives," which are:

to use their world-wide control of money to destroy Christianity and set up a World Government in the framework of the United Nations, and erase all national boundaries and eliminate all racial distinctions except the so called Jewish race, which will then become the masters -- with their headquarters in the State of Israel and in the UN in New York, and from these two communications centers rule a slave-like population of copper-colored mongrels...

Granted, Crommelin appears to have been on the fringe, but his willingness to entangle so many different fears and prejudices into just a couple of sentences is impressive -- and familiar. More mainstream radical rightists in the 60s, Chmaj wrote, had similar tendences. "The amalgam that always includes the Communists, the Warren Court, and the N.A.A.C.P. was, from the first, designed to take in other enemies of the South as well." Early pamphlets from the Citizens Councils in the South argued along similar lines. "The integration scheme ties right in with the new one world, one creed, one race philosophy fostered by the ultra-idealists and the international left-wingers."

Racial conflict and Communism and and religious sentiment and the federal government's power all stewed together. Where one might imagine that such a movement would fall apart, particularly when business rightist interests conflicted with racial rightist interest. But Chmaj said that the movement was held together by a force to "which the Southern right consistently appeals, one which operates to obfuscate internal differences and effectively paralyze dissent."

Let me call this force naive conservatism and define it as a utopian longing to revive the simpler society of a bygone time; a dogmatic insistence upon the cleavage between good and evil, right and wrong, loyalty and treason; and a capacity to romanticize these dogmatisms with a glow of unreality and an air of innocence that serve to blunt their cutting edges. The politician appealing to naive conservatism characteristically explains that he is fighting the Communist menace because he loves his children. Or because America is a Christian country. Such attitudes have deep roots in Southern history.

Have things changed much?

When we look back at the Tea Party, I think we'll see it like the earlier southern radical movement Chmaj chronicled. It will look like a response to a horrible economy and structural shifts in balance of power in the world. This country weathered that far right surge -- and our moderate politics will prove resilient to this one, too. But as both Chmaj and Sean Willentz noted, it's going to take dedicated people within the right itself to contain the ultraright.

"[T]oo hard a push to the right may boomerang, if enough responsible conservatives are forced to repudiate their more extreme colleagues, leaving the impression that the resurgent right-wing movement is little more than a collection of fanatics and demagogues trading on the fear of Communism," Chmaj wrote.

Close of Business is a new video series that we're trying out. The idea is simple: at 5 p.m. (or thereabouts), we post a quick video summarizing the top three news stories of the day. Some of them we'll have written about; others will just be what people were talking about on the Internet.

Links to stories mentioned in this video:

Dells Boomi Buy: Here's What it Means [GigaOm]

Will Netflix Destroy the Internet? [Slate]

Sealed Courtroom Sought in High-Speed-Trading-Code-Theft Case [Wired]

Voting has seen its share of technological change since the founding of our Republic. From paper ballots filled out by hand to the Votomatic to the latest in touchscreen computerized voting, the practice of democracy has long been carried out through the technology of the day.

For your 2010 election day viewing pleasure, here's a gallery of the various devices that we've used to cast our ballots. Thanks go to the Smithsonian's National Museum of American History, which maintains specimens of the many iterations of this country's voting machines.

Social science experiments are important. They allow us to ground sociological and economic theory in real human behavior, and helps establish causal links between events.

"But the problem is that they are a a pain in the ass to do," said David Rand, a Berkman Center for the Internet and Society fellow. "You have to get someone to get down to the lab and spend time in transit. There are fixed time costs."

The Berkman Center is a leading institution for important research into the uses and impacts of digital technologies. We'll be previewing their regular brownbag lunches here on The Atlantic Technology Channel.

That's why he and fellow online social experiment pioneers like Harvard's John Horton are turning to Amazon's Mechanical Turk and other online labor markets to find willing participants. The services take the pain out of doing social science experimentation, decreasing costs and increasing the speed of idea development.

For example, Rand submitted a paper to a journal, which asked that he carry out another experiment before accepting his work. "I got the rejection on Thursday. I designed the experiment on Friday and by Sunday I had 500 people recruited," he said.

Rand will be talking about the value of these online research tools at the Berkman Center today. You can tune in live.

Mechanical Turk allows anyone to submit a set of small tasks, which are farmed out to a bunch of anonymous workers all across the world for tiny amounts of money. It tends to draw people from India, who both speak English and can make meaningful cash from completing tasks for a few dimes a piece.

How do these experiments actually play out? Researchers design experiments that mimic the kind of work people on the sites normally do -- say, labeling photos -- but with small manipulations that allow them to test economics ideas.

So, a whole class of economics experiments ask people to split money between themselves and a stranger. People tend to act fairly generously when presented with that opportunity. But Rand said that some have questioned the results of that experiment because the money is seen as "free, manna from heaven." Using Mechanical Turk, Horton designed an experiment where the split came in the form of divvying up a bonus for real work.

Of course, for all these experiments to contribute to the existing literature, people like Rand have to prove that their work has validity in the offline world. He's confident that it does. At his talk, he'll present evidence for "why it's reasonable that the people you find on the online labor markets are representative of normal people."

And, Mechanical Turk workers are less likely to be WEIRD -- Western, Educated, Industrialized, Rich, and Democratic -- than college undergraduates, the normal subjects for economics experiments, which is probably a good thing.

Close of Business is a new video series that we're trying out. The idea is simple: at 5 p.m. (or thereabouts), we post a quick video summarizing the top three news stories of the day. Some of them we'll have written about; others will just be what people were talking about on the Internet.

Today, I also show you a little bit about the way we make art for the Technology Channel.

Links to stories mentioned in this video:

Inside the Google Books Algorithm [The Atlantic]

New Search Engine Is a Great Concept But... [The Atlantic]

Promoted Tweets: Testing in the Timeline [Twitter]

Google is famous for the brilliance of its algorithm for searching web pages. While the company looks at dozens of factors in determining which results to display, the heart of the search engine is using links between pages to rank their relevancy. We have come to depend on Google to give us exactly what we want.

But what about when the company has to reach outside the web? The printed volumes represented on Google Books form a completely different kind of problem. Google's famous algorithm can't be deployed to search through books because they don't link to each other in the way that webpages do. There is no perfect BookRank corollary for PageRank.

All of which made me wonder: How does Google Books work? What makes it tick? It turns out that it's actually a great place for the company's engineers to learn how to function in a linkless, physical world.

"There is a meaningful effort to say, how do we tune for books? We've got a lot of people doing very focused on the web. How do we take the lessons from what we learned on the web and invent new things that are unique to books?" Matthew Gray, lead software engineer of Google Books, told me.

The system they've come up with has become increasingly sophisticated, as highlighted by their latest tweak, Rich Results, which begins rolling out this afternoon. The feature selectively presents you with one extra-large result when it detects that you're probably searching for an individual title and not a specific mote of information or general topic.

Rich Results is the latest in a series of smaller front-end tweaks that have been matched by backend improvements. Now, the book search algorithm takes into account more than 100 "signals," individual data categories that Google statistically integrates to rank your results. When you search for a book, Google Books doesn't just look at word frequency or how closely your query matches the title of a book. They now take into account web search frequency, recent book sales, the number of libraries that hold the title, and how often an older book has been reprinted.

So, if you search "Help" now, you get a big blow-up of Kathryn Stockett's 2009 book, not one of the dozens of other books with the same title. Or if you search "dragon tattoo," you get Stieg Larsson's blockbuster, not the 2008 children's book actually called Dragon Tattoo.

"One of the fundamental things we've learned is that the whole is greater than the sum of the parts," Gray said.

This is deeply Google thinking but without the dominant algorithm. It's a Google subspecies that evolved by feeding on a different corpus. There is less data about books than web pages, but there is more structure to it, and there's less spam to contend with. Yet the focus on optimizing an experience from vast amounts of data remains. "You want it to have the standard Google quality as much as possible," Gray said. "[You want it to be] a merger of relevance and utility based on all these things."

The most difficult part of making Google Books work, said James Crawford, the team's engineering director, was determining the intent of the service's heterogeneous user base. Scholars who search Google Books have very different wants and expectations from casual users looking to find a trade fiction title.

"Sometimes they are looking for a preview. Sometimes they are looking for information about that book. Third, they want to buy a copy of that book," Crawford said.

Rich Results will help people who are looking specifically for a title, but Crawford said that they aren't ruling out other presentations or features for other user types (e.g. quasi-scholars like myself.)

All the Google Books tweaks I've noticed are small. Earlier this year, they introduced a sidebar for customizing your search. This summer, they added a Books-specific "Suggest" function, so when you type "sh" you get the suggestion of "Sherlock Holmes" instead of "Shoppers," which is what you get on the web. Now you can sort by date, too, or restrict your queries by subject.

But you add them all up and apply them to the 15 million books Google has scanned and the truly unprecedented nature of Google Books starts to emerge. It's not perfect -- and the Google Books Settlement is a whole separate issue -- but it is unique.

"We're in the middle of doing something radical. No one has ever pulled together this whole collection, scanning books from 40 different libraries," Crawford said. "I would say our general approach here has been to just get the books scanned because until they are digitized and OCR is done, you aren't even in the game. As we get more and more content on line, the work that Matthew's team gets to be more and more important and more and more doable."

Try Googling for something general like mortgages or health or cancer. What you want are credible sources. What you get is a bunch of SEO'd up websites that are just this side of spam. The intense competition to capture high-value keywords means that the good sites just can't keep up with the constant tweaking of the content farms.

Blekko, a new search engine, hit beta status today. It's goal, according to the Wall Street Journal, is to solve this problem by using human curators. Here's how the company put it:

As the number of Web pages reaches one trillion, "there is an acceleration of spam," said Rich Skrenta, Blekko's chief executive. "We're cleaning this up ... using large-scale human curation" that promotes "trusted" content.

Queries that Blekko identifies as being health-related, for example, are limited to 76 authoritative information sources. So searching "cure for cold," for example, generates links to sites such as MedicineNet.com, WebMD.com and MedlinePlus, a site affiliated to the National Institutes of Health. On Google, the top 10 search results include links to lesser-known sites such as essortment.com, manageyourlifenow.com and home-remedies-for-you.com.

In a way, this is a return to the web of yore, when big lists of site directories were nominally handpicked. It worked for a while. My own UCLA basketball page used to come up first on Yahoo in the mid-'90s -- a clear indication the site knew quality. But as the number of web pages proliferated, it became absolutely impossible to keep up. The humans lost and the bots won, etc. But now, even the bots are losing to scale. Blekko suggests that it's only a combined human and robot force that has a chance of maintaining a healthy information ecosystem.

So ... does it work? It's hard to tell in the early going. I'd want my grandmother searching this site for medical advice before Google. Take a hot-button topic like vaccination. Blekko serves up lots of scientific and medical information. Google gives you lots of pseudoscience and discredited conspiracy theories. I can't help but think that's a good thing.

But there's a problem: the site's design is, to me, unusable. All the little tags and colors on the page make me crosseyed.

It may just be that I'm so used to Google that any other site feels strange, but I think the problem might be embedded in the system. They are trying to do two things: provide search results and elicit human input. And that means the UI has to be more complicated than Google's.

The Lord's Resistance Army has been fighting a violent guerrilla war in northern Uganda for more than 20 years. Today, the Enough Project, an anti-genocide and war crimes group supported by the Center for American Progress, released a detailed report on the current state of the LRA.

In the world outside the conflict, a lot has changed since 1987. Mobile communications and computing, for example, are now widespread all over the world. In a fascinating passage from the Enough Project's report, they describe exactly how the rebel group, which is under increasing pressure from the Ugandan military, uses electronics.

Despite being regarded as a rag tag band of illiterate fighters, LRA commanders have successfully used sophisticated communications technology in the past. Apart from the use
of maps and satellite phones, [leader Joseph] Kony and his commanders also use dual systems phones (using satellite and mobile phone coverage), GPS monitors which the LRA commanders use to navigate and arrive at prearranged meeting places, maps, and laptops.

The laptops and satellite phones were supplied to the LRA during the Juba Talks or were stolen later. The laptops, powered by solar panels, are mostly used to watch movies when groups camp for the night, LRA commanders continue to keep coded notes in notebooks. There have also been rumors that Kony has been using a Blackberry-like device. [emphasis added]