Alexis C. Madrigal

Close of Business is a new video series that we're trying out. The idea is simple: at 5 p.m. (or thereabouts), we post a quick video summarizing the top three news stories of the day. Some of them we'll have written about; others will just be what people were talking about on the Internet.

Today, I also show you a little bit about the way we make art for the Technology Channel.

Links to stories mentioned in this video:

Inside the Google Books Algorithm [The Atlantic]

New Search Engine Is a Great Concept But... [The Atlantic]

Promoted Tweets: Testing in the Timeline [Twitter]

Google is famous for the brilliance of its algorithm for searching web pages. While the company looks at dozens of factors in determining which results to display, the heart of the search engine is using links between pages to rank their relevancy. We have come to depend on Google to give us exactly what we want.

But what about when the company has to reach outside the web? The printed volumes represented on Google Books form a completely different kind of problem. Google's famous algorithm can't be deployed to search through books because they don't link to each other in the way that webpages do. There is no perfect BookRank corollary for PageRank.

All of which made me wonder: How does Google Books work? What makes it tick? It turns out that it's actually a great place for the company's engineers to learn how to function in a linkless, physical world.

"There is a meaningful effort to say, how do we tune for books? We've got a lot of people doing very focused on the web. How do we take the lessons from what we learned on the web and invent new things that are unique to books?" Matthew Gray, lead software engineer of Google Books, told me.

The system they've come up with has become increasingly sophisticated, as highlighted by their latest tweak, Rich Results, which begins rolling out this afternoon. The feature selectively presents you with one extra-large result when it detects that you're probably searching for an individual title and not a specific mote of information or general topic.

Rich Results is the latest in a series of smaller front-end tweaks that have been matched by backend improvements. Now, the book search algorithm takes into account more than 100 "signals," individual data categories that Google statistically integrates to rank your results. When you search for a book, Google Books doesn't just look at word frequency or how closely your query matches the title of a book. They now take into account web search frequency, recent book sales, the number of libraries that hold the title, and how often an older book has been reprinted.

So, if you search "Help" now, you get a big blow-up of Kathryn Stockett's 2009 book, not one of the dozens of other books with the same title. Or if you search "dragon tattoo," you get Stieg Larsson's blockbuster, not the 2008 children's book actually called Dragon Tattoo.

"One of the fundamental things we've learned is that the whole is greater than the sum of the parts," Gray said.

This is deeply Google thinking but without the dominant algorithm. It's a Google subspecies that evolved by feeding on a different corpus. There is less data about books than web pages, but there is more structure to it, and there's less spam to contend with. Yet the focus on optimizing an experience from vast amounts of data remains. "You want it to have the standard Google quality as much as possible," Gray said. "[You want it to be] a merger of relevance and utility based on all these things."

The most difficult part of making Google Books work, said James Crawford, the team's engineering director, was determining the intent of the service's heterogeneous user base. Scholars who search Google Books have very different wants and expectations from casual users looking to find a trade fiction title.

"Sometimes they are looking for a preview. Sometimes they are looking for information about that book. Third, they want to buy a copy of that book," Crawford said.

Rich Results will help people who are looking specifically for a title, but Crawford said that they aren't ruling out other presentations or features for other user types (e.g. quasi-scholars like myself.)

All the Google Books tweaks I've noticed are small. Earlier this year, they introduced a sidebar for customizing your search. This summer, they added a Books-specific "Suggest" function, so when you type "sh" you get the suggestion of "Sherlock Holmes" instead of "Shoppers," which is what you get on the web. Now you can sort by date, too, or restrict your queries by subject.

But you add them all up and apply them to the 15 million books Google has scanned and the truly unprecedented nature of Google Books starts to emerge. It's not perfect -- and the Google Books Settlement is a whole separate issue -- but it is unique.

"We're in the middle of doing something radical. No one has ever pulled together this whole collection, scanning books from 40 different libraries," Crawford said. "I would say our general approach here has been to just get the books scanned because until they are digitized and OCR is done, you aren't even in the game. As we get more and more content on line, the work that Matthew's team gets to be more and more important and more and more doable."

Try Googling for something general like mortgages or health or cancer. What you want are credible sources. What you get is a bunch of SEO'd up websites that are just this side of spam. The intense competition to capture high-value keywords means that the good sites just can't keep up with the constant tweaking of the content farms.

Blekko, a new search engine, hit beta status today. It's goal, according to the Wall Street Journal, is to solve this problem by using human curators. Here's how the company put it:

As the number of Web pages reaches one trillion, "there is an acceleration of spam," said Rich Skrenta, Blekko's chief executive. "We're cleaning this up ... using large-scale human curation" that promotes "trusted" content.

Queries that Blekko identifies as being health-related, for example, are limited to 76 authoritative information sources. So searching "cure for cold," for example, generates links to sites such as MedicineNet.com, WebMD.com and MedlinePlus, a site affiliated to the National Institutes of Health. On Google, the top 10 search results include links to lesser-known sites such as essortment.com, manageyourlifenow.com and home-remedies-for-you.com.

In a way, this is a return to the web of yore, when big lists of site directories were nominally handpicked. It worked for a while. My own UCLA basketball page used to come up first on Yahoo in the mid-'90s -- a clear indication the site knew quality. But as the number of web pages proliferated, it became absolutely impossible to keep up. The humans lost and the bots won, etc. But now, even the bots are losing to scale. Blekko suggests that it's only a combined human and robot force that has a chance of maintaining a healthy information ecosystem.

So ... does it work? It's hard to tell in the early going. I'd want my grandmother searching this site for medical advice before Google. Take a hot-button topic like vaccination. Blekko serves up lots of scientific and medical information. Google gives you lots of pseudoscience and discredited conspiracy theories. I can't help but think that's a good thing.

But there's a problem: the site's design is, to me, unusable. All the little tags and colors on the page make me crosseyed.

It may just be that I'm so used to Google that any other site feels strange, but I think the problem might be embedded in the system. They are trying to do two things: provide search results and elicit human input. And that means the UI has to be more complicated than Google's.

The Lord's Resistance Army has been fighting a violent guerrilla war in northern Uganda for more than 20 years. Today, the Enough Project, an anti-genocide and war crimes group supported by the Center for American Progress, released a detailed report on the current state of the LRA.

In the world outside the conflict, a lot has changed since 1987. Mobile communications and computing, for example, are now widespread all over the world. In a fascinating passage from the Enough Project's report, they describe exactly how the rebel group, which is under increasing pressure from the Ugandan military, uses electronics.

Despite being regarded as a rag tag band of illiterate fighters, LRA commanders have successfully used sophisticated communications technology in the past. Apart from the use
of maps and satellite phones, [leader Joseph] Kony and his commanders also use dual systems phones (using satellite and mobile phone coverage), GPS monitors which the LRA commanders use to navigate and arrive at prearranged meeting places, maps, and laptops.

The laptops and satellite phones were supplied to the LRA during the Juba Talks or were stolen later. The laptops, powered by solar panels, are mostly used to watch movies when groups camp for the night, LRA commanders continue to keep coded notes in notebooks. There have also been rumors that Kony has been using a Blackberry-like device. [emphasis added]

Bell Labs is probably the most storied technological research center in the world. The list of innovations that came from the outpost is amazing. The transistor, the silicon photovoltaic solar cell, and the CCD all were developed at Bell Labs. And that's just the big stuff. Tons of small improvements to existing devices also flew out of the labs. For example, Labs' staffer Louis Lanzerotti proved how dangerous magnetic storms could be to the nation's electrical grid.

Now, you can read 60 years of technical journals, thanks to Alcatel-Lucent. The full text of every article published between 1922 and 1983 is available online. Now, I'm not going to lie to you: this is not riveting reading for the lay person. Most of these papers have none of the countercultural oddity of Manfred Clynes' paper on cyborgs, for example. But it's still a treasure trove and a gift to the future to have all these papers accessible. I'd love to see more research parks (say, PARC) create similar archives.

Note: I'm not sure exactly when the whole archive went up, but I think it must have been very recently. It was pointed out to me by Bernhard Schulte via Twitter.

Jon Stewart's Rally for Sanity generated the most check-ins in Foursquare's short history, the company confirmed with me today.

The total number of check-ins topped 25,000. I'm not sure if that number is impressive or not, given that it seemed like every young person in DC was was walking in the direction of the Mall on Saturday. On the other hand, cell phone service was truly abysmal that day, so imagine how many people might have checked in if they'd been able to get their phones to work.

In the wake of new evidence that Halliburton knew or should have known that the cement they used in drilling the Macondo well was faulty, the precise stakes for the company are becoming clear.

Yesterday, the National Commission created to investigate the tragedy in the Gulf pointed a finger at the oil services firm, saying that several of the company's own tests showed their cement mix might have been unstable.

Halliburton issued a long denial with a multilayered defense of the company's actions. They argue first that there were substantial differences between the mixes they tested and the specific formulation used at the site. Further, they say that BP requested a change in the composition of the mix, but that the stability of the new batch was never tested. And in any case, BP didn't take reasonable safety measures despite warnings from Halliburton and didn't do the proper tests to ensure that the cement job had been done properly.

While Halliburton and BP are trying to deflect blame to each other, in the court of public opinion, there is more than enough blame to go around. Halliburton admitted that it poured a cement mix down a notoriously tough well without testing the stability of its exact composition, even though they had evidence in hand that not all mixes were stabilizing well. As the legal wrangling continues, let's not lose sight of that fact.

But what really matters to the companies, though, is the legal matter of who has to pay for damages. BP indemnified Halliburton, according to their contract. In the quote below, COMPANY is BP and CONTRACTOR GROUP is Halliburton:

COMPANY shall save, indemnify, release, defend, and hold harmless CONTRACTOR GROUP against all claims, losses, damages, costs (including legal costs) expenses and liabilities resulting from:
(a) loss or damage to any well or hole (including the cost to re-drill);
(b) blowout, fire, explosion, cratering, or any uncontrolled well condition (including the costs to control a wild well and the removal of debris);

One assumes, however, that if the contractor group did something shoddy, such indemnities would fall away, or at least that's what BP would argue. Halliburton, in fact, seemed to signal that BP might try to do so in its last quarterly earnings report:

"We believe that the indemnification obligations contained in our contract are valid and binding against BP Exploration. BP Exploration contractually assumed responsibility for costs and expenses relating to this event, including claims for gross negligence. Given the potential amounts involved, however, BP Exploration and other indemnifying parties may seek to avoid their indemnification obligations. In particular, while we do not believe there is any justification to do so, BP Exploration, in response to our request for indemnification, has generally reserved all of its rights and stated that it is premature to conclude that it is obligated to indemnify us. In doing so, BP Exploration has asserted that the facts are not sufficiently developed to determine who is responsible, and have cited a variety of possible legal theories based upon the contract and facts still to be developed. In addition, the financial analysts and the press have speculated about the financial capacity of BP, and whether it might seek to avoid indemnification obligations in bankruptcy proceedings. We consider the likelihood of a BP bankruptcy to be remote."

You see used IKEA furniture for sale every day on the Internet. But what you don't see for sale every day is a used IKEA furniture plant. In all my years on the web, I'd never seen even one Ikea plant for sale until I opened up my email today and found this gem.

Let me know if you want in on this deal. I'm pretty sure I'd get a sweet commission.

Immediately Available

Used IKEA Furniture Plant For Sale
2,000,000 m2 of Flat Pack Annually

Hello,   

I am pleased to offer a complete furniture plant for immediate sale. The plant produced up to Two million square metres of flat pack pine furniture per annum for IKEA and is complete from "top to bottom".

 

The plant is highly automated and features a huge range of modern equipment installed between 2001 and 2008. To build this plant new today would cost in the region of Euro €25m. 

The plant is offered as a complete plant only - we cannot offer individual machines. Please contact me for more information. Serious enquiries only please.

 

Equipment Highlights:

Hall A (Tooling)

- Puttying Line (Awutek, Viet, Sisal)

- Feeder Equipment (Pinomatic)

- Receiving Equipment (Pinomatic)

- Turnover Equipment (Pinomatic, System TM, Reinbold)

- Glue Sheet Line (Pinomatic, AT, Innova, Weinig, Heesemann)

- (2x) Pegging Line  (Homag, Weeke)

- Sequence Line (Bottene, Weinig, Pinomatic, Homag, Weeke)

Hall B (Surface Treatment)

- Spray Line (CEFLA, Heesemann, Quickwood)

- (2x) Sorbini Roller Coater Line (Pinomatic, CEFLA, Heesemann, Sorbini)

- Vacuum Line (Ligmatech, Siipotec)

- Packing Line (Ferroplan, Formeca, Motoman, Camline, Pinomatic)

- Factory Systems (Ferroplan, Orfer, Tosa, Robopack)

 

Hall C

- Spraying Line (CEFLA, Viet Italia, Barberan, Routronic, Altendorf)

- Glue Board Line (CEFLA, Viet, Innova)

Condition: Excellent

Location: Europe

Inspection: on request

Available: October 2010

Price: TBA

If you have any questions about the plant please contact me.

Close of Business is a new video series that we're trying out. The idea is simple: at 5 p.m. (or thereabouts), we post a quick video summarizing the top three news stories of the day. Some of them we'll have written about; others will just be what people were talking about on the Internet.

Links to stories mentioned in this video:

Report: BP and Halliburton Knew Macondo Well Cement Might Be Unstable [The Atlantic]

Rumor: Samsung Will Announce Nexus Two November 8 [Android and Me]

Microsoft posts solid earnings [CNET]

If you use Google to find places out there in the real world a lot, you'll be happy to note that the company has made some small changes to the way that they present local searches.

Now, searches for "Ethiopian food, Washington, D.C." actually return a list of restaurants with single, definite pages. In the past, the results were a little sloppier. Some suggest this could hurt the local review site, Yelp.

But here's the thing about place-based searching: I want to be able to limit the geography in which I'm looking. If I search for Ethiopian food in D.C., there are dozens of places. I want to zoom the map in to just my neighborhood (Shaw near Howard University, FWIW). This is what Yelp.com does -- and it's the only thing that keeps me going back to that site.

Update: My oversight is your good news! As Jackie Bavaro, Google Place Search's Product Manager points out below, you can get that Yelp-like functionality by clicking on the map. So, it's just on the Places page itself that the map is static. 

Halliburton knew that a particular cement mixture it pumped into BP's Macondo well had been found unstable in laboratory tests, according to a new letter from the National Commission on the Deepwater Horizon Oil Spill and Offshore Drilling.

The cement was meant to help seal the well. Halliburton has strongly denied that its cement job played any role in the disaster. Earlier this month, Thomas Roth, Halliburton's vice president of cementing, defended his company to Oil & Gas Journal:

Problems with the cement could have been traced to its possible contamination, incomplete laboratory testing beforehand, or use of an unstable foam slurry which would have resulted in nitrogen breakout, none of which apparently occurred, he explained. Roth said Halliburton supplied the cement based on BP's specifications, tested it in its own laboratory, and recommended a formulation based on that information. Tests took more than 400 hr and indicated that the foam system was stable on delivery, he said.

Halliburton and BP both had results in March showing that a very similar foam slurry design to the one actually pumped at the Macondo well would be unstable, but neither acted upon that data; and Halliburton (and perhaps BP) should have considered redesigning the foam slurry before pumping it at the Macondo well.

After years of American dominance, China now hosts the fastest computer in the world. It's the first time a Chinese machine has held the title.

As Elizabeth Weingarten pointed out earlier today, the Tianhe-1A can execute 2.5 petaflops, or thousand trillion calculations per second. The fastest U.S. machine, the Jaguar XT-5, can only carry out 1.75 petaflops.

But there's something else that's interesting about the Tianhe-1A besides pure speed: it features impressive domestic interconnect technology. It uses Intel and Nvidia processors, but the system for allowing all 21,000 chips to communicate and work together is Chinese.

"This machine is a sign that we're going to see more machines like this," said Jack Dongarra, a computer scientist at the University of Tennessee who tracks the top supercomputers. That is to say, we're going to see faster Chinese computers that are built using more and more Chinese technology.

"We know the Chinese are developing processor technology and their hope is that they can replace not just the interconnects but also the processors with ones they make in their own country," Dongarra said.

This need not be seen as a bad thing. Certainly, American scientists and engineers can learn from their counterparts across the Pacific. But Dongarra also saw it as a challenge to the U.S. scientific establishment.

"It is a wake up call in the sense that the U.S. needs to make more of an investment in high-performance computing," he said. "If we're not dominant in that area, we're going to lose whatever advantages are conferred by that."

Supercomputers allow us to push the scientific edge. There are a wide variety of fields that depend on the astounding simulation capabilities of today's supercomputers. Dongarra, who just returned from China, said that the scientists who built the machine are planning to use it to study petroleum formations, biomedical research, and climate forecasting.

But it's worth noting that in the United States, these high-performance machines are primarily used to simulate nuclear weapons. It wouldn't be surprising if the Chinese computers are given that task, too.

The machine is located in China's National Supercomputing Center in Tianjin.

Online matchmaking is getting better at telling us whom we ought to like—and that's not good.

Barnes & Noble introduced its improved Nook e-reader today, a 7-inch, full-color tablet running the Android operating system.

I can't say that I was burning with excitement about a new Nook. After all, the iPad has a kind of cachet and polish that the Nook just doesn't. On the other hand, at $250, just half of the iPad's price, I could see buying one. Not as a keyboardless mini-computer but just as a content reader for book, magazine, and newspaper content. For that, regardless of what Chairman Steve says, the 7-inch size (about the same as a paperback book) is perfect for tossing into a bag.

Perhaps the larger point is that perhaps the tablet market won't be quite as winner-take-all as it seems after the iPad's incredible run in the last few months. Maybe multiple devices will carve out and sustain niches serving a specific set of customers, even as the iPad becomes the all-purpose device in the center of the industry.

Close of Business is a new video series that we're trying out. Today, we've got a special edition about The Atlantic's Green Intelligence Forum. I spent the day with policy and energy experts, and this video is a quick collection of thoughts about what I saw and heard.

See more video from The Atlantic Technology Channel.

The New Yorker's Seymour Hersh has one of his classic deep investigations in this week's issue. His topic this time is "cyber war," or at least the packet of behaviors by foreign governments that are sometimes classed under that heading.

While the article feels a tiny bit diffuse to me, you can't help but come away from it feeling like the idea we'd battle another country in cyberspace is mostly a useful fiction for the military establishment. If we're at war, they get to control the nation's cyber security apparatus, and all its attendant turf and riches. And bonus: if we are in a cyber war, the less able and likely we are to fight for our civil liberties and privacy online.

The story is well worth your time. I excerpt the incredibly compelling anecdote that begins the piece about a National Security Agency spy plane captured and (apparently) reverse engineered by the Chinese. You'll have to read to the end to find out the surprising coda to the story.

On April 1, 2001, an American EP-3E Aries II reconnaissance plane on an eavesdropping mission collided with a Chinese interceptor jet over the South China Sea, triggering the first international crisis of George W. Bush's Administration. The Chinese jet crashed, and its pilot was killed, but the pilot of the American aircraft, Navy Lieutenant Shane Osborn, managed to make an emergency landing at a Chinese F-8 fighter base on Hainan Island, fifteen miles from the mainland. Osborn later published a memoir, in which he described the "incessant jackhammer vibration" as the plane fell eight thousand feet in thirty seconds, before he regained control.

The plane carried twenty-four officers and enlisted men and women attached to the Naval Security Group Command, a field component of the National Security Agency. They were repatriated after eleven days; the plane stayed behind. The Pentagon told the press that the crew had followed its protocol, which called for the use of a fire axe, and even hot coffee, to disable the plane's equipment and software. These included an operating system created and controlled by the N.S.A., and the drivers needed to monitor encrypted Chinese radar, voice, and electronic communications. It was more than two years before the Navy acknowledged that things had not gone so well. "Compromise by the People's Republic of China of undestroyed classified material . . . is highly probable and cannot be ruled out," a Navy report issued in September, 2003, said.

The loss was even more devastating than the 2003 report suggested, and its dimensions have still not been fully revealed. Retired Rear Admiral Eric McVadon, who flew patrols off the coast of Russia and served as a defense attaché in Beijing, told me that the radio reports from the aircraft indicated that essential electronic gear had been dealt with. He said that the crew of the EP-3E managed to erase the hard drive--"zeroed it out"--but did not destroy the hardware, which left data retrievable: "No one took a hammer." Worse, the electronics had recently been upgraded. "Some might think it would not turn out as badly as it did, but I sat in some meetings about the intelligence cost," McVadon said. "It was grim."