Recently Jaron Lanier, an essayist on technology, launched a broadside against this faith and set off a major debate within the tech community. At the end of May the online publication Edge published Lanier’s essay “Digital Maoism,” which predicted that collective intelligence would have the same deadening and anticreative effect as political collectivism in general. The heart of this argument was that measures of mass popularity could be accurate in certain limited circumstances, but not in a large variety of others. Edge also published many rebuttals, and the debate goes on. The opposing camps and positions are amazingly similar to those in the endless economic debate between libertarian free-market absolutists, who think that any market outcome must be right, and those who say, “Yes, but …” and start listing cases of market failure.
My sympathies are with Lanier, but here is the intriguing part: even as we debate the limits on how much, and how many kinds of, intelligence human beings can ultimately build into their networks and machines, we have to recognize what computers can do already—and how that eventually may change us.
The most obvious and unquestionable achievement in Internet “intelligence” is the Jeopardy!-style retrieval of “spot knowledge.” If you want to know what other movies Vincent Schiavelli was in before and after Ghost, any search engine will point you to a list. As computing power becomes smaller, cheaper, and easier to embed in other products, nearly everything we use will eventually come with the ability to pull relevant data from search engines. The GPS receivers in cars that tell us about the restaurant or rest stop we’re passing are early indicators. Refrigerators will retrieve recipes for the ingredients inside; household appliances will download pages from repair manuals.
There is a second area in which Internet-borne knowledge is becoming steadily more impressive: categorization, or pattern recognition. In general, deciding how different things are similar, and similar things are different, is extremely difficult for computers. Any three-year-old can instantly tell a cow from a horse; few computers can. But related developments in several search engines have provided the beginnings of useful machine-created categorization.
A search engine called Clusty, founded by Carnegie Mellon computer scientists and based in Pittsburgh, returns its search results grouped by topic category. Type in “theory of evolution,” for instance, and it will tell you which sites discuss Charles Darwin, which cover modern developments in the theory, and which discuss its relationship with the Bible. An experimental search engine developed at the University of Maryland, at tinyurl.com/qkpht, also provides useful categorized results. Ask.com, formerly known as Ask Jeeves, has a very useful “Zoom” feature. Type in “theory of evolution” there, and it will suggest that you might want to narrow the search for information about the mechanics of natural selection, or broaden it to a general query about the beginnings of life. Cnet’s news site has a feature called “Big Picture.” After you enter a query, it produces a concept map showing the related topics to explore. Grokker was a pioneer in producing such concept maps, and remains a useful and reliably interesting way to display the overlaps and divergences among items discovered by a Web search. Raymond Kurzweil’s site employs an idea map of its own. When I’m doing a search to develop a theme and not to check spot knowledge, I have found that these categorizing sites—especially Ask.com—save me time in getting where I want to go.