Elsewhere, Ferrucci has been more circumspect about Watson's level of "understanding." In an interview with IBM's own magazine ForwardView, he said, "For a computer, there is no connection from words to human experience and human cognition. The words are just symbols to the computer. How does it know what they really mean?" In other words, for all of the impressive NLP programming that has gone into Watson, the computer is unable to penetrate the semantics of language, or comprehend how meanings of words are shot through with allusions to human culture and the experience of daily life.
Such a sober assessment doesn't jibe with popular perceptions of artificial intelligence, of course. We want our "smart computers" to engage with us linguistically like HAL did in Stanley Kubrick's 2001 -- well, except for the murderous rampage part. (Ferrucci and his team prefer to use a different pop-cultural point of reference: the original series of Star Trek, with its more benign on-board talking computer.) But let's remember how HAL was introduced, via a BBC interview watched by the spaceship crew in the film:
The sixth member of the Discovery crew was not concerned about the problems of hibernation. For he was the latest result in machine intelligence -- the HAL 9000 computer, which can reproduce, though some experts still prefer to use the word 'mimic,' most of the activities of the human brain, and with incalculably greater speed and reliability.
Is Watson, despite its limitations, nonetheless a precursor to a HAL-esque machine that can "mimic" natural language, if not "reproduce" it? Baby steps, baby steps. First we need a computer that doesn't give Toronto as an answer to a clue about "U.S. Cities," as Watson memorably did for Final Jeopardy in the first game. And we'd also want it to know that the "anatomical oddity" of Olympian gymnast George Eyser was not his leg, but his missing leg.
Those were two isolated gaffes in a pretty clean run by Watson against his human foes, but they'll certainly be remembered at IBM. For proof, see Stephen Baker's book Final Jeopardy, an engaging inside look at the Watson project, culminating with the Jeopardy! showdown in the final chapter. (In a shrewd marketing move, the book was available electronically without its final chapter before the match, and then the ending was given to readers as an update immediately after the conclusion of the tournament.) Baker writes:
As this question-answering technology expands from its quiz show roots into the rest of our lives, engineers at IBM and elsewhere must sharpen its understanding of contextual language. Smarter machines will not call Toronto a U.S. city, and they will recognize the word "missing" as the salient fact in any discussion of George Eyser's leg. Watson represents merely a step in the development of smart machines. Its answering prowess, so formidable on a winter afternoon in 2011, will no doubt seem quaint in a surprisingly short time.
Baker's undoubtedly right about that, but we're still dealing with the limited task of question-answering, not anything even vaguely approaching full-fledged comprehension of natural language, with all of its "nuance, slang and metaphor." If Watson had chuckled at that "computer overlords" jab, then I'd be a little worried.