Noam Chomsky on Where Artificial Intelligence Went Wrong

So is that a worthy goal of unification or the fields should proceed in parallel?

Chomsky: Well, unification is kind of an intuitive ideal, part of the scientific mystique, if you like. It's that you're trying to find a unified theory of the world. Now maybe there isn't one, maybe different parts work in different ways, but your assumption is until I'm proven wrong definitively, I'll assume that there's a unified account of the world, and it's my task to try to find it. And the unification may not come out by reduction -- it often doesn't. And that's kind of the guiding logic of David Marr's approach: what you discover at the computational level ought to be unified with what you'll some day find out at the mechanism level, but maybe not in terms of the way we now understand the mechanisms.

And implicit in Marr it seems that you can't work on all three in parallel [computational, algorithmic, implementation levels], it has to proceed top-down, which is a very stringent requirement, given that science usually doesn't work that way.

Chomsky: Well, he wouldn't have said it has to be rigid. Like for example, discovering more about the mechanisms might lead you to change your concept of computation. But there's kind of a logical precedence, which isn't necessarily the research precedence, since in research everything goes on at the same time. But I think that the rough picture is okay. Though I should mention that Marr's conception was designed for input systems...

information-processing systems...

Chomsky: Yeah, like vision. There's some data out there -- it's a processing system -- and something goes on inside. It isn't very well designed for cognitive systems. Like take your capacity to take out arithmetical operations..

It's very poor, but yeah...

Chomsky: Okay [laughs]. But it's an internal capacity, you know your brain is a controlling unit of some kind of Turing machine, and it has access to some external data, like memory, time and so on. And in principle, you could multiply anything, but of course not in practice. If you try to find out what that internal system is of yours, the Marr hierarchy doesn't really work very well. You can talk about the computational level -- maybe the rules I have are Peano's axioms [Editor's note: a mathematical theory (named after Italian mathematician Giuseppe Peano) that describes a core set of basic rules of arithmetic and natural numbers, from which many useful facts about arithmetic can be deduced], or something, whatever they are -- that's the computational level. In theory, though we don't know how, you can talk about the neurophysiological level, nobody knows how, but there's no real algorithmic level. Because there's no calculation of knowledge, it's just a system of knowledge. To find out the nature of the system of knowledge, there is no algorithm, because there is no process. Using the system of knowledge, that'll have a process, but that's something different.

But since we make mistakes, isn't that evidence of a process gone wrong?

Chomsky: That's the process of using the internal system. But the internal system itself is not a process, because it doesn't have an algorithm. Take, say, ordinary mathematics. If you take Peano's axioms and rules of inference, they determine all arithmetical computations, but there's no algorithm. If you ask how does a number theoretician applies these, well all kinds of ways. Maybe you don't start with the axioms and start with the rules of inference. You take the theorem, and see if I can establish a lemma, and if it works, then see if I can try to ground this lemma in something, and finally you get a proof which is a geometrical object.

But that's a fundamentally different activity from me adding up small numbers in my head, which surely does have some kind of algorithm.

Chomsky: Not necessarily. There's an algorithm for the process in both cases. But there's no algorithm for the system itself, it's kind of a category mistake. You don't ask the question what's the process defined by Peano's axioms and the rules of inference, there's no process. There can be a process of using them. And it could be a complicated process, and the same is true of your calculating. The internal system that you have -- for that, the question of process doesn't arise. But for your using that internal system, it arises, and you may carry out multiplications all kinds of ways. Like maybe when you add 7 and 6, let's say, one algorithm is to say "I'll see how much it takes to get to 10" -- it takes 3, and now I've got 4 left, so I gotta go from 10 and add 4, I get 14. That's an algorithm for adding -- it's actually one I was taught in kindergarten. That's one way to add.

But there are other ways to add -- there's no kind of right algorithm. These are algorithms for carrying out the process the cognitive system that's in your head. And for that system, you don't ask about algorithms. You can ask about the computational level, you can ask about the mechanism level. But the algorithm level doesn't exist for that system. It's the same with language. Language is kind of like the arithmetical capacity. There's some system in there that determines the sound and meaning of an infinite array of possible sentences. But there's no question about what the algorithm is. Like there's no question about what a formal system of arithmetic tells you about proving theorems. The use of the system is a process and you can study it in terms of Marr's level. But it's important to be conceptually clear about these distinctions.

It just seems like an astounding task to go from a computational level theory, like Peano axioms, to Marr level 3 of the...

Chomsky: mechanisms...

...mechanisms and implementations...

Chomsky: Oh yeah. Well..

..without an algorithm at least.

Chomsky: Well, I don't think that's true. Maybe information about how it's used, that'll tell you something about the mechanisms. But some higher intelligence -- maybe higher than ours -- would see that there's an internal system, its got a physiological basis, and I can study the physiological basis of that internal system. Not even looking at the process by which it's used. Maybe looking at the process by which it's used maybe gives you helpful information about how to proceed. But it's conceptually a different problem. That's the question of what's the best way to study something. So maybe the best way to study the relation between Peano's axioms and neurons is by watching mathematicians prove theorems. But that's just because it'll give you information that may be helpful. The actual end result of that will be an account of the system in the brain, the physiological basis for it, with no reference to any algorithm. The algorithms are about a process of using it, which may help you get answers. Maybe like incline planes tell you something about the rate of fall, but if you take a look at Newton's laws, they don't say anything about incline planes.

Right. So the logic for studying cognitive and language systems using this kind of Marr approach makes sense, but since you've argued that language capacity is part of the genetic endowment, you could apply it to other biological systems, like the immune system, the circulatory system....

Chomsky: Certainly, I think it's very similar. You can say the same thing about study of the immune system.

It might even be simpler, in fact, to do it for those systems than for cognition.

Chomsky: Though you'd expect different answers. You can do it for the digestive system. Suppose somebody's studying the digestive system. Well, they're not going to study what happens when you have a stomach flu, or when you've just eaten a big Mac, or something. Let's go back to taking pictures outside the window. One way of studying the digestive system is just to take all data you can find about what digestive systems do under any circumstances, toss the data into a computer, do statistical analysis -- you get something. But it's not gonna be what any biologist would do. They want to abstract away, at the very beginning, from what are presumed -- maybe wrongly, you can always be wrong -- irrelevant variables, like do you have stomach flu.

But that's precisely what the biologists are doing, they are taking the sick people with the sick digestive system, comparing them to the normals, and measuring these molecular properties.

Chomsky: They're doing it in an advanced stage. They already understand a lot about the study of the digestive system before we compare them, otherwise you wouldn't know what to compare, and why is one sick and one isn't.

Well, they're relying on statistical analysis to pick out the features that discriminate. It's a highly fundable approach, because you're claiming to study sick people.

Chomsky: It may be the way to fund things. Like maybe the way to fund study of language is to say, maybe help cure autism. That's a different question [laughs]. But the logic of the search is to begin by studying the system abstracted from what you, plausibly, take to be irrelevant intrusions, see if you can find its basic nature -- then ask, well, what happens when I bring in some of this other stuff, like stomach flu.

It still seems like there's a difficulty in applying Marr's levels to these kinds of systems. If you ask, what is the computational problem that the brain is solving, we have kind of an answer, it's sort of like a computer. But if you ask, what is the computational problem that's being solved by the lung, that's very difficult to even think -- it's not obviously an information-processing kind of problem.

Chomsky: No, but there's no reason to assume that all of biology is computational. There may be reasons to assume that cognition is. And in fact Gallistel is not saying that everything is in the body ought to be studied by finding read/write/address units.

It just seems contrary to any evolutionary intuition. These systems evolved together, reusing many of the same parts, same molecules, pathways. Cells are computing things.

Chomsky: You don't study the lung by asking what cells compute. You study the immune system and the visual system, but you're not going to expect to find the same answers. An organism is a highly modular system, has a lot of complex subsystems, which are more or less internally integrated. They operate by different principles. The biology is highly modular. You don't assume it's all just one big mess, all acting the same way.

No, sure, but I'm saying you would apply the same approach to study each of the modules.

Chomsky: Not necessarily, not if the modules are different. Some of the modules may be computational, others may not be.

So what would you think would be an adequate theory that is explanatory, rather than just predicting data, the statistical way, what would be an adequate theory of these systems that are not computing systems -- can we even understand them?

Chomsky: Sure. You can understand a lot about say, what makes an embryo turn into a chicken rather than a mouse, let's say. It's a very intricate system, involves all kinds of chemical interactions, all sorts of other things. Even the nematode, it's by no means obviously -- in fact there are reports from the study here -- that it's all just a matter of a neural net. You have to look into complex chemical interactions that take place in the brain, in the nervous system. You have to look into each system on its own. These chemical interactions might not be related to how your arithmetical capacity works -- probably aren't. But they might very well be related to whether you decide to raise your arm or lower it.

Though if you study the chemical interactions it might lead you into what you've called just a redescription of the phenomena.

Chomsky: Or an explanation. Because maybe that's directly, crucially, involved.

But if you explain it in terms of chemical X has to be turned on, or gene X has to be turned on, you've not really explained how organism-determination is done. You've simply found a switch, and hit that switch.

Chomsky: But then you look further, and find out what makes this gene do such and such under these circumstances, and do something else under different circumstances.

But if genes are the wrong level of abstraction, you'd be screwed.

Chomsky: Then you won't get the right answer. And maybe they're not. For example, it's notoriously difficult to account for how an organism arises from a genome. There's all kinds of production going on in the cell. If you just look at gene action, you may not be in the right level of abstraction. You never know, that's what you try to study. I don't think there's any algorithm for answering those questions, you try.

So I want to shift gears more toward evolution. You've criticized a very interesting position you've called "phylogenetic empiricism." You've criticized this position for not having explanatory power. It simply states that: well, the mind is the way it because of adaptations to the environment that were selected for. And these were selected for by natural selection. You've argued that this doesn't explain anything because you can always appeal to these two principles of mutation and selection.

Chomsky: Well you can wave your hands at them, but they might be right. It could be that, say, the development of your arithmetical capacity, arose from random mutation and selection. If it turned out to be true, fine.

It seems like a truism.

Chomsky: Well, I mean, doesn't mean it's false. Truisms are true. [laughs].

But they don't explain much.

Chomsky: Maybe that's the highest level of explanation you can get. You can invent a world -- I don't think it's our world -- but you can invent a world in which nothing happens except random changes in objects and selection on the basis of external forces. I don't think that's the way our world works, I don't think it's the way any biologist thinks it is. There are all kind of ways in which natural law imposes channels within which selection can take place, and some things can happen and other things don't happen. Plenty of things that go on in the biology in organisms aren't like this. So take the first step, meiosis. Why do cells split into spheres and not cubes? It's not random mutation and natural selection; it's a law of physics. There's no reason to think that laws of physics stop there, they work all the way through.

Well, they constrain the biology, sure.

Chomsky: Okay, well then it's not just random mutation and selection. It's random mutation, selection, and everything that matters, like laws of physics.

So is there room for these approaches which are now labeled "comparative genomics", like the Broad Institute here [at MIT/Harvard] is generating massive amounts of data, of different genomes, different animals, different cells under different conditions and sequencing any molecule that is sequenceable. Is there anything that can be gleaned about these high-level cognitive tasks from these comparative evolutionary studies or is it premature?

Chomsky: I am not saying it's the wrong approach, but I don't know anything that can be drawn from it. Nor would you expect to.

You don't have any examples where this evolutionary analysis has informed something? Like Foxp2 mutations? [Editor's note: A gene that is thought be implicated in speech or language capacities. A family with a stereotyped speech disorder was found to have genetic mutations that disrupt this gene. This gene evolved to have several mutations unique to the human evolutionary lineage.]

Chomsky: Foxp2 is kind of interesting, but it doesn't have anything to do with language. It has to do with fine motor coordinations and things like that. Which takes place in the use of language, like when you speak you control your lips and so on, but all that's very peripheral to language, and we know that. So for example, whether you use the articulatory organs or sign, you know hand motions, it's the same language. In fact, it's even being analyzed and produced in the same parts of the brain, even though one of them is moving your hands and the other is moving your lips. So whatever the externalization is, it seems quite peripheral. I think they're too complicated to talk about, but I think if you look closely at the design features of language, you get evidence for that. There are interesting cases in the study of language where you find conflicts between computational efficiency and communicative efficiency.

Take this case I even mentioned of linear order. If you want to know which verb the adverb attaches to, the infant reflexively using minimal structural distance, not minimal linear distance. Well, it's using minimal linear distances, computationally easy, but it requires having linear order available. And if linear order is only a reflex of the sensory-motor system, which makes sense, it won't be available. That's evidence that the mapping of the internal system to the sensory-motor system is peripheral to the workings of the computational system.

But it might constrain it like physics constrains meiosis?

Chomsky: It might, but there's very little evidence of that. So for example the left end -- left in the sense of early -- of a sentence has different properties from the right end. If you want to ask a question, let's say "Who did you see?" You put the "Who" infront, not in the end. In fact, in every language in which a wh-phrase -- like who, or which book, or something -- moves to somewhere else, it moves to the left, not to the right. That's very likely a processing constraint. The sentence opens by telling you, the hearer, here's what kind of a sentence it is. If it's at the end, you have to have the whole declarative sentence, and at the end you get the information I'm asking about. If you spell it out, it could be a processing constraint. So that's a case, if true, in which the processing constraint, externalization, do affect the computational character of the syntax and semantics.

There are cases where you find clear conflicts between computational efficiency and communicative efficiency. Take a simple case, structural ambiguity. If I say, "Visiting relatives can be a nuisance" -- that's ambiguous. Relatives that visit, or going to visit relatives. It turns out in every such case that's known, the ambiguity is derived by simply allowing the rules to function freely, with no constraints, and that sometimes yields ambiguities. So it's computationally efficient, but it's inefficient for communication, because it leads to unresolvable ambiguity.

Or take what are called garden-path sentences, sentences like "The horse raced past the barn fell". People presented with that don't understand it, because the way it's put, they're led down a garden path. "The horse raced past the barn" sounds like a sentence, and then you ask what's "fell" doing there at the end. On the other hand, if you think about it, it's a perfectly well formed sentence. It means the horse that was raced past the barn, by someone, fell. But the rules of the language when they just function happen to give you a sentence which is unintelligible because of the garden-path phenomena. And there are lots of cases like that. There are things you just can't say, for some reason. So if I say, "The mechanics fixed the cars". And you say, "They wondered if the mechanics fixed the cars." You can ask questions about the cars, "How many cars did they wonder if the mechanics fixed?" More or less okay. Suppose you want to ask a question about the mechanics. "How many mechanics did they wonder if fixed the cars?" Somehow it doesn't work, can't say that. It's a fine thought, but you can't say it. Well, if you look into it in detail, the most efficient computational rules prevent you from saying it. But for expressing thought, for communication, it'd be better if you could say it -- so that's a conflict.

And in fact, every case of a conflict that's known, computational efficiency wins. The externalization is yielding all kinds of ambiguities but for simple computational reasons, it seems that the system internally is just computing efficiently, it doesn't care about the externalization. Well, I haven't made that a very plausible, but if you spell it out it can be made quite a convincing argument I think.

That tells something about evolution. What it strongly suggests is that in the evolution of language, a computational system developed, and later on it was externalized. And if you think about how a language might have evolved, you're almost driven to that position. At some point in human evolution, and it's apparently pretty recent given the archeological record -- maybe last hundred thousand years, which is nothing -- at some point a computational system emerged with had new properties, that other organisms don't have, that has kind of arithmetical type properties...

It enabled better thought before externalization?

Chomsky: It gives you thought. Some rewiring of the brain, that happens in a single person, not in a group. So that person had the capacity for thought -- the group didn't. So there isn't any point in externalization. Later on, if this genetic change proliferates, maybe a lot of people have it, okay then there's a point in figuring out a way to map it to the sensory-motor system and that's externalization but it's a secondary process.

Unless the externalization and the internal thought system are coupled in ways we just don't predict.

Chomsky: We don't predict, and they don't make a lot of sense. Why should it be connected to the external system? In fact, say your arithmetical capacity isn't. And there are other animals, like songbirds, which have internal computational systems, bird song. It's not the same system but it's some kind of internal computational system. And it is externalized, but sometimes it's not. A chick in some species acquires the song of that species but doesn't produce it until maturity. During that early period it has the song, but it doesn't have the externalization system. Actually that's true of humans too, like a human infant understands a lot more than it can produce -- plenty of experimental evidence for this, meaning it's got the internal system somehow, but it can't externalize it. Maybe it doesn't have enough memory, or whatever it may be.


Graham Gordon Ramsay

I'd like to close with one question about the philosophy of science. In a recent interview, you said that part of the problem is that scientists don't think enough about what they're up to. You mentioned that you taught a philosophy of science course at MIT and people would read, say, Willard van Orman Quine, and it would go in one ear out the other, and people would go back doing the same kind of science that they were doing. What are the insights that have been obtained in philosophy of science that are most relevant to scientists who are trying to let's say, explain biology, and give an explanatory theory rather than redescription of the phenomena? What do you expect from such a theory, and what are the insights that help guide science in that way? Rather than guiding it towards behaviorism which seems to be an intuition that many, say, neuroscientists have?

Chomsky: Philosophy of science is a very interesting field, but I don't think it really contribute to science, it learns from science. It tries to understand what the sciences do, why do they achieve things, what are the wrong paths, see if we can codify that and come to understand. What I think is valuable is the history of science. I think we learn a lot of things from the history of science that can be very valuable to the emerging sciences. Particularly when we realize that in say, the emerging cognitive sciences, we really are in a kind of pre-Galilean stage. We don't know what we're looking for anymore than Galileo did, and there's a lot to learn from that. So for example one striking fact about early science, not just Galileo, but the Galilean breakthrough, was the recognition that simple things are puzzling.

Take say, if I'm holding this here [cup of water], and say the water is boiling [putting hand over water], the steam will rise, but if I take my hand away the cup will fall. Well why does the cup fall and the steam rise? Well for millennia there was a satisfactory answer to that: they're seeking their natural place.

Like in Aristotelian physics?

Chomsky: That's the Aristotelian physics. The best and greatest scientists thought that was answer. Galileo allowed himself to be puzzled by it. As soon as you allow yourself to be puzzled by it, you immediately find that all your intuitions are wrong. Like the fall of a big mass and a small mass, and so on. All your intuitions are wrong -- there are puzzles everywhere you look. That's something to learn from the history of science. Take the one example that I gave to you, "Instinctively eagles that fly swim." Nobody ever thought that was puzzling -- yeah, why not. But if you think about it, it's very puzzling, you're using a complex computation instead of a simple one. Well, if you allow yourself to be puzzled by that, like the fall of a cup, you ask "Why?" and then you're led down a path to some pretty interesting answers. Like maybe linear order just isn't part of the computational system, which is a strong claim about the architecture of the mind -- it says it's just part of the externalization system, secondary, you know. And that opens up all sorts of other paths, same with everything else.

Take another case: the difference between reduction and unification. History of science gives some very interesting illustrations of that, like chemistry and physics, and I think they're quite relevant to the state of the cognitive and neurosciences today.

Presented by

Yarden Katz is a graduate student in the Department of Brain and Cognitive sciences at MIT, where he studies the regulation of gene expression in the developing nervous system and in cancer. 

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register with Disqus.

Please note that The Atlantic's account system is separate from our commenting system. To log in or register with The Atlantic, use the Sign In button at the top of every page.

blog comments powered by Disqus


Confessions of Moms Around the World

A global look at the hardest and best job ever


A Stop-Motion Tour of New York City

A filmmaker animated hundreds of still photographs to create this Big Apple flip book


The Absurd Psychology of Restaurant Menus

Would people eat healthier if celery was called "cool celery?"


This Japanese Inn Has Been Open for 1,300 Years

It's one of the oldest family businesses in the world.

More in Technology

Just In