AM: How do most people work with data now?
JH: What most people do today is put data in big databases and analyze the correlations. Say you have 1 billion users on Facebook, and you’re trying to figure out what advertisement to feed to 20 percent of them. You want one big model on all these data. What we do is different: Say someone has 10,000 smart meters, and they’re trying to figure out what energy consumption is going to be two hours from now. We build 10,000 models. You can’t have a data analyst doing that. If you want to model every machine in a factory or every windmill in a windmill farm, it’s all about automation. We build lots of little models—that’s the future of data.
Most advanced analytics require substantial human expertise and are done in batch fashion—data are gathered for some time period and then processed in big chunks. This process can be slow and is hard to apply to a broad range of problems as the world changes. It also means using huge databases, which are expensive—it’s complex to maintain and move large amounts of data around. Grok, like your brain, is a streaming system. Data pass through Grok, predictions are made—but Grok doesn’t need to store the data to function. With millions of devices generating billions of data streams, the ability to store only what’s critical can be a significant advantage.
In industrial applications, more-basic approaches—say, looking at a sensor to ensure a temperature does not exceed a certain value—can be used to monitor equipment and alarms. These approaches have limitations: By the time an alarm is triggered, it may be too late. Or what’s normal for one machine may not be normal for the next.
Grok lets you automate processes that previously required manual adjustment. Heating or cooling systems can be turned on or off intelligently. Applications can be migrated between servers based on load. Network traffic can be rerouted. Unusual behavior of heavy machinery can generate alerts that recommend specific action. Instead of reacting to problems, you can anticipate them.
AM: Who else might use your software?
JH: People who want to do anomaly prediction. Grok works as if it’s listening to very noisy melodies and going, “I recognize some of this. That sounds a little familiar.” And all of a sudden it says, “This sounds totally different. I’ve never seen this before,” so it goes beep, beep. There are a lot of people looking for anomalous behavior in credit-card and security applications. It turns out this might be even bigger than prediction. But of course, anomalies are the flip side of predictions—if I can’t predict well, then I have an anomaly.
AM: So how does this relate to your previous work on the brain?
JH: First of all, we have a very complex brain; it’s got all these different components. But we’re just talking neocortex here. Every mammal, from a human to a mouse to a dolphin, has one. What is the neocortex doing? It’s building a model of the world, of what we call sensory motor contingencies or sensory motor patterns: Why are you wearing glasses and what does that mean? Or: If I turn my head to the right, I have expectations about what I’m going to see. Most of what we learn about the world is how it behaves when we interact with it. The neocortex builds a model of what should happen in a particular context. A bigger neocortex lets you make a more complex model, and it lets you have more sensors. And that’s what intelligence is: it’s learning this model of the world.