"I am finding it increasingly plausible that existential risk is the biggest moral issue in the world, even if it hasn’t gone mainstream yet," Bostrom told Ross Andersen recently in an amazing profile in Aeon. Bostrom, along with Hawking, is an advisor to the recently-established Centre for the Study of Existential Risk at Cambridge University, and to Tegmark’s new analogous group in Cambridge, Massachusetts, the Future of Life Institute, which has a launch event later this month. Existential risks, as Tegmark describes them, are things that are “not just a little bit bad, like a parking ticket, but really bad. Things that could really mess up or wipe out human civilization.”
The single existential risk that Tegmark worries about most is unfriendly artificial intelligence. That is, when computers are able to start improving themselves, there will be a rapid increase in their capacities, and then, Tegmark says, it’s very difficult to predict what will happen.
Tegmark told Lex Berko at Motherboard earlier this year, "I would guess there’s about a 60 percent chance that I’m not going to die of old age, but from some kind of human-caused calamity. Which would suggest that I should spend a significant portion of my time actually worrying about this. We should in society, too."
I really wanted to know what all of this means in more concrete terms, so I asked Tegmark about it myself. He was actually walking around the Pima Air and Space Museum in Tucson with his kids as we spoke, periodically breaking to answer their questions about the exhibits.
"Longer term—and this might mean 10 years, it might mean 50 or 100 years, depending on who you ask—when computers can do everything we can do," Tegmark said, "after that they will probably very rapidly get vastly better than us at everything, and we’ll face this question we talked about in the Huffington Post article: whether there’s really a place for us after that, or not." I imagined glances from nearby museum-goers.
"This is very near-term stuff. Anyone who’s thinking about what their kids should study in high school or college should care a lot about this."
"The main reason people don’t act on these things is they’re not educated about them,” Tegmark continued. "I’ve never talked with anyone about these things who turned around and said, 'I don’t care.'" He’s previously said that the biggest threat to humanity is our own stupidity.
Tegmark told me, as he has told others on more than just this occasion, that more people know Justin Bieber than know Vasili Arkhipov—a Soviet naval officer who is credited with single-handedly preventing thermonuclear war during the Cuban Missile Crisis. That knowledge differential isn’t surprising at all. More people know Bieber than know most historic figures, including Bo Jackson. That’s especially hard to swallow after learning this week from Seth Rogen that, in fact, "Justin Bieber is a piece of shit."