As email, documents, and almost every aspect of our professional and personal lives moves onto the “cloud”—remote servers we rely on to store, guard, and make available all of our data whenever and from wherever we want them, all the time and into eternity—a brush with disaster reminds the author and his wife just how vulnerable those data can be. A trip to the inner fortress of Gmail, where Google developers recovered six years’ worth of hacked and deleted e‑mail, provides specific advice on protecting and backing up data now—and gives a picture both consoling and unsettling of the vulnerabilities we can all expect to face in the future.
On April 13 of this year, a Wednesday, my wife got up later than usual and didn’t check her e‑mail until around 8:30 a.m. The previous night, she had put her computer to “sleep,” rather than shutting it down. When she opened it that morning to the Gmail account that had been her main communications center for more than six years, it seemed to be responding very slowly and jerkily. She hadn’t fully restarted the computer in several days, and thought that was the problem. So she closed all programs, rebooted the machine, and went off to make coffee and have some breakfast.
When she came back to her desk, half an hour later, she couldn’t log into Gmail at all. By that time, I was up and looking at e‑mail, and we both quickly saw what the real problem was. In my inbox I found a message purporting to be from her, followed by a quickly proliferating stream of concerned responses from friends and acquaintances, all about the fact that she had been “mugged in Madrid.” The account had seemed sluggish earlier that morning because my wife had tried to use it at just the moment a hacker was taking it over and changing its settings—including the password, so that she couldn’t log in again. The bogus message that had just gone out to me and everyone else in her Gmail contact list was this:
From: Deb Fallows
Date: Wed, Apr 13, 2011 at 8:45 AM
now this might come as a suprise to you,but I made a quick trip to Madrid in Spain and was mugged.My bag,valuables,credit cards and passport all gone.The embassy has cooperated by issuing a temporary passport.I need funds to settle outstanding hotel bills,ticket and other expenses.
To be honest,i don’t have money with me at the moment. I’ve made contact with my bank but the best they could do was to send me a new card in the mail which will take 2-4 working days to arrive here from DC. I need you to lend me some Money to sort my self out of this predicament, i will pay back once i get this over with because i need to make a last minute flight.
Western Union or MoneyGram is the fastest option to wire funds to me. Let me know if you need my details(Full names/location) to effect a transfer. You can reach me via hotel’s desk phone and the number is, +34 981 600916867.
I’m using her real e‑mail address because at this stage there’s no point in “protecting” it. Someone had obviously taken over her account and was using it as a crude spamming tool—or at least what we considered crude. Who hadn’t seen countless messages like this before? Which of her friends would really think that Deb would capitalize “Money,” type a paragraph’s worth of sentences with no spaces separating them, or say that she had gone to “Madrid in Spain”? And, indeed, the great majority of notes were warnings that her account had been hacked and was being used to send out fraud-spam; some included gratuitous tips about the need to be more careful in online life.
But a touching handful of the notes came from people who took the plea at face value. They wrote to me; they replied to Deb’s account; a few even phoned from South America, Asia, the Midwest, and Australia to find out how they could help. That was the first indication that this would be more than a minor nuisance. What if some of them actually sent cash?
The more serious sign of the potential scale of our problems came later in the day. Google offers a variety of automated ways for users to regain control of Gmail and other accounts they think have been hacked. The automated routines, plus an online forum moderated by Google employees, are the only help Google offers. With hundreds of millions of active Gmail accounts to manage—that’s as specific as Google will be about its user base—operating in 54 languages worldwide, the relative handful of human beings on Gmail’s support staff could not even pretend to offer live one-on-one service. The same is true of Yahoo, Microsoft’s Hotmail, Facebook, Skype, eBay, and the other big operators of “cloud”-based systems.
As a reminder: in cloud-based systems, users turn the management and protection of crucial data and services over to third parties, and then call up information as necessary via the Internet. For individuals, the appeal is that e-mail held “in the cloud” by Google, Yahoo, Microsoft, et cetera, is available wherever there is an Internet connection, rather than being lodged on any one machine. If a corporation is large enough, it may operate its own internal cloud, or turn to large-scale suppliers like Amazon—which has a cloud-server business apart from its familiar retail functions—to store and protect data.
The first and easiest automated step was to fill out a password-reset form. Doing so prompted Google to send reset instructions to the mobile-phone number or alternate e‑mail address listed as “recovery options” for Deb’s account. That alternate e‑mail account, with AOL, was no longer active, and in any case whoever had taken over her Gmail settings seemed to have removed or changed the information. The next line of defense was to submit a form reporting that an account had been taken over or compromised. We had sent in that form within 30 minutes of discovering the problem, giving my Gmail address as the new contact point. Meanwhile my wife logged into a secondary Gmail account she had previously created, and began writing to friends and family as quickly as she could to explain what was going on.
While we waited for results from Google, we began to hear, by phone and via our other e‑mail accounts, from people who had written back to Deb “in Madrid” to find out more about her predicament. They had all quickly gotten responses, from an account meant to look similar to hers but with a one-letter difference: firstname.lastname@example.org. We learned later that, as a predictable part of a hacking attack on Gmail, Yahoo, Hotmail, or any other e‑mail service, an attacker will change the settings so that all incoming mail is forwarded to a new, similar-seeming address—and then deleted from the real account, to make it harder for the real user to know, later on, who had responded to the scam. And whoever was on the other end of the exchange had gathered at least enough personal info to sustain a round or two of exchanges with concerned friends. For instance, Deb’s mother in Florida, 89 years old, had written back immediately to say that the message was a fraud, signing the note “Mom.” Her message went instead to debbfallows, who sent this reply:
From: “Deb Fallows”
Date: Wed, 13 Apr 2011 14:23:23 +0000
Subject: Re: terrible scam
I am too old to raise a false alarm. I was mugged last night under gun point.
I need €1,500. Below are my details you for the transfer,kindly get this done from any Western union office close to you:
Receiver Name: Deborah Fallows
Don’t forget to email the Money control number(MTCN) to enable me pick up the funds. I promise to pay back once I return to DC. Expecting to hear from you.
Other than this,how are you doing?
We thought that “other than this” was a nice touch. Other friends who replied got other variations, all with the same basic social-engineering info—the knowledge that our regular home was “in DC,” and that I was her spouse. A friend in New York was ready to send money, if he could only talk with Deb on the phone to be sure. His exchange with the hacker finally petered out this way:
I had to check out of the hotel due to accumulated bills. I am in a cyber cafe at the moment,sorry I cannot afford a call card.
So far, this was embarrassing, and possibly costly to the most openhearted or trusting among our acquaintances, but not worse than that. We’d returned only a few days earlier from a two-month stay in China. Perhaps this was one more predictable aftereffect, like my chronic cough? Things seemed to be improving when, around 2 p.m., a message from Google’s help system arrived in my account, with instructions on how Deb could at last reset her password and regain control of her information.
She did so, and logged into her Gmail account with enormous relief, which lasted perhaps five seconds. When she looked at her Inbox, and her Archives, and even the Trash and Spam folders in her account, she found—absolutely nothing. Of her allocated 7 gigabytes of storage, 0.0 gigabytes were in use, versus the 4+ gigabytes shown the day before. Six years’ worth of correspondence and everything that went with it were gone. All the notes, interviews, recollections, and attached photos from our years of traveling through China. All the correspondence with and about her father in the last years of his life. The planning for our sons’ weddings; the exchanges she’d had with subjects, editors, and readers of her recent book; the accounting information for her projects; the travel arrangements and appointments she had for tomorrow and next week and next month; much of the incidental-expense data for the income-tax return I was about to file—all of this had been erased. It had not just been put in the “Trash” folder but permanently deleted.
In some other circumstances, we might have had a calmly reasoned discussion about whether it made sense to have so much emotionally and practically precious information in a single, now evidently vulnerable, place. Even in these real circumstances, we realized that with enough persistent effort she could have eventually rewoven parts of the missing fabric. Her mother still had some of the messages Deb had sent about her father, I had some of what she’d written and done in China; bit by bit she might get some things back. For the moment, all we could do was clean up some of the traces of the attack that remained in her account—the command to forward all incoming messages to DebbFallows, the bogus e‑mail address and phone numbers for the password-recovery routines—and fill out another form on the Google help site, this one to request an automated recovery of deleted e‑mail.
It was at about this time that I started thinking about the ramifications of this problem beyond our own situation, desperate as that situation felt to us just then. Through more than 30 years of computing, I’d had my ups and downs with data storage. My very first computer, a Processor Technology SOL-20, was nearly incinerated along with all of its electronic contents when a lightning bolt hit our house in the early 1980s. (The contents included the notes and drafts for my book National Defense, which fortunately I’d printed out on paper.) Hard disks fail; laptops get dropped. But I’d never before imagined the chance of total, catastrophic, years’ worth of loss. This was a loss whose sweeping magnitude was possible only because my wife had entrusted her data exclusively to the most professional of pros: Google’s operation in the cloud. If we had thought that data security was strictly up to us, we’d have made backups of some sort to limit the potential damage—much as we would lay in our own firewood and keep our own chickens and cows to be sure we’d never freeze or starve if normal supplies were cut off. In my own version of Depression-style thinking, and with that lightning strike in mind, I had always made triply redundant backups of anything that mattered to me, including e‑mail. Local on-disk backups of Gmail archives, via programs like Eudora and Thunderbird—or both. Online backups of those local backups, through SugarSync and Dropbox—and then more local backups on my other machines. But my wife had trusted the cloud and Google. And now?
Her move to the cloud had coincided with the larger and irreversible shift of business, personal, governmental, and every other sort of activity to the cloud. The shift is irreversible because it brings so many advantages. Who would go back to searching for addresses on paper maps after using online mapping services? Needing to save and file canceled paper checks rather than inspecting them online, or doing a thousand other chores in pre-cloud form? In addition to these corporate and public services, whose users are increasingly conducting their business and storing their data in the cloud rather than on paper, our personal data has moved to the cloud as well, with the premise that we’ll be able to retrieve and work on our correspondence, our contacts, our photos and documents, from any computer connected to the Internet. But, of course, the more we rely on the cloud, the more we expose ourselves to its vulnerabilities. These include the breakdowns that affect any complex system. When much of Washington had a multiday power outage after a snowstorm last January, the loss of Internet service seemed almost as crippling as the loss of light and heat. They also include deliberate attacks—for criminal gain, spying, or sabotage—that are sure to increase as the value of cloud-based information does. “Where the money is, that is where the criminals will go,” a former National Security Agency official named Ken Silva, who now works as an online-security specialist for Booz Allen Hamilton, told me this summer. “Where the sensitive information is concentrated, that is where the spies will go. This is just a fact of life.” The more important online storage becomes, the more relentlessly it will be under attack.
For instance: Chastened by my wife’s experience, I decided to make my online passwords “stronger,” and to shift to an online storage site to manage them. The following week, that site—LastPass.com—was itself hacked and some of its data stolen. (I still use it, as I’ll explain.) At around the same time, the anonymous hacker group LulzSec, operating under the motto “Laughing at your security since 2011” (the first part of the name is phonetic for “LOLs”; the second stands for “security”), started functioning as a kind of tech-world version of WikiLeaks, penetrating corporate sites and then publishing large numbers of usernames and passwords.
Sony, Citibank, Veterans Affairs, major hospitals, tech firms like Intel, Cisco, and Google—I stopped keeping track of the institutions that announced intrusions, after security experts told me that essentially every major organization suffers ongoing attacks. But I used the shock of my wife’s experience as an occasion to educate myself about the vulnerabilities and new rules of operation in the cloud era, as they involve corporations and institutions as well as individuals. What I found is not all good news, but it is better than I might have feared. It includes some hopeful signs about the way corporations and governments are defending their data, and manageable practical steps individuals can take to avoid scares like the one my wife had that day.
I say “scare” rather than “trauma” because—to skip ahead in the story—my wife eventually got her e‑mail back, through Google’s recent “Undeletion Project,” as I called it when I learned of it. But it was a long time before that happened, and our attitude toward Google got much worse before it got better. I concentrate on Google here because that’s where we had our problem, and more generally because of its exceptional international role. But everyone I spoke with there and at other organizations emphasized that our experiences with Gmail—the brush with disaster and subsequent revelation of the gulf between data professionals’ view of reality and what the rest of us assume—were not exceptional at all but were variations on a cloud-wide theme. And our experience and revelation would apply to most people using most online services, including Apple’s pending “iCloud” services and Microsoft’s continuing movement of Windows services to the cloud.
I felt antsy rather than sleepy on that first night after the attack, as I kept fielding calls and e‑mails from friends and spending time on hold trying to change our credit-card numbers. So I was still at the computer a little after 2 a.m., monitoring both of our e‑mail accounts, when Google’s recovery team sent its response to our “My e‑mail is missing” form. I’ve boldfaced the parts that jumped out at me:
From: The Google Team
Date: Thu, Apr 14, 2011 at 2:01 AM
Subject: Re: [#791225671] (no subject)
We have processed your request to recover mail that may have been inappropriately purged from your Gmail account. Any previously deleted messages that we were able to recover will now be in your account in a newly added label called ‘recovered