Moving .PST files into Gmail: you can try this at home!

Challenge was announced last week, here: how to get many years' worth of email stored in old Outlook .PST files into an online Gmail account.

Main reason for change: wanting to have email archives searchable and available from any computer, rather than worrying about which .PST is on which machine. They'd also be searchable and usable from mobile devices with Gmail apps -- which I know first-hand includes BlackBerry and Android/Nexus One, and which I believe now includes iPhone and SideKick. Other motives discussed in the original post.

Promising-sounding solution that's not right for me: 
Google offers its own "Google Email Uploader" (left) available for free download here. Uploader.pngIt sounds as if it exactly fills the bill: "It uploads email and contacts from desktop email programs (like Microsoft Outlook® ) into your Google Apps mailbox. It preserves information such as sent dates and sender/recipient data, as well as the folder structure used by email programs." Unfortunately, it doesn't work with normal, free Gmail accounts. It requires a different "Google Apps" account. These cost $50 per year (details here), and while that's hardly prohibitive, for the same money I could buy 200 GB of online storage from Google (see here). And this approach would defeat my purpose of concentrating my mail in one, main, existing Gmail account.

Conceptual basis of the real solution. It involves IMAP accounts. If everything about IMAP and POP3 is already obvious to you, skip to the step-by-step list further down. If not, read the next paragraph.

Outlook (and other email handlers) can interact with online email services, like Gmail or Hotmail, in two main ways, via POP3 or IMAP. (There are others, which we'll ignore for now.) Fundamentally, POP3 is a way of collecting messages from Gmail or Hotmail and transferring them to your computer, whereas IMAP is a way of mirroring what's in your online account on your own computer. Here's the practical difference: With a POP3 account, what you do to messages on your own computer has no direct effect on the versions of those messages stored online. You can collect all your Gmail message into Outlook and save or delete them there, but the originals will still be sitting in your Gmail account. But if you've set up an IMAP connection between Outlook and Gmail, what happens to a message in either place affects it in both. If you delete a message in Outlook, it also disappears from Gmail. And if you add a message to an IMAPed Outlook folder, it will also be added to the corresponding Gmail account. That's the key to the .PST -> Gmail transfer.
Step-by-step procedure: This is all you have to do. It's easier than it looks. If you have a lot of messages, the full transfer takes some time -- but it's mainly "the computer is thinking" time, rather than "you have to pay attention" time.
1. Load Outlook, and create a "New Email Account." Procedures vary with different releases of Outlook. For instance, in Outlook 2007 it's Tools / Account Settings / E-mail / New. Give this new account the settings (user name / pw) of the Gmail account in which you want to store your old .PST messages.

2. When setting up this new account, specify it as an IMAP account, not a POP3 account. (That's the choice Outlook will give you.) Note: this can be the same Gmail account that you're already using with Outlook, via a POP3 connection. There's no problem with Outlook having two of its own email accounts addressing the same Gmail account. For years I've collected my Gmail messages into Outlook via POP3. For the transfer, I just created a new IMAP connection to that same account. Outlook happily keeps both of those open at the same time.

3. Follow very carefully the configuration instructions here and here for your new IMAP account. They're easy, but you have to get the details right. The first step is to enable IMAP within Gmail (see the first link); the second is to configure your new Outlook IMAP account, as explained in the second link. Mine didn't work until I was sure to put in the "993" / "587" port settings, below, on the "Advanced" tab:

4) Open Outlook, where you will find a new folder that is IMAP'd (mirrored) to your Gmail account. Whatever is new and unread in your Gmail inbox will look the same way in Outlook. Here is the way the new account looks in my Outlook right now:
It has the name "Gmail IMAP" because that's the one I chose -- you can call it anything. The important point to note here is the folder structure of this new email account. I didn't create any of the subfolders shown here -- Inbox, &Answer, etc. Those are all automatically created, as part of the IMAP process, based on labels in Gmail. A label in Gmail is exactly equivalent to a subfolder in an IMAP Outlook account. We'll see why this matters in the next step.

5) Create a new label in your Gmail account -- let's call it "Transfer." OR, create a new subfolder in your Outlook IMAP account, like "Transfer."  Right-click on your Outlook account and choose "Update Folder List," and labels in Gmail will be synchronized with the subfolders in Outlook. You're almost there.

6) Using Outlook, open the .PST file whose messages you would like to transfer to Gmail. Drag-and-drop those messages over to your new "Transfer" folder. You can mass-select a lot of messages and do this at one time. It may take Outlook a little while to get them all transferred, but it will happen.

7) Go do something else.

8) When you come back to the computer, all the messages that you added to your "Transfer" folder will now be in Gmail.  They will all have a Gmail label corresponding to the folder you created -- "Transfer," etc.  What I have done is to set up these folders by year. "Transfer 2003" etc. Once they're over in Gmail, you can do with them whatever you want. Remove those labels; add new ones; archive; and of course search. From any of your computers or mobile devices, you can rummage through whatever you've stored in your Gmail account.

What comes next. For a future installment, what to do after you've gotten everything you care about transferred across to Gmail storage. One immediate step is to close that IMAP account you used for the transfers. In my experience, Outlook bogs down considerably when it is handling all the synchronization needed for an IMAP account. (That is worth it when you're making the transfer, but not as a standard practice.) Should you erase those old .PST files? No way! More backups are better. (But now you don't have to copy them machine-to-machine.) Should you still use Outlook as a mail-handler? If you want -- but without worrying about where or whether you've copied its archive of old files.

More details on other fronts when I regain energy for this topic. (For instance: what about finding old messages in Gmail when you're off line? And how should you feel about having all your correspondence in this one new basket? Much more to discuss.) In the meantime, thanks and promised rewards soon on the way to those who were early on recommending IMAP.