Where Online Services Go When They Die

Rebuilding Prodigy, one screen at a time

Michael Doino approached the late hours of October 1, 1999, with a lingering sense of dread.  It was finally time, after 11 years, to pull the plug on Prodigy Classic, a commercial online service he had helped shepherd from a plucky upstart into a nationwide giant.

"It was very bittersweet, very sad," recalls Doino, a veteran project manager at the company. "I had been there before the Prodigy service went live."

Some time before midnight, Doino logged into the main Prodigy Classic server and, as instructed, uploaded a file to redirect Prodigy Classic users to the company's newer Prodigy Internet service.  At that moment, the written record of a massive, unique online culture, including millions of messages and tens of thousands of hand-drawn pieces of digital art, seemingly vanished into thin air.

Doino, front and center, 1999 (Prodigy)

It had no where to go but away. That data was never on the Internet; it existed in a proprietary format on a proprietary network, far out of reach from the technological layman.  It was then shuffled around, forgotten, and perhaps overwritten by a series of indifferent corporate overlords.

Fifteen years later, a Prodigy enthusiast named Jim Carpenter has found an ingenious way to bring some of that data back from the dead. With a little bit of Python code and some old Prodigy software at hand, Carpenter, working alone, recently managed to partially reverse-engineer the Prodigy client and eke out some Prodigy content that was formerly thought to have been lost forever.

"Honestly, I wasn't a huge fan of Prodigy," says Carpenter, a 38 year-old freelance programmer based in Massachusetts, recalling his time on the service around the turn of the 1990s. "I had already been using the Internet for a couple of years and Prodigy seemed so closed in. But I still used Prodigy every single day. It was the graphics."

It was Carpenter's drive to see those graphics once again that got him fiddling with Prodigy clients in late 2012. "Finding decent color screen shots of Prodigy is nearly impossible," says Carpenter.

He knew the sign-on screen was stored on the hard drive, so he began to wonder what else he might find in the client software. Using a hex editor, Carpenter fiddled with the client software until he found even more graphical data. "As far as I knew, the only thing I might be able to get is a screenshot of the set-up options dialog."

And he did.  But what he found next blew his mind.

* * *

When any sizable online service disappears, a piece of our civilization's cultural fabric goes with it. In this case, the missing cultural repository is Prodigy, a consumer-oriented online service that launched in 1988 as a partnership between Sears and IBM. Users accessed it by dialing into regional servers with a personal computer and a modem over traditional telephone lines. Once connected, they could trade emails, participate in online message board discussions, read the daily news, shop for mail-order items, check the weather, stocks, sports scores, play games, and more.

Prodigy even devoted a portion of the user's screen to graphical banner ads. It was very much like a microcosm of the modern Internet—if the entire World Wide Web was published by a single company. Over its 11-year lifespan, a generation of Americans grew up with Prodigy as part of their shared cultural heritage. In an earlier era, we may have spoken about another common cultural experience—say, Buster Keaton films—as a cultural frame of reference for an entire generation. Everybody saw them, everybody referenced them. And while Prodigy was nowhere near as popular as Buster Keaton among the general public, hundreds of thousands of people with a computer and a modem in the early 1990s tried Prodigy at least once. What those early online explorers saw when they logged in was, to them, glorious: colors, fonts, illustrations, and a point-and-click interface—features which, at Prodigy's launch in 1988, were entirely new. Prior to Prodigy, competitors like CompuServe and GEnie forced users to type obtuse commands to get any meaningful result (and that result also happened to be a screen full of lifeless text).

Prodigy gained its distinctive flair from a now-forgotten graphical protocol called North American Presentation Level Protocol Syntax, or NAPLPS for short. NAPLPS was a product of the brief Teletext era of the late 1970s, when TV networks sought to piggyback extra digital information such as weather forecasts or sports scores using something called the "vertical blanking interval" of a TV broadcast signal. The vertical blanking interval could only hold a small amount of data, so engineers devised a way to present digital color graphics and text in the most economical way possible. NAPLPS did this by reducing an image into a set of mathematical instructions (i.e. "draw an oval at this location and fill it with blue") instead of storing data on every pixel in a bitmap image like JPEG or GIF files do today.

Screenshot of Where in the World is Carmen San Diego? on Prodigy circa 1988 (Prodigy)

The NAPLPS method required a custom piece of hardware or software, commonly called a "terminal" or "client," on the receiving end to receive the drawing instructions and to translate them into an image or page layout on the user's screen. Teletext never caught on in the US (although it did flourish in Europe), nor did Videotex, the two-way interactive version of the concept that required remote computers accessed by modem and corresponding terminals hooked to TV sets.

* * *

Coming on the heels of Videotex mania, which swept the Western world in the late 1970s and early 1980s, Sears, CBS, and IBM joined together in 1984 to craft a Videotex service of its own.  They called their partnership Trintex: "Tri" for the three companies, and "tex" for Videotex. The plan, as conceived from a corporate standpoint, was almost naively simple: the world's largest retailer (Sears) would provide online shopping. The world's largest media conglomerate (CBS) would provide content and information, and the world's largest computer company (IBM) would provide the underlying technology.

Kim Moser

How the trio got there, however, would turn out to be far more complicated. A very expensive technological effort (which, among other minor hiccups, required creating a nationwide proprietary telecommunications network with hundreds of nodes), would end up inadvertently crafting a consumer online world for the everyman that eerily presaged the Internet we know today—if in a Bizarro Superman type way.

Looking back, Prodigy's technology felt like a centralized, parallel universe Internet where technologies looked very, very similar to what we know now but are in fact fundamentally different—like lifting the hoods of two identical-looking cars and finding a diesel engine under one and a gasoline engine under the other. They both get you there, but in different ways. Even so, the similarities were close enough that patents, legal precedents, and online techniques forged from the Trintex and Prodigy partnerships still loom over the Internet in ways that few in the public understand.

To put the Trintex partnership in modern terms, it was as if Wal-Mart, Comcast, and Apple were to team up today and rewrite the rules of media distribution and general retail commerce. It's a terrifying prospect. But the online landscape back then was raw and rough, undefined and relatively new, so few feared a partnership from such a trinity of giants in 1984.

And, as giants are wont to do, it took them four years to bring the service to the market. Along the way, CBS dropped out, and Sears and IBM were left to fend alone. The remaining pair changed the name from "Trintex" to "Prodigy" to reflect not only the lack of a third partner, but also to reposition the company with a mass-market name that would appeal to the general public.

After launch in 1988, Prodigy soared in the consumer online space until it was handily surpassed in subscribership by AOL in the early-mid 1990s. Then, of course, it was wholly trampled in the late 1990s by that hungry, all-consuming digital maw called the Internet.

* * *

At the time it was finally shuttered, Prodigy was an absolute dinosaur technologically. Built from systems that were state-of-the art at the dawn of the 1980s, and existing on top of a complex and proprietary network infrastructure that was always separate from the Internet, Prodigy existed in spite of itself.

Eager to pivot entirely to its burgeoning ISP business, Prodigy's parent sought a convenient escape route from Prodigy Classic in the late 1990s. Subscribership to its Classic service had dwindled to 208,000—down from 1 million a few years earlier—and the infrastructure was costly to maintain. Conveniently citing the "Y2K problem," Prodigy's CEO, Samer Salameh, announced a Classic shutdown in early 1999.

After that shutdown, loyal Prodigy customers, who had hung on to the bitter end, were suspicious about the stated reasons for the closing. And they were mad. Fifteen years later, we can now confirm that their suspicions were correct: "As far as I know, Prodigy Classic being shut down was not influenced by Y2K issues," recalls Doino, the Prodigy employee who actually pulled the plug on the service in 1999.

But even an insurmountable tide of customer goodwill cannot stop one of the most sacred laws of the free market: that unprofitable products, even if they happen to be one-of-a-kind repositories of digital human culture, eventually meet their end at the hands of a corporation that needs to make money to survive. When Prodigy Classic shut down, its servers entirely shifted over to servicing the ISP portion of Prodigy's business. The network of regional servers—called Prodigy Local Sites—was dismantled. The actual Prodigy Classic data became neglected, and its whereabouts are uncertain, although I'm trying my best to track it down.

Even if Prodigy's archives are found, various former Prodigy employees say the data is trapped behind a technological minefield of obsolete storage formats, protocols, programming languages, and computer systems. And, naturally, each one must be present and working in tandem to have any hope of ever accessing the information. In other words, to resurrect some of Prodigy, you'd have to make all of it work again.

"Numerous attorneys fighting patent cases have asked me about this," says Les Briney, Prodigy's former executive director of technology and architecture.  (Briney is describing IBM's existing portfolio of lucrative Prodigy-related patents, which apply to many parts of the modern Web.)  "My estimate is in the million dollar range to do this."

But plenty of things seem costly-to-impossible until you actually do them. Just ask Carpenter, the programmer who stumbled upon something stunning when he was tinkering with Prodigy client software. He’d uncovered some of the old graphical images he was after, but then he found an unexpected trove leading into the past: "Then I discovered STAGE.DAT."

Prodigy in 1992, as captured on a home video by the author's father. (Benj Edwards)

STAGE.DAT, as it turns out, was one of the Prodigy client's two cache files. These two files, CACHE.DAT  and STAGE.DAT, stored both temporarily and frequently used data on the user's machine to speed up page load times. (This same STAGE.DAT got Prodigy in trouble in the early 1990s when users discovered that it could contain fragments of data culled from their PCs. As it turns out, Prodigy's client was filling in "empty" portions of STAGE.DAT with random snippets of system memory. Users were convinced Prodigy was spying on them, uploading this data to its servers (it wasn't); Prodigy denied this and released a tool for the paranoid to zero out their STAGE.DAT files.)

Prodigy's entire architecture, in fact, was based on this caching system, with a central server at the hub in Yorktown Heights, New York, and hundreds of regional caching servers spread throughout the US. That way, server load was distributed, and loading times were minimized from the user's standpoint. Data would propagate from the top down. Whenever a Prodigy user called up a page, a portion of that data was ultimately downloaded to his machine, much in the way web browsers cache HTML and image data today.

So here's the key to Carpenter's discovery: Whenever a user last dialed into Prodigy before it shut down in 1999, the data saved to STAGE.DAT was frozen in time like a mosquito stuck in digital amber. Carpenter found a way to tap into that amber and extract the data. His series of Python programs reads through a previously used STAGE.DAT file, generates a list of pointers to the pages or object data contained within, then directs the Prodigy client to display them one at a time so he can take screenshots.

The fact that Carpenter has to go through such a Rube Goldbergian series of steps to get at these images is a result of Prodigy's complexity. As it turns out, the service didn't use the vanilla NAPLPS standard to render its graphics. Prodigy's graphically rich screens—most of which were hand-drawn by Prodigy's staff of artists—existed as a proprietary, object-oriented superset of NAPLPS. Different portions of any given Prodigy page, including graphics, text, and interactive elements existed as separate "objects" that the Prodigy client assembled onto the user's display.

Like the caching system, this object-oriented behavior was born out of both technological necessity and a desire to reduce business costs, explains Robert Filepp, one of the original engineers who designed Prodigy's backend technology. Prior to the birth of Prodigy, AT&T held a limited Videotex subscription service trial in New Jersey. During the trial, says Filepp, "[AT&T] had a game called 'Make a Monster' in which users could take out body parts and decide to put them together into different pieces." But the monsters didn't actually have separate body parts, data-wise. "The artists had to create every possible combination and permutation of content, one per page."

At the end of the trial, AT&T had almost as many people developing content as they did subscribers, since each and every new piece of information introduced on the service required an entirely different screenful of NAPLPS artwork. To avoid this problem, David Waks, Prodigy's former head of research and development—who could also be called the godfather of Prodigy's architecture—conceived of the object-oriented approach where different portions of the screen could be updated independently of each other.  But this was only one part of his greater plan.

In sketching out the specifications for Prodigy, Waks envisioned a world in which, instead of dumb terminals hooked up to TV sets, users of a Videotex service would use custom client software running on personal computers—a dramatic leap forward, conceptually. In order for the same interactive Videotex content to be displayed on multiple incompatible platforms, Waks imagined a sort of virtual machine environment that would execute an embedded programming language while also displaying resolution-independent vector graphics to that machine's best ability. So, not only to save artist manpower, but to save bandwidth, it was best to break each page into chunks that could be shuffled around to regional servers and cached, then fed into graphical templates hosted on users' machines.

Screenshot of a weather forecast on Prodigy circa 1988 (Prodigy)

With Waks' approach, later adopted by Prodigy with some modifications, every page became a package of objects: some of the objects could be programs, some of them could be text, and some of them could be graphics. It was up to the client to reassemble them in the correct manner. Even today, the vintage Prodigy "Reception System" client software (as Prodigy called it) is the key to unlocking the service's graphically rich pages. With 250,000 lines of C++ code created and modified by 30 engineers for over the course of a decade, the Reception System is very difficult to reverse engineer and duplicate. Carpenter, spry as he is, isn't up to that challenge yet. But his gears are turning.

"Some day I'd like to create something to emulate the Prodigy backend and serve up requested objects to the client," says Carpenter, who plans to use the existing Prodigy client as the interface. "Perhaps I could bring Prodigy back online like hobbyists have brought QuantumLink back."

We may see a day, thanks to the efforts of devoted hobbyists like Carpenter, where anyone on the Internet can view original Prodigy on the web. Imagine a Javascript-based Prodigy Reception System in the vein of Javascript MESS that runs in any browser and renders original Prodigy pages in all their original, dynamic glory. After all, each Prodigy page is not just a static screen, but a collection of potentially interactive objects.

A few limited simulations of Prodigy already exist, like this MadMaze recreation, hosted on my website. But to appreciate the full cultural value of what Prodigy brought to millions of users for over a decade, the public deserves to have continued access to the real thing.

To do so, Carpenter needs more data, and he's turning to the Internet for help.

"I need everything in people's old C:\PRODIGY directory," he says. "The whole thing zipped up, because a STAGE.DAT is meant to work with a specific version of a Reception System. The objects within it may use features only found in certain versions of the RS."

The Prodigy log-in screen as it looked in 1991, pulled from a STAGE.DAT file by Jim Carpenter. (Prodigy)

The key, as he mentioned, is to find the files from an existing Prodigy installation. Note that the Prodigy client installation disks themselves are not sufficient because they were never used to connect to the Prodigy service, and thus they do not contain downloaded content in the cache files. (If you have anything you think would help, email me.)

Carpenter also needs technical specifications and source code for the Reception System, which may be floating out there somewhere. Needless to say, finding working Prodigy installations from over 20 years ago is tricky. Luckily, I found one of my working Prodigy installations and gave it to Carpenter last year.  He managed to extract many of the Prodigy on-screen illustrations that you see accompanying this article.

* * *

As we invest more of our lives into the electronic realm, corporate decisions to shut down online services without recourse are beginning to resemble digital acts of Nero burning Rome—cultural history and entire communities are trashed in the process.

"From the beginning, online services have visualized what they do as some sort of product, but this is wrong," says Jason Scott, an archivist at the Internet Archive. "They have become critical public spaces and libraries of community speech. In some cases, you can have hundreds of thousands or even millions of people represented in this data."

We, as members of the public, have to be vigilant about how our commercially-driven society treats our shared cultural heritage. As we've seen from previous shutterings of services like Geocities, the world of business can seem painfully callous and indifferent to the needs of history, and that seems unlikely to change any time soon. "I am skeptical that these businesses will spontaneously decide to think in terms of archives and long-term preservation," says Scott. "It's just not in their DNA. Legacy is a luxury to the modern business."

Ultimately, it's up to us, the public, to save what we can.