A MACHINE," SAID ED RASALA, ONE DAY IN THE spring of 1979. "The machine. It's what everyone calls it. That's the whole thing, to build the machine."
See the first installment of this article from the July, 1981, Atlantic.
Rasala's machine was the new 32-bit minicomputer, nicknamed Eagle, that Data General's Eclipse Group was building. By January of 1979, after about six months of intense labor, the team had designed Eagle and two prototypes had been assembled. But computers are complicated, and theirs was more complicated than many. As expected, Eagle's design was far from perfect. The prototypes did not work. The Eclipse Group had to find and repair the flaws in their creation; they had, as they said, to "debug" it. A debugging could be harrowing, in part because it had to be performed with great thoroughness. The hardware, the actual circuitry, of modern computers is remarkably reliable, and needs to be. Eagle would do a cycle of work every 220 billionths of a second. If they left in the machine bugs that caused it to fail only once in every billion cycles, Eagle would be a very unreliable contraption indeed.
The debugging of Eagle did not go well at first, and the Eclipse Group's leader, Tom West, fretted, his stomach queasy, for a couple of months. Gradually, the team surmounted the first barriers to success, and in March West declared, "Most of the fear is gone now." But the debugging was not nearly finished. In effect, West merely passed on the responsibility for worrying to his lieutenant, Ed Rasala.
Computer engineers, like poets, usually blossom early, and Ed Rasala was at thirty-five an old and experienced tradesman. He was the captain of "the Hardy Boys," a dozen or so engineers who worked on Eagle's hardware. When the debugging began, Rasala's Hardy Boys virtually went to live with the prototype Eagles, in a small and windowless laboratory, behind locked doors, at Data General's headquarters in Westborough, Massachusetts. Soon most of the crew were spending the majority of their waking hours in the lab, for six days of every week, and even when they went elsewhere, in their minds they continued to grope around inside the new, still defective machine. For the people who were building it, the incipient computer defined a small world. Rasala was the leader of the debugging; it was, in effect; his job to see that this little world was rid of uncertainty.
From the top of his head, which was bald, to his feet, which were usually shod in construction boots, Rasala looked big. He wore a thin beard. He delivered firm handshakes, and he spoke rapidly. "It's almost a speech impediment," said one of his oldest professional friends. "Sometimes I feel like giving him a shot of Valium and then telling him, 'Okay, Ed, say it again."'
Of Ed Rasala Tom West once said, "His biggest strength is, the guy does not give up." Rasala's parents were immigrants from Poland; he grew up in modest circumstances in Brooklyn. It was his misfortune to have a brilliant older brother, against whom all of his teachers measured him. "From grade school on, it was always my rap, I never lived up to my potential," Rasala told me.
Rasala had been reluctant to work on Eagle when the project began; he had just completed another computer and he wanted a rest. Some months later, I asked him why he had changed his mind. He ticked off the reasons on his fingers. "I was looking for opportunity, responsibility, visibility."
What did those words mean to him?
Rasala shrugged his shoulders. "I wanted to see what I was worth," he said.
Rasala got along well with most of the Hardy Boys. Often he jokingly insulted them, and they replied in kind. Rasala liked a slightly contentious atmosphere. "Smart, opinionated, and non-sensitive, that's a Hardy Boy," he declared. Above all, he wanted around him engineers who took an interest in the entire computer, not just in the parts they had designed.
One evening at the end of January, on the lower level of Data General's headquarters building, Rasala walked up to a doorway labeled "RESTRICTED AREA." He unlocked it and led me down a corridor, around a bend, through another door, and into a chamber about the size of a living room. The floor was linoleum-tiled; the ceiling was a network of heating pipes and steel girders, from which many thick black electrical cables hung; the furnishings were gray metal. The chief attractions, standing side by side at shoulder height along a wall of cement blocks, and surrounded by a variety of adjunctive devices, were the two prototype Eagles. Their blue metal frames housed a multitude of wires. "Here are two state-of-the-art computers, sitting there," Rasala said. "They're not too impressive at this point in time." In each machine was a shelf filled with "wire-wrapped" prototype circuit boards. On these boards, which were rectangular and about the size of an atlas, were implanted the essential hardware of the computer. They rendered an accurate visual impression of what the debuggers confronted. One side of each board contained orderly looking rows of small boxes. A nest of tiny wires covered the other side.
The Hardy Boys were debugging Eagle by exercising the prototypes and by fixing them when they didn't work. "Exercising" meant running programs, which are the linked lists of instructions and data that direct a computer's actions. These exercise programs were called "diagnostics." They were designed to make the prototypes fail on account of both glaring and subtle bugs. When a failure occurred during the running of a diagnostic, the Hardy Boys searched for its cause and invented a repair. Often, their repairs required alterations in the wiring of one or more boards. Rasala had made it a rule that they perform this labor promptly, and he had come to the lab this evening to enforce his commandment and to participate in the process itself.
Everyone, including Rasala, disliked this part of the job. Altering the wires was dreary labor that strained the eyes. Rasala sat down to work on a board. In a little while he retrieved with a pair of tweezers a tiny strand of wire. Holding the tweezers aloft, he explained that it was very easy to drop a piece of wire into the nest of wires on a board, and an extraneous piece of wire could cause the machine to commit failures of the "flaky" variety; debuggers could spend hours, even days, searching for the cause of such a problem.
"We do this ourselves rather than have a wire person do it," said Rasala. "We may be a little more cautious." He turned back to the board he was working on, and added, with characteristic humility toward the gods of chance, "Then again, we may not be."
Rasala maintained that he was not the most inventive of the team's engineers. "But I'm a good designer, I think, and I'm a better debugger. I may not be the smartest guy in the world ... but I'm dumb enough to stick with it to the end."
In the first months of debugging, Rasala's main technical role was to play the brake on the Hardy Boys' enthusiasm. The sheer physical complexity of the computer required that they plod sometimes; otherwise, they would never gain control over this machine. Were signals just barely making their appointed rounds in the 220 nanoseconds that lay between two ticks of the computer's clock? If so, Rasala would make the engineers return to problems they thought they had solved. Was there a lot of noise inside the prototypes? "Noise," Rasala explained, is what makes the TV go haywire when the blender is turned on; it consists of stray voltages that are propagated through the machine. Too much noise fouls performance.
Once in a while, a Hardy Boy would find a problem, fix it temporarily, and move on. The engineer would plan to make a proper repair later, but, absorbed in new bugs, he would forget to do so, and weeks afterward his oversight would cause some mysterious failure. Such omissions were inevitable. By April, constant handling of the prototype was beginning to make unreliable some of the thousands of connections among components on the boards; loose wires and sockets began to cause some flaky failures. These were often hard to diagnose; as the debuggers liked to say, it's hard to fix something when it's working. Now and then an improperly manufactured component impeded their progress. "Just stuff you never account for in a schedule," Rasala said. "You assume it's not going to happen, and it always does."
In addition, the Hardy Boys were unearthing many flaws in the logic of their design. "We went with an imperfect design," said Rasala, early in the spring. "We knew we were pushing it." When debugging had started, in January, Rasala's boss, West, had promised the front office that Eagle would be finished in April. Rasala was skeptical, but, believing in the need for haste and being a loyal lieutenant, he had worked up a debugging schedule for that date. However, by March it was clear that they would not finish by April. So the group's leader had named a new deadline, and then another, and for each revised deadline Rasala had drawn up a new debugging schedule. In May, when his third revised schedule was hanging on a wall in his cubicle, Rasala remarked, "The way to stay on schedule is to make another one." Asked to evaluate the team's progress, he said, "It's coming, it's coming." He added, "There's still a good chance that we've totally blown it."
Near the beginning of the debugging he had said, "I believe there's always a focal point in any machine." And by May, the focal point of the problems had identified itself to him. It was the part of Eagle known as the instruction processor, or IP.
WHEN A COMPUTER RUNS A PROGRAM, THE MACHINE has to perform, again and again, two general sorts of tasks. First, it has to find and prepare an instruction; then it has to execute that instruction. Often a great deal more time is consumed in the preparation than in the execution. In order to make Eagle a fast computer, its designers arranged for it to perform the two operations simultaneously. The IP was, in the local idiom, an "accelerator." Instructions fall into oft-repeated patterns; the IP was designed to recognize the pattern and to figure out what, at any moment, the next instructions would be. In the 220 billionths of a second that lay between two ticks of Eagle's clock, the IP would send one instruction out for execution and at the same time make ready the next several instructions that the program is going to require.
The IP was a complex device, obviously; it was a sort of computer inside the computer, and like every computer, the IP had its own storage compartment, called the "I-cache." Another part of Eagle, called the "system cache," also contained a storage compartment, considerably larger than the I-cache. The system cache kept on hand commonly used instructions, so that if the IP made a wrong assumption and did not have the next instruction in its I-cache, it could get that instruction quickly from the system cache. Eagle had other storage compartments, including a "main memory," and all of these storage compartments had to be kept consistent with one another. Consistency, however, was not easily achieved. As Eagle maneuvered through a program, the IP and the system cache were constantly throwing out blocks of instructions and bringing new ones into their storage compartments. Although Eagle's designers had created mechanisms for preserving consistency among storages, they could not foresee all the possible sources of trouble. And the bugs that arose were particularly troublesome and devilishly hard to diagnose.
If some unforeseen electronic event should cause the I-cache to contain a block of instructions that "looked" the same but was in fact slightly different from a block of instructions in the system cache, Eagle would be bound to fail. Sooner or later, the IP would draw upon the outdated, inconsistent block of instructions in its cache and would order the computer to execute a wrong instruction. Usually, the IP would place this wrong order a long time after its cache had become inconsistent with the rest of the memory system. Then the debuggers would not be able to identify the problem by examining the failure. It would be as if a time bomb had been placed inside the machine.