History is mostly silent to us now.
Thousands of years of human stories have been told in paintings, and sculptures, and sheet music, and text; in shards and shells, and other fragments of things left behind. But because the history of recorded sound is only 160 years old, the original sounds of the distant past are lost to time.
“And the history of recorded sound, it’s not even very high-quality recorded sound,” said Christos Kyriakakis, the director of the Immersive Audio Laboratory at the University of Southern California.
Kyriakakis is part of a team of researchers who recently set out to analyze the acoustics of Byzantine era churches. “You think about wax cylinders from the Edison era [in the 1890s],” Kyriakakis said. “But we’re talking about trying to analyze sounds that were happening in the fourth century.”
The project began because Sharon Gerstel, an art history and archaeology professor at UCLA, realized something was missing from her already-deep understanding of Byzantine art. “What struck me was, we always look at paintings without thinking about the sonic accompaniments,” she told me. “So many paintings of a certain period contain representations of hymns and hymnographers, but people were looking at these paintings as if they were mute.”
The more Gerstel thought about it, the more this bothered her. The music of the Byzantine era, she decided, was a key to understanding her area of expertise—and not just the music itself, but understanding the experience of hearing it, and what it would have been like 700 years ago. “As an art historian, I could look at the pictures and say, ‘this is a nice painting of the hymn,’ but I couldn’t say anything about how the audience perceived that painting within a ritual setting.”
Which means she also couldn’t fully appreciate why churches began to change shape and size in the 13th century. Perhaps, she realized, it was to optimize the sound of chanting. “It seemed to me that the only way to think about these paintings and their meaning was to think about the music,” Gerstel said.
So she teamed up with Kyriakakis and James Donahue, an associate professor of music production and engineering at Berklee College of Music, and together they devised a plan to map the acoustic fingerprint of several churches, starting in Thessaloniki, Greece. Even before their technical analysis began, it was clear that these ancient spaces were designed to shift a person’s sensory experience.
“You cross the threshold and your eyes immediately have to adjust,” Gerstel said. “It seems pitch black inside. The first thing you notice is images of saints, who are your size, staring at you. Gold halos against dark background, and they seem to loom. It smells of incense. You’re in this world of myrrh. The temperature is different as well. Inside, you’re in a much cooler space. Your entire body adjusts ... and then to have music at the same time? That hits every sense.”
“What was truly surprising for me,” Donahue said, “was going into a space that was ancient, and to crawl around the ceiling and look at the walls and realize that they were looking at things acoustically. It wasn’t just about the architecture. They had these big jugs that were put up there to sip certain frequencies out of the air ... They built diffusion, a way to break up the sound waves by putting striations in the walls. They were actively trying to tune the space.”
“They also discovered something that we call slap echo,” Donahue added, “when you have walls fairly close to one another and the frequencies go back and forth. It goes ta-ta-ta, ta-ta-ta, ta-ta-ta, ta-ta-ta, ta-ta-ta. [In the ancient world,] they described it as the sound of angels’ wings.”
When Donahue and his colleagues were in Hagia Sophia—once the cathedral of Thessaloniki——they used a test tone at different frequencies to see how the space responded to sound.* The tone ranged from about 50 hertz, which sounds like a low buzz, to 20 kilohertz, a high-pitched whine. “I heard the standard sweep tone until it hit 6 kilohertz, and then it just spread out everywhere,” Donahue said. “I could hear the fluttering. I said, ‘Wow, those are the angels.’”
To map the acoustics of ancient spaces, to understand how a church was designed to reverberate at certain frequencies, Kyriakakis and Donahue gathered what’s called an impulse response. To do that, they placed loudspeakers omni-directionally throughout a church. Then, over the loudspeakers, they broadcast a test signal, like the one Donahue described in Hagia Sophia. “It’s a very long chirp that starts at low frequencies and goes up to high frequencies and it just sweeps through, like a whooooop,” Kyriakakis said. “And you record from various locations with microphones to see what happens to that chirp as it bounces around the church.”
The data showing what happened to the chirp in each part of the church is fed to a computer, which then registers the impulse response for the unique space. And here’s where it gets really interesting: Once you have a building’s impulse response, you can apply it to a recording captured in another space and make it sound as though that recording had taken place in the original building.
“So you can take chanters with the original [Byzantine era] music and put them in a studio that has no acoustics,” Kyriakakis said. “They can sing a chant, and then we can process it ... and all of the sudden we have performances happening in medieval structures. It’s like time travel to me.”
The implications go far beyond the ancient world. Kyriakakis, Donahue, and Gerstel imagine creating a catalog of impulse responses for historic buildings, then recreating the sounds of those structures in what would be, essentially, a museum of lost sound. With an integration of virtual reality technology, visitors could even get the experience of how the sound would have changed as people moved through a given space. (Theoretically, they could share these recordings online, too, but both Kyriakakis and Donahue say it’s harder to render the sound authentically over headphones. They talk more about the idea in a USC Engineering podcast.)
The museum they’re envisioning would include churches, like the ones they’ve already mapped, but other structures, too—everything from ancient theaters and the Parthenon (an experiment that would also require mathematical modeling to bring back the missing part) to modern baseball stadiums and train stations. “If we open up this idea,” Kyriakakis told me, “there’s no limit as to what can be measured and recreated.”
“Some of these buildings may not exist later,” he added. “Some of these historic buildings are being destroyed.”
At Princeton, the historian of technology Emily Thompson has worked on her own recreation of lost sounds, using a combination of maps, recordings, and public records. She created an interactive map of New York City, overlaid with audio and video recordings of various places from the 1920s and 1930s. So you can hear sound of city traffic outside of the old Penn Station on a July afternoon, and the clank of an automatic dishwasher at the New York Life Insurance Company Building, and the shouts of children at a contest to see who can hold their breath the longest in a Central Park swimming pool. Thompson also included reports of noise complaints, mapped to where they originated, so you can read descriptions of what neighborhoods sounded like. (“But, as with all historical sources,” Thompson told me in 2013, “the vast majority of the past is gone.”)
As for Gerstel, studying the acoustics of Byzantine churches has fundamentally changed the way she thinks about art history. In addition to producing an impulse response, she and her colleagues recorded several hours of chanters singing in the churches, to test their theories about architectural design, an experience that Gerstel called “awe-inspiring” and “transportive.” Hearing the chants performed in the original churches brought into relief the idea that certain works of art were meant to be understood in conjunction with song. The hymn might instruct a person to “stand trembling,” for instance, and the angels in a painting nearby seem to be responding.
“When you enliven that text, it changes the way you look at the art,” she told me. “That’s the problem with the way we study art. We don’t think about the ritual that’s unfolding, linking emotions and music, inscriptions and art.”
It’s strange, in a way, that museums are quiet as tombs. Perhaps silence is a form of reverence, but it still seems unnatural. I asked Gerstel what she thought it said about our culture and our relationship with art, that museum spaces are so often hushed. “I can’t say,” she said. “Except it’s almost as if museums have become churches.”