Spend any time performing classical music and you are told that appearance matters. A choir can enter a hall and, through their demeanor alone, receive applause. A band strolling on to a darkened stage gets cheers. We live with YouTube music videos as much as we live with invisible MP3s, and what we see prepares us, excites us, primes us, for what we’re about to hear.
So the results of a University College London study, published last week, seemed intriguing. The psychologist Chia-Jung Tsay showed novice musicians and professional adjudicators different types of media from an international piano competition. She had them only listen to audio, only view silent video, or only watch video clips with sound of the contestants performing. Then she asked them to identify, from experiencing just one of those types of media, which of the contestants had won the competition.
And, as NPR recently reported:
"What was surprising was that even though most people will say sound matters the most, it turned out that it was only in the silent videos, the videos without any sound, that participants were able to identify the actual winners," Tsay says.
Incredibly, the volunteers were better able to identify the winners when they couldn't hear the music at all, compared with when they could only hear the music. In fact, it was even worse than that: When the volunteers could see the musicians and hear the music, they became less accurate in picking the winners compared with when they could only see the performers. The music was actually a distraction.
At first glance, it looks like a staggering result. The study’s abstract declares: “[T]he findings demonstrate that people actually depend primarily on visual information when making judgments about music performance.”
But looks can be deceiving. Tom Stafford, a cognitive scientist and author of the book Mind Hacks, found the study, and its coverage, problematic*. The finalists at an international piano competition are some of the finest musicians in the world, and, he writes on his website:
In all probability there is a minute difference between their performances on any scale of quality. The paper itself admits that the judges themselves often disagree about who the winner is in these competitions.
The experimental participants were not scored according to some abstract ability to measure playing quality, but according to how well they were able to match real-world competition outcome.
The experiments show that matching the judges in these competitions can be done based on sight but not on sound. This isn’t because sight reveals playing quality, but because sight gives the experimental participants similar biases to the real judges. The real expert judges are biased by how the performers look – and why not, since there is probably so little to choose between them in terms of how they sound?
The science blog Arcsecond agrees. If all the players were the best in the world, their auditory performance would be among the best in the world and thus very close, but:
the variation in how the musicians move and express themselves physically could potentially be large – 50, 70, 90, for example. So even if judges base their scores mostly on the quality of playing, the visual aspect can still dominate the final rankings.
But how to explain the silent video viewers outperforming the audio-and-video watchers? Perhaps they don’t need to. “[I]t’s not like people with visual information did very well,” writes Arcsecond: “They got to roughly 50% accurate.”