Come on Feel the Data (and Smell It)

Digital interaction will engage all of our senses simultaneously, including smell and taste, to help us feel the impact of information in our guts
Google Patents/The Atlantic

In February of 1960, Scent of Mystery flopped. The film was touted as the first Hollywood production to deploy Smell-O-Vision, inventor Hans Laube’s automated system for introducing scents into a movie theater in coordination with the pictures on screen. While critics savaged Scent of Mystery’s plot, they also complained that Laube’s glitch-prone machine wasn’t worth the hype: “motion pictures and synthetic smells do not mix,” fumed the New York Times. Smell-O-Vision became a punchline.

Scent of Mystery print ad, featuring prominent
mention of Smell-O-Vision

More than fifty years later, Haruka Matsukura and his colleagues at Tokyo University of Agriculture and Technology announced they had reinvented scented media for the digital age. In March of 2013, Matsukura unveiled a system for producing scents built into LCD advertising screens, creating the illusion that a Big Mac smells as good as it looks. Meanwhile, Spanish chef Andoni Luiz Aduriz recently announced a collaborative project with Scentee, a smartphone accessory that signals alerts and alarms via scent. Like the Smell-O-Vision before them, these products suggest more potent possibilities for our digitally mediated lives: transforming a sea of disembodied information we struggle to interpret visually or aurally into more “visceral” data that we see, hear, feel, breathe and even ingest.

The Internet of Things promises to bring network connectivity and ubiquitous digital sensors in a wide variety of everyday materials and devices. This plethora of inputs produces data, and lots of it. We already stretched to the limit processing, internalizing, and understanding the data we have today. In the future, the sophisticated data visualizations—graphs, flowcharts, and infographics—that are staples of contemporary digital media products will become increasingly insufficient. Instead, the burgeoning Internet of Things will rely increasingly on what I call “data visceralizations.” Data visceralizations are representations of information that don’t rely solely and primarily on sight or sound, but on multiple senses including touch, smell, and even taste, working together to stimulate our feelings as well as our thoughts.

Of course it’s possible to prompt a visceral reaction through sight alone. Designer Sha Hwang recently suggested that designers strive to make data visualizations hit harder emotionally, by reframing the scope of their graphics and employing more emotion in their products. But relying on seeing by itself isn’t enough: influential design theorist Donald Norman’s theory of emotional design helps explain why.

In his 2004 book Emotional Design, Norman suggests that design elicits an emotional response at three different levels: the visceral, the behavioral, and the reflective. The reflective level connects a design with our conscious mind; the behavioral level is how we use a product; the visceral level, according to Norman, is made up of a design’s “look, feel and sound,” its “heft” and “sensuality.” Great visceral designs, like Apple’s original iMac, are all about “immediate emotional impact” because they engage multiple senses: Apple’s candy-colored computers were also rounded and smooth to the touch, distinguishing their feel as well as their look.

Experiments in visceral design have a long history in the creation of computing technologies. In the middle of the last century, cybernetics pioneer Grey Walter was exploring the possibilities of multi-sensory perception and interaction with computing. In the early 1950s Walter developed Flicker—a device that used EEG hookups to synchronize a user’s brainwaves with colored lights and music in a self-regulating feedback loop. Though his project never made it into large-scale production, today’s scientist-entrepreneurs are bringing a similar vision to the market.

The Muse EEG headband was exhibited at the 2014 Consumer Electronics Show in Las Vegas and is available now for pre-order. It allows a smartphone or tablet to take a person’s brainwaves as direct input. In an earlier product, Toronto-based Interaxon, Inc. created an app that gives a person instant feedback on their brain’s state in order to teach meditation techniques to calm and focus the mind. The company imagines a wide range of uses from immersive gaming to pouring a “nice cold mug of beer” “simply by using the power of your brain.”

The theory of sentics, developed by Dr. Manfred Clynes in the early 1970s, offers another prescient example. A neuroscientist and musician, Clynes developed a machine that correlated and tracked the relationship between bodily gestures and different emotional states. Clynes argued that particular motions were universally associated with particular emotions, and that these movement patterns could be seen in art and even heard within musical compositions.

Along with his colleague Nathan S. Kline, Clynes had coined the term “cyborg” in 1960. While he made engineering contributions to a number of medical fields, his vision of engaging multiple senses in the pursuit of a richer experience and understanding of the data we produce was ahead of its time compared to developments in consumer electronics.

Nintendo Rumble Pak attached to controller.
(Wikimedia Commons)

Today however, mainstream digital media products and interaction designs increasingly strive to create visceral reactions as a primary experience. Part of this new emphasis stems from attending to multiple channels of interaction: not just visual and aural cues, but also gestural and touch-based (haptic) feedback. Digital gaming has played a major role in incorporating tactile feedback, among other visceral design methods, into its products. The Rumble Pak, introduced commercially by Nintendo in 1997, makes game controllers shake and vibrate in time with the on-screen action. While initially derided as a clunky frill, rumble quickly became a standard of game play. Contemporary mobile devices like the iPhone and iPad lack active tactile feedback, but their gestural interfaces have allowed the designers of games like Angry Birds and Candy Crush to exploit visceral design principles effectively. Game designer Kyle Gabler uses the term “juiciness” to describe the positive feedback games like Candy Crush provide their players: shimmering colors and visual elements that move organically with some, but minimal, built-in resistance are hallmarks of a “juicy” gaming interface. Increasingly, interface designers of all kinds are combining this visual “juiciness” with appealing feedback for the other senses: products like Sifteo interactive tiles combine tactile, auditory and visual reactions in an attempt to captivate users by injecting the principles of “juicy” feedback into the domains of touch and hearing.

Sifteo Cubes (Sifteo Inc.)

All of the design strategies described above seek to prompt a visceral response in users through engaging one sense or another in order to trigger a physical or emotional reaction, making an end-run around rational thought.  A commitment to the concept of visceral data, in contrast, involves focusing on how these different tools and technologies can work together to help make abstract information have a meaningful visceral impact on users—one that’s appropriate and compelling for the context and the data involved.

Presented by

Luke Stark is a doctoral candidate in media, culture, and communication at New York University.

Never Tell People How Old They Look

Age discrimination affects us all. Who cares about youth? James Hamblin turns to his colleague Jeffrey Goldberg for advice.

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register.

blog comments powered by Disqus

Video

Never Tell People How Old They Look

Age discrimination affects us all. James Hamblin turns to a colleague for advice.

Video

Would You Live in a Treehouse?

A treehouse can be an ideal office space, vacation rental, and way of reconnecting with your youth.

Video

Pittsburgh: 'Better Than You Thought'

How Steel City became a bikeable, walkable paradise

Video

A Four-Dimensional Tour of Boston

In this groundbreaking video, time moves at multiple speeds within a single frame.

Video

Who Made Pop Music So Repetitive? You Did.

If pop music is too homogenous, that's because listeners want it that way.

More in Technology

Just In