Behind the Scenes: Making Sesame Street's iPad App

Shortly after the iPad's launch, when curiosity overwhelmed reason, my wife purchased a Wi-Fi model, which was promptly nationalized by our then-twenty-month-old daughter, Margaux. Thinking that she would grow frustrated and tire of it quickly, I indulged her desire to play with the shiny newness. But only a few tentative swipes later, she was fully in command, navigating through apps like a pro. It dawned on me: This is her first computer. And it was so intuitive, so elegant, so refined, that it was completely usable by someone who eight months before was just learning to walk. Seeing the future playing with the future was magical and unsettling. I realized that if I was going to be a part of her world, I wanted to help create it.

Developing an app is very much like trying to carry a tray full of champagne flutes across a raging torrent by log rolling. It is precarious, requires balance and nimble footwork, and seems like a cool job to someone who is watching from dry land. It also involves frequent dunkings in frigid water. I lucked out in the fact that the hardest part of the process had already been taken care of by the time I joined the team at Callaway Digital Arts; they had already commenced the log rolling by producing several apps, including "Miss Spider's Tea Party" for both the iPad and iPhone. Having a well-received app in the store afforded us the opportunity to work with some incredible partners, including Martha Stewart, Thomas the Tank Engine, and Sesame Street. Most recently, we released the app version of The Monster at the End of This Book, Sesame Street's classic children's book starring "loveable, furry old Grover." First published in 1971, it has entertained and enthralled generations of kids, including myself, with Grover's admonition: "Please do not turn the page."

Story continues after the gallery.

Working with Sesame is a privilege; it's rare to find anyone in any field more passionate about their work. They truly understand kids -- and hold themselves (and their partners) to the highest standards. They also have the experience, and weight, of forty years of tradition bearing down on them. Monster is one of their crown jewels, and from the outset we felt an immense responsibility bringing this classic to the new medium of the iPad.

Our early brainstorming sessions included members of the development team recounting their memories of being read the book as children. Our head of technology, Nelson Gomez, even graced us with an impromptu rendition of his reading the book to his four sons. I confessed that I had memorized the book as a child and would recount it, in Grover's voice, to my younger sisters. Despite a thirty-year hiatus, I revived my rendition of Grover, which was deemed passable, albeit slightly haggard -- as though Grover had spent a few rough nights out on the town.

When I worked at Pixar I witnessed a lecture by Hayao Miyazaki describing his creative process. He explained that he locked himself in a small room, sketched, and dreamed, awash in pure creativity. He would then emerge, and for the next three years: "much suffering." We wanted to get the app out for the holidays, and it was early October. It wouldn't be three years, but we wouldn't get much sleep in the next few months.

Nothing quite like Monster existed in the app store. We wanted it to be a fully interactive reading experience, as though the book itself had come alive. Our creative mandate was to preserve the appeal and charm of Jon Stone's original story as well as capture the whimsy and funk of Michael Smollin's illustrations. Unfortunately, most of the tools available to us were focused on creating 3D games, not interactive, illustrated storybooks. So we engineered a hybrid solution to preserve the line quality of the original illustrations while animating Grover, the text, the pages, and the myriad obstacles Grover throws in our way. Using the original book illustrations, our illustrator re-drew Grover in a variety of poses: holding a hammer, walking, gesturing frantically, cowering. Our animators then sliced Grover into hundreds of pieces, down to the individual finger, and created incredibly intricate character rigs in After Effects from which to animate the scenes. Through much painful trial and error we managed to get the first few scenes onto the iPad.

Developing an app is very much like trying to carry a tray full of champagne flutes across a raging torrent by log rolling.

Sesame arranged to test a very early build of three scenes from the app with parents and children so that we could refine the work in progress. This research is a rarity in the app world, where development constraints don't allow for much beta testing. Unfortunately, we weren't able to get Eric Jacobson, the actual voice of Grover, to record all of the necessary lines for our focus group. I was conscripted to record my aging-castrati-coalminer Grover in the coat closet of our offices (where I scared a coworker who unknowingly opened the door mid-record). In addition to great feedback from kids and parents, one child tester rejoiced after knocking down Grover's brick wall, "He said that I was very strong!" Somehow, the agony of sprite sheet optimization doesn't seem so burdensome after that.

Presented by

Gabriel Schlumberger is a senior app producer at Callaway Digital Arts. Before joining Callaway, he worked at Pixar and Blue Sky studios. His film credits include The Incredibles, Cars, and Horton Hears a Who.

Never Tell People How Old They Look

Age discrimination affects us all. Who cares about youth? James Hamblin turns to his colleague Jeffrey Goldberg for advice.

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register.

blog comments powered by Disqus

Video

Never Tell People How Old They Look

Age discrimination affects us all. James Hamblin turns to a colleague for advice.

Video

Would You Live in a Treehouse?

A treehouse can be an ideal office space, vacation rental, and way of reconnecting with your youth.

Video

Pittsburgh: 'Better Than You Thought'

How Steel City became a bikeable, walkable paradise

Video

A Four-Dimensional Tour of Boston

In this groundbreaking video, time moves at multiple speeds within a single frame.

Video

Who Made Pop Music So Repetitive? You Did.

If pop music is too homogenous, that's because listeners want it that way.

More in Technology

Just In