Behind the Scenes: Making Sesame Street's iPad App

Shortly after the iPad's launch, when curiosity overwhelmed reason, my wife purchased a Wi-Fi model, which was promptly nationalized by our then-twenty-month-old daughter, Margaux. Thinking that she would grow frustrated and tire of it quickly, I indulged her desire to play with the shiny newness. But only a few tentative swipes later, she was fully in command, navigating through apps like a pro. It dawned on me: This is her first computer. And it was so intuitive, so elegant, so refined, that it was completely usable by someone who eight months before was just learning to walk. Seeing the future playing with the future was magical and unsettling. I realized that if I was going to be a part of her world, I wanted to help create it.

Developing an app is very much like trying to carry a tray full of champagne flutes across a raging torrent by log rolling. It is precarious, requires balance and nimble footwork, and seems like a cool job to someone who is watching from dry land. It also involves frequent dunkings in frigid water. I lucked out in the fact that the hardest part of the process had already been taken care of by the time I joined the team at Callaway Digital Arts; they had already commenced the log rolling by producing several apps, including "Miss Spider's Tea Party" for both the iPad and iPhone. Having a well-received app in the store afforded us the opportunity to work with some incredible partners, including Martha Stewart, Thomas the Tank Engine, and Sesame Street. Most recently, we released the app version of The Monster at the End of This Book, Sesame Street's classic children's book starring "loveable, furry old Grover." First published in 1971, it has entertained and enthralled generations of kids, including myself, with Grover's admonition: "Please do not turn the page."

Story continues after the gallery.

Working with Sesame is a privilege; it's rare to find anyone in any field more passionate about their work. They truly understand kids -- and hold themselves (and their partners) to the highest standards. They also have the experience, and weight, of forty years of tradition bearing down on them. Monster is one of their crown jewels, and from the outset we felt an immense responsibility bringing this classic to the new medium of the iPad.

Our early brainstorming sessions included members of the development team recounting their memories of being read the book as children. Our head of technology, Nelson Gomez, even graced us with an impromptu rendition of his reading the book to his four sons. I confessed that I had memorized the book as a child and would recount it, in Grover's voice, to my younger sisters. Despite a thirty-year hiatus, I revived my rendition of Grover, which was deemed passable, albeit slightly haggard -- as though Grover had spent a few rough nights out on the town.

When I worked at Pixar I witnessed a lecture by Hayao Miyazaki describing his creative process. He explained that he locked himself in a small room, sketched, and dreamed, awash in pure creativity. He would then emerge, and for the next three years: "much suffering." We wanted to get the app out for the holidays, and it was early October. It wouldn't be three years, but we wouldn't get much sleep in the next few months.

Nothing quite like Monster existed in the app store. We wanted it to be a fully interactive reading experience, as though the book itself had come alive. Our creative mandate was to preserve the appeal and charm of Jon Stone's original story as well as capture the whimsy and funk of Michael Smollin's illustrations. Unfortunately, most of the tools available to us were focused on creating 3D games, not interactive, illustrated storybooks. So we engineered a hybrid solution to preserve the line quality of the original illustrations while animating Grover, the text, the pages, and the myriad obstacles Grover throws in our way. Using the original book illustrations, our illustrator re-drew Grover in a variety of poses: holding a hammer, walking, gesturing frantically, cowering. Our animators then sliced Grover into hundreds of pieces, down to the individual finger, and created incredibly intricate character rigs in After Effects from which to animate the scenes. Through much painful trial and error we managed to get the first few scenes onto the iPad.

Developing an app is very much like trying to carry a tray full of champagne flutes across a raging torrent by log rolling.

Sesame arranged to test a very early build of three scenes from the app with parents and children so that we could refine the work in progress. This research is a rarity in the app world, where development constraints don't allow for much beta testing. Unfortunately, we weren't able to get Eric Jacobson, the actual voice of Grover, to record all of the necessary lines for our focus group. I was conscripted to record my aging-castrati-coalminer Grover in the coat closet of our offices (where I scared a coworker who unknowingly opened the door mid-record). In addition to great feedback from kids and parents, one child tester rejoiced after knocking down Grover's brick wall, "He said that I was very strong!" Somehow, the agony of sprite sheet optimization doesn't seem so burdensome after that.

Presented by

Gabriel Schlumberger is a senior app producer at Callaway Digital Arts. Before joining Callaway, he worked at Pixar and Blue Sky studios. His film credits include The Incredibles, Cars, and Horton Hears a Who.

How to Cook Spaghetti Squash (and Why)

Cooking for yourself is one of the surest ways to eat well. Bestselling author Mark Bittman teaches James Hamblin the recipe that everyone is Googling.

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register.

blog comments powered by Disqus


How to Cook Spaghetti Squash (and Why)

Cooking for yourself is one of the surest ways to eat well.


Before Tinder, a Tree

Looking for your soulmate? Write a letter to the "Bridegroom's Oak" in Germany.


The Health Benefits of Going Outside

People spend too much time indoors. One solution: ecotherapy.


Where High Tech Meets the 1950s

Why did Green Bank, West Virginia, ban wireless signals? For science.


Yes, Quidditch Is Real

How J.K. Rowling's magical sport spread from Hogwarts to college campuses


Would You Live in a Treehouse?

A treehouse can be an ideal office space, vacation rental, and way of reconnecting with your youth.

More in Technology

Just In