Extreme Netflix‽ Some Users Have Rated 50,000 Movies

More

Netflix Raters.jpg

So, you think you've honed the Netflix recommendation engine by rating a thousand movies? That's nothing, according to the company's internal statistics.

Several hundred Netflix members have rated more than 50,000 filmed entertainment programs. 50,000! To watch all those at a pace of one movie or TV show per day, it would take 136 years.

But those users are just the extreme end of a broader behavioral pattern. About a tenth of one percent (0.07%) of Netflix users -- more than 10,000 people --  have rated more than 20,000 items. And a full one percent, or nearly 150,000 Netflixers, have rated more than 5,000 movies. By contrast, only 60 percent of Netflix users rate any movies at all, and the typical person only gives out 200 starred grades.

Who are the subset of users who choose to make evaluating movies into an obsession instead of a casual exercise? They are nurturing the Netflix algorithm, training it. But why?

The two biggest raters I was able to track down had each reviewed in the neighborhood of 6,500 programs. Both are long-time users and neither intended to end up putting so much data into the system. But they were aware that there was an algorithm out there awaiting their input to reshape itself to their desires.

Mike Reilly, a producer, has rated more than 6,500 movies. At first, he just rated movies as they showed up, but then he heard about the Netflix Prize, a high-profile competition to improve the accuracy of the service's predictions.

"I became fascinated with the concept, the different approaches people were taking, and the practicality of these applied theories," Reilly told me.

He didn't employ a consistent methodology, rating in spurts and usually while searching for something to watch. What's fascinating is that Reilly noticed changes in the quality of the Netflix predictions as he rated more and more movies.

"The recommendations are better by far [than at the beginning]. I would say that from 0 to about 500 was pretty useless, at 1,000 to 2,000 it got a lot better -- then tailed off to about 5,000. From then on it's been pretty fantastic," he said. "It's really difficult to find something you simply don't know about -- this new system not only finds it, but can really pinpoint why it thinks you'd like it -- there's not just content, but the context as well, and that's really helpful."

That said, even after 6,500 ratings, the system still recommends bad choices occasionally.

"At this point it's just throwing, like, every Star Trek episode at me -- I've never really seen [that program] and am not interested, but it's like 'this is all that's left so we're going to keep asking, oh, and are you sure you still don't want to watch Mystery Science Theater 3000?'" Reilly said. "It's the same with kids movies."

Lorraine Hopping Egan, a book author, has rated 6,471 movies, but feels that the recommendations she gets aren't commensurate with the time she's invested.

"When I first joined, I went into a ratings frenzy because it was fun to say 'I saw that! I loved that! Overrated!' But mostly, I've rated movies as they popped up, in part so that they would stop coming up and I'd see more missed gems," she wrote to me. "But after 10 years, the recommendations are pretty thin and off-track."

Egan has found herself relying on regular old word-of-mouth and professional movie critics more than the algorithmic recommendations.

Some less intense users seem to get better results. Josh West, a developer here at The Atlantic, had a particularly elegant way of training his algorithm. He got to 416 ratings and consciously stopped starring movies.

"I felt like it knew my taste perfectly. It would predict I'd give a movie 3.6 stars -- and that is exactly how I would feel about it," West told me. "It predicted my rating more precisely than I could because you can only give something 3 or 4 stars, so I just stopped doing it."

Other people have adopted more complicated training techniques. Culture writer and co-founder of HiLoBrow.com, Josh Glenn, rated 2,638 movies in a single morning.

"I decided to rate as many as I could, really quickly, because I was sick of having movies suggested to me that I've either seen already or would never want to watch," Glenn wrote to me in an e-mail. "So I rated every movie I don't like or don't want to see with one star -- for some reason, I don't like clicking the NOT INTERESTED button. I try to save four stars for my all-time favorites. I don't have a system for two vs. three stars, and I don't use half stars."

His system may have worked too well. Now, Glenn, who only watches the movies available online from Netflix, gets very few recommendations. But that doesn't bother him too much.

"Maybe I'm an enabler, but I make excuses for Netflix," he said. "I watch a lot more movies than most people (I think) so I understand why they can't keep me satisfied."

The practice of rating Netflix movies can be hypnotic. If you go into the official page for rating movie, it displays your number of reviews in the upper right. As you rate movie after movie, your score goes up and up. When you really think about it, the Netflix rating system works on the world's simplest game mechanic: do something, get a point, move to a slightly more complex situation.

It's not unlike a casual game, perhaps like Zynga's smash hit, Farmville, a Facebook game in which you raise virtual crops. Except in this case, what you're growing isn't a virtual representation of wheat or tomatoes, but your own personal movie-picking servant, a savant twin of yourself that knows nothing but you and movies.

We'd love to hear your algorithmic training methodologies, or about any extreme feats of Netflixing. Get in touch in the comments or send an e-mail to me at amadrigal[at]theatlantic.com.

H/T to Mike McCaffrey for suggesting the idea of algorithmic farming on Twitter one day.

Jump to comments
Presented by

Alexis C. Madrigal

Alexis Madrigal is the deputy editor of TheAtlantic.com, where he also oversees the Technology Channel. He's the author of Powering the Dream: The History and Promise of Green Technology. More

The New York Observer has called Madrigal "for all intents and purposes, the perfect modern reporter." He co-founded Longshot magazine, a high-speed media experiment that garnered attention from The New York Times, The Wall Street Journal, and the BBC. While at Wired.com, he built Wired Science into one of the most popular blogs in the world. The site was nominated for best magazine blog by the MPA and best science Web site in the 2009 Webby Awards. He also co-founded Haiti ReWired, a groundbreaking community dedicated to the discussion of technology, infrastructure, and the future of Haiti.

He's spoken at Stanford, CalTech, Berkeley, SXSW, E3, and the National Renewable Energy Laboratory, and his writing was anthologized in Best Technology Writing 2010 (Yale University Press).

Madrigal is a visiting scholar at the University of California at Berkeley's Office for the History of Science and Technology. Born in Mexico City, he grew up in the exurbs north of Portland, Oregon, and now lives in Oakland.

Get Today's Top Stories in Your Inbox (preview)

Tracing Sriracha's Origin to a Seaside Town in Thailand

Ever wonder how the wildly popular hot sauce got its name? It all started in Si Racha.


Elsewhere on the web

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register. blog comments powered by Disqus

Video

Where the Wild Things Go

A government facility outside of Denver houses more than a million products of the illegal wildlife trade, from tigers and bears to bald eagles.

Video

Adults Need Playtime Too

When was the last time you played your favorite childhood game?

Video

Is Wine Healthy?

James Hamblin prepares to impress his date with knowledge about the health benefits of wine.

Video

The World's Largest Balloon Festival

Nine days, more than 700 balloons, and a whole lot of hot air

Writers

Up
Down

More in Technology

Just In