So, you think you've honed the Netflix recommendation engine by rating a thousand movies? That's nothing, according to the company's internal statistics.
Several hundred Netflix members have rated more than 50,000 filmed entertainment programs. 50,000! To watch all those at a pace of one movie or TV show per day, it would take 136 years.
But those users are just the extreme end of a broader behavioral pattern. About a tenth of one percent (0.07%) of Netflix users -- more than 10,000 people -- have rated more than 20,000 items. And a full one percent, or nearly 150,000 Netflixers, have rated more than 5,000 movies. By contrast, only 60 percent of Netflix users rate any movies at all, and the typical person only gives out 200 starred grades.
Who are the subset of users who choose to make evaluating movies into an obsession instead of a casual exercise? They are nurturing the Netflix algorithm, training it. But why?
The two biggest raters I was able to track down had each reviewed in the neighborhood of 6,500 programs. Both are long-time users and neither intended to end up putting so much data into the system. But they were aware that there was an algorithm out there awaiting their input to reshape itself to their desires.
Mike Reilly, a producer, has rated more than 6,500 movies. At first, he just rated movies as they showed up, but then he heard about the Netflix Prize, a high-profile competition to improve the accuracy of the service's predictions.
"I became fascinated with the concept, the different approaches people were taking, and the practicality of these applied theories," Reilly told me.
He didn't employ a consistent methodology, rating in spurts and usually while searching for something to watch. What's fascinating is that Reilly noticed changes in the quality of the Netflix predictions as he rated more and more movies.
"The recommendations are better by far [than at the beginning]. I would say that from 0 to about 500 was pretty useless, at 1,000 to 2,000 it got a lot better -- then tailed off to about 5,000. From then on it's been pretty fantastic," he said. "It's really difficult to find something you simply don't know about -- this new system not only finds it, but can really pinpoint why it thinks you'd like it -- there's not just content, but the context as well, and that's really helpful."
That said, even after 6,500 ratings, the system still recommends bad choices occasionally.
"At this point it's just throwing, like, every Star Trek episode at me -- I've never really seen [that program] and am not interested, but it's like 'this is all that's left so we're going to keep asking, oh, and are you sure you still don't want to watch Mystery Science Theater 3000?'" Reilly said. "It's the same with kids movies."
Lorraine Hopping Egan, a book author, has rated 6,471 movies, but feels that the recommendations she gets aren't commensurate with the time she's invested.
"When I first joined, I went into a ratings frenzy because it was fun to say 'I saw that! I loved that! Overrated!' But mostly, I've rated movies as they popped up, in part so that they would stop coming up and I'd see more missed gems," she wrote to me. "But after 10 years, the recommendations are pretty thin and off-track."
Egan has found herself relying on regular old word-of-mouth and professional movie critics more than the algorithmic recommendations.
Some less intense users seem to get better results. Josh West, a developer here at The Atlantic, had a particularly elegant way of training his algorithm. He got to 416 ratings and consciously stopped starring movies.
"I felt like it knew my taste perfectly. It would predict I'd give a movie 3.6 stars -- and that is exactly how I would feel about it," West told me. "It predicted my rating more precisely than I could because you can only give something 3 or 4 stars, so I just stopped doing it."
Other people have adopted more complicated training techniques. Culture writer and co-founder of HiLoBrow.com, Josh Glenn, rated 2,638 movies in a single morning.
"I decided to rate as many as I could, really quickly, because I was sick of having movies suggested to me that I've either seen already or would never want to watch," Glenn wrote to me in an e-mail. "So I rated every movie I don't like or don't want to see with one star -- for some reason, I don't like clicking the NOT INTERESTED button. I try to save four stars for my all-time favorites. I don't have a system for two vs. three stars, and I don't use half stars."
His system may have worked too well. Now, Glenn, who only watches the movies available online from Netflix, gets very few recommendations. But that doesn't bother him too much.
"Maybe I'm an enabler, but I make excuses for Netflix," he said. "I watch a lot more movies than most people (I think) so I understand why they can't keep me satisfied."
The practice of rating Netflix movies can be hypnotic. If you go into the official page for rating movie, it displays your number of reviews in the upper right. As you rate movie after movie, your score goes up and up. When you really think about it, the Netflix rating system works on the world's simplest game mechanic: do something, get a point, move to a slightly more complex situation.
It's not unlike a casual game, perhaps like Zynga's smash hit, Farmville, a Facebook game in which you raise virtual crops. Except in this case, what you're growing isn't a virtual representation of wheat or tomatoes, but your own personal movie-picking servant, a savant twin of yourself that knows nothing but you and movies.
We'd love to hear your algorithmic training methodologies, or about any extreme feats of Netflixing. Get in touch in the comments or send an e-mail to me at amadrigal[at]theatlantic.com.
H/T to Mike McCaffrey for suggesting the idea of algorithmic farming on Twitter one day.
