Mike Reilly, a producer, has rated more than 6,500 movies. At first, he just rated movies as they showed up, but then he heard about the Netflix Prize, a high-profile competition to improve the accuracy of the service's predictions.
"I became fascinated with the concept, the different approaches people were taking, and the practicality of these applied theories," Reilly told me.
He didn't employ a consistent methodology, rating in spurts and usually while searching for something to watch. What's fascinating is that Reilly noticed changes in the quality of the Netflix predictions as he rated more and more movies.
"The recommendations are better by far [than at the beginning]. I would say that from 0 to about 500 was pretty useless, at 1,000 to 2,000 it got a lot better -- then tailed off to about 5,000. From then on it's been pretty fantastic," he said. "It's really difficult to find something you simply don't know about -- this new system not only finds it, but can really pinpoint why it thinks you'd like it -- there's not just content, but the context as well, and that's really helpful."
That said, even after 6,500 ratings, the system still recommends bad choices occasionally.
"At this point it's just throwing, like, every Star Trek episode at me -- I've never really seen [that program] and am not interested, but it's like 'this is all that's left so we're going to keep asking, oh, and are you sure you still don't want to watch Mystery Science Theater 3000?'" Reilly said. "It's the same with kids movies."
Lorraine Hopping Egan, a book author, has rated 6,471 movies, but feels that the recommendations she gets aren't commensurate with the time she's invested.
"When I first joined, I went into a ratings frenzy because it was fun to say 'I saw that! I loved that! Overrated!' But mostly, I've rated movies as they popped up, in part so that they would stop coming up and I'd see more missed gems," she wrote to me. "But after 10 years, the recommendations are pretty thin and off-track."
Egan has found herself relying on regular old word-of-mouth and professional movie critics more than the algorithmic recommendations.
Some less intense users seem to get better results. Josh West, a developer here at The Atlantic, had a particularly elegant way of training his algorithm. He got to 416 ratings and consciously stopped starring movies.
"I felt like it knew my taste perfectly. It would predict I'd give a movie 3.6 stars -- and that is exactly how I would feel about it," West told me. "It predicted my rating more precisely than I could because you can only give something 3 or 4 stars, so I just stopped doing it."
Other people have adopted more complicated training techniques. Culture writer and co-founder of HiLoBrow.com, Josh Glenn, rated 2,638 movies in a single morning.