How do you begin to teach a machine to analyze an image for how memorable it will be to humans? First you need to find out, from the humans themselves, what images are most memorable. So the researchers had people them play a game in which they were presented with a series of images, some of which were eventually repeated. The dataset was huge: It included some 60,000 diverse images—sunsets, clocks, ballerinas, selfies, dogs, trees, etc. Participants were told to press a key if they recognized an image they’d already been shown.
Then, the researchers trained a computer, using a model they call MemNet, to classify those same images 1,000 different ways, based on what humans found memorable. Here’s where it gets really crazy: “By visualizing the learned representation of the layers of MemNet, we discover the emergent representations, or diagnostic objects, that explain what makes an image memorable or forgettable.”
In other words, once they knew the machine could accurately predict what humans found memorable, they were able to figure out why.
Using a demo of this model, for instance, I was able to learn that this image of me is pretty memorable. It scores 0.85 on a possible scale from 0 (least memorable) to 1 (most memorable), with the red section representing the most memorable aspect of the photo:
This doesn’t have much to do with my face, per se. Which is probably a good thing: Researchers found that images that evoke disgust, anger, and fear tend to be more memorable than those that evoke awe or contentment. Overall, negative images are more memorable than positive ones—with the exception of images that people find amusing, which also score high for memorability.
The reason the photo of me scored so high on the memorability scale, Khosla told me, probably has more to do with the framing of the photo, since the computer model was not trained to be face-centric. (He and his colleagues are working on building a face-centric model that will distinguish memorability based on facial features.) Using the current model, the most memorable images have salient focal points, and are often close-ups.
This guy, for example, was almost as memorable as my face:
This dog was pretty memorable, too:
But this townscape wasn’t very memorable at all:
On the other hand, these eggs with little faces painted on them were super memorable:
One intriguing finding was that memorability appears correlated with popularity. Images in the dataset that came from Flickr, and were popular on that site, ranked higher in memorability than images that were less popular. This link suggests people may eventually be able to design images so they’re optimized for sharing across social networks. It also raises a new and complicated question about the nature of memorability. What would happen if people saw only memorable images? “Would they start forgetting everything and only remember the most memorable of the lot?” Khosla said. “Our initial experiments suggest that this might not be the case. We found that, for at least a small set of images, the average number of images remembered by people increased when we showed them more memorable images.”