In a video, a rodent reaches out and grabs a morsel of food, while small, colored dots highlight the positions of its knuckles. In another clip, a racehorse gallops along a track; again, small, colored dots track the position of its body parts. In a third video, two human dancers circle around each other as those same dots unfailingly follow the sometimes fluid, sometimes jerky movements of their limbs.
These videos are showcases for DeepLabCut, a tool that can automatically track and label the body parts of moving animals. Developed this year by Mackenzie Mathis and Alexander Mathis, a pair of married neuroscientists, DeepLabCut is remarkable in its simplicity. It has allowed researchers to download any video from the internet and digitally label specific body parts in a few dozen frames. The tool then learns how to pick out those same features in the rest of the video, or others like it. And it works across species, from laboratory stalwarts like flies and mice to … more unusual animals.
Here’s one striking example. This video, shot in Costa Rica, shows a lichen katydid, an insect whose white protrusions perfectly camouflage its body against the white lichen on which it walks. DeepLabCut sees through the insect’s ruse, successfully labeling its feet, joints, and antennae. “I think this method is going to revolutionize behavioral science, including neuroscience and psychology,” says Venkatesh Murthy from Harvard University, who works with the Mathises.