Ranking pain isn’t a simple thing. The standard scale that goes from one to 10, often accompanied by smiley faces that become increasingly distressed, has been lampooned by many as being difficult to use. What does it mean to be a five? Or a three? What is that mildly sad frowny face saying? Do you have to be crying for it to really be a 10?

And for some people, it’s even harder to put a number to a subjective experience. Patients with autism, for example, often struggle to express the pain they’re feeling. “We do see many members of our community who either experience altered pain perception, or who have difficulties communicating about and reporting pain,” Julia Bascom, the director of programs at the Autistic Self Advocacy Network, told me in an email. “So someone might experience acid reflux not as burning pain, but as pressure in their throat, and then struggle to interpret a numerical pain scale, or not realize they should bring the issue to the attention of those around them—or what words to use to be taken seriously.” Autism can also mean a difficulty interpreting facial expressions, so the happy and sad faces wouldn't be the most helpful visual cues. And some autistic patients aren’t verbal at all.

In fact, for a long time, people thought that kids with autism didn’t feel pain at all, because they often didn’t show reactions to it the same way other people do. “They might not understand the words other people use to describe pain, even if they are feeling the exact same sensation, and their outward reactions might seem to indicate much more pain than they are actually feeling,” Bascom said.

Shannon Des Roches Rosa, a founding editor at Thinking Person’s Guide to Autism, told me about her son Leo, who is 15. “The pain scales don't work for him, not any of them,” she said in an email. “That's not to say they don't work for other autistic kids or adults; it's just we've not found one he finds useful.” Rosa says that they’re trying to come up with a reliable system of communication, but it’s still a work in progress. And she says that while Leo does communicate with spoken language (however sparing) his words can’t always be taken literally. “Leo's spoken language is not always an accurate representation of what he means to say. He will ask for the same solution—Tylenol—for a minor pimple or a major earache.”

It turns out there are tons of pain measurement systems designed for patients who can’t use the typical smiley face scale. So many, in fact, that it can be hard to tell which ones to pay attention to. And while these pain scales do help nurses understand the discomfort of patients who have trouble communicating, they're not perfect, either.

* * *

For a long time, nurses used rubrics designed for newborns and toddlers to assess patients who couldn’t communicate verbally, even though the way an infant experiences and expresses pain is quite different from how an older child or adult might. Then, in the early 1990s, some new national guidelines pushed nurses to think harder about how they were recording and measuring pain, particularly in non-verbal patients. “We were looking at pain for years, but we didn’t document it,” said Terri Voepel-Lewis, a researcher at the University of Michigan. So all across the country hospitals and researchers began designing their own scales. “Everybody jumped on the bandwagon and started developing these tools.”

The result was a lot of alternative ways of measuring pain. A lot. There are so many tools out there, you could pick from a barrel and choose one and use it,” Voepel-Lewis said. There’s the Non-Communicating Child's Pain Checklist-Postoperative Version, the Echelle Douleur Enfant San Salvador, the Individualized Numeric Rating scale, the Pediatric Pain Profile, the revised Face, Leg, Activity, Cry, and Consolability scale and more. But not all these tests are well-understood or well-developed. “Even though there is a sea of these tools out there, a lot of them are understudied, underdeveloped, and not necessarily ready to be used,” said Quinn Crosta, a nurse at the Swedish Neuroscience Institute in Seattle.

Most of these new scales lump all non-communicating patients together, often under the banner of “cognitive impairment.” So kids with cerebral palsy who can’t speak are measured the same way as kids with autism who are non-verbal. Without the ability to use words to communicate a number on a scale, measuring pain comes down to observing behaviors. That’s what each of these scales do, albeit slightly differently: look for behaviors that might suggest that a child is in pain. Some patients may express pain by withdrawal, others through repetitive behaviors, some may guard the area that hurts, others may stop sleeping or stop eating, or they may simply move differently.

This variability is important. And along with a list of possible behaviors that might suggest pain, alternative pain scales are all, in some way, individualized. Because these pain scales treat cerebral palsy and autism under the same umbrella, Voepel-Lewis says, they rely heavily on parents and caregivers to tell doctors which behaviors to look for in their child. “We could create a little list of behaviors that may relate to pain or may typically relate to pain,” she said. “But when you start looking at a kid with cerebral palsy or autism, they may have behaviors that are so unique to them that we would not know that, as a caregiver that just interacts with them maybe only once.”

Rosa, for example, knows what to look for in Leo. “We've learned that it's better to pay attention to his behavior, because if he is in pain, his behavior will change,” she said. “If he's really in pain, he will cry. This is far from perfect, but until we get a communication system in place (and not that that's a guarantee anyhow), it's what we've got.”

Knowing how a particular child reacts to pain is key to the success of all of these alternative pain scales. If we’re going to assess pain, we need input from those who know these children,” Voepel-Lewis said. But how does a nurse pick which scale to use?

Crosta is one of many researchers who has tried to compare the effectiveness of the pain measures out there. She published a review study that looked at four different choices. But even these review comparisons don’t agree. Crosta and Voepel-Lewis conducted independent comparisons. They both came to the conclusion that the revised Face, Leg, Activity, Cry, and Consolability (rFLACC) scale was the best. This version has nurses run through a checklist of the five categories listed in the scale's name, looking for things like how consolable the patient is, how much they're kicking their legs, and how much they're crying.

But another review paper, by Giovanna Ferrara, looked at a similar set of scales and picked the Non-Communicating Child's Pain Checklist-Postoperative Version, one that has more categories and questions. "In my opinion the NCCPC-PV allows us to capture the slightest change in the state of the patient and therefore a better rating of pain, as it contains more items," Ferrara said. She worried that the rFLACC scale might miss something in its simplicity.

No matter what scale you pick, lumping together all patients who have trouble picking a number from one to 10 to express pain means simplifying the needs of each patient. There are more basic problems too. Doctors are trained to think about pain on a 0-10 scale, for better or worse. Several of these alternate pain scales work on a totally different numerical system. One is out of 81, another is out of 54.

And some scales require more input than others. The Pediatric Pain Profile (PPP), for example, is predominantly made up of an extensive, parent-generated history, in which caregivers describe what the child is like “at their best” and “at their worst.” This works really well if the parents continually update the document, and put the work in. But if a child is coming into the emergency room, if the parent isn’t present, or the caregiver simply doesn’t yet know what behaviors to look for, it might not be the best choice. Crosta says that the PPP might be a great document for parents with autism to start working on before any kind of medical need arises (the form is free and available online), but it might not be the one nurses turn to at the hospital.

* * *

At the Sheikh Zayed Institute for Pediatric Surgical Innovation at Children’s National Hospital in Washington D.C., Julia Finkel and her team take a different approach to understanding pain in those who have trouble communicating. She points out that while ranking pain is important, the number you get out of any pain scale shouldn’t be taken alone. “Unidimensional scales do nothing to define the type of pain,” Zayed said. A doctor might come to a numerical pain conclusion (6, or 34, or 57, depending on which scale they’ve picked) but figuring out what to do with that information can be difficult if they don’t understand the nature of the pain. How much of the discomfort is physical, how much of it is mental? “On the scale of 1-10, a pain [ranking] of 6 or more, is meaningless when a part of that 6 is that you’re afraid of the puncture that’s about to happen.” Finkel said. Nobody likes being in a hospital, but for patients with autism the hospital environment can be particularly overwhelming and distressing.

So Finkel and her team are trying to develop ways to figure out what is feeding into a person’s discomfort. “Our focus is developing devices, tools, and methodology to objectively measure pain,” she said. One way they’re doing that is through something they call the Human Algometer—a device that sits atop the head and monitors brain activity. The idea is that by looking at the activity in different parts of the brain, doctors could see exactly what percent of a patient’s distress is coming from physical pain being transmitted by pain nerves themselves, and how much of it is coming from the emotional centers of the brain. Then doctors can better decide what drugs to use, or whether to treat with drugs at all.

The Human Algometer isn’t a proven technique just yet—it's still in the early stages of clinical trials. But Finkel hopes it will eventually be a way for doctors to feel more secure in precisely treating problems. “This device also allows us to see what kind of pain the patient is experiencing, and then what intervention would make sense in addressing it.”

Another project at Sheikh Zayed uses video games and a Microsoft Kinect system to watch a patient and look for signs of distress. Finkel described a non-verbal autistic child who came into their clinic with his mother. “To examine him is very difficult, and his mother said he was having back pain. I would not have been able to discern that was the case looking at him. But he has an affinity for this video world, and we put him in front of this game that we designed.” In the game, the player is asked to reach for things and move around. While he played, the Kinect system tracked his movement, specifically looking at his back movements and facial expressions, helping the doctors see where he pulled up, and which movements made him tense. All the while, Finkel said, the patient was simply enjoying the game. At the end of the session, the kid, who she had never head speak, said the words “more game.”

Obviously these tools aren’t going to work in an emergency room setting. But Finkel says that for the most part if someone comes in with a broken leg, regardless of their ability to communicate, the doctor should know they’re in pain and know how to treat that kind of injury.

Ultimately, assessing pain in each patient is going to be an individual experience, and likely involve some combination of measures. Everyone I spoke with stressed the importance of not stopping at a number, and not using a numerical score to justify clinical decisions. “Most practitioners know that you can’t just rely on those pain numbers that you get for those tools,” said Voepel-Lewis. “But some don’t, and that’s the problem.”

And despite there being a bevy of options, there’s no one surefire way to assess pain in patients who can’t tell you what they’re feeling directly. Researchers are still  testing and developing alternate scales and devices, and Finkel says they’ll keep doing so until they find something that works well. “There’s a desperate need to develop something that helps. We keep developing these things because nothing we do helps.”