Can we diagnose autism using a brain scan?

[Crossposted from Medium].

It seems like every day, a news story breathlessly claims that neuroscientists have found a way to reliably diagnose autism using neuroimaging (usually, fMRI). So, could you go to a hospital, ask for an MRI scan, and be told within a few days whether you or your child are autistic?
Not right now, and probably not any time soon.
Why not?
Above: Neuroscientist Russell Poldrack in an MRI scanner.
Limitations of neuroimaging
Structural neuroimaging won’t tell us anything about autism, because autistic gross anatomy is usually normal. You won’t find any missing parts, tumors, or swollen, inflamed areas. Sometimes, very young children have unusually large heads, but it’s uncommon, and rarely lasts long.
Many of the studies in the news use functional neuroimaging, or fMRI. Participants perform a task while the scanner measures the changes in blood flow that follow neural activity (see this post by Neurologism for more information on what exactly fMRI measures and how it relates to brain activity). A statistical analysis then determines whether an autistic brain’s activity during the task differs from a non-autistic brain’s more than you would expect by chance.

Currently, functional neuroimaging can tell us a lot about groups, but not so much about individuals. In other words, an entire group of autistic brains looks reliably different from a group of neurotypical brains. But we might not be able to look at an individual participant’s brain and determine whether it belongs to an autistic or neurotypical person. Why?

In order to get statistically significant changes during a task, you need to collect many trials from many people. The changes in brain activity you’re looking for are very small, and embedded in noise, so you need large numbers to reach statistical significance. Furthermore, each time any given person performs the task, her brain activity is a little different. One time, she might be thinking about the task; another time, what she’s going to have for dinner; another time, the scanner noise. You need a lot of trials from a lot of participants to make up for that variability.
In order to determine where group differences in brain activity are located, you have to map everybody’s brain onto a 3D coordinate system. But everybody’s brain is different — even neurotypicals of the same age. So, a little distortion necessarily occurs.
[TANGENT FOR PEOPLE WITH MORE OF A NEUROSCIENCE BACKGROUND: One could also use resting state fMRI, where participants lie in the scanner doing nothing instead of performing a task. The participant’s experience is similar to an anatomical scan, but the data resembles a functional scan — you’re still looking at changes in blood flow that reflect brain activity. Resting state scans look at patterns in functional connectivity — which parts of the brain activate (or deactivate) together. It takes a long time to collect data in these studies, because the variability over time is even greater (since participants aren’t even performing a consistent task). Interpreting resting state fMRI can be controversial, and right now, autism studies using this technique are plagued by conflicting results and confounds. We’re not ready to diagnose autism yet using resting state fMRI, either.]
Autism is too heterogeneous.
Autism is heterogeneous. As they say, “if you’ve met one person with autism, you’ve met one person with autism.” It probably isn’t a single type of brain at all. More likely, it’s a bunch of different disabilities with different causes that all happen to lead to the same sorts of behavior. If the autisms have different causes, then the brain signatures are likely to be different, too.
The difference between the brains of any two autistic people might be at least as big as the difference between an autistic and a non-autistic brain. And if so, it will be impossible to find a test that can cleanly separate the two groups.
Autism overlaps with other conditions.
Most of the studies in the news compare autistic to neurotypical brains. That’s the easy comparison. In practice, people would only be scanned if they had some sort of developmental problem that caused concern. So autistic children would be compared to those with various developmental delays, or those with overlapping disabilities, such as ADHD, sensory processing disorder, or specific language impairment. Adults seeking a diagnosis, if not autistic, are also likely to have another disability that affects them enough to make them seek help. Not only does autism resemble other developmental delays and disabilities, but can co-occur with them, and share genetic causes. If a scan can’t reliably distinguish an autistic brain from these other, similar sorts of brains, it’s not ready for real-world use.
Autism occurs in a tiny percentage of the population.
So far, all these problems seem like they could be fixed with better methods and better technology. But a statistical issue — the rarity of autism in the population — places a hard limit on the accuracy of diagnostic tests in principle.
Despite the media hysteria about rising autism rates, autism still affects only about 1–2% of the population. Even the best tests will give you a lot of false positives: most of the people diagnosed with autism will not have it. Even with a 90% accurate test, only 8% of people with “autistic” brains would actually be autistic, as shown in the graph below. This post by autism researcher Jon Brock explains the problem (and the graph) in more detail.
The likelihood of false positives remains high until a test has well over 90% accuracy. It may not be realistic to develop a test that accurate. Source: Jon Brock.
Importantly, the same argument applies no matter what sort of screening test one uses. It doesn’t matter if it’s based on neuroimaging, genetics, a parent survey or the child’s behavior. The same signal detection problem applies.
Whether a high rate of false positives is acceptable is a moral issue, not a scientific one. But autistic children are subjected to stigma, described as burdens who are “missing” from their own lives (perhaps a reason why people fear labels, saying things like “labels are for soup cans”). The hours of therapy they endure involve an opportunity cost — denying them the opportunity to relax, play, and interact with their families. Children with an autism diagnosis are often denied an equal education with their peers, put in isolation rooms or physical restraints, and are physically manhandled and denied the right to say no to such treatment, which some autistic people and parents fear set them up for later abuse. No child should be treated this way, autistic or not. But certainly, nobody would want to risk this sort of treatment for a child who society does not even deem to “need” it.
Since we are unwilling to risk false positives, an autism screening test will probably not exist any time soon.
Does that mean we must give up on recognizing autism altogether? Not necessarily. Although, as we’ve seen, a perfect screening test isn’t possible, clinicians do a pretty good job of recognizing autism in children, based on their observed behavior and parent reports, and they can do so as early as two years old. Furthermore, diagnoses are pretty stable. That is, a child diagnosed with autism often meets criteria for the diagnosis a few years later. So, we can diagnose autism reasonably well — and we don’t even need an expensive MRI scan to do it.
Take Home Message: We cannot diagnose a person with autism using brain scans for several reasons. These include the limits of the technology, the variability within autism, the overlap between autism and other conditions, and the statistical problems with diagnosing any rare condition.