Photo via max-kegfire/GettyImages (Licensed)
Photos taken, edited, and shared to Instagram can show signs of depression through filters, faces, and colors, according to Harvard University's Andrew Reece and Christopher Danforth from the University of Vermont.
Computers trained on signals like hue, facial recognition, and saturation were also better at predicting depression than average humans who analyzed photo attributes like happiness, sadness, and likability. Further results show that through this method, depression can be predicted even before an individual receives a diagnosis. Algorithmically generated results had a more successful diagnostic success rate for depression than that of general practitioners.
Predictive markers of depression in Instagram photos wouldn't be used for treatment; rather, they could potentially become part of procedures to catch symptoms of depression early and prompt people to get care sooner.
"We foresee a more mature version of this tool being used more in the context of screening and assessment, rather than treatment," Danforth and Reece explained in an email. "The algorithm we used looks for complex, systemic patterns across many data points to infer clues about individual psychology. If someone posts a dark, bluish photo to Instagram, it shouldn't necessarily be a red flag for their therapist—that person could just like photos of whales, or blueberries."
The research, a pre-print study published on arxiv.org that has not yet been peer-reviewed, analyzed almost 44,000 Instagram photos from 166 individuals. Surveys were assigned to depressed and healthy individuals; depressed people were asked to share their mental health history, including the date of their diagnosis.
Once connected to Instagram, researchers analyzed photos in two different ways: Workers on Mechanical Turk were asked to rate images on a scale of zero to five based on how likable, happy, interesting, and sad they were. Then a computer program analyzed pictures based on the number of faces in each photo, colorization, vividness, and brightness of images to find out what image qualities apply to photos shared by depressed individuals.
Researchers found darker, bluer, and more faded images predicted depression, and the more comments a post received increased the likelihood of being posted by a depressed person; more likes, however, signified the opposite. The "Inkwell" filter was the most common filter used by depressed participants, who were also less likely to apply filters at all, while healthy individuals favored "Valencia" and warmer tones.
In the comparison below, the image on the right contains the type of photo features exhibited by someone with depression—darker, grayer, and bluer, researchers explained.
While computers could pick up on the markers of depression, you probably won't be able to recognize symptoms in a similar way while scrolling through your friends' Instagram accounts. In the study, human raters (who were unaware of whether or not a person was depressed) exhibited a markedly different photo analysis.
"Human ratings exhibited extremely low correlation with computational features," like hues and facial presence, researchers said. "In other words, the bluer, darker, grayer photos weren't the sad photos, according to the people we asked. So while our computational approach is capable of screening for depression, it uses different information signals than those captured by human judgments of what makes a photo sad."
Surprisingly, the study found depressed participants posted more photos with faces, but the number of people in each photo was lower. Danforth said they're still considering what this finding suggests, but it might indicate smaller social activities.
Machine learning is increasingly used in clinical settings, and researchers continue to investigate how artificial intelligence can impact mental health. For instances, therapy bots are used to diagnose, assess, and comfort patients with mental disorders. Early tests showed that some people feel more comfortable talking to a screen about mental health, post-traumatic stress disorder, and anxiety as opposed to human therapists. AI is in development that detects physical ailments, too. Doctors are reportedly using Watson, IBM's cognitive system, to predict cancer in patients.
However, researchers cautioned against making a formal comparison to human physicians, or thinking that machine learning will replace your therapist. The computer's findings were not directly tested against clinicians analyzing the same photos of depressed patients.
"The intuition and experience of human physicians are still very important, especially in mental health care settings, and machine learning methods seem to work best when they complement, rather than replace, human knowledge," they said. "In that sense, 'augmenting', rather than 'displacing,' might be a better way to describe the new approaches to human-computer interaction that are advancing health care."
So perhaps in the future, a computer might analyze your mental health before you visit a doctor's office, and once markers are recognized, a real human being can begin treatment.