- Fortnite streamer Tfue sues gaming organization FaZe Clan over contract dispute Today 12:28 AM
- Report finds some users can’t opt out of Facebook’s face recognition Monday 7:27 PM
- Get emotional over this real-life pastor baptizing an anime girl in virtual reality Monday 6:53 PM
- Twitter wants to know what Jack in the Box did to offend Kim Kardashian Monday 6:38 PM
- ‘Game of Thrones’ meme claims King’s Landing is an ‘inside job’ Monday 6:06 PM
- Report: Personal data of 49 million Instagram influencers exposed online Monday 4:57 PM
- ‘Stranger Things’ season 3 trailer teases a wet, hot American summer Monday 4:02 PM
- What Daenerys’ biggest ‘Game of Thrones’ scenes have in common with Nazi propaganda Monday 3:12 PM
- Here’s what’s coming to Amazon Prime in June Monday 2:11 PM
- Where did Jon Snow go? Unpacking the ‘Game of Thrones’ ending Monday 2:04 PM
- So, did anyone actually win ‘Game of Thrones’? Monday 1:29 PM
- The surprising religious subtext of ‘John Wick: Chapter 3’ Monday 12:53 PM
- Robin Arryn got hot—and the internet is seriously shook Monday 12:40 PM
- Tana Mongeau is going to VidCon a year after TanaCon disaster Monday 12:12 PM
- What have 2020 Democrats said about Alabama’s abortion ban? Monday 11:36 AM
An AI can recognize features as markers of depression.
Photos taken, edited, and shared to Instagram can show signs of depression through filters, faces, and colors, according to Harvard University’s Andrew Reece and Christopher Danforth from the University of Vermont.
Computers trained on signals like hue, facial recognition, and saturation were also better at predicting depression than average humans who analyzed photo attributes like happiness, sadness, and likability. Further results show that through this method, depression can be predicted even before an individual receives a diagnosis. Algorithmically generated results had a more successful diagnostic success rate for depression than that of general practitioners.
Predictive markers of depression in Instagram photos wouldn’t be used for treatment; rather, they could potentially become part of procedures to catch symptoms of depression early and prompt people to get care sooner.
“We foresee a more mature version of this tool being used more in the context of screening and assessment, rather than treatment,” Danforth and Reece explained in an email. “The algorithm we used looks for complex, systemic patterns across many data points to infer clues about individual psychology. If someone posts a dark, bluish photo to Instagram, it shouldn’t necessarily be a red flag for their therapist—that person could just like photos of whales, or blueberries.”
The research, a pre-print study published on arxiv.org that has not yet been peer-reviewed, analyzed almost 44,000 Instagram photos from 166 individuals. Surveys were assigned to depressed and healthy individuals; depressed people were asked to share their mental health history, including the date of their diagnosis.
Once connected to Instagram, researchers analyzed photos in two different ways: Workers on Mechanical Turk were asked to rate images on a scale of zero to five based on how likable, happy, interesting, and sad they were. Then a computer program analyzed pictures based on the number of faces in each photo, colorization, vividness, and brightness of images to find out what image qualities apply to photos shared by depressed individuals.
Researchers found darker, bluer, and more faded images predicted depression, and the more comments a post received increased the likelihood of being posted by a depressed person; more likes, however, signified the opposite. The “Inkwell” filter was the most common filter used by depressed participants, who were also less likely to apply filters at all, while healthy individuals favored “Valencia” and warmer tones.
Andrew Reece, Christopher Danforth
In the comparison below, the image on the right contains the type of photo features exhibited by someone with depression—darker, grayer, and bluer, researchers explained.
Andrew Reece, Christopher Danforth
While computers could pick up on the markers of depression, you probably won’t be able to recognize symptoms in a similar way while scrolling through your friends’ Instagram accounts. In the study, human raters (who were unaware of whether or not a person was depressed) exhibited a markedly different photo analysis.
“Human ratings exhibited extremely low correlation with computational features,” like hues and facial presence, researchers said. “In other words, the bluer, darker, grayer photos weren’t the sad photos, according to the people we asked. So while our computational approach is capable of screening for depression, it uses different information signals than those captured by human judgments of what makes a photo sad.”
Surprisingly, the study found depressed participants posted more photos with faces, but the number of people in each photo was lower. Danforth said they’re still considering what this finding suggests, but it might indicate smaller social activities.
Machine learning is increasingly used in clinical settings, and researchers continue to investigate how artificial intelligence can impact mental health. For instances, therapy bots are used to diagnose, assess, and comfort patients with mental disorders. Early tests showed that some people feel more comfortable talking to a screen about mental health, post-traumatic stress disorder, and anxiety as opposed to human therapists. AI is in development that detects physical ailments, too. Doctors are reportedly using Watson, IBM’s cognitive system, to predict cancer in patients.
However, researchers cautioned against making a formal comparison to human physicians, or thinking that machine learning will replace your therapist. The computer’s findings were not directly tested against clinicians analyzing the same photos of depressed patients.
“The intuition and experience of human physicians are still very important, especially in mental health care settings, and machine learning methods seem to work best when they complement, rather than replace, human knowledge,” they said. “In that sense, ‘augmenting’, rather than ‘displacing,’ might be a better way to describe the new approaches to human-computer interaction that are advancing health care.”
So perhaps in the future, a computer might analyze your mental health before you visit a doctor’s office, and once markers are recognized, a real human being can begin treatment.
Selena Larson is a technology reporter based in San Francisco who writes about the intersection of technology and culture. Her work explores new technologies and the way they impact industries, human behavior, and security and privacy. Since leaving the Daily Dot, she's reported for CNN Money and done technical writing for cybersecurity firm Dragos.