- A police union is urging its officers to post ‘The Punisher’ logo Monday 7:33 PM
- Redditors call for a Nestlé boycott through memes Monday 6:16 PM
- How a 10-second Disney jingle became a meme in Thailand Monday 4:48 PM
- Instagram users share photos showing gruesome killing of 17-year-old Bianca Devins Monday 4:33 PM
- The horror game banned for mocking China’s president probably isn’t coming back Monday 3:31 PM
- Cheap vibrators, condoms, and lube: The most satisfying Amazon Prime Day deals Monday 3:07 PM
- George R.R. Martin says fan backlash won’t affect his ‘Game of Thrones’ ending Monday 3:03 PM
- The very finest Area 51 memes Monday 2:52 PM
- Tweet map ranks states where people are boycotting Amazon Prime Day Monday 1:54 PM
- Lil Nas X says he will perform at Area 51 for free Monday 12:56 PM
- The best Prime Day deals for gamers Monday 12:53 PM
- How Republicans are dancing around Trump’s racist tweets Monday 12:42 PM
- Not even anti-immigrant groups are defending Trump’s ‘go back’ tweets Monday 12:37 PM
- Netflix’s latest chase thriller ‘Point Blank’ lacks electricity Monday 12:27 PM
- Jay Inslee floats Megan Rapinoe as his secretary of state pick Monday 11:33 AM
An AI can recognize features as markers of depression.
Photos taken, edited, and shared to Instagram can show signs of depression through filters, faces, and colors, according to Harvard University’s Andrew Reece and Christopher Danforth from the University of Vermont.
Computers trained on signals like hue, facial recognition, and saturation were also better at predicting depression than average humans who analyzed photo attributes like happiness, sadness, and likability. Further results show that through this method, depression can be predicted even before an individual receives a diagnosis. Algorithmically generated results had a more successful diagnostic success rate for depression than that of general practitioners.
Predictive markers of depression in Instagram photos wouldn’t be used for treatment; rather, they could potentially become part of procedures to catch symptoms of depression early and prompt people to get care sooner.
“We foresee a more mature version of this tool being used more in the context of screening and assessment, rather than treatment,” Danforth and Reece explained in an email. “The algorithm we used looks for complex, systemic patterns across many data points to infer clues about individual psychology. If someone posts a dark, bluish photo to Instagram, it shouldn’t necessarily be a red flag for their therapist—that person could just like photos of whales, or blueberries.”
The research, a pre-print study published on arxiv.org that has not yet been peer-reviewed, analyzed almost 44,000 Instagram photos from 166 individuals. Surveys were assigned to depressed and healthy individuals; depressed people were asked to share their mental health history, including the date of their diagnosis.
Once connected to Instagram, researchers analyzed photos in two different ways: Workers on Mechanical Turk were asked to rate images on a scale of zero to five based on how likable, happy, interesting, and sad they were. Then a computer program analyzed pictures based on the number of faces in each photo, colorization, vividness, and brightness of images to find out what image qualities apply to photos shared by depressed individuals.
Researchers found darker, bluer, and more faded images predicted depression, and the more comments a post received increased the likelihood of being posted by a depressed person; more likes, however, signified the opposite. The “Inkwell” filter was the most common filter used by depressed participants, who were also less likely to apply filters at all, while healthy individuals favored “Valencia” and warmer tones.
Andrew Reece, Christopher Danforth
In the comparison below, the image on the right contains the type of photo features exhibited by someone with depression—darker, grayer, and bluer, researchers explained.
Andrew Reece, Christopher Danforth
While computers could pick up on the markers of depression, you probably won’t be able to recognize symptoms in a similar way while scrolling through your friends’ Instagram accounts. In the study, human raters (who were unaware of whether or not a person was depressed) exhibited a markedly different photo analysis.
“Human ratings exhibited extremely low correlation with computational features,” like hues and facial presence, researchers said. “In other words, the bluer, darker, grayer photos weren’t the sad photos, according to the people we asked. So while our computational approach is capable of screening for depression, it uses different information signals than those captured by human judgments of what makes a photo sad.”
Surprisingly, the study found depressed participants posted more photos with faces, but the number of people in each photo was lower. Danforth said they’re still considering what this finding suggests, but it might indicate smaller social activities.
Machine learning is increasingly used in clinical settings, and researchers continue to investigate how artificial intelligence can impact mental health. For instances, therapy bots are used to diagnose, assess, and comfort patients with mental disorders. Early tests showed that some people feel more comfortable talking to a screen about mental health, post-traumatic stress disorder, and anxiety as opposed to human therapists. AI is in development that detects physical ailments, too. Doctors are reportedly using Watson, IBM’s cognitive system, to predict cancer in patients.
However, researchers cautioned against making a formal comparison to human physicians, or thinking that machine learning will replace your therapist. The computer’s findings were not directly tested against clinicians analyzing the same photos of depressed patients.
“The intuition and experience of human physicians are still very important, especially in mental health care settings, and machine learning methods seem to work best when they complement, rather than replace, human knowledge,” they said. “In that sense, ‘augmenting’, rather than ‘displacing,’ might be a better way to describe the new approaches to human-computer interaction that are advancing health care.”
So perhaps in the future, a computer might analyze your mental health before you visit a doctor’s office, and once markers are recognized, a real human being can begin treatment.
Selena Larson is a technology reporter based in San Francisco who writes about the intersection of technology and culture. Her work explores new technologies and the way they impact industries, human behavior, and security and privacy. Since leaving the Daily Dot, she's reported for CNN Money and done technical writing for cybersecurity firm Dragos.