Article Lead Image

Suicide-prevention app faces backlash over user privacy

Designing an app to prevent suicide is harder than it might seem.

 

Gavia Baker-Whitelaw

Tech

Posted on Nov 5, 2014   Updated on May 30, 2021, 6:45 am CDT

Mental health charity Samaritans wants to help people on social media who show signs of depression and suicidal thoughts. With all the concerns about young people becoming isolated online, this sounds like a pretty good idea. So, why are so many people upset by the Samaritans’ new suicide prevention app?

Samaritans Radar was designed to sift through people’s Twitter feeds for phrases like “hate myself” and “need someone to talk to,” which might be warning signs for depression. People who use the app will then be notified whenever someone they follow tweets one of these phrases. Samaritans will offer advice on how to approach flagged users if it looks like they need support.

Obviously, this system will bring up plenty of false positives. Tthe Samaritans Radar website points out that the app is “in its infancy and won’t get it right every time—it’s not good at sarcasm or jokes yet!” But people aren’t objecting to the app misidentifying innocuous tweets—they’re more concerned about privacy, and whether flagging “worrying” phrases actually helps vulnerable people online.

With #SamaritansRadar, I’m worried about vulnerable people experiencing complex thoughts being inundated with harmful platitudes.

— #BlackDogRunner (@bdogrunner) October 29, 2014

The concern is twofold. First of all, malicious people could easily abuse the app to cherry-pick vulnerable Twitter users and cyberbully them. Then there’s the issue of the app being “opt-out,” meaning that your tweets will automatically be scanned by unless you specifically request to be put on the app’s whitelist. So far the only way to get on the Samaritans Radar whitelist is to send the charity a direct message on Twitter, which will only work if the Samaritans account is following you. (Some people have had success by just tweeting @Samaritans.) There’s also no way to know whether someone who follows you is using the app, so most Twitter users will likely never know it exists.

People with mental health problems objected to being monitored by #SamaritansRadar as soon as it was announced.

— Tentacle Sixteen (@latentexistence) November 4, 2014

Thing I didn’t think about at first with #SamaritansRadar is also notion of compulsory happiness.

— HardFemmeGavroche (@CeCeSadist) November 4, 2014

Exploiting the privacy of those who are already marginalised and exploited in society doesn’t help anyone, it harms them.#SamaritansRadar

— Sarah Mitchell (@Smitchalot) November 4, 2014

I’ve seen groups here on Twitter target people suffering depression and mock them for talking about suicide. #SamaritansRadar enables that.

— chiller (@chiller) November 4, 2014

Samaritans Radar could save lives. It could very well be the reason why a friend notices that you’re depressed or lonely and offers to help. But the backlash on Twitter makes it abundantly clear that many people see it as actively harmful toward people with mental illness—or even just people having a bad day. While tweets are technically public, that doesn’t mean every Twitter user wants details of their emotional state to be collected, sorted, and possibly reported to total strangers.

Sometimes you just want to vent your emotions on social media, but it’s not always a cry for help. The existence of Samaritans Radar might actually discourage some people from tweeting about their depression, for fear of being monitored through the app.

Samaritans Radar attempts to find a simple, automated solution for a complex problem, so it’s really no surprise that some people are already trying to get it shut down.

Photo via Heinrich Böll Stiftung/Wikimedia (CC BY-SA 2.0)

Share this article
*First Published: Nov 5, 2014, 8:50 am CST