Latest Articles
Newsletter
Our Story

Tumblr’s busted NSFW algorithm suggests a dark future for online sex work

By

Ana Valens

Opinion

No one seems to be a fan of Tumblr’s adult content ban. It’s hurting users who turned to Tumblr for nonmainstream porn. It’s harming the sex workers and adult artists that fostered its popularity. It’s sending the message to marginalized communities that their work and desires don’t matter. But this is only the beginning. The way Tumblr plans to moderate NSFW content spells trouble for the internet’s future.

For starters, Tumblr’s current algorithm is abysmal. Tumblr users have been astonished with its sheer number of false positives over the past week, with SFW content regularly labeled as explicit. Twitter has since created the hashtag #TooSexyForTumblr and posted the site’s false flags. These include dragons, pets, Minecraft characters, boot scrubbing designs, puking unicorns, dinosaurs wearing flower crowns, holiday sweaters, and Vice President Joe Biden.

https://twitter.com/stewped/status/1069984007134228481

But this kind of disastrous NSFW algorithm—that threatens sex workers, adult content creators, and queer users who struggle to find an affirming 18+ community to call home—is likely the new standard for machine-based content moderation for porn.

During 2018’s third quarter, Facebook took down 30.8 million pieces of adult content, with 95.9 percent caught “before users reported it” thanks to the site’s detection technology. In an apparently recent move, Facebook’s Community Guidelines now ban “vague suggestive statements,” “sexual hints,” and “explicit sexual solicitation,” as well as drawings that depict “explicit sexual activity” or “suggestively posed persons.”

https://twitter.com/suchnerve/status/1070391585458806784

Facebook isn’t the only social media platform that relies on machine learning. In 2015, Twitter acquired the AI company Madbits, which developed a visual intelligence system that automatically identifies pornographic content. Twitter now relies on machine learning for everything from organizing users’ timelines to finding relevant search results. While Twitter allows NSFW content, it’s easy to imagine how quickly Twitter could remove adult work if it decided to censor its platform.

For adult artists, sex workers, and queer users leaving Tumblr amid its NSFW crackdown, the site’s drastic policy change is a reminder that sexual content is only safe to the extent that it is profitable. Discord pulled the plug on NSFW content in Partnered Servers just one year after receiving a $50 million investment round. And in Tumblr’s case, one anonymous former staff engineer suggested the NSFW ban’s roots come from Verizon, Tumblr’s parent company, after the phone company struggled to sell ads next to pornographic content on the site, according to Vox.

Not just that, but future NSFW moderation may be open-sourced, letting communities work together to implement adult content filters that censor sex workers. In 2016, Tumblr’s owner Yahoo published its algorithm “open_nsfw” on GitHub, letting any internet user study a deep neural network that “takes an image as input and outputs a probability which can be used to detect and filter NSFW images.”

Gabriel Goh, a University of California, Davis graduate student, was able to use open_nsfw to generate random images with a visualization technique that activates classifier neurons to study how open_nsfw identifies NSFW content. It’s a bit like reverse engineering an algorithm to see how it thinks—and it’s creepily good at its job. As filmmaker Tim Nicholas accurately describes it, the generated photos are “totally abstract yet distinctly pornographic, like HR Geiger or something.”

READ MORE

While it’s unclear if open_nsfw is the same algorithm behind Tumblr’s NSFW bans, it isn’t hard to imagine a dedicated development team building off the open-source algorithm. In the near future, NSFW content algorithms could be as readily available as SEO plugins, letting social media platforms exile sex workers from day one with advanced image recognition technology.

In a world increasingly infatuated with controlling, identifying, and removing adult content, it’s hard to know which start-up social media platforms will actively care about protecting sex workers. As Tumblr’s Dec. 17 deadline to remove NSFW content looms closer, a replacement run by and for sex workers is needed. Only then will social media have adult creators’ true interests at heart.

Ana Valens

Ana Valens is a reporter specializing in online queer communities, marginalized identities, and adult content creation. She is a former Daily Dot staff writer. Her work has appeared at Vice, Vox, Truthout, Bitch Media, Kill Screen, Rolling Stone, and the Toast. She lives in Brooklyn, New York, and spends her free time developing queer adult games.