- Lyft received a whopping 7 sexual assault lawsuits in a day Wednesday 10:00 PM
- High school reopens investigation into Nazi salute video after other racist videos emerge Wednesday 7:14 PM
- Facebook content moderators continue to suffer from brutal working conditions Wednesday 5:58 PM
- #RIPReese: Man bullied for relationship with trans woman dies by suicide Wednesday 4:46 PM
- Redaction error reveals ICE is paying Palantir $49 million Wednesday 4:25 PM
- People are using social media to raise awareness about the Amazon fires Wednesday 4:24 PM
- How to watch ‘Detective Pikachu’ right now Wednesday 3:56 PM
- Walmart is suing Tesla over fires at stores with solar panels Wednesday 3:44 PM
- Jeremy Renner asks nicely for Sony to let Spider-Man back in the MCU Wednesday 2:51 PM
- The best and safest torrenting sites you should be using in 2019 Wednesday 2:47 PM
- ‘Beyoncé’s Assistant for a Day’ creator is releasing more games on storytelling app Yarn Wednesday 1:54 PM
- Why does everyone keep falling for that Instagram and Facebook hoax? Wednesday 1:46 PM
- A bunch of celebrities fell for that viral Instagram hoax Wednesday 1:17 PM
- Former Die Antwoord crew member says video shows ‘homophobic attack’ Wednesday 1:13 PM
- How to stream all the MLS Rivalry Week matches Wednesday 1:13 PM
Twitter search now includes any tweet ever
You can finally comb through your old tweets on Twitter.
Twitter isn’t ephemeral—each tweet is etched into the Internet ether as soon as you send it out. But sometimes it feels like tweets disappear, thanks to Twitter’s historically awful search function.
That’s all about to change. On Tuesday, Twitter announced it’s indexing every tweet ever sent since 2006, which means everyone’s tweet history is now searchable on Twitter proper. Previously, the only way to find old tweets was to use services like Apple-owned Topsy, Tweet Machine, or Snapbird.
The search feature was a massively technical undertaking, which began all the way back in 2012. Because of the huge amount of information and data that’s produced on Twitter each day, the company had to figure out how to scale the search infrastructure to aggregate Twitter’s history without using expensive RAM, or memory, and create an interface that thinks about all the separate data chunks (like tweets and engagements) as one endpoint.
According to a profile of Twitter’s search indexing team in Wired, the guys behind the undertaking initially asked themselves if it was even possible—the hard part would be not only scraping the entire history of tweets, but also including the millions of tweets sent each day.
Twitter’s new search feature can comb through half a trillion documents and display tweets and data in less than one second.
Photo via mkhmarketing/Flickr (CC BY 2.0)
Selena Larson is a technology reporter based in San Francisco who writes about the intersection of technology and culture. Her work explores new technologies and the way they impact industries, human behavior, and security and privacy. Since leaving the Daily Dot, she's reported for CNN Money and done technical writing for cybersecurity firm Dragos.