This post has gone completely viral on Reddit and it’s no surprise why. Perhaps more important, it asserts the increasingly dangerous issue of AI-created disinformation. Journalist Casey Newton, of Platformer, had the story first. An UberEats employee badge and an internal Uber document with 18 pages explaining driver desperation scores were first found on Reddit. As a result, the post went viral, receiving more than 87,000 upvotes. It further jumped onto channels such as X (formerly Twitter), where it garnered an outstanding 208k likes and achieved 36.8 million impressions.
After contacting the original Reddit user, Newton chronicled his attempts to trace the post back through the internet. The two continued to communicate over Signal. This realization opened our eyes to the incredible measures people would take to craft convincing but deceptive materials. The event involves a hoax gone viral, posing important questions regarding the trustworthiness of information posted on social media. This worry intensifies as AI tools continue to be more widely used.
Max Spero, founder of Pangram Labs, emphasized the challenges of distinguishing between real and fabricated content in today’s digital landscape. In a recent interview with TechCrunch, he underscored the growing crisis of AI-fueled posts. These posts can be extremely misleading to users and highly impactful on public sentiment.
“AI slop on the internet has gotten a lot worse, and I think part of this is due to the increased use of LLMs, but other factors as well,” – Max Spero
The implications of such content are significant. Businesses with deep pockets can use ‘dark patterns’ methods on sites like Reddit. Then, they generate 100s or even 1000s of AI generated posts that mentioned their brand names in hopes of going viral. This tactic is an affront to genuine user-generated content and adds layers of confusion around the fact-checking process.
Newton reflected on the credibility of the document shared with him, stating, “For most of my career up until this point, the document shared with me by the whistleblower would have seemed highly credible in large part because it would have taken so long to put together.” He further questioned the motivations behind such elaborate hoaxes: “Who would take the time to put together a detailed, 18-page technical document about market dynamics just to troll a reporter? Who would go to the trouble of creating a fake badge?”
Then the whole situation turned on a dime. These posts were quickly revealed to be one of a number of similar AI-generated hoaxes that had circulated through Reddit over the weekend. In line with this trend, DoorDash recently bragged that it had banned a driver for faking a delivery with AI tools at their disposal.
As AI technology rapidly advances, the same is true for the demand for new and robust fact-checking practices. Amanda Silberling @amsilbwrites — senior writer at TechCrunch covering the space where tech and culture collide — expressed this support. It’s getting harder and harder for reporters—and you as a regular user—to tell the truth in a world full of lies.
Google’s Gemini was instrumental in confirming that an image associated with one of these hoaxes was indeed generated by AI, aided by Google’s SynthID watermark. This state-of-the-art watermark is robust to many image processing attacks, such as cropping and blurring. It becomes extremely invaluable for establishing the legitimacy of more content.
“There’s companies with millions in revenue that can pay for ‘organic engagement’ on Reddit, which is actually just that they’re going to try to go viral on Reddit with AI-generated posts that mention your brand name,” – Max Spero
As misinformation continues to spread rampant across the internet, platforms are increasingly being called on to improve their content moderation resources. Recent events remind us that social media can be a force for good, allowing powerful tools to connect advocates and share important information. They also bring significant risks that require vigilance from users and companies alike.