John Keeble/Getty Images
- Generative AI is becoming a headache for the marketplace quality team at TikTok Shop.Fraudsters are using generative AI to try to list fake brands or sell dupe products on Shop.TikTok exec Nicolas Waldmann said the company is mixing AI and human moderation to combat fraud.
AI-generated "slop" isn't just mucking up our social feeds with deepfakes and other unsavory content. It's also becoming a real problem for marketplace quality teams at e-commerce platforms like TikTok Shop.
Fraudulent sellers are using generative AI tools to make fake brands or dupe products in an attempt to get users to pay for goods that don't actually exist, said Nicolas Waldmann, who leads TikTok Shop's governance and experience external affairs team.
"It's organized crime, to be honest," Waldmann said. "They're trying to basically go through and sell, and of course, never deliver anything, and then run with the money."
While this type of e-commerce fraud has been around for years, generative AI has increased the sophistication of the methods that bad actors use to try to trick moderation teams on platforms like TikTok Shop or Amazon, Waldmann said.
TikTok uses a mix of human and AI moderation to help track down fraudulent accounts and listings. The company has its own in-house detection tools, as well as partnerships with outside firms to manage tasks like authenticating pre-owned luxury goods.
"We use AI to basically deal with AI," Waldmann said.
In a new report published Thursday, the company said it had rejected 70 million products and removed 700,000 sellers for various policy violations in the first six months of 2025.
TikTok Shop is a top priority at the social-video company, which has set ambitious growth targets that have been challenging for its US team to hit. Last year, the company drove $100 million in single-day sales on Black Friday in the US.
The company has run into a variety of marketplace quality challenges since it began testing its e-commerce tool in the US in 2022. While TikTok has a lengthy list of prohibited items listed on its website, savvy operators have still managed to break through its filters to sell items like THC syrup and sex toys. "Dupes," or knockoff versions of household-name items, have also created headaches for the social-shopping app. The company's automated enforcement has irked some sellers in the past who said they were hit with violations without a clear explanation.
