How Platforms Can Fight AI Impersonation, Synthetic Identities, and Online Abuse
- Glen Wise, Co-Founder & CEO of Cinder and Alexandra Ryabova, Head of Operations at Wizz App
- AI and Automation, Customer Experience (CX), Digital Transformation, Managing Org Transformation, Other Martech, Platform and Data Integration
As AI tools become more accessible, the risks facing digital platforms are growing just as quickly. Impersonation, phishing, synthetic identities, harmful content, and nonconsensual imagery are no longer edge cases. They are becoming everyday trust and safety challenges for social platforms, marketplaces, and AI-driven products alike.
In a recent Martechify conversation, Glen Wise, Co-Founder & CEO of Cinder, and Alexandra Ryabova, Head of Operations at Wizz App, shared how platforms are responding to this new era of online abuse. Their discussion made one point especially clear: Companies cannot afford to treat safety as an afterthought.
Watch the full interview with Cinder’s Glen Wise and Wizz App’s Alexandra Ryabova to hear how platforms are racing to stay ahead of AI-powered impersonation, synthetic identities, and online abuse. Learn how faster enforcement, sharper policies, and smarter detection tools are becoming essential to protecting users in a rapidly changing digital landscape.
Why online safety can no longer be reactive
Glen described Cinder as a company built to help platforms protect users from abuse, manipulation, and harmful content. He said the idea for the company grew out of his work at Meta on Facebook’s threat intelligence team, where he saw how slowly large platforms often translated emerging harms into policy and enforcement responses. “There must be a much faster way of doing it than taking nine months to go through that full cycle,” he said.
That need for speed matters more now because AI has sharply lowered the barrier to abuse. Glen noted that bad actors no longer need deep technical skills to launch convincing attacks. “The barrier to entry for conducting highly sophisticated attacks has dropped down to basically zero,” he said. In practice, that means more realistic phishing attempts, more polished impersonation, and more synthetic content created with little more than a laptop and a few prompts.
For Glen, effective platform protection starts with understanding the attacker’s mindset. “We approach this by looking at how the adversary, or how bad actors, approach the space,” he said. That perspective has shaped how Cinder works across industries, from social platforms and marketplaces to generative AI applications. Rather than applying a one-size-fits-all moderation system, Cinder helps customers define the experience they want users to have, translate that into policy, and then enforce those policies through Cinder and human review where needed.
That policy-first approach came up repeatedly in the conversation. Glen said, “Policies really are the center of the universe.” He explained that companies first need to define what they are trying to prevent, whether that is adult content, hate speech, fraud, spam, scraping, or some entirely new form of abuse. Only then can they measure whether human moderators or AI systems are making the right decisions.
This matters because content moderation is rarely black and white. Alexandra shared a simple but revealing example from Wizz App, where safety tools once interpreted “I want to dye my hair” as “I want to die.” The mix-up was a false positive, but she said it reflected a common tradeoff in trust and safety. In sensitive cases, platforms often prefer catching too much rather than missing genuine harm.
That need for speed matters more now because AI has sharply lowered the barrier to abuse. Glen noted that bad actors no longer need deep technical skills to launch convincing attacks. “The barrier to entry for conducting highly sophisticated attacks has dropped down to basically zero,” he said. In practice, that means more realistic phishing attempts, more polished impersonation, and more synthetic content created with little more than a laptop and a few prompts.
For Glen, effective platform protection starts with understanding the attacker’s mindset. “We approach this by looking at how the adversary, or how bad actors, approach the space,” he said. That perspective has shaped how Cinder works across industries, from social platforms and marketplaces to generative AI applications. Rather than applying a one-size-fits-all moderation system, Cinder helps customers define the experience they want users to have, translate that into policy, and then enforce those policies through Cinder and human review where needed.
That policy-first approach came up repeatedly in the conversation. Glen said, “Policies really are the center of the universe.” He explained that companies first need to define what they are trying to prevent, whether that is adult content, hate speech, fraud, spam, scraping, or some entirely new form of abuse. Only then can they measure whether human moderators or AI systems are making the right decisions.
This matters because content moderation is rarely black and white. Alexandra shared a simple but revealing example from Wizz App, where safety tools once interpreted “I want to dye my hair” as “I want to die.” The mix-up was a false positive, but she said it reflected a common tradeoff in trust and safety. In sensitive cases, platforms often prefer catching too much rather than missing genuine harm.
Catching risk before it reaches users: Liveness checks and other tools
At Wizz App, scale makes those decisions even more urgent. Alexandra said the social platform handles enormous message volume, citing “around eight hundred billion messages per year,” and has to act quickly to protect users, especially minors. “We want the content to be caught somewhere in the sky before the user who’s intended to receive it actually gets it on their phone,” she said. That pre-moderation approach helps stop explicit or harmful content before it reaches the end user, rather than waiting for reports after the damage is done.
She also emphasized that AI detection does not always need to be overly complex. “Sometimes the most simple solution is the best way to detect AI,” she said. At Wizz App, one tactic involves comparing a user’s instant age-verification image with the profile image they upload. Because the verification image is captured live, it becomes easier to identify synthetic imagery, impersonation attempts, or profile manipulation. Alexandra said this type of liveness check has become useful not only for age estimation but also for detecting AI-generated identities. That approach gives Wizz App another layer of defense beyond traditional content moderation. It helps the platform verify that the person behind the profile is real, rather than a synthetic or manipulated identity.
The conversation also explored how synthetic content and impersonation are changing platform risk. Glen said many scams are not entirely new, but AI has made them easier, cheaper, and more convincing. In the past, a bad actor might have needed strong design skills or language fluency to create a believable fake. Now, those barriers are fading. That is why, he noted, companies increasingly need to combine content analysis with behavioral detection to catch suspicious activity that looks authentic on the surface.
Alexandra added that safety principles still apply even when the content itself is synthetic. If a piece of content is harmful, explicit, or abusive, it should still be treated as such regardless of whether it was created by a human or generated by AI. That kind of consistency helps platforms avoid getting distracted by novelty and stay focused on user protection.
She also emphasized that AI detection does not always need to be overly complex. “Sometimes the most simple solution is the best way to detect AI,” she said. At Wizz App, one tactic involves comparing a user’s instant age-verification image with the profile image they upload. Because the verification image is captured live, it becomes easier to identify synthetic imagery, impersonation attempts, or profile manipulation. Alexandra said this type of liveness check has become useful not only for age estimation but also for detecting AI-generated identities. That approach gives Wizz App another layer of defense beyond traditional content moderation. It helps the platform verify that the person behind the profile is real, rather than a synthetic or manipulated identity.
The conversation also explored how synthetic content and impersonation are changing platform risk. Glen said many scams are not entirely new, but AI has made them easier, cheaper, and more convincing. In the past, a bad actor might have needed strong design skills or language fluency to create a believable fake. Now, those barriers are fading. That is why, he noted, companies increasingly need to combine content analysis with behavioral detection to catch suspicious activity that looks authentic on the surface.
Alexandra added that safety principles still apply even when the content itself is synthetic. If a piece of content is harmful, explicit, or abusive, it should still be treated as such regardless of whether it was created by a human or generated by AI. That kind of consistency helps platforms avoid getting distracted by novelty and stay focused on user protection.
The Take It Down Act
Another major topic was the Take It Down Act and its implications for platforms. Alexandra described it as an important step forward for victims of nonconsensual image sharing, including synthetic content. “If a victim of nonconsensual image sharing reaches out to a platform, the platform needs to take down the content within 48 hours,” she said.
She noted that the same principles still apply even when content is AI-generated. If content is harmful, explicit, or abusive, Wizz App treats it the same way and works to intercept it before it reaches another user.
She noted that the same principles still apply even when content is AI-generated. If content is harmful, explicit, or abusive, Wizz App treats it the same way and works to intercept it before it reaches another user.
Industry collaboration and digital literacy matter
Both experts also pointed to the broader role of industry collaboration. Alexandra said platforms need the right tools and partners to investigate incidents quickly, respond to victims, and remove harmful content before it spreads. Glen added that newer verification, hashing, and detection technologies are making that response more scalable than the manual systems many companies relied on in the past.
Still, technology alone is not enough. Alexandra said user education has to become part of the solution, especially as younger users interact more frequently with AI systems. She argued that digital literacy should be discussed not only on platforms, but also at home and in schools, so people understand that AI outputs can be flawed, hypothetical, or unsafe.
The biggest takeaway from the discussion was that safer platforms do not happen by accident. They require clear policy, fast enforcement, thoughtful use of AI, and a genuine commitment from the companies building online experiences. As Glen put it, the first step is simply recognizing that safety matters and deciding to act on it.
Still, technology alone is not enough. Alexandra said user education has to become part of the solution, especially as younger users interact more frequently with AI systems. She argued that digital literacy should be discussed not only on platforms, but also at home and in schools, so people understand that AI outputs can be flawed, hypothetical, or unsafe.
The biggest takeaway from the discussion was that safer platforms do not happen by accident. They require clear policy, fast enforcement, thoughtful use of AI, and a genuine commitment from the companies building online experiences. As Glen put it, the first step is simply recognizing that safety matters and deciding to act on it.
Related resources
Want more like this resource? Find helpful links below.
