Scammers, hackers, and cybercriminals on underground forums are voicing a complaint that security teams might find darkly satisfying: their own platforms are now being flooded with AI-generated garbage. New research analyzing over 100 million posts from dark web forums shows that most cybercriminals are struggling to adopt AI meaningfully, and that safety guardrails on mainstream models continue to hold — but the picture also contains a concrete warning that points squarely at defenders, not attackers.
Underground AI Adoption: What the Research Shows
Two significant research efforts have converged this week to paint a detailed picture of how artificial intelligence is reshaping the criminal underground — and the findings are more nuanced than the standard "AI is supercharging hackers" narrative.
The most substantial evidence comes from a peer-reviewed study by researchers at the University of Edinburgh, the University of Strathclyde, and Cambridge. Their analysis drew from the CrimeBB database — a longitudinal academic dataset scraped from underground and dark web forums, encompassing over 100 million posts from November 2022 (the month ChatGPT launched) onward. The researchers found no significant evidence that cybercriminals are successfully leveraging AI to meaningfully improve their hacking operations.
The study, due to be presented at the Workshop on the Economics of Information Security in Berkeley in June 2026, found that AI assistance primarily benefits actors who already possess strong technical skills. For the majority of forum participants — who lack deep programming knowledge — AI tools offer little practical uplift. One forum post captured the prevailing frustration bluntly: "You've gotta first learn the ropes of programming by yourself before you can use AI and ACTUALLY benefit from it."
A separate analysis by Professor Benoît Dupont at the University of Montreal examined 163 discussion threads across 21 active underground forums — including XSS.is, BreachForums, Dread, and Exploit.in — covering 2,264 posts contributed by 1,661 participants over the first half of 2025. That study found AI adoption clustering almost entirely around low-sophistication tasks, with experienced attackers expressing consistent skepticism about AI-generated code quality.
What AI Criminal Operators Are Actually Doing
When underground forum members do use AI, their applications are heavily skewed toward fraud and social engineering rather than technical exploitation. Among forums referencing mainstream AI services, ChatGPT appeared in 52.5% of threads, followed by DeepSeek (27.9%), Anthropic's Claude (19.7%), and Grok (18%). Specific uses cited across threads include:
- Drafting phishing emails with natural-sounding language to bypass grammar-based filters
- Generating boilerplate code snippets for malware stubs
- Coaching scripts for social engineering calls and vishing (voice phishing) operations
- Automating call center fraud workflows at scale
- Producing variable spam content to defeat signature-based detection
The criminal AI product market mirrors mainstream offerings in name if not in capability. WormGPT — an uncensored large language model (LLM, an AI system trained on text to generate human-like responses) marketed specifically to cybercriminals — led product mentions at 26% of criminal tool discussions, followed by FraudGPT (18%), DarkGPT (16%), and a cluster including ChaosGPT, GhostGPT, and SpamGPT at roughly 6% each. These tools exist specifically to circumvent the safety guardrails that block mainstream models from assisting with attacks — but forum discussions consistently describe them as error-prone, unreliable for complex tasks, and resource-intensive compared to mainstream alternatives.
The Irony: AI Is Now a Spam Problem for Cybercriminals Too
Here is where both studies converge on a finding with a sharp edge: AI is generating so much low-quality content on criminal forums that veteran members are complaining. The same dynamics that plague legitimate platforms — AI-written posts flooding discussions, degrading the signal-to-noise ratio, and eroding trust in posted information — are now playing out in spaces where criminals share tradecraft.
On forums where technical reputation is currency, AI-generated noise creates a real operational problem. Experienced attackers rely on trusted, high-reputation contributors for verified exploit techniques, up-to-date credentials, and operational security (OPSEC — practices that prevent investigators from identifying criminals by their digital footprint) advice. AI spam dilutes that signal. Worse, AI-generated disinformation about tools or techniques — whether intentional or simply the product of hallucination (an AI model confidently stating incorrect information) — creates genuine operational risk for the criminals who act on it.
The researchers note that cybercriminals attempting to use mainstream AI models face a layered challenge: the best models refuse harmful requests, and the open-source jailbroken (safety-restrictions-removed) alternatives deliver inferior output. The practical result is a friction point that is discouraging broad, sophisticated adoption.
Why Safety Guardrails Are Holding
Dr. Ben Collier of the University of Edinburgh's School of Social and Political Science, one of the CrimeBB study's lead researchers, summarized the team's message to the security industry: "Don't panic yet. The immediate danger comes from companies adopting poorly secured AI systems."
Major AI providers — including Anthropic, OpenAI, and Google — have invested substantially in safety guardrails: filters and training-time restrictions built into models to block them from producing functional malware, step-by-step attack instructions, or content that assists with exploitation. The Edinburgh research found early evidence that these controls are working. Cybercriminals who attempt to misuse mainstream models are being blocked, and the workaround — migrating to self-hosted, jailbroken open-source models — comes with meaningful quality and resource penalties.
This does not mean the risk is zero. Guardrails are being probed continuously, and adversarial prompt injection (crafting inputs designed to bypass safety filters) remains an active research area on both defensive and offensive sides. But the current state of evidence is that the barrier is real and is discouraging casual misuse.
The Real Threat: Vibe-Coded Products and Agentic AI
Both research teams agree on where the genuine near-term risk lies — and it sits on the defender's side of the table.
The first risk vector is what practitioners are calling "vibe-coded" products: software built rapidly using AI coding assistants by developers who do not fully understand the code being produced. A developer who uses an AI assistant to ship a product they could not have built manually is producing code with unknown properties, untested edge cases, and potentially missing input validation. These applications enter production without the scrutiny that catches subtle logic flaws, insecure defaults, or authentication bypasses — and they become attack surface.
The second vector is insecurely deployed agentic AI systems. Agentic AI refers to AI models granted the ability to take autonomous actions: calling external APIs, reading and writing files, executing code, or sending messages on a user's behalf. If these systems are deployed without proper sandboxing (isolation preventing them from accessing systems beyond their intended scope), least-privilege access controls, or defenses against prompt injection (an attack where malicious instructions hidden in documents or web content hijack the agent's actions), they become high-value targets. An attacker who can redirect an agentic AI's instructions can potentially use that agent to exfiltrate data or move laterally through internal infrastructure.
Professor Dupont's conclusion from the Montreal study frames the near-term threat precisely: "Social engineering and scamming operations will probably leverage AI capacities more systematically, profitably and sooner than malware development." Mass-scale, convincing phishing and fraud campaigns are the immediate AI uplift zone — not novel exploitation frameworks.
What Security Teams Should Do Right Now
- Prioritize AI-enhanced phishing defenses. The data confirms social engineering is the immediate AI-uplift risk. Review email security controls, strengthen user awareness training with AI-generated phishing examples, and test fraud detection pipelines against LLM-generated content.
- Establish code review gates for AI-generated output. If developers ship AI-written code without manual review or automated static analysis, establish gating processes. Tools like Semgrep and Snyk can catch common vulnerability classes in AI-generated code before it reaches production.
- Apply least-privilege and sandboxing to all agentic AI deployments. Any AI agent that can execute actions — API calls, file reads, external HTTP requests — needs explicit scope limits, comprehensive logging, and human approval gates for high-impact operations.
- Monitor for WormGPT/FraudGPT-generated phishing campaigns. These tools excel at producing grammatically correct, context-aware phishing emails at scale. Add LLM-generated-content detection to email filtering pipelines where available, and flag atypical phishing language patterns that signature-based tools miss.
- Subscribe to underground forum threat intelligence. The criminal AI tooling market is evolving rapidly. Dedicated threat intelligence feeds that monitor forums like XSS, BreachForums, and Exploit.in provide early warning when new AI tools appear in criminal hands.
Background: How Underground Forum Reputation Economies Work
Understanding why AI slop is a real problem for cybercriminals requires understanding how underground forums function. Platforms like XSS and BreachForums operate as knowledge-sharing marketplaces where participants trade technical credibility for access to tools, exploits, and stolen data. High-reputation contributors — those who have demonstrated skill and reliability over time — hold genuine influence. Their posts are trusted as technically accurate and operationally sound.
AI-generated spam degrades this reputation economy directly. When low-effort, hallucination-ridden posts flood discussion threads, experienced actors cannot efficiently find useful signal. The irony runs deep: the same problem that AI is creating for legitimate social media and journalism — an undifferentiated flood of plausible-sounding but unreliable content — is now also degrading the information quality of criminal ecosystems. The criminals built information platforms that depend on epistemic trust, and AI is undermining that trust from the inside.
The CrimeBB database, maintained for academic research with appropriate ethics oversight, represents one of the richest longitudinal datasets of criminal communication available to researchers. Its 100 million posts span multiple years across several platforms, enabling trend analysis that individual forum snapshots cannot provide.
Conclusion
The data says cybercriminals are struggling to make AI work for sophisticated attacks, AI safety guardrails on mainstream models are holding, and the primary criminal AI uplift is in mass-scale social engineering and fraud — not advanced exploitation. The bigger risk sits on the defender's side: in poorly secured AI-coded products and agentic systems that attackers will exploit as they proliferate. Security teams that focus threat modeling on AI-enhanced phishing today, and start auditing agentic AI deployments now, are addressing the threat that the evidence actually supports.
For any query contact us at contact@cipherssecurity.com
