Artificial intelligence is transforming the way organizations work and how cybercriminals operate.

Google recently filed a federal lawsuit against Outsider Enterprise, a China-based cybercrime network accused of abusing Google’s Gemini AI platform to create phishing websites and power large-scale SMS phishing (smishing) campaigns. The case marks Google’s first lawsuit involving the misuse of its Gemini AI tools and highlights a growing concern across the cybersecurity industry: AI is making scams quicker to create as well as more convincing, and more difficult to detect.

AI Is Changing the Phishing Landscape

According to Google’s complaint, the cybercriminal group developed and distributed a phishing-as-a-service (PhaaS) toolkit that allowed other attackers to launch sophisticated phishing campaigns with minimal technical expertise.

The phishing kits reportedly included instructions for using AI tools like Gemini to:

  • Generate realistic phishing websites
  • Create convincing fraudulent content
  • Build scam infrastructure faster
  • Scale attacks against thousands of victims simultaneously

Rather than spending hours writing convincing emails or designing fake login pages, attackers can now leverage AI to automate much of the process.

The result is phishing attacks that look increasingly legitimate—and are far more likely to fool unsuspecting users.

Millions of Fraudulent Messages

Google estimates the operation affected hundreds of thousands of victims and generated millions of dollars in losses.

The investigation linked the organization to:

  • More than 9,000 fake websites
  • Over 1 million fraudulent URLs
  • Approximately 2.5 million scam text messages sent during a two-week period
  • More than 55,000 spam reports from Android users during the same timeframe

Victims received urgent text messages claiming there was an issue with an account, a missed package, or a reward waiting to be claimed. Clicking the embedded link directed users to counterfeit websites designed to steal usernames, passwords, payment information, and other sensitive data.

Why AI Makes These Attacks More Dangerous

As FBI Cyber Division Assistant Director Brett Leatherman noted, criminals are increasingly using AI to make fraud “more convincing and harder to detect.”

Traditional phishing attacks often contained poor grammar, awkward wording, or obvious design flaws.

Today’s AI-generated phishing campaigns can produce:

  • Professionally written messages
  • Convincing branding and logos
  • Realistic website layouts
  • Personalized content
  • Rapidly generated variations that bypass traditional filters

This significantly lowers the barrier to entry for cybercriminals while increasing the sophistication of their attacks.

Google’s Response

Google is taking a multi-layered approach to disrupting the operation. In addition to filing the lawsuit, the company is working with the FBI on law enforcement actions while partnering with AT&T, T-Mobile, and Verizon to block malicious text messages before they reach consumers. Google has also expanded its AI-powered scam detection features for Android, continues to strengthen messaging protections that already intercept more than 10 billion malicious messages each month, and is advocating for federal legislation to establish stronger, long-term protections against AI-enabled fraud. Together, these efforts demonstrate that technology companies are responding more aggressively to the growing misuse of AI platforms.

The case demonstrates that technology companies are beginning to respond aggressively to the misuse of AI platforms.

What Businesses Should Do Now

While technology providers continue strengthening their defenses, organizations cannot rely solely on software to stop phishing attacks.

Businesses should take a layered cybersecurity approach that includes:

  • Ongoing employee security awareness training
  • Multi-factor authentication (MFA)
  • Email and SMS security filtering
  • Continuous monitoring for suspicious activity
  • Strong password policies
  • Incident response planning
  • Regular security assessments

Employees remain one of the most targeted attack vectors, making user education just as important as technical controls.

AI Is Both a Tool and a Target

Artificial intelligence offers tremendous opportunities for innovation, productivity, and automation. Unfortunately, cybercriminals are embracing those same capabilities to improve the speed and effectiveness of their attacks.

The Google lawsuit serves as a reminder that as AI technology evolves, cybersecurity strategies must evolve alongside it.

Organizations that invest in proactive security measures, employee education, and modern threat detection will be far better positioned to defend against the next generation of AI-powered phishing attacks.

At Enfortra, we help businesses strengthen their cybersecurity posture with proactive security solutions, user awareness, and managed IT services designed to reduce risk in an increasingly sophisticated threat landscape. As AI-driven attacks continue to evolve, partnering with a trusted cybersecurity provider can help keep your business one step ahead.