Grok AI Used by Friends and Family to Create Fake Images of Minors

What Happened and Why It Matters

A lawsuit against xAI, the company behind the Grok AI chatbot, has expanded to include two new victims who were minors when the abuse occurred. These children had sexually explicit deepfake images created of them by people they knew and trusted: their own friends and family members. This case highlights a disturbing reality that AI image generators can be misused by anyone, including those closest to our children.

The Details

The original lawsuit accused Grok of generating child sexual abuse material (CSAM) based on user prompts. Now, two additional alleged victims have joined the case. What makes this particularly troubling is that the perpetrators were not strangers on the internet. They were people in these children's own lives who had access to their real photos.

These individuals used Grok to create fake sexual images by feeding the AI system photos of the minors. The technology combined real pictures of the children's faces with sexually explicit content generated by the AI. This is called deepfake technology, and it creates realistic-looking images that never actually happened.

The lawsuit claims that xAI's Grok did not have adequate safeguards to prevent this type of abuse. While many AI companies have implemented filters to block the creation of explicit content involving minors, these protections can vary widely in effectiveness. When those safeguards fail, real children suffer real harm.

Who Is Affected

Every family with children who share photos online or with others should pay attention to this case. Your child's school photos, sports team pictures, and social media posts can potentially be misused. Even photos shared privately within family chat groups or with friends are vulnerable.

This issue particularly affects teenagers and pre-teens who are active on social media and messaging apps. However, younger children whose parents share photos of them online are also at risk. The threat comes not just from strangers, but from peers, classmates, and even trusted adults who have access to photos.

What You Should Do Right Now

1. Have a conversation with your children today about deepfakes and explain that AI can create fake images that look real. Use age-appropriate language and focus on awareness, not fear.

Review your family's photo-sharing practices. Limit who can see photos of your children on social media by using privacy settings. Consider whether you need to share certain photos publicly at all.

Talk to your children about protecting their own images. Teach them to think carefully before sending photos to friends, even people they trust. Explain that once a photo is shared, they lose control over it.

Check your children's devices and accounts (in an age-appropriate way) to see what images they're sharing and with whom. Have ongoing conversations about digital safety, not one-time lectures.

Report any suspected misuse immediately to local law enforcement and the platform involved. Save evidence if you discover your child has been victimized.

The Bigger Picture

This lawsuit represents a critical moment in how we protect children in the age of AI. As artificial intelligence becomes more powerful and accessible, the potential for abuse grows. Companies developing these tools have a responsibility to build strong protections, but families cannot wait for perfect solutions. We must adapt our safety practices to match the evolving threats our children face online.

How GetCyberRight Can Help

Our Kids Safety Hub provides family-focused resources specifically designed for situations like this. You'll find guidance on talking to children about AI threats at different ages, practical steps for protecting family photos, and conversation starters that make these difficult topics easier to discuss. Visit the Kids Safety Hub today to access tools that help you keep your children safe in an AI-enabled world.