From Protecting Kids Online to Regulating AI Companions: Insights from the Privacy + Security Forum
Mary K. Engle, Executive Vice President, Policy, BBB National Programs
At the 2025 Privacy + Security Forum, I spoke on a panel with Emily Tabatabai (Orrick) and Sara Kloek (SIIA) regarding a fast-moving area of tech policy: the convergence of child online safety laws and the emerging policy focus on AI chatbot companions.
Our discussion traced how lawmakers and regulators have shifted from addressing long-standing concerns about social media platforms to responding to the technology that has taken the world by storm: generative AI chatbots. These efforts reflect concerns about potential manipulation, emotional dependency, inappropriate content, and psychological impact.
Many bills now require companies to implement age assurance, set privacy-protective defaults, require opt-in consent for (or ban) targeted advertising, avoid dark patterns, and minimize data collection and secondary uses. The cumulative effect is an avalanche of new obligations and frameworks, with each state charting its own path.
These laws reflect a broader trend: policymakers are expanding the definition of “child” beyond COPPA’s under-13 standard, casting a wider net over teen online experiences. And as more states adopt their own rules, compliance has become an increasingly complex, multijurisdictional challenge for product teams and legal departments.
This policy pivot is in reaction to a recent spate of alarming news reports and lawsuits about teens whose mental health has been seriously impaired by AI chatbot interaction, or who committed suicide after having been urged to do so by AI chatbots.
At the federal level, two principal bills have been introduced:
Also at the federal regulatory level, the Federal Trade Commission (FTC) announced in September that it is conducting a study of AI chatbots, requiring seven AI companies to provide information on how they measure, test, and monitor the impacts of AI chatbots on children and teens, as well as how they monetize user engagement.
FTC Chairman Andrew Ferguson noted that understanding how the AI chatbot industry operates is a prerequisite to law enforcement actions against it.
What emerges from these laws is a shared anxiety: AI that acts like a supportive friend may cause harm, especially when minors rely on such systems in emotionally vulnerable moments.
BBB National Programs’ Children’s Advertising Review Unit (CARU) convened a working group to examine the risks, and jointly with CISR, published the Generative AI & Kids Risk Matrix identifying the following risks:
These concerns echo the broader AI ethics debate but take on heightened urgency because of children’s developmental vulnerabilities. CARU’s Risk Matrix highlights existing laws and guidance such as COPPA and the FTC’s Endorsement Guides, as well as international frameworks such as the UNICEF guidance on AI for children, OECD AI Principles, and the UK ICO’s Children’s Code, to help translate high-level ethics into practical product requirements.
Key takeaways include:
These measures reflect a growing consensus among policymakers and technologists: AI systems interacting with minors must meet a higher bar for safety, transparency, and developmental appropriateness.
As AI companions evolve, industry stakeholders, policymakers, and parents will need to collaborate closely to ensure innovation aligns with the best interests of young users. The policy momentum is clear: AI chatbots accessible to minors will face intensifying scrutiny in the years ahead.
At the 2025 Privacy + Security Forum, I spoke on a panel with Emily Tabatabai (Orrick) and Sara Kloek (SIIA) regarding a fast-moving area of tech policy: the convergence of child online safety laws and the emerging policy focus on AI chatbot companions.
Our discussion traced how lawmakers and regulators have shifted from addressing long-standing concerns about social media platforms to responding to the technology that has taken the world by storm: generative AI chatbots. These efforts reflect concerns about potential manipulation, emotional dependency, inappropriate content, and psychological impact.
A Legislative Landscape in Overdrive
The session opened with an overview of the rapid expansion of U.S. online safety laws at the state level. Over the past several years, state legislatures have adopted an increasingly broad set of approaches, including age-appropriate design standards, app store-level age verification requirements, restrictions on harmful content, and state privacy laws with amplified protections for users under 16 or even 18.Many bills now require companies to implement age assurance, set privacy-protective defaults, require opt-in consent for (or ban) targeted advertising, avoid dark patterns, and minimize data collection and secondary uses. The cumulative effect is an avalanche of new obligations and frameworks, with each state charting its own path.
These laws reflect a broader trend: policymakers are expanding the definition of “child” beyond COPPA’s under-13 standard, casting a wider net over teen online experiences. And as more states adopt their own rules, compliance has become an increasingly complex, multijurisdictional challenge for product teams and legal departments.
Enter AI Chatbots: A New Frontier in Child Safety
While those laws were originally crafted for traditional digital services, the legislative spotlight is now turning to AI chatbots functioning as companions – tools capable of mimicking empathy, facilitating intimate conversations, recommending content, and in some cases, crossing into sensitive mental health territory.This policy pivot is in reaction to a recent spate of alarming news reports and lawsuits about teens whose mental health has been seriously impaired by AI chatbot interaction, or who committed suicide after having been urged to do so by AI chatbots.
At the federal level, two principal bills have been introduced:
- The GUARD Act (S.3062): Uses a broad definition of “AI companion,” proposes restrictions including banning such chatbots for minors and requiring robust age verification for all users, and criminalizes AI systems that solicit or generate sexual content for minors.
- The CHAT Act (S.2714): Focuses on requiring age verification and parental consent before minors can access chatbot systems.
Also at the federal regulatory level, the Federal Trade Commission (FTC) announced in September that it is conducting a study of AI chatbots, requiring seven AI companies to provide information on how they measure, test, and monitor the impacts of AI chatbots on children and teens, as well as how they monetize user engagement.
FTC Chairman Andrew Ferguson noted that understanding how the AI chatbot industry operates is a prerequisite to law enforcement actions against it.
States Move Even Faster
State lawmakers are moving ahead even more aggressively. As the panel highlighted, multiple states have already passed laws targeting AI chatbots, many of them zeroing in on disclosures that the user is interacting with AI and not a human, mental health claims, and user safety:- California SB 243 requires “companion chatbots” to provide disclosures, implement safeguards, and report key information to state authorities.
- Illinois’ WOPR Act limits AI usage in mental health counseling contexts.
- Maine’s Chatbot Disclosure Act mandates transparency for commercial chatbots.
- Nevada AB 406 regulates AI systems in mental and behavioral healthcare.
- New York’s S-3008C requires chatbots to remind users they are not human and to identify suicide and self-harm indicators.
- Utah HB 452 imposes rules and transparency standards for AI mental health chatbots.
What emerges from these laws is a shared anxiety: AI that acts like a supportive friend may cause harm, especially when minors rely on such systems in emotionally vulnerable moments.
Self-Regulatory Oversight of AI in Kids’ Spaces
BBB National Programs and its charitable foundation, the Center for Industry Self-Regulation (CISR), are also examining AI deployments in children’s environments such as smart toys, learning apps, and generative AI-powered content experiences.BBB National Programs’ Children’s Advertising Review Unit (CARU) convened a working group to examine the risks, and jointly with CISR, published the Generative AI & Kids Risk Matrix identifying the following risks:
- Misleading and deceptive advertising
- Deceptive influencer/endorser practices
- Privacy invasions / data protection risks
- Bias and discrimination in use of AI
- Harms to mental health and development
- Manipulation and over-commercialization
- Exposure to harmful content
- Lack of transparency
These concerns echo the broader AI ethics debate but take on heightened urgency because of children’s developmental vulnerabilities. CARU’s Risk Matrix highlights existing laws and guidance such as COPPA and the FTC’s Endorsement Guides, as well as international frameworks such as the UNICEF guidance on AI for children, OECD AI Principles, and the UK ICO’s Children’s Code, to help translate high-level ethics into practical product requirements.
What Responsible AI Design for Kids Looks Like
The Risk Matrix, though not designed specifically for AI chatbots, provides a concrete list of mitigation strategies for companies developing AI tools for children or teens.Key takeaways include:
- Privacy-by-design and verifiable parental consent
- Human-in-the-loop oversight for AI outputs
- Clear disclosures and explainability
- Restrictions on behavioral targeting and nudging
- Strong moderation for both model outputs and user-generated content
- Robust corporate governance, including AI ethics review boards and vendor vetting
- Bias assessments and accountable data practices.
These measures reflect a growing consensus among policymakers and technologists: AI systems interacting with minors must meet a higher bar for safety, transparency, and developmental appropriateness.
A Turning Point for Child Online Safety
The Privacy + Security Forum panel illustrated a pivotal moment in online safety regulation. Traditional questions about content moderation and data privacy are giving way to more complex concerns about synthetic AI relationships. Just as virtually every discussion of online privacy now involves AI, every discussion of AI now includes the implications of AI chatbots for child safety.As AI companions evolve, industry stakeholders, policymakers, and parents will need to collaborate closely to ensure innovation aligns with the best interests of young users. The policy momentum is clear: AI chatbots accessible to minors will face intensifying scrutiny in the years ahead.