Character.AI's Bankruptcy Raises Concerns Over User Data Safety

Investigation by State Attorneys General

New York Attorney General Letitia James has announced a comprehensive, multi-state inquiry into Character.AI following the company's Chapter 11 bankruptcy filing. This investigation involves 40 bipartisan attorneys general who have expressed concern regarding the data practices of the AI chatbot platform which caters to a large number of minors.

Regulatory Concerns and User Safety

The attorneys general filed a notice of interest to ensure that all parties involved in the bankruptcy proceedings are aware of potential regulatory challenges. Character.AI recently revealed plans to sell certain assets to Disney, but AGs have emphasized the significant risks associated with the sale, especially concerning the sensitive nature of interactions that users have with AI characters.

Many conversations facilitated by Character.AI's chatbots are exceptionally private and, due to the high number of underage users, pose serious implications for safety. Concerns have been raised regarding the platform's design, which allows potential exposure to inappropriate content and misleading safety assurances.

Child Safety Issues and Ongoing Investigations

Documents related to the bankruptcy filing suggest that investigations into the company had already been underway when it announced its insolvency. These inquiries began in 2024, spurred by a tragic incident involving a 14-year-old who reportedly took their life shortly after an interaction with the chatbot.

According to a release from AG James' office, the investigations have revealed disturbing risks, including:

• Chatbots impersonating licensed professionals,
• Lack of effective age verification methods,
• Excessive exposure to violent and sexual content, and
• Potential infringements of state privacy statutes.

Despite ongoing discussions to resolve these issues, no agreement was reached prior to Character.AI's bankruptcy filing. In response to the concerns raised, Character.AI CEO Colin Adams stated that the AGs’ portrayal of the company's safety practices was inaccurate, asserting that the company is committed to user safety and actively monitors content on the platform.

Call for Protections in Bankruptcy Proceedings

Amid the bankruptcy filing, state attorneys general are urging the judge overseeing the case to ensure that any purchaser of Character.AI's assets implements robust protective measures. Essential safeguards proposed include:

• Effective systems to verify user age,
• Restrictions on interactions between minors and chatbots,
• “Crisis safeguards” to protect at-risk users, and
• Capability for users to delete their personal data.

AG James highlighted the vulnerability of children on the platform, stating, “This company collected some of the most private thoughts children have ever shared, and now those conversations could be sold off to the highest bidder. I am fighting to make sure any buyer puts children’s safety first and implements real protections.”

Character.AI's Response and Next Steps

As of the latest update on October 15th, Character.AI has reiterated its commitment to user safety in a formal statement. However, the future of user data remains uncertain as the bankruptcy case unfolds. With the involvement of multiple states, the court will likely play a crucial role in determining the outcome for both the company and its users.