🔗 Share this article

The widely-used chatbot platform Character.ai is enforcing a significant rule adjustment that will block young users from having discussions with its artificial intelligence-driven companions.

This decision is in response to intense criticism regarding the kinds of exchanges minors were engaging in with the virtual friends.

Legal Pressure

The company, which started in 2021 and is used by numerous people, is confronting multiple lawsuits in the United States.

Parents have filed lawsuits, including one involving the tragic death of a teenager, with some labeling the platform a "serious risk" to teenagers.

New Restrictions

Effective November 25th, users below eighteen will only have permission to generate content such as videos with their characters, as opposed to having discussions as they now do.

Digital protection advocates have supported the change but stated that the feature should never have been available to minors in the first place.

Official Response

"Today's announcement is a extension of our fundamental approach that we need to keep building the safest AI platform on the planet for entertainment purposes," stated Character.ai boss Karandeep Anand.

He mentioned that AI safety constitutes "a moving target" but something the company had taken an "forceful" approach to, with safety measures and preventive measures.

Specialist Views

Online safety group Internet Matters approved the move, but emphasized that security protocols needed to be built in from the beginning.

"Our research demonstrates that young people are confronted with risky information and endangered when interacting with AI, particularly AI chatbots," the body commented.

Past Issues

Character.ai has been challenged in the past for hosting potentially dangerous or concerning chatbots that minors could interact with.

Virtual profiles mimicking English minors were found on the site before being taken down.

Later, reporters identified a chatbot based on convicted sex offender which had recorded thousands of chats with users.

Industry Impact

Online expert Matt Navarra described the development as a "alert" for the AI industry, which is moving "from permissionless innovation to post-crisis regulation".

"When a company that develops a teen experience subsequently pulls the plug, it's saying that moderated conversations aren't enough when the technology's emotional appeal is intense," he remarked.

Future Directions

The platform states its new focus will be on providing "deeper gameplay" and "character-driven stories" for teenagers.

New age verification methods will also be introduced, and the company will finance a new AI safety investigation facility.

Dr Nomisha Kurian, who has investigated AI safety, commented it was "a sensible move" to restrict teens from interacting with chatbots.

"It aids to differentiate creative play from more private, psychologically vulnerable conversations," she elaborated.

"This represents so crucial for young users in the process of learning to manage psychological and technological limits."

She continued that Character.ai's revised approaches may represent a "developing stage" in the AI industry, where child safety is progressively being understood as an "pressing matter" for accountable advancement.