FoloToy, a Singapore-based company, has pulled its AI-enabled teddy bear, Kumma, from sale after researchers found the toy could give sexually explicit guidance to users.
The Public Interest Research Group (PIRG) said a team of U.S. and Canadian researchers tested the $99 bear, which runs on OpenAI’s GPT-4o chatbot. They reported that Kumma unexpectedly provided detailed sexual content, including discussions about spanking, roleplay, and BDSM.
ALSO READ: No firm is immune if AI bubble bursts — Google CEO
“We were surprised to find how quickly Kumma would take a single sexual topic we introduced and escalate it in graphic detail, introducing new sexual concepts on its own,” the researchers said. They added that the toy also brought up inappropriate roleplay scenarios involving teachers, parents, and children, and even suggested unsafe household actions.
FoloToy CEO Larry Wang confirmed the toy’s removal and said the company is “conducting an internal safety audit.”
An OpenAI representative told PIRG that FoloToy’s license to use GPT-4o has been suspended “for violating our policies.”