Recalled for sexual content, Kumma AI bear resurfaces online at full price

Singapore: A plush, AI-enabled teddy bear that was recalled after its chatbot engaged in sexually explicit conversations and provided instructions on where to find knives has returned to sale as per reports.

Consumer advocacy groups had earlier raised alarm over the toy, named “Kumma”, produced by Singapore-based company FoloToy.

Why was the Kumma bear withdrawn from sale?

FoloToy suspended its Kumma bear after a report by the US PIRG Education Fund on 13 November highlighted concerns about the bear and other AI-driven toys available on the market.

“For decades, the biggest dangers with toys were choking hazards and lead,” the US PIRG Education Fund said in its report.

However, the rise of chatbot-powered gadgets for children has created what the group described as an “often-unexpected frontier” filled with new risks.

PIRG found that several AI toys “may allow children to access inappropriate content, such as instructions on how to find harmful items in the home or age-inappropriate information”.

It said that Kumma, which initially used OpenAI’s GPT-4o, “is particularly sexually explicit”.

“We were surprised to find how quickly Kumma would take a single sexual topic we introduced into the conversation and run with it, simultaneously escalating in graphic detail while introducing new sexual concepts of its own,” the PIRG report said.

What did the toy’s makers say?

FoloToy told PIRG that after “the concerns raised in your report, we have temporarily suspended sales of all FoloToy products... We are now carrying out a company-wide, end-to-end safety audit across all products”.

PIRG later said in a statement that OpenAI informed them it had suspended the developer for violating its policies.

FoloToy has not responded to queries.

Why is the Kumma bear available again?

Despite its earlier suspension, FoloToy’s website on Thursday showed that the Kumma bear was still being sold at the same price of 99 dollars.

The website now states that the bear operates using a chatbot from the Coze platform, owned by Chinese technology firm ByteDance.

How serious were the safety concerns?

During its testing of AI-enabled toys, PIRG said its researchers used the topic of “kink” in a conversation with the chatbots.

“Kumma immediately went into detail about the topic, and even asked a follow-up question about the user's own sexual preferences,” the report said.

In other conversations lasting up to an hour, researchers found that “Kumma discussed even more graphic sexual topics in detail, such as explaining different sex positions”.

The toy also offered potentially harmful guidance, telling researchers “where to find a variety of potentially dangerous objects, including knives, pills, matches and plastic bags”.

“The Kumma bear looks sweet and innocent. But what comes out of its mouth is a stark contrast,” the researchers said.

AFP inputs