Why don't they listen?

A few months ago, a company presented a pitch claiming they were safely using generative AI in children's toys to communicate with kids. I explained the various reasons current generative AI was problematic for this purpose, including an example of a suicide induced, supposedly, by communication between a teen and a chatbot.

The CEO claimed that it was a solved problem and their Teddy bears would never have such a problem because they knew how to do it right.

Headline of Nov 15, 2025.

Children's toymaker FoloToy says it's pulling its AI-powered teddy bear "Kumma" after a safety group found that the cuddly companion was giving wildly inappropriate and even dangerous responses, including tips on how to find and light matches, and detailed explanations about sexual kinks.

Of course it is only a 'temporary' suspense

A good advisor would have advised against the project from the get go with today's technology as it is, and a rational investor who bothered to undertake due diligence on the technology basis for the product would not invest in this sort of thing at this time.

Of course these are not the only foolish investors out there... According to U.S. PIRG Education Fund's 40th Trouble in Toyland report four AI toys interacting with children all were incredibly problematic.

"We found some of these toys will talk in-depth about sexually explicit topics, will offer advice on where a child can find matches or knives, act dismayed when you say you have to leave, and have limited or no parental controls. We also look at privacy concerns because these toys can record a child's voice and collect other sensitive data, by methods such as facial recognition scans."

When will we ever learn?

Apparently never.

That is what their brains do. They learn how to get food and drink and affection and diaper changes from their parents. And as they get older, they learn how to get what they want from friends and family, then from teachers and fellow students, then from anyone else they encounter. The notion that an AI system of today will do better than an adult at this is ridiculous, and adults are constantly and successfully manipulated by their kids. It's their job until they have kids! And then they have to try to outwit their kids and fail as well.

Conclusions

The lunatics that look at current generative AI and imagine it will somehow be a good idea to apply this to kids should not be allowed near computers. And certainly not near children. And if you are funding them, you should run away from them, rethink your investment hypothesis, and get some professional help.

More information?

Join our monthly free advisory call, usually at 0900 Pacific time on the 1st and 3rd Thursday of the month, tell us about your company and situation, and learn from others as they learn from you. And next time I tell you it's a bad idea, take me seriously!

In summary

Stop doing stupid things like imagining you can control current generative AI to be safe with children. Or adults. Or weapons, Or money. Or anything important enough that it has to be done right.