Few would argue fake money poses no threat to the financial system. If you can’t trust that currency is real, buying and selling gets more difficult, with implications for the economy. But according to one noted author, not enough attention goes to another danger: A.I. bots on social media pretending to be real people.

“Now it is possible, for the first time in history, to create fake people—to create billions of fake people,” Israeli historian and author Yuval Noah Harari said last week. “You interact with somebody online, and you don’t know if it’s a real human being or a bot.”

The author of Sapiens—a history of humanity that Bill Gates calls one of his favorite books—made the comments while addressing the UN’s AI for Good summit in Geneva.

He continued, “What happens if you have a social media platform when it’s not just bots that retweet what a human created, but you have millions, potentially billions, of bots that can create content that in many ways is superior to what humans can create, like more convincing, more appealing, whatever—more tailored to your specific personality and life history. If we allow this to happen, then basically humans have completely lost control of the public conversation, and things like democracy will become completely unworkable.”

He warned that “it will do to society what fake money threatens to do to the financial system. If you can’t know who is a real human and who is a fake human, trust will collapse, and with it, at least free society. Maybe dictatorships will be able to manage somehow, but not democracies.”

“AI bot swarms taking over”

Twitter owner Elon Musk is also aware of the bot problem. He tweeted in March that “only verified accounts will be eligible to be in For You recommendations,” calling it “the only realistic way to address advanced AI bot swarms taking over. It is otherwise a hopeless losing battle.”

OpenAI CEO Sam Altman, meanwhile, cofounded a startup called Worldcoin, which offers a way for people to prove they’re a human being and not an A.I. bot, with the help of an iris-scanning device and crypto system. The venture announced a $115 million Series C funding round in May.

Harari called for “very strict rules” against “faking people.”

“If you fake people, or if you allow fake people on your platform without taking effective countermeasures, so maybe we don’t execute you, but you go to 20 years in jail,” he said.

Facing such consequences, tech giants would quickly “find ways to prevent the platforms from being overflown with fake people,” he said.

As for why such rules don’t exist already, he noted that until now creating fake people in such a way “was technically impossible.” Counterfeiting money, by contrast, has long been possible, and governments have enacted “very strict rules” against it to “protect the financial system.”

He noted that he wasn’t calling for laws against creating such bots, but rather, “you’re not allowed to pass them in public as real people.” For example offering an A.I. doctor is fine and “can be extremely helpful,” he said, but only “provided it’s very clear that this is not a human doctor…I need to know whether it’s a real human being or an A.I.”