Chinese police on Sunday detained a man for allegedly disseminating fake news on a train crash using the artificial intelligence software ChatGPT. The arrest was reportedly the first for misuse of ChatGPT.
The South China Morning Post reported that the Gansu police arrested a man with the surname Hong on charges of “using artificial intelligence technology to concoct false and untrue information.” The publication reported that the 'fake' news was spotted by the cyber division of a county police bureau and the news reportedly claimed nine died in a local train accident on April 25.
Kongtong County's cybersecurity officers also found the article simultaneously posted by more than 20 accounts on Baijiahao, a blog-style platform run by Chinese search engine giant Baidu. According to reports, the stories received over 15,000 clicks by the time it caught authorities' attention.
This was the first time an arrest under China's new regulations to check the use of “deepfake” technology was made public. Chinese government in January introduced the 'Administrative Provisions on Deep Synthesis for Internet Information Service' to check the use of deep synthesis. “The provisions define deep synthesis as the use of technologies – including deep learning and augmented reality – to generate text, images, audio and video and to create virtual scenarios,” SCMP reported. According to reports, the regulation mandates users should clearly label to prevent public confusion.
The Gansu public security department said Hong was suspected of the crime of "picking quarrels and provoking trouble", a charge that normally carries a maximum sentence of five years. But in cases that are deemed especially severe, offenders can be jailed for 10 years and given additional penalties, the Post reported.
A statement issued by police said, “Hong confessed to bypassing Baijiahao's duplication check function to publish on multiple accounts he had acquired. He input the elements of trending social stories in China from past years into ChatGPT to quickly produce different versions of the same fake story and uploaded them to his Baijiahao accounts.”
Though ChatGPT isn't directly available in China, it could be accessed using a reliable VPN connection.
China's top internet regulator has long voiced concern that unchecked development and use of deep synthesis technology could lead to its use in criminal activities such as online scams or defamation.
As ChatGPT has gone viral in recent months, China's law enforcement agencies have repeatedly voiced suspicion, and even warnings, about the technology.
"In one of the first comments on the chatbot made by the Chinese security apparatus, police in Beijing specifically warned the public in February to be wary of rumours generated by ChatGPT," the Post reported.
(With PTI inputs.)
