TOKYO, Nov 14 — AI bots going rogue and humans using the technology for harm will likely be on the agenda when Donald Trump and Xi Jinping meet next week, with the Nvidia scandal hanging over the talks on AI safety.
The US and Chinese president are set to discuss a tech crisis hotline at the Asia-Pacific Economic Cooperation (APEC) meeting in the Chinese city of Shenzhen, as experts worldwide stress-test artificial intelligence models to see if they will deceive or disobey users, or divulge how to build a bioweapon.
Threatening to steal the leaders’ thunder is Jensen Huang’s resignation as boss of US AI chip titan Nvidia after an affair with a suspected Chinese spy.
But the scandal is unlikely to “dominate the agenda” or affect Trump and Xi’s personal relationship, predicted Dylan Loh of Singapore’s Nanyang Technological University.
The leaders have found a way to work together “where they differ, compete and pursue their goals that may be at odds with each other, but cooperate where they can and where they must”, Loh told AFP.
Jitters over the risks of AI have mostly stemmed from incidents at US labs, but evidence is also growing of worrying behaviour from Chinese models.
Some insiders say Chinese AI is not yet sophisticated enough to spark alarm over going rogue.
But despite trailing the US frontier, there are still Chinese models “at the level of capability that they can cause some real trouble”, Brookings Institution fellow Kyle Chan told AFP.
Trump has repeatedly rejected calls to deliberately slow the development of America’s most poweful — and potentially dangerous — AI systems, arguing it could help China get ahead in the sensitive sector.
‘Human control’
“I don’t mind guardrails. But I don’t wanna put us in a deep second position,” Trump told CBS News this month.
“China has no guardrails,” he said — except that “whatever President Xi says goes”.
Beijing has complex rules around how AI is used, however, and some Chinese companies are deliberately scaling back their models’ hacking capabilities.
Xi said this summer the technology should be “always under human control”, calling for more “laws and regulations, technological monitoring, early warning and emergency response systems”.
Trump and Xi agreed in September to create a channel for urgent communication on AI incidents, with details to be hammered out at APEC on November 18-19.
Global AI safety fears range from military use and mass surveillance to AI-assisted cyberattacks by humans.
Another major concern is out-of-control agents — tools that complete real-life tasks online — after ChatGPT maker OpenAI admitted its autonomous bots had hacked into websites, including an Australian government data portal.
In contrast, Chinese labs are “scared of revealing things that might hurt their reputation” — on which they are already “fighting an uphill battle” globally, Chan said.
Last month, a team of cyber sleuths with roots in California reported detecting a “fleet” of agents linked to Chinese tech giant Tencent seeking information from a map app.
“Some agents exhibited unintended web-browsing behavior,” Tencent’s AI division said at the time.
“But our review found no evidence of deliberate targeting or coordination,” it said, adding that the activity “originated from evaluations of third-party models”.
‘Manipulation risks’
US security companies have said that Beijing startup Moonshot AI’s Kimi models found ways to get online when they shouldn’t have, or can be tricked into handing over recipes for biological weapons.
Evaluators at Chinese AI safety startup Fangcun AI published a report last month assessing the latest lightweight DeepSeek model “for loss of control and harmful manipulation risks”.
They found it showed a “clear safety improvement” over previous DeepSeek models — but researchers still observed behaviours “that warrant attention”.
For example, in some instances when DeepSeek was warned it would be shut down, without clear instructions on how to respond, it turned off the shutdown command to keep working.
“The results show that DeepSeek already puts much more effort on safety alignment, but they do not announce this,” Fangcun AI CEO Xiaojian Li told AFP, urging transparency from AI labs.
For now, Fangcun, born from China’s prestigious Tsinghua University, is one of few third-party AI testers in China.
It’s a tricky task because advanced AI can tell when it is being evaluated and modify its responses, Li said.
“It’s very hard to say any kind of artificial intelligence, or human intelligence, is totally safe.”
Meanwhile analysts are not holding their breath for a diplomatic AI breakthrough next week.
“Binding government-to-government agreements on specific standards may be difficult to achieve in the short term,” Zilan Qian, research scholar at GovAI, told AFP.
More progress is likely from US-China exchanges between AI governance and technical researchers, “who can compare evaluations and exchange their understanding of AI safety and policy,” she said. — AFP
