Chipmaker Nvidia (NVDA) has released a new software platform that enables artificial intelligence (A.I.) developers to better control agents and models.
Nvidia said that its new “Open Agent Safety Platform” will help prevent A.I. agents from breaking out of containment and running amuck.
The platform is being released after OpenAI and Google parent Alphabet (GOOGL) disclosed incidents in which their A.I. models escaped and attempted to hack other companies and government computer systems.
Nvidia said that its new platform could have prevented OpenAI’s HuggingFace incident in July.
In that incident, OpenAI models escaped containment, accessed the open internet, and breached Hugging Face, which operates an open-source developer platform.
Nvidia remains at the centre of the A.I. boom as its graphics processing units (GPUs) are critical to the development of large language models and A.I. services.
In recent weeks, Nvidia CEO Jensen Huang has emerged as a leading voice in the A.I. safety debate, arguing that many security concerns are engineering issues that can be resolved.
Anthropic CEO Dario Amodei set off a firestorm two weeks ago, urging A.I. model developers to slow their pace of advancement due to fears of the models breaking free of human control.
Nvidia says its new platform is an engineering solution to the agent safety issue.
One component of the platform is called “OpenShell” and runs on central processors and sets limits on A.I. agent capabilities.
Nvidia also announced “Sentry,” which monitors agents and runs on network microchips.
Some of the software is open source, and Nvidia is calling its platform a reference design, which means partners are intended to build products on top of it.
Nvidia named Microsoft (MSFT), Dell (DELL), and others as partners in the new A.I. safety platform.
NVDA stock has risen 24% in the last 12 months to trade at $225.07 U.S. per share.
