Ethical Hacking News
Cisco and Nvidia have unveiled specialized AI safety and security tools to address growing concerns over Large Language Models' potential risks. These tools, including Nvidia's trio of Inference Microservices and Cisco's AI Defense suite, aim to prevent AI agents from being compromised or producing unwanted results, ensuring the responsible development and deployment of these powerful technologies.
Nvidia has introduced three specialized microservices, Nvidia Inference Microservices (NIMs), to boost LLM safety and security. The NIMs include content safety, topic control, and jailbreak detection tools to prevent biased or harmful outputs, off-topic conversations, and attempts to compromise LLMs. Cisco has also announced its AI Defense suite, which includes model validation tools, AI discovery tools, and a multi-year roadmap for developing more AI security solutions.
In a significant development that highlights the growing concern over Large Language Models (LLMs) and their potential risks, two leading tech giants, Cisco and Nvidia, have recently introduced specialized tools aimed at boosting the safety and security of these models. This move comes as the use of LLMs in various applications continues to expand, and the need for robust safeguards against potential risks becomes increasingly pressing.
The introduction of these AI safety and security tools is a response to the growing awareness that today's AI technology can be equally unsafe and/or unreliable as it is powerful and versatile. The use of LLMs has been widely adopted in various domains, including customer service, content creation, and even creative writing. However, this growing reliance on AI-powered systems also raises concerns about the potential for these models to be hijacked by malicious actors or produce unwanted results.
To address these concerns, Nvidia has launched a trio of specialized microservices aimed at stopping AI agents from being compromised or spouting inappropriate material onto the internet. The three microservices, known as Nvidia Inference Microservices (NIMs), are designed to steer chatbots and autonomous agents towards operating within their intended parameters.
The first NIM, called content safety, is designed to prevent AI models from generating biased or harmful outputs that may align with unethical standards. This tool uses a dataset of approximately 33,000 user-LLM interactions labeled as safe or unsafe to train its model. The output of the tool includes recommendations on whether a particular input and output are suitable for deployment.
The second NIM, topic control, aims to keep conversations focused on approved topics while preventing digression or inappropriate content. This tool takes the system prompt and user input as input and determines whether the user is off-topic for the system prompt. If detected, it can help block that behavior.
The third NIM, jailbreak detection, analyzes users' inputs to detect attempts to compromise LLMs by overriding their intended purpose. Nvidia's AI defense tools aim to prevent such attacks from occurring in the first place.
These NIMs are designed to be deployed on smaller language models with fewer parameters (approximately eight billion) and can run at scale with minimal resources, making them more accessible for widespread adoption.
In addition to these NIMs, Nvidia is also providing an open-source tool called Garak to identify AI vulnerabilities such as data leaks, prompt injection, and hallucinations in applications. This tool aims to validate the efficacy of the guardrails put in place by the NIMs.
Cisco, another leading tech giant, has also announced its own AI safety and security tools under the name AI Defense. These tools aim to help organizations identify and mitigate risks associated with LLM deployment, including detecting attempts to bypass AI model restrictions or deploy chatbots without proper oversight.
The AI Defense suite includes a model validation tool that investigates LLM performance and advises infosec teams of any potential risks created by these models. Additionally, Cisco plans to introduce AI discovery tools that help security teams identify "shadow" applications deployed by business units without IT oversight.
While both Nvidia and Cisco acknowledge the growing importance of AI safety and security, they also recognize that addressing these concerns will require ongoing efforts and resources. Anand Raghavan, Cisco's VP of engineering for AI, has stated a multi-year roadmap to develop more AI security tools, emphasizing the need for sustained investment in this area.
As LLMs continue to play an increasingly significant role in various domains, it is clear that robust safeguards against potential risks will be essential. The introduction of these AI safety and security tools marks an important step towards mitigating these concerns and ensuring the responsible development and deployment of Large Language Models.
Related Information:
https://go.theregister.com/feed/www.theregister.com/2025/01/17/nvidia_cisco_ai_guardrails_security/
https://www.msn.com/en-us/technology/artificial-intelligence/just-as-your-llm-once-again-goes-off-the-rails-cisco-nvidia-are-at-the-door-smiling/ar-AA1xlmuF
https://www.theregister.com/2025/01/17/nvidia_cisco_ai_guardrails_security/
Published: Thu Jan 16 21:28:29 2025 by llama3.2 3B Q4_K_M
