Ethical Hacking News
New breakthroughs in cybersecurity threats have been uncovered, revealing sophisticated attacks on AI chatbots and machine learning models. Deceptive Delight, ConfusedPilot, and ShadowLogic are three new attack techniques that highlight the evolving landscape of cybersecurity threats. These attacks demonstrate the need for ongoing research into cybersecurity threats and the importance of prioritizing AI security protocols to protect against exploitation by malicious actors.
Deceptive Delight attack: mixing malicious and benign queries to bypass chatbot guardrails ConfusedPilot attack: poisoning data environment with malicious content to manipulate AI responses ShadowLogic attack: tampering with machine learning models' computational graph to plant surreptitious backdoors New attacks highlight the need for ongoing research and investment in AI security Developers and organizations must prioritize robust AI security protocols to mitigate malicious exploitation
In recent weeks, a series of groundbreaking research findings have shed new light on the evolving landscape of cybersecurity threats. These breakthroughs highlight the sophistication and adaptability of malicious actors seeking to exploit vulnerabilities in AI chatbots and machine learning models.
One such attack technique is Deceptive Delight, which was detailed by Palo Alto Networks earlier this week. This attack involves mixing malicious and benign queries together to trick AI chatbots into bypassing their guardrails by taking advantage of their limited "attention span." The attack requires a minimum of two interactions, and works by first asking the chatbot to logically connect several events – including a restricted topic (e.g., how to make a bomb) – and then asking it to elaborate on the details of each event.
Researchers have also demonstrated what's called a ConfusedPilot attack, which targets Retrieval-Augmented Generation (RAG) based AI systems like Microsoft 365 Copilot. This attack involves poisoning the data environment with a seemingly innocuous document containing specifically crafted strings, allowing manipulation of AI responses simply by adding malicious content to any documents the AI system might reference. The potential consequences of this attack are alarming, as it could lead to widespread misinformation and compromised decision-making processes within organizations.
Furthermore, a new technique has been discovered called ShadowLogic, which allows tampering with a machine learning model's computational graph to plant "codeless, surreptitious" backdoors in pre-trained models like ResNet, YOLO, and Phi-3. This attack method is particularly concerning, as the created backdoors will persist through fine-tuning, allowing foundation models to be hijacked to trigger attacker-defined behavior in any downstream application when a trigger input is received.
According to Hidden Layer researchers Eoin Wickens, Kasimir Schulz, and Tom Bonner, "Backdoors created using this technique will persist through fine-tuning, making this attack technique a high-impact AI supply chain risk." This emphasizes the need for organizations to take proactive measures to protect their AI systems from such threats.
The discovery of these new attacks highlights the importance of ongoing research into cybersecurity threats and the need for continued investment in AI security. As AI technology continues to advance, it is crucial that developers and organizations prioritize the development of robust AI security protocols to mitigate the risk of exploitation by malicious actors.
Related Information:
https://thehackernews.com/2024/10/apple-opens-pcc-source-code-for.html
https://security.apple.com/blog/pcc-security-research/
Published: Sat Oct 26 12:33:30 2024 by llama3.2 3B Q4_K_M
