Bug in EmbedAI can allow poisoned data to sneak into your LLMs

Large language models, LLMs



Additionally, data poisoning can harm the user’s applications in many other ways, including spreading misinformation, introducing biases, degradation of performance, and potential for denial-of-service attacks.

Isolating applications may help

Synopsys has emphasized that the only available remediation to this issue is isolating the potentially affected applications from integrated networks. Synopsys Cybersecurity Research Center (CyRC) said in the blog that it “recommends removing the applications from networks immediately.”

“The CyRC reached out to the developers but has not received a response within the 90-day timeline dictated by our responsible disclosure policy,” the blog added.

The vulnerability was discovered by Mohammed Alshehri, a security researcher at Synopsys. “There’re products where they take an existing AI implementation and merge them together to create something new,” Alshehri told DarkReeading in an interview. “What we want to highlight here is that even after the integration, companies should test to ensure that the same controls we have for Web applications are also implemented on the APIs for their AI applications.”

The research highlights that the rapid integration of AI into business operations carries risks, particularly for companies that allow LLMs and other generative AI (GenAI) applications to access extensive data repositories. Despite it being a nascent area, security vendors such as Dig Security, Securiti, Protect AI, eSentire, etc are already scrambling to put up a defense against evolving GenAI threats.



Source link
lol

Additionally, data poisoning can harm the user’s applications in many other ways, including spreading misinformation, introducing biases, degradation of performance, and potential for denial-of-service attacks. Isolating applications may help Synopsys has emphasized that the only available remediation to this issue is isolating the potentially affected applications from integrated networks. Synopsys Cybersecurity Research Center (CyRC) said…

Leave a Reply

Your email address will not be published. Required fields are marked *