It’s often quipped that “humans are the weakest link,” however that doesn’t need to be the case if an organization properly integrates AI security and privacy training into their generative AI and LLM adoption journey.

This involves helping staff understand existing generative AI/LLM initiatives, as well as the broader technology and how it functions, and key security considerations, such as data leakage. Additionally, it is key to establish a culture of trust and transparency, so that staff feel comfortable sharing what generative AI and LLM tools and services are being used, and how.

This one may be surprising, but much like with the cloud before it, most organizations don’t actually establish coherent strategic business cases for using new innovative technologies, including generative AI and LLM. It is easy to get caught in the hype and feel you need to join the race or get left behind. But without a sound business case, the organization risks poor outcomes, increased risks and opaque goals.

Without Governance, accountability and clear objectives are nearly impossible. This area of the checklist involves establishing an AI RACI chart for the organization’s AI efforts, documenting and assigning who will be responsible for risks and governance and establishing organizational-wide AI policies and processes.

While obviously requiring input from legal experts beyond the cyber domain, the legal implications of AI aren’t to be underestimated. They are quickly evolving and could impact the organization financially and reputationally.

This area involves an extensive list of activities, such as product warranties involving AI, AI EULAs, ownership rights for code developed with AI tools, IP risks and contract indemnification provisions just to name a few. To put it succinctly, be sure to engage your legal team or experts to determine the various legal-focused activities the organization should be undertaking as part of their adoption and use of generative AI and LLMs.

To build on the legal discussions, regulations are also rapidly evolving, such as the EU’s AI Act, with others undoubtedly soon to follow. Organizations should be determining their country, state and Government AI compliance requirements, consent around the use of AI for specific purposes such as employee monitoring and clearly understanding how their AI vendors store and delete data as well as regulate its use.

Using LLM solutions requires specific risk considerations and controls. The checklist calls out items such as access control, training pipeline security, mapping data workflows, and understanding existing or potential vulnerabilities in LLM models and supply chains. Additionally, there is a need to request third-party audits, penetration testing and even code reviews for suppliers, both initially and on an ongoing basis.

The TEVV process is one specifically recommended by NIST in its AI Framework. This involves establishing continuous testing, evaluation, verification, and validation throughout AI model lifecycles as well as providing executive metrics on AI model functionality, security and reliability.

To ethically deploy LLMs, the checklist calls for the use of model and risk cards, which can be used to let users understand and trust the AI systems as well as openly addressing potentially negative consequences such as biases and privacy.

These cards can include items such as model details, architecture, training data methodologies, and performance metrics. There is also an emphasis on accounting for responsible AI considerations and concerns around fairness and transparency.

Retrieval-augmented generation (RAG) is a way to optimize the capabilities of LLMs when it comes to retrieving relevant data from specific sources. It is a part of optimizing pre-trained models or re-training existing models on new data to improve performance. The checklist recommended implementing RAG to maximize the value and effectiveness of LLMs for organizational purposes.

Lastly, the checklist calls out the use of AI red teaming, which is emulating adversarial attacks of AI systems to identify vulnerabilities and validate existing controls and defenses. It does emphasize that red teaming alone isn’t a comprehensive solution or approach to securing generative AI and LLMs but should be part of a comprehensive approach to secure generative AI and LLM adoption.