Network Trace
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

12) Does your organisation have processes in place to identify, triage and remediate the effects of AI model updates such as output accuracy or bias?

September 11, 2024
Artificial Intelligence
Service AI application

Answer yes if your organisation evaluates the effects of changes of the underlying AI Model, whether that model is created and maintained by you or is adopted and applied from an external source (e.g. Amazon Bedrock AI as a Service). Change impacts can include changes in output accuracy or bias and the potential need to reprocess historic data for analysis consistency. Please describe how you evaluate the effects of these changes or upload supporting documentation (as a PDF file).

What is the control?

Information Security is in large parts about risk management. We improve security by removing risks as best we can within a certain scope and level or resource.

AI model updates are a specific case where the effects of change may not be immediately apparent and should be practically evaluated.

Why should I have it?

Any significant change to your service delivery environment — including external factors such as changes to legislation, best practice guidance, emerging threats, and changes of processing scope — should be subject to an assessment to determine its impact. Few changes are potentially as significant as those as services which change information processing, such as the provision of AI models and services.

Where your clients have adopted use of AI-supported services, it is important to evaluate the risks of changes to the model as a result of training - or where options to update an adopted third party AI model are offered.

Changing a model can materially change the characteristic of outputs including processing result accuracy compared with past results.  Other changes may be unclear at the outset and only become apparent over time, such as ‘drift’ in accuracy or indications of apparent bias in the results of processing.

How to implement the control

Testing for the effect of AI model changes - to enable risk assessment -  may involve replay of past data (e.g. LLM RAG data with prompts) to compare and contrast the processing results. However, the identification of more subtle drift or bias effects may involve a longer period of either parallel running (the old model processing the same data and prompts as a proposed new model, with results retained internally for analysis) and statistical comparison of data-based processing results.

There are a growing number consultancies or individual consultants that will be able to assist in crafting a policy and process that meets your business and technical requirements. But bear in mind that as with all novel technologies this is a rapidly evolving area of academic research.

If you would like to contribute to this article or provide feedback, please email knowledge@riskledger.com. Contributors will be recognised on our contributors page.

Pattern Trapezoid Mesh

Defend against supply chain attacks with Defend-As-One.

No organisation is an island.