How to respond to an AI incident

The stages below assume you already know what counts as an incident. If that threshold is not settled in your organisation, start with what is an AI incident, because a response process only runs if something triggers it.

Before an incident: be ready for one

A mature response depends on decisions you make before anything goes wrong. Agree who the accountable owner for AI incidents is, and who they can call on. Decide what severity levels you use and what each one commits you to. Write down the point at which an incident becomes reportable to a regulator or to an affected person, so that call is not being made under pressure for the first time. Keep this alongside your AI register, so that when a specific system is involved you already know who owns it and what it does.

Triage the event

When something is reported, the first task is to decide whether it is an incident, a hazard or noise, and how serious it is. Record what was seen, which system is involved, when it started and who raised it. Assign a severity using the levels you set in advance. Triage is also the point where you start the record, because the earliest facts are the ones most likely to be lost.

Contain the harm

Containment is about stopping the incident getting worse while you work out what caused it. Depending on the system, that can mean pausing it, routing decisions to a human, rolling back to a known-good version or narrowing what the system is allowed to do. Note what you changed and when, because a containment action taken quickly is easy to forget and important to be able to explain later.

Investigate the cause

With the harm contained, work out what actually happened. Look at the decision the system made, the input data behind it, the model version in use and any recent change to the system or its data. The aim is a cause you can state plainly and evidence, not a guess. Keep the working, including the versions and data you examined, so the conclusion can be stood behind rather than merely asserted.

Remediate

Remediation is the fix, both the immediate one and the durable one. The immediate fix restores safe operation. The durable fix addresses the cause so the same incident does not recur, which might be a change to the model, the data, the controls around the system or the approval it was running under. Record what you changed and the reasoning, so the link between the incident and the fix is visible.

Notify where you are required to

Some AI incidents carry notification duties. Where an incident involves personal information and amounts to an eligible data breach that is likely to result in serious harm, the Notifiable Data Breaches scheme requires you to notify the OAIC and the affected individuals. Other duties can arise from your sector’s rules or your contracts. Work out early whether a duty applies, because these obligations are time-bound, and record the assessment you made and the basis for it whether or not you end up notifying.

Review and close

After the incident is contained and fixed, review it. Confirm the cause, confirm the fix held and capture what the incident revealed about the system and about the process. This is where near misses earn their place, because the lesson from an event that was caught in time is as useful as the lesson from one that was not. Close the incident with a record that a person who was not in the room could follow.

How Aicura fits

Aicura’s Incidents surface is where each of these stages is recorded, against the system in your register that the incident involves, so the response and the system it concerns stay together. When you seal the incident record in the Evidence Vault, the snapshot is cryptographically anchored, so its contents can be verified as unchanged by someone outside Aicura without taking Aicura’s word for it, which is what makes it evidence you don’t have to trust us for.

Aicura helps you record and evidence the response. It does not adjudicate the incident, decide its severity or certify that you handled it well. Those calls stay with you and your accountable owner. To keep the record so it holds up when someone asks to see it, read building the evidence trail for an AI incident.

A note on this page

This is general information, not legal advice. The response shape above follows the NIST AI Risk Management Framework (AI RMF 1.0, January 2023), in particular its MANAGE 4.3 outcome on tracking, responding to, recovering from and documenting incidents. For notification duties involving personal information, the primary source is the Notifiable Data Breaches scheme under Part IIIC of the Privacy Act 1988 (Cth), administered by the OAIC. Read the primary sources, and take your own advice on how they apply to your organisation and your obligations.