What is an AI incident?

The distinction matters because it sets the threshold for when your incident process should run. A model that starts drifting is a hazard. The same model approving loans it should have declined, or rejecting a group of applicants it should not have, is an incident.

What counts as an AI incident

An AI incident is not only an outage. A system that keeps running while producing wrong, unfair or unsafe results is often the more serious case, because it can go unnoticed for longer. The events that usually meet the threshold include a system making decisions that harm people or breach their rights, a model behaving in a way its owners did not intend or cannot explain, an AI system exposing or misusing personal information and an AI tool being used outside the scope it was approved for.

A near miss belongs in the same conversation. If a system almost produced a harmful outcome and was caught in time, that is a hazard worth recording, because the record is what lets you fix the weakness before it becomes an incident.

Why an AI incident is different from an ordinary IT incident

An ordinary IT incident is usually about availability. The system is down, and the response is to bring it back up. An AI incident can happen while everything is technically working. The model is available, the pipeline is green and the harm is in the output.

That changes what you need to record. For an IT outage the useful record is largely technical. For an AI incident the useful record also covers the decision the system made, the data and version behind it, who was affected, what a human did about it and why. This is the record a board or a regulator asks to see, and it is rarely reconstructable after the fact if you did not keep it at the time.

Why the record matters

The Voluntary AI Safety Standard asks organisations to monitor AI systems once they are deployed, not only to test them before release, so that unexpected behaviour is caught rather than assumed away. Monitoring is what surfaces an incident. The record is what lets you show, later, that you saw it, understood it and acted.

This is where incident handling connects to the wider assurance story. An incident is the moment when being able to show what happened, and what you did about it, carries the most weight. A record that someone outside the organisation can verify carries further than one they are asked to trust.

How Aicura fits

Aicura’s Incidents surface is where you record an AI incident and the response to it, so the account of what happened lives alongside the register entry for the system involved rather than in a separate document that drifts out of date. When you seal that record in the Evidence Vault, the snapshot is cryptographically anchored, which means its contents can be verified as unchanged by someone outside Aicura without taking Aicura’s word for it. That is the line on the Vault, evidence you don’t have to trust us for.

Aicura helps you record and evidence the response. It does not adjudicate the incident or certify that you handled it well. Those judgements stay with you and your accountable owner.

To work through a response step by step, read how to respond to an AI incident. To keep the record so it holds up later, read building the evidence trail for an AI incident.

A note on this page

This is general information about AI incidents, not legal advice. The primary source for the definitions above is the OECD report “Defining AI incidents and related terms” (OECD, 2024). For the expectation to monitor deployed systems, see the Voluntary AI Safety Standard published by the Department of Industry, Science and Resources (August 2024). Read the primary sources, and take your own advice on how they apply to your organisation.