What counts as AI audit evidence

What an auditor means by evidence

ISO 19011 describes audit evidence as records, statements of fact or other information that are relevant to the audit criteria and verifiable. Two words in that do the work. Relevant means the evidence actually bears on the control being tested, not adjacent material that looks reassuring. Verifiable means the auditor can confirm it, rather than taking it on the organisation’s word. A great deal of what organisations offer as governance evidence is relevant but not verifiable, a document that proves its own contents but not when it was written or whether it has changed, and that shortfall is where audit findings come from.

The qualities that make a record count

A record that stands up as evidence has four qualities. It is relevant to the specific control. It is sufficient, meaning there is enough of it to support the conclusion rather than a single artefact standing in for a practice. It is attributed, so it is clear who did the work and who owned the decision. And it is reliable, meaning the auditor can trust its integrity, that it is the record it claims to be and has not been altered since. A record can be present and still fail on any of these, and the one organisations most often miss is reliability, because a file that could have been edited last night carries less weight than one that can be shown unchanged since its date.

Common forms of AI audit evidence

For an AI management system, the evidence an auditor asks for tracks the controls you claim. It includes the register of AI systems in scope, the impact assessment behind each material system, the record of the owner and the accepted risk, the log of a governed agent’s actions, the incidents recorded with what was done about them and the reviews carried out over each system’s life. Guardrail 9 of the Voluntary AI Safety Standard asks you to keep records that let a third party assess your governance, and this is the set of records that guardrail is describing. Each one has to be the record captured when the work happened, not one reconstructed for the audit.

Why integrity is the hard part for AI evidence

The reliability of AI evidence is harder than it looks, for a reason specific to AI. These systems change without a formal release, so the date on a record matters, an assessment written before a model was retrained does not evidence the system as it runs now. An auditor reading a governance record wants to know when it was captured and whether it has been touched since. A record held in a way that proves its history answers that question. A record in a shared drive does not. This is why mature functions move their governance evidence into a form where its integrity can be shown, rather than asserted.

Evidence that fails, and why

• A policy offered as evidence that a control operates. The policy is the intention, not the operation.
• A single artefact standing in for a whole practice. One assessment does not evidence a program.
• A record with no owner. An unattributed document answers a weaker question than a signed one.
• A record that cannot be dated or shown unchanged. The auditor cannot rely on what it cannot verify.
• Evidence assembled the week before the audit. Reconstruction reads as reconstruction.

Frequently asked questions

Is a policy document audit evidence? A policy is evidence that a control was designed, not that it operated. An auditor testing whether a control works asks for the records the control produced when it ran, the assessments, the approvals, the reviews and the logs, not only the policy that describes it.

Does the evidence have to be in a particular system? No. Evidence can live anywhere, provided it is relevant, sufficient, attributed and reliable. What matters is whether the auditor can trust it and confirm it. The reason organisations move governance evidence into a purpose-built place is to make the reliability and the attribution easy to show rather than argue.

What makes a record verifiable? That an outside reader can confirm it independently, in particular that it is the record it claims to be and has not changed since capture. A record whose integrity can be checked without relying on the organisation’s assurance is verifiable in the sense an auditor means.

Does Aicura decide whether your evidence is sufficient? No. Aicura holds the records and makes their integrity and attribution easy to show. It does not judge whether the evidence is sufficient, test your controls or form an audit conclusion. The auditor evaluates the evidence and the accountable owner stands behind it.

Where Aicura fits

Aicura is your AI Register, and the register is the first piece of evidence an auditor asks for, the population of AI systems in scope, current rather than reconstructed. Impact Assessments, the agent records in the AI Register and Incidents hold the assessments, the agent action records and the incident records against the systems they concern, each attributed to the person who did the work, which covers relevance, sufficiency and attribution.

The Evidence Vault covers reliability. It seals each record so it can be shown unchanged since capture, evidence you don’t have to trust us for, cryptographically anchored and verifiable without Aicura in the loop, so the record’s integrity is something the auditor confirms rather than accepts on your word. The Trust Centre shares that evidence with the auditor directly, and Attestation lets the accountable owner sign against it. Aicura holds the records and makes their integrity easy to show. It does not judge sufficiency, test your controls or form a conclusion. The auditor does that.

For the related work, read what is AI assurance and how to keep AI governance records that stand up. When an audit is coming, how to prepare for an AI governance audit sets out the preparation. The AI governance software overview sets out what Aicura supports.

Sources

• ISO 19011:2018, Guidelines for auditing management systems, International Organization for Standardization, iso.org
• ISO/IEC 42001:2023, AI management system, International Organization for Standardization, iso.org
• Voluntary AI Safety Standard, Department of Industry, Science and Resources, industry.gov.au

A note on this page

This explainer is general information on what counts as AI audit evidence against current Australian and international guidance. It is not legal advice and it does not tell you whether particular evidence satisfies a particular audit or obligation. For how this applies to your organisation, read the primary sources above and take your own professional advice.