Guide

How to evaluate an AI governance platform

Vendor feature lists in this category all use the same words. These are the questions that reveal what each product actually is, on enforcement, evidence, discovery, agents, frameworks and buying, asked of any vendor including Aicura.

How should we evaluate an AI governance platform?

Evaluate against questions that expose what a product actually does rather than what its feature list claims, because the category's vocabulary has converged while the products have not. The questions that separate products fastest are these. Does enforcement act on live traffic or is it approval workflow, can a third party verify the audit trail without access to the vendor's systems, does discovery correlate real signals from identity, device, network and finance sources or rely on self-reporting, does agent governance constrain what an agent may do at runtime or only record what it is, is the framework content you are obliged to meet actually loaded as usable data, and can your organisation buy and start without a six-figure enterprise sales process. Ask every vendor the same questions and make them show you rather than tell you.

Every vendor in this category claims discovery, assessment, enforcement, monitoring and audit. The words have converged because the analyst category has, but underneath the words are three different product lineages with different strengths, and a feature checklist cannot tell them apart. Questions can. This guide sets out the ones that expose the differences fastest, and they work on any vendor, including Aicura.

On enforcement: does it act on traffic, or on paperwork?

Ask: when a prompt violates policy at 2pm on a Tuesday, what happens to that prompt? For most of the category the answer, once unwound, is that nothing happens to the prompt, because “enforcement” means an approval workflow that ran before deployment or a review task created after the fact. Real runtime enforcement sits in the request path and blocks, redacts or constrains the request itself. Both models are legitimate, but they are different products, and a vendor should be able to say plainly which one they are. Follow up by asking to see an enforcement event fire live, and where the record of it lands.

On evidence: who has to be trusted for it to hold?

Ask: can a third party verify your audit trail without access to your systems? The category’s standard words here are immutable, tamper-evident and system of record, and they almost always describe an append-only log inside the vendor’s own infrastructure. That is a promise about the vendor’s database, and verifying it requires trusting the vendor. Cryptographically verifiable evidence is different in kind, using hashes anchored externally, timestamps from an external service, signatures with external keys and a verifier the third party can run themselves. If a vendor’s answer to the question is a description of access controls, you have your answer. The companion guide on what makes AI evidence independently verifiable covers the distinction in full.

Then ask the continuity version: if your company disappears in five years, what happens to our evidence? The answer reveals whether the record’s durability depends on the vendor’s survival.

On discovery: signals or self-reporting?

Ask: name the sources your discovery actually reads, and show us a finding traced from signal to register entry. Discovery ranges from a form teams fill in, to scans of a single surface, to correlation across identity providers, device management, network egress, cloud inventories and finance systems. The width matters because shadow AI does not announce itself in any single source, and the correlation matters because raw hits without resolution just move the work. Ask what percentage of findings in a reference deployment came from sources other than self-reporting.

On agents: is the governance a record or a constraint?

Ask: can your platform stop an agent doing something, or record that it did? Agent registries are now common. Runtime constraint of tool permissions and action scope, suspension of a misbehaving agent and a decision log of what an agent actually did are much rarer, and they are what the risk of systems that act calls for. Ask to see an agent’s permitted tools changed and the change take effect, and ask what record exists when someone later asks why an agent did something.

On frameworks: loaded data or a logo wall?

Ask: for the frameworks we are obliged to meet, show us the content operating inside the product, not the logo on the website. Framework coverage claims range from content genuinely wired into assessments and reviews, to mappings maintained on request, to a list. For Australian organisations the question bites hard, because most of the category’s content is EU and US and Australian-specific AI governance content is scarce. Ask to see an assessment run against the framework that matters to you, on your systems.

On buying: can you start, or only be sold to?

Ask: what is the smallest real commitment, and can we make it without a sales process? Most of the category is enterprise-only, with mandatory sales engagement and floors from the mid five figures upward. That is a fine model for some buyers and a wall for others. A free or low-floor entry means the evaluation can be your own hands on your own systems rather than a scripted demo, which improves every other question on this list.

Running the evaluation

Ask every vendor the same questions, in writing, and make the shortlist demonstrate rather than describe. One real piece of work end to end beats any demo, so register real systems, run a real assessment against your real framework, fire a real guardrail, produce a real evidence pack and hand it to someone sceptical. Where a vendor is weak, listen for whether they say so plainly. A vendor who concedes a real limitation and explains the mitigation is telling you more about the next five years of the relationship than one who has no weaknesses at all.

Where Aicura fits

Aicura publishes its answers rather than saving them for a demo. Enforcement is in-line, through an AI gateway and a guardrail compiler producing signed bundles, with every intervention recorded as an enforcement event. Evidence is anchored with Merkle trees, externally timestamped, signed with external keys including post-quantum schemes and checkable with the open-source evverify verifier, so the record does not depend on trusting Aicura or on Aicura’s survival. Discovery correlates identity, device, cloud, ITSM, finance and workspace signals into suggestions a person decides on. Agents live in the same AI Register as systems, with runtime constraints, suspension and re-scoping, and decision logs on Enterprise. Australian framework content is loaded as data, with EU AI Act, UK, Singapore and US state content available on Enterprise. And the floor is a free Register tier with self-serve billing.

Aicura provides guidance, not legal advice, and does not certify, audit or issue a compliance verdict. It issues no compliance verdicts or scores by design, so if your evaluation requires a product that asserts pass or fail, that is a real difference to weigh, and we would rather say so here than surprise you in month three.

Common questions

Should we just shortlist from an analyst quadrant? Quadrants are useful for drawing the category boundary and knowing who is in scope. They are less useful for fit, because placement rewards scale and breadth while your decision turns on depth in the two or three areas your organisation actually needs. Use the analyst view to build the longlist and these questions to cut it.

How much should a proof of concept prove? One real piece of work end to end, with your systems and your people. Register real systems, run a real assessment, fire a real guardrail, produce a real evidence pack. A demo on the vendor’s data proves the demo works.

What if a vendor answers every question well? Be more suspicious, not less. The lineages are real, and depth everywhere is not how this category has developed. Press hardest on the areas furthest from the vendor’s origin, and ask them to show rather than describe.

How does Aicura answer these questions? In-line enforcement through a gateway and guardrail compiler with recorded enforcement events, evidence anchored with Merkle trees and checkable with an open-source verifier without trusting Aicura, discovery correlated from identity, device, cloud, ITSM and finance signals, agents governed with runtime constraints in one register with systems, Australian framework content loaded as data with EU, UK, Singapore and US state content on Enterprise, and a free Register tier with self-serve billing. And we would rather you asked us these questions in a walkthrough than took a webpage’s word for it.

A note on this page

This guide is written by a vendor in the category it describes, so read it the way you would read any vendor’s evaluation advice, as a set of questions to ask everyone, us included. It is general information, not advice, and the right weighting of these questions depends on your organisation’s obligations and risk.

Related guides

Ask us the hard ones

The fastest way to evaluate Aicura is to bring this list to a walkthrough and make us show you each answer against your own scenarios. We built the product to survive exactly these questions.

Talk to us