Skip to content
NL Plan a start sprint

Compliance and security

Human oversight and logging: what the AI Act asks and how to build it

David Homan · 3 min read

The short answer

For high‑risk AI, the AI Act requires that people can understand and monitor the system and, where needed, step in or override its decision, and that the system automatically records what it does. For the applications in Annex III that applies from 2 December 2027. For other AI it is wise too: it makes mistakes visible and keeps responsibility with people.

What does the law ask?

For high-risk AI, the AI Act sets two requirements that end up in the design of a system:

  • Human oversight (Article 14). The system must be built so that people can understand it, monitor it and step in where needed. They must be able to disregard or reverse an outcome, and to stop the system. Organisations that use the system appoint people with the knowledge, authority and time to do this (Article 26).
  • Logging (Article 12). The system must automatically record what it does, so you can trace afterwards what happened. Deployers keep those logs for at least six months.

For the high-risk applications in Annex III these requirements apply from 2 December 2027; for AI in regulated products from 2 August 2028. Those dates were set in 2026 by the Digital Omnibus. We check them every quarter.

What that means in a system

Human oversight is a design choice. These are the patterns we use most:

  1. A person approves. The system makes a proposal; a member of staff sends it or decides. Suits anything that goes out or affects a person.
  2. Hand over when in doubt or urgent. The system handles standard cases and passes difficult, urgent or sensitive cases straight to a member of staff.
  3. A confidence threshold. Below a set confidence level the case goes to a person. You adjust the threshold based on the logs.
  4. An emergency stop. One button that halts the automated processing, while the manual work carries on.
  5. An explanation with each outcome. The system shows what an outcome is based on, so the person checking it can see whether it is right.

What to log, and for how long

A good log answers four questions afterwards:

QuestionWhat you record
What went in?The input, or a reference to it
What came out?The outcome and, where possible, the confidence or the sources
What happened to it?Approved, edited, handed over or overridden, and by whom
When and with which version?Time, version of the system and of the model

For high-risk AI you keep the logs for at least six months. If they contain personal data, the GDPR also applies: no longer than necessary, and well secured. Read more in AI and the GDPR.

An example from practice: the Medux AI voice

For Medux we built an AI voice that handles phone calls: answering, booking appointments and responding to questions. The figures: 4,000 calls a month, 80% handled by the AI, difficult or urgent questions passed straight to a member of staff.

This is what human oversight looks like in practice: the AI for the standard work, a person where it matters, and every call transcribed for quality and review. Read the case.

Adding it later or building it in from the start

Adding oversight and logging afterwards is expensive. You have to find decision points in a system that was not designed for them, and reconstruct logs that do not exist. If you include it from the start, it becomes part of the application, and you can adjust it later based on what the logs show.

What AI Heroes does here

In every build we decide the risk class and the decision points during the start sprint. Hand-off to a person, logging with fixed retention periods and an emergency stop are in the system from the first version. That is what we mean by compliance from the start.

Sources

Frequently asked questions

Is human oversight always mandatory?

Legally only for high‑risk AI. For the applications in Annex III, such as AI in recruitment or credit scoring, it applies from 2 December 2027. For other applications it is a design choice, and often a sensible one.

What should you log in an AI system?

At least when the system ran, on which input, what the outcome was and who checked or overrode it. For high‑risk AI the system must be able to record this automatically.

How long do you keep logs?

For high‑risk AI, deployers must keep the automatically generated logs for at least six months, unless other legislation requires otherwise. If the logs contain personal data, under the GDPR you also keep them no longer than necessary.

Does human oversight mean someone has to check everything?

No. It means a person can follow the system and step in. How you set that up depends on the risk: from spot checks at low risk to mandatory approval for important decisions.

Who is responsible if the AI makes a mistake?

The organisation that deploys the system, and within that organisation the people tasked with oversight. The AI Act asks that they have the authority, knowledge and time to actually exercise that oversight.

Building an AI application that prepares decisions? In every build, oversight and logging are built into the design from the start. Start with a start sprint.

Go to compliance and security

Further reading