industry
Evaluate AI tools before relying on their outputs.
Health systems differ, but the questions about AI documentation tools are converging. The World Health Organization's 2024 guidance on large multi-modal models warns of automation bias in health professionals and patients. The good machine learning practice principles, first published by national regulators, ask for testing in clinically relevant conditions, attention to how clinician and AI perform together, and monitoring once a model is deployed. In the EU, AI that is a medical device, or a safety component of one, is high-risk from 2 August 2028 (as amended in July 2026) where the device needs third-party conformity assessment. Each country's regulator decides when an AI scribe becomes a medical device. We don't make that call; we measure what the scribe gets wrong, and your regulatory team decides.
Which rules apply to you is your compliance or legal team's call. We map our evidence to them; we don't certify against them.
In Australia, a scribe arrives through a vendor demo or a software update, and what it leaves out of notes in your specialties hasn't been measured.
Each AI tool is tested on your own scripted consultations before rollout and after each update, with omissions and additions counted.
Vendor AI Evaluation before you sign
Scribe & Clinical AI Evaluation
Model-Change Regression after vendor updates
Non-clinical Policy & Procedure Assistant
Credentialing and referral intake registers
Administrative documents only.
Sources
Lead solutions
The health form of the Independent Evaluation Report: selection against the regulator and safety guidance that applies to you (in Australia: TGA, Ahpra and ACSQHC), local validation on your own specialties, omission audits and a monitoring plan for after go-live. We evaluate these products; we never build them.
1–3 vendor tools tested before you sign, side by side, on your own content and questions, in accounts your organisation controls.
One policy and procedure library for one staff group, delivered inside Microsoft 365 (Copilot, SharePoint or Teams) where you license it, otherwise on the web. Answers cite the policy, section and version, and say "not in our documents" when the library is silent.
Reads credentialing documents from outside the national practitioner register (Ahpra in Australia), such as college fellowships and scope-of-practice letters, or referrals that arrive as PDFs or faxes, into a register a person checks. Administrative only: a person makes every credentialing or referral decision.
If your staff have Microsoft Copilot licences, SharePoint's built-in AI can already answer from a policy library within each user's permissions. Try it first; we'll say whether it's good enough.
Illustrative scenario: not a client engagement
Problem
A fictional day-surgery group licensed a scribe after a GP-based demo. Its specialists say details go missing, and an update added next steps nobody asked for.
Approach
Clinicians role-play scripted consultations and write reference notes. Two clinicians label what the scribe leaves out or adds, and a scope test checks for any diagnosis, test or treatment never said.
How we'd measure
What ships first
The week-0 threshold sheet and the scripted consultation set.
Once the scripted consultations and thresholds are agreed in week 0, and before testing starts, you decide whether to continue. If you stop, you keep the test design.
Australia is our home market and the one we know best.
Works alongside
| Obligation | What it means for an AI system | How we design for it | Evidence you get | Source |
|---|---|---|---|---|
| TGA Digital scribes guidance: a scribe that analyses or interprets is a medical device | A scribe intended only to transcribe or translate a consultation into a written record, without analysis or interpretation, is not a medical device. One that analyses or interprets, for example by generating a diagnosis, differential diagnosis or treatment recommendation the clinician didn't state, is a medical device and must be in the ARTG before supply. A software update can move a product across that line. | The scribe evaluation includes a scope test for any diagnosis, test or treatment the clinician never said, re-run on each vendor update. | A scope-test log with the product version tested | Source, as at 30 January 2026 |
| ACSQHC AI Clinical Use Guide v1.0 and Ambient scribe scenario: 'while you use' and 'after you use' steps | The guide and scenario ask clinicians to review scribe summaries for hallucinations, over-summarisation and missing information, label records AI helped create, and take part in regular reviews of how the scribe performs. | We count omissions, additions and errors separately, and hand over the test set so your clinicians can re-run it. | An evaluation report and the re-runnable test set | Source, as at 1 August 2025 |
| Ahpra and National Boards AI guidance: Accountability and Informed consent principles | The practitioner must apply human judgement to any AI output and check the accuracy of records a scribe creates, and generally needs the patient's informed consent before a scribe records. Ahpra says tools should be tested by the user or organisation for fitness for purpose before clinical use. | Testing happens before clinical use, against thresholds your clinicians agree in week 0. It never replaces a clinician's review of each note. | A pre-use test report with the agreed thresholds and every failure | Source, as at 22 August 2024 |
| Privacy Act s 6D(4)(b) and OAIC guidance on commercially available AI products | Private health service providers that hold health information are covered by the Privacy Act at any turnover, so AI that handles health information is subject to the APPs. The OAIC expects due diligence on whether a product was tested for your use, and regular performance reviews rather than 'set and forget'. | Evaluations run on scripted or consented material inside your environment, and we record where each processing step runs. | A one-page data note naming where each step runs | Source, as at 17 January 2025 |
Last reviewed
General information, not legal advice.
Regulation (EU) 2026/1744 moved Annex III high-risk duties to 2 December 2027 and Annex I to 2 August 2028. What applies now to AI suppliers outside the EU.
From 10 December 2026, privacy policies must name decisions a computer program makes or substantially assists. How to find yours, licensed software included.
The TGA, ACSQHC and Ahpra ask for AI scribes to be checked before use and reviewed after updates. How to test one on scripted consultations in your specialties.
Let’s discuss your workflow.
Talk to us about AI for healthcare