Measure Copilot before the licences go out.

To decide whether Microsoft Copilot is worth keeping, time three to five recurring tasks and count the errors in them before any licence is assigned. Then measure the same tasks the same way on a review date you set in advance. Microsoft’s reports show who used Copilot and estimate the hours it assisted, but they don’t measure whether your work got faster or better, and neither did the Australian Government’s own trial.

Microsoft says Microsoft 365 Copilot is now Microsoft Copilot, and Microsoft 365 Copilot Chat is now Microsoft Copilot Chat (page updated 1 October 2026). Copilot Chat is included with eligible Microsoft 365 plans; fuller access inside the apps and to organisational data depends on licensing. So the real question is whether the add-on licence beats the Copilot Chat you already have.

What did the Australian Government trial find?

The Digital Transformation Agency coordinated a trial of Microsoft 365 Copilot across agencies from January to June 2024, and released the evaluation on 23 October 2024. Agencies nominated the staff who received licences; the trial wasn’t randomised. Evidence came from interviews, focus groups and surveys, including 831 post-use responses, 330 of them linked to a pre-use response (Executive summary, Methodology).

The Productivity chapter reports that 69% of post-use respondents agreed Copilot improved the speed at which they completed tasks, and 61% agreed it improved the quality of their work. Respondents estimated savings of around an hour a day on summarising, first drafts and information searches (Table 4). Quality gains were smaller, because editing “was almost always needed to tailor content for the audience or context”. Up to 7% said Copilot added time to some activities, partly because outputs had to be verified (Table 7).

Only a third of post-use respondents used Copilot daily (Employee-related outcomes, Table 3). The adoption chapter found that “Use of Copilot enabled some participants to access documents that they should not have had permission to access.” Among the unintended outcomes, participants worried that relying on Copilot could erode summarising and writing skills.

What couldn’t the trial measure?

The headline savings are participants’ own estimates, as the report’s limitations say:

The evaluation methodology relied on trial participants self-assessing the impacts of Copilot, which may naturally under or overestimate the benefits – particularly time savings.

The Productivity chapter adds that the hour-a-day figures “likely quote the upper bound of time savings Copilot could contribute”. The DTA’s evaluation briefing, last updated 25 October 2024, says the evaluation didn’t assess the cost-benefit ratio at a whole-of-government level, and:

The trial did not benchmark or technically evaluate the accuracy of Microsoft 365 Copilot’s outputs.

So the trial measured perceived productivity, not how long work took or how often outputs were wrong. The briefing says agencies may choose to run cost-benefit evaluations specific to their operating environment; a firm can do that on a handful of its own tasks.

What do Microsoft’s Copilot reports show?

Of the four reporting sources Microsoft lists, two matter for a licence decision.

• The usage report in the Microsoft 365 admin centre (updated 18 August 2026) counts enabled users, active users, prompts and active days per user. An active user is a licensed user who tried a user-initiated Copilot feature at least once in the period; opening the Copilot pane doesn’t count. People without the add-on appear in a separate Copilot Chat usage report.
• The Copilot Dashboard in Viva Insights (updated 29 September 2026) needs no paid Viva Insights licence to view. Its Copilot assisted hours figure is an estimate built from multipliers derived from Microsoft’s research: a summarised meeting counts for its length, and each search, summary or creation action counts as 6 minutes.

These are broad approximations based on the best available research rather than precise calculations.

The dashboard turns those hours into a value using a default hourly rate drawn from US labour statistics. The business impact report relates usage to outcome data you upload, but needs Viva Insights licences, and Microsoft recommends at least 65 Copilot and 65 non-Copilot users. It also warns the differences shown “have not been evaluated for statistical significance”.

Question	Where the answer comes from
Who has a licence, and how much do they use it?	Usage report: active users, prompts, active days
How many hours were saved?	Dashboard: a modelled estimate. Only your own timing measures it
Are the outputs right?	No report. A reviewer’s check
Should we keep paying?	A decision rule written down first

How do you measure Copilot ROI in your own firm?

1. Pick three to five recurring tasks with a clear finish, such as turning a client meeting into file notes, drafting a standard letter from a precedent, or summarising a correspondence thread. Include one task you suspect Copilot handles poorly.
2. Baseline time and errors. For at least a week before licences go out, the people who’ll use Copilot log start and finish times, and the reviewer records every correction before approval.
3. Time to an approved output. The clock stops at sign-off, not when a draft appears. The trial found checking and editing ate into the savings.
4. Keep a comparison group if you can. Give the add-on to some staff and leave others doing the same task on Copilot Chat. Microsoft’s business impact guidance notes that seasonality can move results; the comparison group shows what changed anyway.
5. Set the review date now. Day 30 after training gets past novelty and leaves time to act.
6. Write the decision rule first. For example: keep a role’s licence if median time to an approved output falls by the margin you agreed, with no rise in corrections. Otherwise reassign it or drop it.

Cost the measured hours at your own staff rates and set them against the licence: that’s the return, in your numbers. A flat result from someone with few active days in the usage report is an adoption problem, not a verdict on the tool.

For tax practitioners, the TPB’s TPB(GS) 55/2026, issued on 22 July 2026, says AI-generated content should be verified at each step of the workflow, with each step documented (paragraph 16). Entering client information into AI tools can be a disclosure needing each client’s permission, depending on how the tools are configured and used (paragraph 23). This isn’t legal advice; check the guidance itself. Lawyers can start with what the courts’ practice notes ask.

What should you fix or switch off before rollout?

Microsoft says Copilot retrieves data from Microsoft Graph and respects existing permissions, sharing settings and policies, so a badly shared folder becomes an easily found one. The same page, updated 16 July 2026, warns:

By default, SharePoint sets sharing settings to the most permissive option.

Before the first licence:

• Run the sharing reports. SharePoint’s data access governance reports include a site permissions baseline, an Everyone except external users report and sharing links activity. SharePoint Advanced Management is included with the Copilot licence.
• Hide high-risk sites. Restricted Content Discovery keeps a site out of Copilot and organisation-wide search without changing its permissions.
• Follow the setup checklist. Microsoft’s setup guide lists pilot testing with a select group, multifactor authentication, unified audit logging and turning off Everyone except external users at the tenant level.
• Decide the meeting setting. In Teams, the default policy is On with saved transcript required, which organisers can’t change (updated 11 August 2026). Under the On setting, organisers default to using Copilot only during the meeting, from speech-to-text data that isn’t saved. Some trial participants felt pressured to consent to transcription.
• Decide on the value card. The dashboard’s assisted value is on by default. A Global admin can turn it off so a modelled figure isn’t mistaken for a result.

A practical rollout plan

• This week: pick the tasks and reviewer, and write the decision rule.
• Next week: log the baseline and fix sharing.
• Then: license and train the trial group, and keep the comparison group on Copilot Chat.
• Day 30: measure the same tasks the same way and apply the rule.

Before buying anything, check what your current plans already include. The AI Switch-On method follows the same pattern: a one-week time log from two or three staff before anything changes, and the same measure at day 30.