Two open toolboxes side by side, one nearly empty with a single basic tool and the other fully equipped, representing the gap between a token AI trial and a properly resourced one
AI-generated image

Why “We Gave Everyone Copilot” Isn't a Real AI Trial

PM
Peter Mangin
Founder, AI Innovisory
7 min read

In workshops and projects across New Zealand, one line comes up on repeat: "We've given everyone Copilot. They need to demonstrate value before we spend anything."

The logic is understandable. But usually, "everyone has Copilot" means everyone has the included, free Copilot Chat, which has limited capability relative to what the business then expects them to achieve with it. That decision shapes the outcome before the trial has even started.

Why do free AI tool trials fail to show real value?

Because the free tier is often not the tool the business is actually judging. Staff try it, find it underwhelming for real work, and stop, or they find value in a different tool and quietly keep using it rather than admit they stepped outside the approved environment. Either way, leadership sees low adoption and draws the wrong conclusion.

What Leadership Actually Sees

People try the free tool, find it underwhelming for the work they were asked to do, and stop. Or they find real value in a different tool entirely and keep using it quietly, reluctant to admit they've stepped outside the approved environment. Leadership sees low adoption. Governance loses visibility. Neither side gets the full picture, and the "proof" the business was waiting for never really tests the thing it meant to test.

We cover the second half of that pattern, the quiet, unapproved tool use that builds up underneath a low-adoption headline, in our guide to managing covert AI use. The two problems reinforce each other: an underpowered official tool pushes people toward unofficial ones, and the resulting adoption numbers make the case for spending even less.

Fund a Proper Trial Instead

How do you run an AI trial that actually tests the tool's value?

Choose a small group with clear opportunities, give them suitable tools and practical training with clear boundaries, start with work that involves no personal or commercially sensitive data, and measure time saved, quality, and rework rather than just usage counts.

There is a fairly straightforward alternative to handing out the free tier and waiting. Fund a proper trial:

  • Choose a small group with clear opportunities: marketers, researchers, knowledge workers spending real hours finding, interpreting, and producing information.
  • Give them suitable tools: that might mean paid Copilot, or another approved business tool with appropriate privacy and IP protections. The included free tier rarely is that tool.
  • Set clear boundaries: start with work involving public information, without personal or commercially sensitive data. Our five-question data checklist and data classification checklist are built for exactly this decision.
  • Measure the right things: time saved, quality, and rework, not just logins or licence uptake.
  • Make it safe to report what doesn't work: a trial that only welcomes good news isn't measuring anything.

The Paint Test

You wouldn't insist your designers use Microsoft Paint because the business runs Microsoft Office. Apply that same judgement to AI. Copilot Chat has useful applications, and for some roles it may be enough. But expecting one included tool to meet everyone's needs can leave substantial value unrealised, and NZ$0 in licence fees does not mean zero cost to the business. The cost shows up later, in a trial that quietly proved the wrong thing.

Where This Sits on the Adoption Ladder

This decision usually surfaces at the point organisations move from individual experimentation into something more deliberate, what we call the Augment stage of AI adoption: building daily fluency across a team, not just tolerating scattered individual use. Get the tool selection right here, and the case for the next stage, wiring AI into actual workflows at Automate, rests on evidence instead of a guess dressed up as a pilot.

Ready to run a trial that actually answers the question?

AI Innovisory helps New Zealand leaders design AI trials that test the real tool, with the right group, the right boundaries, and the right measures, through advisory and hands-on workshops.

Book a strategy call

Enjoyed this article?