AI Governance, and the Checklist Nobody Wrote Down Until It Was Too Late

By The Demand Co.ai | Signal | 8 min read

A company we know found out their customer support team had been pasting client contracts into a public AI chatbot for months, trying to summarize them faster. Nobody told them not to. Nobody told them to either. There was no policy telling them not to, so they just did the thing that saved them twenty minutes. Nobody found out until a client’s lawyer asked, mid-call, exactly where their contract had been pasted in the last few months. The room went quiet.

That’s not a one-off either. Samsung’s engineers did basically the same thing in 2023, pasting actual source code into ChatGPT because it was the fastest way to debug something. The company found out, panicked, and banned generative AI outright. The tool wasn’t really the problem in either case. The absence of a single written rule was.

We’re writing this because most AI governance content out there is either a 40-page framework written for a compliance department that nobody outside legal will ever read, or a LinkedIn post insisting you need “responsible AI principles” without saying what that actually means on a Tuesday. Neither one tells a leader what to actually do this quarter. So here’s the honest version — what AI governance actually requires, and a real checklist you can use instead of a values statement nobody will reference again.

First, the thing that will humble you immediately

AI governance isn’t a document. It’s the set of questions someone has to actually ask before a tool goes live, every time, not just once.

That distinction sounds small until you see what happens without it. Air Canada found this out in a small claims court in 2024. Their chatbot told a grieving passenger the wrong thing about bereavement fares, and when it got challenged, the airline actually tried arguing the chatbot was its own legal entity, separate from the company, somehow responsible for its own words. The judge was not amused. Air Canada ended up honoring the bad advice their own tool had given. Nobody had decided in advance who was accountable when the chatbot got something wrong. The lawsuit decided it for them.

The companies handling this well made one decision early that most others skip: before any AI tool touches a customer or a piece of sensitive data, someone specific is named as accountable for what it does, and there’s a written answer to what happens when it’s wrong. Without that, you don’t have an AI governance problem. You have an accountability vacuum that an AI tool happens to be standing in.

The version of this conversation that made us want to leave the room

There’s a version of the AI governance conversation where the answer is just “build an AI ethics committee” and consider the problem solved. We’ve sat in meetings where that was the entire plan — a quarterly committee, a slide deck of principles, no actual process for what happens when someone in marketing wants to connect a new AI tool to the customer database next Tuesday. It made us want to leave.

The reality, reported by the companies who’ve actually had something go wrong, is that governance failures rarely happen at the committee level. They happen in the gap between the committee meeting and the actual Tuesday when someone needed an answer and didn’t have one, so they just proceeded. A mid-size insurer we’re familiar with had a beautifully written AI principles document and zero process for reviewing a new tool before procurement signed the contract. The document sat in a shared drive. The actual decision got made in a vendor call nobody from governance was on.

That gap between the policy and the moment someone needs it? That’s where the real risk lives. A committee that meets quarterly can’t help someone making a decision on a Tuesday afternoon.

Where the actual leverage is hiding

Let’s get specific, because “establish responsible AI governance” as a phrase tells nobody what to do before lunch.

Know what AI tools are actually in use, not what you think is in use. Most companies are surprised by their own AI footprint when they actually audit it — tools adopted by individual teams, browser extensions installed without IT’s knowledge, AI features quietly turned on inside existing software they already pay for. The starting point isn’t a policy. It’s an honest inventory, because you can’t govern what you don’t know exists.

Classify by what the AI touches, not by how impressive the tool sounds. A tool that drafts internal meeting notes carries a different risk than one that makes a lending decision or talks to a customer. The companies doing this well sort their AI use by the actual stakes involved — does it touch personal data, does it make a decision that affects someone, does it represent the company publicly — and apply review proportional to that, instead of treating every tool with the same heavy process or, worse, the same total absence of one.

Assign a named owner to every tool, not a department. “IT owns it” or “marketing owns it” means nobody owns it the day something goes wrong. The companies handling this well name an actual person accountable for each AI tool in active use — someone who knows what data it touches, who approved it, and who gets called first if it misbehaves.

Build the review into procurement, not after launch. The EU AI Act, now in active enforcement, requires risk assessments for AI systems before deployment in certain categories, and similar requirements are showing up in U.S. state-level legislation. Companies bolting a review process onto a tool that’s already live are doing the hardest version of this work. The ones building review into the procurement and launch process itself are doing the easy version, because the questions get asked before anyone’s emotionally and financially committed to the tool already being live.

The actual checklist

Skip the framework. Here’s what a real review needs to answer before any AI tool goes live, for any team:

• What data does this tool see, and is any of it customer data, employee data, or anything regulated?
• Who is the named, individual owner of this tool, not a department?
• What happens if it gives someone wrong information, and who is accountable for that outcome?
• Can a human override or stop it, and does anyone actually know how?
• Has anyone outside the team that wants this tool reviewed it?
• And six months from now, would anyone actually notice if it started being used for something it was never approved for?

If a team can’t answer all six on the spot, that’s the answer. The tool isn’t ready, no matter how good the demo looked.

The part that will actually trip you up

Here’s the implementation reality that the responsible AI conference circuit skips over.

A governance process that takes six weeks to approve a tool will get bypassed, quietly, by teams under deadline pressure who go around it rather than through it — which is functionally worse than having no process at all, because now there’s a false sense of control. The companies getting this right built a fast lane for low-risk tools, reviewed in days, and reserved the slower, deeper process for the tools that actually touch sensitive data or make consequential decisions. Speed and rigor aren’t opposed here either, the same way they weren’t in the security conversation. Unmanaged speed is the actual problem.

The organizations doing this well also treat governance as something that has to be re-checked, not approved once and forgotten. A tool approved for internal meeting notes can quietly start handling customer emails six months later, with nobody re-reviewing it because the original approval is still technically valid.

And then there’s the trust problem between the teams who want to move fast and the people asking the governance questions — which is where this ultimately lives or dies.

The person in legal or compliance asking who’s accountable if the AI gets something wrong isn’t your blocker. That’s the person who has actually read what happened to Air Canada and doesn’t want your company to be the next case study. Don’t route around that instinct — build a process fast enough that nobody feels the need to. The companies that figured this out early ship AI tools just as fast as everyone else, with a real answer ready the day something goes wrong instead of finding out in a deposition.

A word on what’s coming

Regulation is moving faster than most companies’ internal processes. The EU AI Act’s risk-tiered requirements are already in force for the highest-risk categories, with more obligations phasing in through 2026 and beyond. Colorado, California, and other states have passed or are advancing their own AI accountability laws. The era of governance being optional, a nice-to-have for companies with spare compliance budget, is closing fast.

That shift is forcing a conversation that used to happen only after a lawsuit to happen during the procurement meeting instead, which is exactly where it always should have been.

But here’s what we keep landing on. The companies handling AI governance well are not the ones with the most polished principles document. They’re the ones who turned governance into a fast, repeatable set of questions someone actually answers before a tool goes live, instead of a value statement that sits in a drive nobody opens until a lawyer asks why not.

AI governance was never really about the policy. It was always about whether someone could answer “who’s accountable for this” before it became a question a court had to answer instead.

The Demand Co.ai covers what’s actually moving the needle in B2B marketing and demand generation. Subscribe to Signal — our weekly intelligence brief — and skip the noise.