Every CX leader I talk to is feeling the same pressure.

On top of the usual plate spinning—forecasting demand, supporting new product launches, reworking processes and inflows, sharing customer insights, coaching teammates through the increasingly difficult work of supporting customers—we’re being given a new charge: do AI, and do it fast.

So, we scramble. We pilot chatbots, We slam AI-assisted agent tools in front of our teams. We train models on our knowledge base, then funnel customer data into LLMs to approximate human decisions. And then we wait for the magic to happen: costs drop, resolutions soar, our customers are happier than ever because they’re getting fast, frictionless responses.

…Right?

We're Having Déjà Vu

If you’ve been around long enough, you’ll have seen all of this before.

When chatbots were first introduced in the 2000s, they kicked off a gold rush of vendors and consultants promising to reduce costs, increase sales, and prevent churn. The best implementations made it look so easy, so smooth—and droves of businesses went all in on chatbots as their CX strategy, and they moved fast to get the edge on their competitors.

And so, you had all of these chatbots popping up, and they all sucked. They sucked! You felt like you were screaming at a brick wall just to get from Point A to Point B. It took a decade of chatbot ubiquity before consumers went from “oh god, not this” to begrudged expectation.

That’s where we’re at with AI right now—a wild west landgrab, where we’re doing first and figuring out later. And many consumers are at that instinctual rejection stage. They’ve seen too much slop, they’re being trapped in too many endless LLM loops—and they’re looking for any opportunity to circumvent your AI.

It's completely understandable why businesses are going to hard on AI. Executives are getting the same pressure—from board meetings, competitor reviews, every time they step foot onto LinkedIn. But, as they say, haste makes waste, and it's easy to find yourself in an AI dumpster.

Irresponsible AI deployment isn’t just costing you money in token fees and contractual minimums, it’s taxing the trust you’ve established with your CX team and your customers alike.

The Reality

Here’s what I’ve learned from watching dozens of AI implementations across the industry: the technology is rarely the bottleneck. The organizational readiness is.

When you drop AI into an existing support operation, you’re not just adding a new tool. You’re fundamentally changing:

  • How work gets distributed between humans and machines
  • What skills your agents need to succeed
  • How you measure success (condolences to your dashboards)
  • How your content needs to be structured for AI consumption
  • What your quality assurance process looks like when AI is in the loop
  • The tone and patience of the customer who makes it through AI triage

Using a manual dispositioning system for tracking trends? That’s dead. Tracking first reply time as an operational metric? Not anymore you aren’t. WFM? WTF.

The Shift

The organizations getting AI right aren’t just deploying technology—they’re redesigning their operations around human-AI partnership. Grafting AI into a system not ready for it is just going to lead to that system rejecting it like an incompatible organ.

True AI readiness looks like this:

Rethinking Agent Roles

Your best agents aren’t going to be the ones who handle the most tickets, they’re going to be the ones who:

  • Train and improve AI systems
  • Handle the complex escalations that AI can’t (and prevent them from happening again)
  • Provide quality assurance on AI-generated responses
  • Develop and maintain the content AI draws from

But, if you’re operating lean, they aren’t going to have time to do that work. You need to find ways to engage your team, draw clear swimlanes, and give them space to live in these new ways of work.

Investing in Content Operations

Garbage in? Garbage out.

AI is only as good as the knowledge it draws from and its understanding of the context that the knowledge is meant to address. Most support organizations have content scattered across wikis, aging Google Docs, arcane Slack threads, and—almost certainly—a random fact sitting in the back of one of your agents’ brains.

AI systems need hand-holding. They’re not absorbing the breadth of your corporate culture and pattern matching against a lived experience. Which means you need to be obsessive about the nuts and bolts of how content actually functions within your team. And this isn’t just a CX problem—it’s a marketing and product problem too.

“Don’t support agents write responses all day? Can’t they just write all the content? Why bother with content operations?”

Writing content and creating a system so that you never miss an opportunity to create, improve, or deprecate content ingested by AI are completely different things. You can create content all day, but if it’s redundant, counterfactual, or lacks detail, not only will you do more harm than good—you might not know the total impact of that harm for weeks or months.

Building Feedback Loops

Something that we don’t spend enough time talking about is how quickly AI models change. When ChatGPT went from 4o to 5, its entire personality changed. It processed information differently, it felt different. If your platform switched models, it’d be like if your most productive agent got bonked on the head and changed their whole way of operating.

Even outside of these changes, AI systems need continuous feedback to improve. That means building processes for:

  • Monitoring AI accuracy and confidence
  • Capturing cases where AI fails
  • Routing insights back to training data
  • Measuring customer outcomes, not just resolution rates
  • Making sure that your issue disposition data doesn’t get muddied

And whatever you do—don’t trust the AI platform to be honest about its resolution rate. You need to take an active role in steering your AI implementation’s continuous improvement.

Pulling It Together

There are a lot of ways that these ideas can work in harmony, and it depends entirely on how your organization operates, your industry, and the expectations of your customers.

Some practical steps that you might take to fortify your operations ahead of AI transformation might include:

  1. Start (slowly!) shifting the responsibilities of your agent towards continuous improvement. Can you deprioritize anything on their plates to give them space for new work? Do it. Your goal is to transform this from chore to opportunity.
  2. Make content request and recommendation intake easy. Doesn't matter if you use Confluence, Asana, Linear, or a Slack channel. Whatever you choose, make it simple and stay consistent on updates.
  3. If your agents are manually updating dispositions, rethink your strategy. Manual dispositions are great because they get at the reality of what a customer is asking about. It's a goldmine for product and should influence what the business is prioritizing. But, these dispositions often have the same problem as any other piece of information—they lack context. Your AI platform probably won't use your dispositions, which means you need to think about normalizing now.
  4. Clearly define what "success" looks like in an AI conversation. Not all human support conversations resolve satisfactorily, but we still consider them to be successful. Clearly defining what a good AI conversation looks like (especially when the customer doesn't get the resolution that they were seeking) will help you make better judgments about quality.
  5. Shrink your QA rubric. You'll be glad you did. Scoring based on accuracy, voice, and resolution status will give you directional information that's critical for making quick decisions. You don't want to spent 10 minutes QAing each AI conversation.

The Bottom Line

AI is a transformative technology and has truly changed the way that we interact with the internet in a way that hasn’t been seen since the dawn of social media. We should be integrating this technology into our organizations.

But, AI is a capability, not a strategy. Coming down from the mountaintop and saying “do AI yesterday” isn’t going to make your team faster, your customers happier, or your revenues increase. The strategy is building an organization that can effectively leverage that capability while preserving what makes human support valuable.

That takes more than a vendor contract and a pilot program. It takes intentional organizational design, investment in your people, and the patience to do it right.