Customer service teams are now product teams

Customers interact with your business in many ways. They talk to your sales team, they use your product, and they reach out for support. But they don’t experience those interactions as separate things – they experience your brand as a cohesive whole.

A single good support interaction can do as much for retention, expansion, and loyalty as a strong product feature. AI makes acting on that possible because it’s changing what a customer service team is at a fundamental level. As AI Agents take on the majority of customer-facing queries, the support org’s focus moves to designing the customer experience and managing the AI system that powers it: determining how the Agent communicates with customers, deciding where it hands off to humans, and owning what the experience looks like end to end.

That’s a product function. Customer experience is a product, and customer service teams are now product teams.

Our Senior Director of AI Support, Ruth O'Brien , wrote a guide to show you how to run support like a product organization and operate in a way that lets you define, build, and own your customer experience just like a product org owns a product.

Here are the main takeaways.

Write down the experience you want customers to have

Start by zooming out. How do you want customers to think and feel about your brand? Codify it in simple language that’s easy for anyone to understand, then circulate it with your leadership team and ask for feedback. Principles are there to align your org by giving people something clear to follow when making decisions or working through challenges, so getting everyone on the same page from the beginning is worth the time.

The three our support org follows at Fin:

Build the system the way a product leader would

Once you’ve set your guiding principles, you need to build a system that helps you work towards your ideal customer experience. Approach it the way a product leader would:

• Define org-level goals so your team can build a roadmap with clear milestones, explain their work in terms the rest of the business cares about, and push back on requests that don’t fit the plan. That means deciding how your vision aligns with broader business objectives, how you prioritize, and where you make tradeoffs.
• Decide what success looks like so your team knows who to go to for a decision, and whether a change actually worked. Name an owner for each initiative, define what progress looks like, and set the target you’ll measure against.
• Map your non-negotiables so your team can move fast without putting the experience at risk. Be explicit about what you won’t compromise on, no matter the efficiency gain.
• Adopt an experimental mindset so your team can try bold things knowing the risks are contained. Decide how much freedom they have during a sprint, what guardrails apply, and who signs off on tests and monitors them.
• Own the outcomes so every initiative feeds the next instead of starting from scratch. Decide how results get shared and how insights flow in to streamline your reporting.

Stand behind the outcomes

Owning the outcomes means taking accountability and being willing to share them publicly. Many product orgs report uptime and outages on public status pages to visibly uphold their high standards. Support can do the same by sharing metrics that demonstrate the quality of their customer experience.

What that looks like will vary. Some orgs might publish live numbers where customers and prospects can easily see them. Vanta, for instance, displays its support metrics on its website.

Others share details about their journey transforming their organization and ways of working or goals they’re trying to achieve. At Fin, we’re documenting our progress toward a 95% automation rate. However you approach it, sharing openly keeps your org accountable for the experience you deliver.

Own the experience end to end

In the reactive customer service model, success means closing tickets fast, planning is about managing volume, and change happens only when something breaks. In the product-led model, planning is mapped to specific outcomes, teams ship experiments often, insights flow between support and other functions, and leadership sees support as a value driver rather than a cost center.

The customer experience is yours to own, so own it. Decide what good looks like, set the targets, run the experiments, measure the outcomes (not just the output), and stand behind the results.

What this looks like at Kalshi

Kalshi is the first federally regulated prediction market exchange in the US, where people trade contracts on real-world events from elections to the weather. With support volumes growing rapidly, the team brought Fin in to help them scale by handling front-line queries on top of their existing helpdesk.

Within weeks of going live with Fin, Super Bowl LX happened. The rush of sign-ups, deposits, and trading drove support volume to record highs, and while Fin held up, the team’s existing platform beneath it did not. Support went dark for 48 hours.

That failure prompted Shannon Magiera , Kalshi's Operations Lead, to rethink her support strategy and rebuild like a product team would:

• Fixed the foundation instead of patching it: Rather than work around the platform that failed, they moved the entire support function onto the Intercom helpdesk, completing the cutover in under three weeks.
• Expanded Fin’s scope gradually: Because they operate in such a regulated environment, the team rigorously tested Fin, with Kalshi’s legal and compliance teams stress-testing every guardrail before setting it live. Once they were confident with its performance on the frontline, they integrated it with more of their systems, allowing it to answer personalized questions. Now, they’re exploring how they can safely involve it in more sensitive conversations, involving fraud investigations, compliance cases, and account restrictions. 
• Established new Fin-focused roles: As Fin takes on more work, the team is focusing more on building and maintaining the AI operation, tracking metrics, filling knowledge gaps, and mapping the broader operational strategy.
• Set ambitious goals: Today, 90-95% of Kalshi’s support conversations come in through live chat, and Fin resolves around 80% of these (more than 80,000 every month) on its own. But the team wants to push this number higher. “What we’re looking for is 100% automation, 100% resolution,” Shannon says. “Obviously there’s always going to be a subset of users who want to speak to a human, and we love that. But if you can automate as much as possible, why not set ambitious goals?”

Read their story here.

News and updates

Announcing: Evals and Releases

We recently announced Evals and Releases. Paired with Monitors, they give you an evaluation system for Fin, so you can test configuration changes, roll them out with control, monitor every live conversation, and have confidence in the experience Fin delivers.

We built these because keeping an AI Agent to your standards in every conversation is hard. Three things make it that way:

• AI Agents are probabilistic: ask the same question twice and you can get two different answers. 
• AI is unpredictable: a small content change can affect behavior in places you wouldn’t expect.
• An Agent’s unit of output is a dynamic conversation, meaning success isn’t just about whether the final answer is right. In the case of Fin, you need to know it used the right data, Procedure, or handed off at the correct time. 

Without an evaluation system, every change you make risks introducing drift. At the scale some of our customers run Fin, a 1% regression could affect thousands of conversations a day. Rigorously testing changes before they go live is essential. 

Evals and Releases was inspired by the system our own machine learning team uses to test updates and monitor performance. We call it Eval-driven delivery, and it's now in the hands of teams running support with Fin.

Join leaders from Anthropic, Kalshi, and Clay at Pioneer

Pioneer, the summit where CX leaders redefine what’s possible, is just seven weeks away.

We’re excited to share that Emily Lampert , Head of Product Support at Anthropic , Shannon Magiera , Operations Lead at Kalshi , and George Dilthey , Head of Support at Clay , will be joining us for an insightful conversation on the current state of AI transformation, and how they’re planning for what’s next.

Join the conversation on October 7th in San Francisco, or attend virtually. Register here.