How a private RAG concierge cut a FinTech's support cost 47%

A step-by-step breakdown of the retrieval architecture, model routing, and guardrails that turned a drowning L1 support queue into a 92% first-contact-resolution engine — in eight weeks.

By , Head of AI and Growth · · 9 min read

The problem: 80% of tickets were repeat questions

A scaling FinTech was fielding ~12,000 support tickets a month. Roughly 80% were repeat policy questions — fees, KYC status, payout timelines — yet they consumed the same human bandwidth as genuinely novel issues. CSAT was sliding and the queue was growing faster than headcount could.

The mandate was blunt: cut cost without cutting quality. Not a chatbot that deflects, but a concierge that actually resolves.

  • −47% Support cost
  • 92% First-contact resolution
  • +18pt CSAT lift

The architecture: retrieval first, generation last

We built a private RAG pipeline over the client's policy corpus and anonymised ticket history. Every answer is grounded in retrieved, cited source chunks — the model is never allowed to free-associate on financial policy.

  • Ingestion — policies, help-centre articles, and 90 days of resolved tickets, chunked semantically and embedded.
  • Hybrid retrieval — dense vector search fused with BM25 keyword recall, then re-ranked.
  • Tiered routing — a cheap fast model answers high-confidence, well-grounded queries; a frontier model handles ambiguous ones.
  • Guardrails — confidence + groundedness thresholds. Below threshold, it hands off to a human with a full context summary.
async function answer(query: string) {
  const ctx = await hybridRetrieve(query, { k: 8 });
  const grounded = await rerank(query, ctx);

  // Route by retrieval confidence, not by guessing.
  const tier = grounded.score > 0.78 ? "fast" : "frontier";
  const draft = await generate(query, grounded.chunks, { tier });

  // Never ship an ungrounded financial answer.
  if (draft.groundedness < 0.7) {
    return escalateToHuman(query, grounded.chunks);
  }
  return draft;
}

Why it worked: the escalation path was the product

The win wasn't the model — it was the handoff. By treating "I'm not sure" as a first-class outcome and routing those cases to a human with the retrieved context attached, we eliminated the two failure modes that kill support AI: confident hallucination and dead-end deflection.

Eight weeks from kickoff to production. The queue stopped growing in week three; cost crossed the −40% line by week six.

Key takeaways

  • Ground every answer in retrieved, cited sources — especially in regulated domains.
  • Route by measured retrieval confidence, not by prompt-engineering guesswork.
  • Design the human handoff first; it's the safety net that lets you ship aggressively.

Book a 20-minute AI Opportunity Call · Get a free AI opportunity map

iqdigital