←Back

Customer Intent Engine

2026 · Snowflake Cortex · Cortex Agents · MLflow · CloudWatch

Business Problem

The Customer Service department handles thousands of conversations every month. The Customer Service Summarization Project already processes every call and writes five structured summary columns per conversation to Snowflake, with PII masked before storage. That pipeline was built to automate wrap-ups and save agent time, not to make insights accessible. The data sits in Snowflake, but querying it requires SQL, which puts it out of reach for the analysts, managers, and researchers who have the most relevant questions.

Customer surveys add a second signal: NPS, satisfaction, and campaign feedback across the group's consumer lottery brands. Nothing connected survey trends to call driver trends. Questions that should take minutes, like "what is driving call volume this week?" or "why do people cancel?", either went unanswered or cost an analyst a day.

Solution

Everything runs inside Snowflake. A scheduled Cortex pipeline classifies each conversation into facets (call reason, sentiment, churn reason, brand) and writes one row per interaction to a fact table, currently around 319,000 interactions from April 2024 onward. A second table, the label map, translates raw model output into a canonical vocabulary using reviewed raw_pattern → canonical_label rules. That split is the core design decision: the taxonomy can change without retraining anything, and every rule carries a reviewer and a first-seen date.

A semantic model sits on top of the facet table and describes the dimensions, measures, and synonyms in business language. Users ask questions in the Cortex Agent interface in Snowflake, which generates SQL against the semantic model and answers with a chart and a count. No separate app to host, no data leaving the warehouse, and access follows the Snowflake roles that already exist.

Answers are aggregated by design. The agent never returns individual conversation records, and every answer names its coverage ("based on 7,300 of 319,000 interactions"), which matters because facet coverage is uneven. That keeps the DPO review scope narrow and keeps the system out of scope for EU AI Act individual-decision obligations.

MLflow holds the evaluation side: a golden question set, per-facet precision and recall against human labels, and coverage as a tracked metric rather than a footnote. CloudWatch holds the operational side. Scheduled Snowflake checks publish freshness, volume, drift, and unmapped-label rates as custom metrics, and EventBridge routes alerts through SNS to the ML platform mailbox for pipeline failures and to the data science Teams channel for quality regressions.

Architecture

Customer Intent Engine architecture
architecture_v7.svg

Technical Challenges

Normalisation belongs in a table, not in the prompt. The first version asked the model to emit final labels directly. Label drift was invisible: a wording change in the prompt silently renamed a category, and week-on-week trends moved for no business reason. Moving normalisation into the reviewed label map made the taxonomy a governed artefact. When a prompt update reclassified "address change enquiries" into a broader "administrative requests" theme, the apparent spike was diagnosable in an hour because every fact row carries the model version that produced it.

Coverage is the metric that actually matters. Only about 7,300 of 319,000 interactions carry an explicit churn reason. Accuracy on the labelled subset looked excellent and was close to useless on its own, because a stakeholder reading "cost is the top cancellation reason" needs to know it rests on 2% of conversations. Coverage now appears in the answer text, in the MLflow evaluation, and as a CloudWatch metric that alerts when the unmapped share rises.

Unmapped and failed are first-class values. Operational states like ongemapt (unmapped) and pipeline_fout (pipeline error) are stored as real facet values instead of nulls. A rising unmapped rate on call reason means the model is emitting patterns the label map does not cover yet, which is a taxonomy gap the data science team can fix in minutes. Hiding those as nulls would have made the same gap look like a volume drop.

Two observability layers answering different questions. CloudWatch covers whether the pipeline ran: Cortex job duration, error rates, freshness lag, volume per brand. Those are the signals that page someone. MLflow covers whether the answers are still right, tracked per model version against the golden set. A silent drift in theme labels shows up as a metric regression in MLflow, not as an infrastructure alert, and the two are joined by the model version stamped on every row.

MVP Status

The Cortex pipeline, facet table, label map, and semantic model are live. CS Analytics and Customer Research are asking questions through the Cortex Agent in an internal beta. The fact table covers 319,000 call interactions from April 2024 to February 2026 on a single model version. Theme extraction sits at 82% agreement with human labels on a 100-record spot-check, above the 80% MVP threshold.

Not yet in production for Marketing. Survey data and external call centre data are modelled as additional source types but not yet ingested, so today the engine answers on calls only. Campaign context is not available to the agent, which means it can report that sentiment dropped 12% this week but not that a campaign launched on Monday.

Next Steps

  • Feedback table: a one-question review in the CS tool ("is this reason correct?") lands corrected labels next to the predicted ones, giving a live agreement rate per facet and a golden set that grows by itself
  • Survey source: surveys bring native CSAT and NPS scores, so model-inferred sentiment can be checked against a self-reported number instead of a hand-labelled sample
  • External call centre source: proves the label map can normalise a second vendor's taxonomy into the same canonical vocabulary
  • Campaign context: a nightly sync of the marketing calendar into the semantic model, so answers name the campaign behind a trend
  • Automated evaluation: faithfulness and answer-relevance scoring on the golden question set each week, replacing manual spot-checks, gated so a new model version only promotes if it beats the current one

Code is proprietary; happy to walk through the architecture in detail.