←Back

Customer Intent Engine

2026 · AWS Agent Core · Fast MCP · AWS Bedrock · Snowflake · Streamlit · MLflow · CloudWatch · X-Ray

Business Problem

The Customer Service department handles thousands of conversations every month. The Customer Service Summarization Project already processes every call and writes five structured summary columns per conversation to Snowflake, with PII masked before storage. That pipeline was built to automate conversation wrap-ups and save agent time, not to make insights accessible to business users. The summary data sits in Snowflake, but querying it requires SQL, which puts it out of reach for the analysts, managers, and researchers who have the most relevant questions.

Customer surveys add a second layer of signal: NPS, satisfaction, and campaign feedback collected across the group's consumer lottery brands. These were also siloed, with no shared query layer connecting survey trends to call driver trends. Operational questions that should take minutes, like "what is driving call volume this week?" or "is survey satisfaction down since we changed the campaign?", either went unanswered or required disproportionate analyst effort.

Solution

I added an AI agent layer on top of these two existing data assets. A daily aggregation pipeline in Snowflake Cortex reads the conversation summaries and survey data, extracts themes, and writes pre-computed aggregations back to Snowflake. From there, a Fast MCP server exposes one dedicated tool per customer service topic (call reason, sentiment, churn reason, customer feedback, agent dynamics) and one tool per survey type.

AWS Agent Core on Bedrock orchestrates tool calls in response to natural-language questions from Marketing, CS Analytics, and Customer Research. Its memory module retains context across sessions: active marketing campaigns, brand identity guidelines, and operational processes. The agent already knows what campaigns ran last Monday before it pulls sentiment trends from the same period, producing answers grounded in business context rather than raw data alone.

All outputs are aggregated-only by design. The system never surfaces individual conversation records. This keeps the DPO review scope narrow, avoids re-identification risk at the query layer, and keeps the system out of scope for EU AI Act individual-decision obligations.

Streamlit is the interface layer. Users type natural-language questions, receive answers with coverage counts and source statistics, and rate each response. Ratings are linked to the prompt version that produced the answer, so a score drop on a known question shows up in MLflow before it reaches the wider user base. The POC ran locally; the MVP is investigating Snowflake-hosted Streamlit to remove the local setup dependency.

Architecture

Customer Intent Engine architecture
architecture_v6.svg

Technical Challenges

One tool per domain, not one general tool. The first version used a broader "query customer data" tool. The agent regularly conflated call reason themes with sentiment signals, and the output schemas were too open-ended to validate. Splitting into domain-specific tools with typed output schemas eliminated the confusion: each tool returns exactly what its domain can answer, and agent reasoning traces became readable. The tradeoff is that multi-topic questions require explicit routing logic, which is now versioned in MLflow so any regression is traceable.

Agent memory does more work than expected. Without campaign context in memory, the agent produced technically correct answers that were operationally useless: "sentiment is down 12% this week" with no connection to the campaign that launched Monday. With memory seeded with active campaigns, the same question returns an answer grounded in the business situation. The lesson is that memory is not a convenience feature for agentic analytics systems; it is what makes the difference between a data lookup and an insight.

Prompt versioning is a data provenance problem. Two weeks into the beta, a prompt update reclassified "address change enquiries" into a broader "administrative requests" theme. Week-on-week trend data showed an apparent spike in administrative contacts that was entirely an artefact of the relabelling. MLflow versioning made this diagnosable in under an hour: every aggregation run is tagged with the prompt version and tool version that produced it. Without that, the team would have spent days investigating a non-event.

The aggregated-only constraint required structural enforcement, not just a system prompt. Telling the agent to avoid individual-level answers was not sufficient: users rephrased questions until answers approached individual-level specificity. The fix was adding explicit coverage counts to every response ("based on 847 conversations") and having the agent actively name the re-scoping when it happens ("I cannot show individual calls, but across 847 conversations this week..."). The count signals statistical weight; the explicit re-scoping prevents users from assuming the agent is withholding data.

Observability at three layers, each answering a different question. CloudWatch covers operational health: Bedrock invocation counts, MCP tool latency per domain, Cortex pipeline duration and error rates. These are the signals that page someone. AgentCore Observability emits per-session reasoning traces to X-Ray: which tool was called, what it returned, and the chain of thought that preceded each decision. That data is too verbose for a dashboard but essential when a user reports a wrong answer. MLflow sits outside both: it tracks quality across prompt versions, not operational health. Every aggregation run is tagged with the prompt version that produced the themes; every golden-set evaluation logs the mean rating and faithfulness score. A silent drift in theme labels shows up as a metric regression in MLflow, not as an infrastructure alert in CloudWatch.

MVP Status

The MCP server is running with tools covering all five CS topic columns and the two current survey types. AWS Agent Core with memory is live and answering questions for CS Analytics and Customer Research in an internal beta. The Streamlit interface is live for the beta; hosting on Snowflake is being evaluated to make it accessible without a local setup. CloudWatch, X-Ray, and MLflow are each wired up for their respective layer: operational metrics, per-session reasoning traces, and quality tracking across prompt versions.

The system is not yet in production for Marketing. Campaign context in agent memory is seeded manually for now, pending the nightly sync from the marketing calendar. Theme extraction accuracy sits at 82% agreement with human labels on a 100-record spot-check, above the 80% MVP threshold. Latency on a typical question, including tool calls and synthesis, is under 8 seconds at P95. The single-agent architecture is hitting limits on complex cross-domain questions; the next phase moves to specialist agents per domain with a dedicated QA agent.

Next Steps

  • QA agent before response delivery: before any answer returns to the user, a separate agent validates that all statistics are traceable to tool output and flags low-confidence responses rather than returning them silently
  • Multi-agent architecture: Intent agent parses and routes, Analytics agent computes themes and trends, Context agent injects campaign and brand context, QA agent evaluates before the answer is returned; this replaces the current single orchestrator that handles all of these in one system prompt
  • Automated campaign memory sync: nightly Lambda pulls active campaigns from the marketing calendar into agent memory, removing the manual seeding step before Marketing rollout
  • RAGAS evaluation suite: automate faithfulness and answer-relevance scoring on a weekly golden question set, replacing manual spot-checks; the goal is to catch prompt version regressions before they reach users
  • Knowledge graph layer: connect aggregated themes, active campaigns, and churn signals into a persistent graph so cross-domain synthesis does not require re-querying all tools on every question
  • Near real-time aggregation: shift from daily batch to streaming so new conversations are reflected within hours, which matters for Contact Centre teams reacting to same-day call spikes

Code is proprietary; happy to walk through the architecture in detail.