1. Why do we summarize conversations?
I needed to implement conversation summaries while designing an AI agent-assist product. Before designing the feature, I asked why agents needed a summary in the first place.
After a call, the agent leaves a conversation record. The summary is one part of it. The next agent needs to know what the customer said and what was done. Managers need a way to review interactions and manage customer and support operations.
A short paragraph can be useful to read, but it can be hard to tell at a glance what situation the customer was in, what they requested, and how the agent handled it. I considered separating the record into three fields.
- Situation: What circumstances was the customer in?
- Request: What did the customer ask for?
- Action: What did the agent do?
This could help the next agent pick up the conversation and give managers a consistent view. But the more I thought about why we keep those records, the more I saw another use for them. The records accumulate over time as a history of how the company works with its customers.
Could we analyze them as well as read them? That was the reason to store situation, request, and action as structured fields.
2. What can we see when we connect those fields?
Customers approaching a loan maturity date may want different things. One wants to apply for an extension, another wants to understand the steps, and another wants to know whether they can complete it remotely. The same request can also lead to different actions, such as a text with document instructions or a referral to a branch.
To see those paths together, I used n8n and a language model to extract situation, request, and action from 30 speech-to-text records about loan maturity extensions. I normalized similar expressions into a codebook and connected the three fields in a Sankey diagram.
The interesting branches were where similar situations and requests led to different actions. Customer conditions may have differed, or agents may have handled the work differently. The diagram showed where handling diverged and what conditions to check next. Thirty records were enough to explore the method. A recurring pattern would need a larger sample and further verification.
3. What problems might a contact category hide?
Contact categories help support teams log and manage their work. One category can contain customers in different situations who need different outcomes.
Take a limit increase. One customer has used their available limit. Another wants to restore a limit they previously lowered. Both can be recorded under the same category, even though the eligibility checks and process may differ.
Keeping situation, request, and action separate makes those differences visible within a category. The unit we use to operate a support queue and the unit we use to investigate a customer problem can serve different purposes. A Sankey is one way to examine that difference.
4. What changes when we add customer state and prior behavior?
A Sankey shows the relationships inside the conversation. To understand why one request led to two different actions, I wanted to know what happened before the call. Which product and conditions applied? What did the customer try on a particular screen? What did the API return? Had the customer just received a notification?
Joining those records could help find customers who encountered the same problem under similar conditions, even if they described it differently. Repeated issues after a policy change or system event would give us a relationship to investigate. Timing alone cannot establish the cause. The conversation and system records would need to be checked against the hypothesis.
5. Who can fix the problem once we find it?
A product feature may need a product owner. A policy condition may need a policy owner. Unclear instructions may need the team that owns the relevant workflow. Each issue can arrive in the same support queue and end up recorded as a category and a handled contact.
I started thinking about combining the structured conversation with the trigger, related feature, customer state, suspected cause, and scope of impact. That evidence could help suggest the teams that might be able to act. The design would map events to teams, detect recurring patterns, and help someone review the evidence and confirm ownership before an improvement request is sent. Later, the same record could show what changed and whether the issue recurred.
6. What has been built, and what needs to be tested?
I prototyped extracting situation, request, and action from speech-to-text records, normalizing the fields with a codebook, and visualizing the flows. Linking customer behavior, state, and events, detecting recurring problems, suggesting owners, and tracking recurrence after a change are still areas to validate.
We would need to decide how to join records across systems, when several contacts count as one issue, how to separate a possible cause from confirmed evidence, and how teams should confirm responsibility.
It started with a conversation summary. Asking why we keep conversation records led me to ask whether the same data could help discover customer problems and connect them to the people who can solve them. The Lab uses synthetic data to explore what would have to work for that to happen.