Skip to main content
source-empty

The Problem

Customer support inboxes get flooded with emails daily. Some need expert human attention, others are routine questions that keep coming up again and again.
source-empty
We wanted to create a system that could:
  • Process incoming customer emails automatically
  • Determine the sentiment and urgency of each message
  • Identify which ones need human expertise
  • Automatically respond to straightforward inquiries
So we rolled up our sleeves and built a solution with Datazone

The Setup: Three Core Components

Our solution consists of three main parts:
Customer Message Automation Architecture
  1. Data Source: A JSON file containing customer support messages
  2. AI Analysis Engine: Anthropic Claude 3.7 with Pydantic AI for structured output
  3. Datazone Pipeline: It’s the simplest part of the flow thanks to Datazone. 😎
Here’s how we built it:

The Data: Raw Customer Messages

Here’s what our sample data looks like:
We have used a sample JSON file to simulate incoming customer emails. You can find it in the example repository which I’ll share in below.

Datazone + Pydantic AI: Perfect Together

We created a Pydantic model to structure the AI’s analysis:
This gives us structured data we can actually use in our workflow, rather than raw text.

The Instructions: Guiding the AI

We wrote a clear set of instructions for the AI in a markdown file:
This helps the AI make consistent assessments about each incoming message. You can access the full instructions in the example repository.

The Pipeline: Orchestrating the Workflow

Here’s the Datazone pipeline we built:
Three transformations is all it takes to build a complete email processing system!

The Decision Engine: Human or Machine?

The key part of the system is the logic that decides which emails need human attention:
When the AI isn’t confident in its response, it flags the message for human review. This ensures customers always get the right answer, whether it comes from AI or the support team.

Final Data ✨

The Dashboard: Real-time Monitoring

We built a dashboard in Datazone to monitor the system:
Customer Message Automation Architecture
The dashboard tracks:
  • Total message volume
  • Average urgency levels
  • Messages requiring human intervention
  • Sentiment distribution across all communications
Please don’t assume I moved the data to another platform or used a traditional BI tool to build this app! 😊 I’m still in Datazone and accomplished this with a simple command to Orion AI. For more details, check out the Intelligent App section.

The Results: Real Business Impact

After deploying the system:
Customer Message Automation Impact
  • Approximately 60% of customer emails are now handled automatically
  • Response time has decreased to under 1 hour
  • Support team can focus on complex issues that truly need human expertise
  • Customer satisfaction has improved due to faster response times
And we built the entire system in just a few hours using Datazone’s platform.

Next Steps for this Project

  • Using tools for enhanced actions by LLM like accessing user detail, previous interactions, and context to improve response accuracy.
  • Give more context about the business case to find better insights and responses.
  • Use some prompt engineering methods like few-shot prompting to categorize and notice correct sentiment.
  • Implement feedback loops to continuously improve the model’s performance based on real-world interactions.

Resources