Spaces

Weekly chat review checklist for firms using an AI agent

This weekly checklist shows what a firm owner should look at in the chats an AI agent handled with clients. It helps you catch wrong answers, slow handovers and missing facts early.

Runs in your browser. Nothing you enter is sent anywhere or saved.

Before you start

In a firm, quality problems hide in small places: a vague promise, a missing file number, a reply that sounded too sure. A weekly look at the real conversations catches these while they are small. It also shows what clients actually ask, which is rarely what the website suggests.

Set aside thirty minutes on a quiet day, such as Monday morning. Read a sample of last week's chats with this list beside you. Note what to fix in the written answers, and who to speak to. Keep the notes in one place so the pattern is visible over time.

Tick each one as you go. 0 of 10 done.

[ ] Read every chat that was handed over, from first message to reply.
[ ] Check how long each handover waited for a person.
[ ] Read a sample of chats the agent answered alone.
[ ] Look for any reply that sounded like advice or an opinion.
[ ] List questions the agent could not answer.
[ ] Check promises about timing against what actually happened.
[ ] Note any client who seemed upset or confused.
[ ] Check that documents received were passed to the right person.
[ ] Update the written facts: hours, staff, fees policy and closures.
[ ] Share one lesson with the team.

What to do with what you find

Fix the cause, not just the chat. If the agent gave the wrong turnaround time, correct the written fact it reads from. If it missed a handover cue, add that cue to your rules. One edit then helps every future client.

Count a few simple things each week: chats handled, chats handed over, and average wait for a person. You do not need charts. A short note in a shared file is enough to spot a trend.

Signs the agent needs tighter rules

Watch for these patterns across several weeks.

  • Clients repeating a question because the first answer was unclear.
  • Handovers arriving with missing details.
  • Staff redoing work the chat should have collected.
  • Complaints that mention being passed around.

FAQ

How many chats should I read each week?

Read all handovers, and a handful of chats the agent answered alone. In a small firm that may be twenty or thirty conversations, which takes around half an hour.

Who should do the review in a larger practice?

Choose someone senior enough to judge whether a reply strayed into advice, such as an office manager with a partner joining monthly.

Can I review chats less often once things settle?

Often yes, but keep a regular rhythm. Services, fees and staff change, and the agent only knows what you last wrote down.

More free tools for professional and technical service firms

Read the guide: An AI agent for professional, scientific and technical firms.