You’ve brought a new wardrobe from Ikea.
You spend hours hammering away, screwing stuff in, swearing and sweating.
And can’t get the damn thing up.
Would you throw it away? Just because you can’t get it done immediately? Smash the bugger up and throw the pieces out?
Nah. Well, I hope not!
Same goes for your tone of voice tool. It might not be perfect immediately. It probably won’t be! Do you drop the project?
Think about it: If you hired a new copywriter who got your voice 80% right on their first assignment, would you immediately fire them?
Of course not. You'd give them feedback. You'd help them understand what wasn't quite landing. You'd train them.
The same principle applies to your AI tone of voice system. That first output isn't the final product—it's the starting point of an iterative training process. Just like a new team member, your AI needs guidance, feedback, and time to really nail your voice.
Let's get started:
First up: your initial outputs won't be perfect. This isn't a failure of the process—it's a natural part of it.
Even the most sophisticated AI systems need training and refinement. Hell, that’s what makes them so good! What we're building is a feedback loop, not a one-time setup. The magic happens through iteration.
It’ll probably look a bit like this:
The best news? This improvement happens quickly. Quicker than with humans!
We'll tackle training in two distinct phases:
This approach lets you refine in a safe space before deploying your voice system in actual production scenarios.
When to make a switch? If you are building this for yourself then you’ll likely go live a lot earlier. If building for a client then you’ll do more testing and refining up front. It depends!
Offline testing means giving your system tasks that mimic real-world needs but aren't actually used in production. It's like a dress rehearsal or a dry run. So many good metaphors here! Or similes? You know what I mean!
To properly test your voice system, you need a diverse set of challenges. Here's a prompt to help you generate appropriate some test tasks to start with:
You are an AI testing specialist. I need to create a comprehensive set of test scenarios for my brand voice AI system. The system will be used for [describe your intended use cases].
Please create 10 diverse test tasks that will thoroughly evaluate the AI's ability to replicate my brand voice across different contexts, tones, and content types.
For each test task:
1. Provide a clear prompt I should give to my AI
2. Explain what specific aspects of voice this test evaluates
3. Include a baseline for success (how to know if it passed)
Make sure the test set includes:
- Different content lengths (short, medium, long)
- Varying emotional tones (positive, neutral, challenging)
- Different content types (based on my intended use cases)
- Edge cases that might be particularly challenging
The goal is to identify where the AI excels and where it needs improvement in capturing my brand voice.Use this prompt to generate a comprehensive set of test tasks tailored to your specific needs.
Now - what to do with these tasks?
Here's the exact process for systematically improving your custom instructions:
We are basically stress testing our tone of voice AI with tasks. And each time telling it what it did well and what it sucked at. From that we get new custom instructions to work with.
This creates a tight feedback loop where each iteration improves your voice system.
Here's a prompt you can copy and paste to start this refinement process:
You'll help me refine my input custom instructions for better voice accuracy. Follow these steps:
1. First, I'll provide my current custom instructions and a test task as inputs.
2. You'll complete the test task using those instructions. Complete the task only and provide no additional information.
3. After seeing your response, I'll provide feedback
4. You'll then update my input custom instructions based on that feedback
When I provide feedback, analyse it carefully to identify patterns and issues with the voice. Then create updated custom instructions that address these issues.
Present the updated instructions in a clear, formatted way that I can easily copy and paste into my system. Use the same basic formatting and structure as the input - just with amendments based on my feedback. Do NOT explain the changes - provide only the reworked custom instructions with no additional information
INPUTS begin:
Current Custom Instructions:
[PASTE YOUR CURRENT CUSTOM INSTRUCTIONS HERE]
Test Task:
[ENTER A TASK YOU WANT THE AI TO COMPLETE]This prompt is a little complex. All you need to know is you copy and paste in your current custom instructions which we created in Part 2 and one of the tasks from above.
Run all of this in your Project, CustomGPT, API model or otherwise deployed tool so it has full access to your knowledge base.
This will run a process and spit out a reworked set of custom instructions.
Take these new custom instructions and add them into your deployed model.
Then run another task. And another. And another.
Each refinement cycle typically takes just a few minutes but dramatically improves your results. Most voice systems require 3-5 cycles to reach excellent accuracy. It’s well worth the investment of time.
When you're satisfied with the refinement, simply copy the final updated instructions into your voice system configuration and you as done for now!
Now comes the crucial part: providing effective feedback. Here's what makes feedback effective:
Poor feedback: "This doesn't sound right. Try again."
Effective feedback: "This is too formal for my brand voice. My writing typically uses contractions (I'm vs. I am), shorter sentences, and more conversational transitions. The paragraph structure is good, but the word choice feels stiff. Reference my blog posts from March for better examples of how I discuss technical topics."
The key elements of effective feedback:
If in doubt imagine you are a teacher providing writing feedback to a student. Keep it simple and focused.
Once your offline tests are consistently producing good results, it's time to implement your voice system in real-world scenarios.
Start small:
Even in production, the training doesn't stop. Here are some pointers though, especially if deploying for a client.