A/B testing
Making two versions of something is the easy part. Deciding what the difference between them is allowed to be is the work.
The short version
What you get. Three thumbnail-and-title concepts per video, with a clear question behind each comparison.
Is this for you? For a channel ready to compare ideas and act on the results. Small samples need longer observation and may not support a clear winner. If no result would change what you make next, there is nothing to test.
What leaves your plate. The test question, the variants and the comparison method. Concept tests compare whole packages; narrow tests isolate one element.
Start with the Free Audit.
What the difference between them is allowed to be
Concept first, then the single variable
We build three thumbnail-and-title concepts, such as a face, the finished result or the ingredients. This compares whole packages. To understand one element’s contribution, we then run a narrower comparison with that element changed and the others held steady.
Run as a native test where the channel can, by hand only where it cannot
Where the channel is eligible, we use YouTube’s native testing tools and report the result using the measure the tool provides. Sequential swaps are labelled separately because the audience and conditions can change between periods. Smaller samples are treated cautiously, and their leads are reported as provisional.
Two voiceovers, and a decision about how to compare them
We agree how to compare alternative voiceovers before recording. Two reads over the same cut offer a closer comparison; versions used on different videos also reflect differences between those videos.
Testing is what a channel does before it commits
When the audience or subject is still being explored, we start with a limited set of lower-cost pieces and review what they teach us before expanding production.
Variants arrive with the video itself
The three thumbnail-and-title concepts arrive with the video. The Thumbnail Design service develops the concepts; this service sets up the comparison and interprets the result.
The shape of a test
| Covers per video | three, each a concept with its own title, tested as whole packages |
|---|---|
| Where it runs | up to three at once in YouTube's native tool, titles, thumbnails or the two together, where the channel is eligible; by hand where it is not |
| How a winner is judged | share of watch time in the native test; click-through and retention read together where it is manual |
| What differs in a concept test | subject, title, colour and type, all at once, on purpose |
| What differs in a narrow test | exactly one thing |
| Voiceover comparison | two versions of one video, or one difference across two videos |
| Agreed before anything is made | the question, and what each possible answer would change |
The two variables worth spending a cycle on
What gets tested first
Thumbnails and titles are common starting points. Hooks follow where retention suggests a question worth testing. The next comparison depends on the evidence, the channel’s priorities and the options available.
Where the answers go
Results go into the monthly report with their limitations and the decision they inform. We agree the question and the possible next actions before producing the variants.
What is not settled this way
Tests compare ideas people have developed. Results inform the next brief alongside the brand’s priorities and creative judgement.
At the bench
The work, demonstrated
Try a decision from the craft: drag, tap or toggle, then read the trade-off.
These are illustrative models; no partner’s results are shown.
More demonstrations at the bench →Agree what the result would change before anything is made.
A test nobody will act on is a preference with a chart attached.
What changes for you
The next round starts with a documented comparison, what it supports and what remains uncertain.
Related: Thumbnails · Hooks and the first seconds · Analytics and reporting · Content systems and planning
Send your channel. We will review recent work and suggest what to address first.
Request your Free Audit.
Three fields, and a written reading back.
Your audit usually arrives within three working days. We use your details only to reply to your enquiry.
What you will get
What is working, what could improve and whether we can help.
Two or three priorities, with the reason for each.
A recommended next step, even if that step is not us.
There is no obligation to continue. We only arrange a call if you ask for one.
The work, written up
The case studies keep their dates, sources and limits visible.
See the casework →