A list of 12,735 profiles sounds like plenty of room to experiment. It is not. Each person sees your first message once, and a poor first impression is hard to undo. Testing well means learning the most from the smallest part of your audience.
The testing plan
- Write down one question. "Does a free mock test get more sign-ups than a free band-score guide?" is a question. "What works best?" is not.
- Pick one metric in advance. Sign-ups, cost per lead or replies. Decide before you look at results, so the numbers cannot talk you into a winner.
- Split at random. Divide the test group randomly, not by city or by upload order, or the split itself will decide the result.
- Keep a holdout. Leave part of the list untouched. It shows what happens without your campaign and keeps fresh contacts for the winning version.
- Run long enough. Give each version the same time and budget. Stopping the moment one version looks ahead is the most common testing mistake.
A worked example
The numbers below are illustrative, to show the structure. The audience size is real.
| Group | Share | Profiles | What they get |
|---|---|---|---|
| Version A | 20% | 2,547 | Ad offering a free mock test |
| Version B | 20% | 2,547 | Ad offering a free band-score guide |
| Holdout | 60% | 7,641 | Nothing yet; receives the winner later |
Say version A brings 64 sign-ups and version B brings 41 on the same budget. Before you call A the winner, check that the difference is not chance: a free online A/B test significance calculator takes a minute. If it holds, roll A out to the holdout group.
What is worth testing
- The offer: what the person gets for opting in. Usually the biggest effect.
- The angle: saving time versus saving money versus avoiding a mistake.
- The segment: the same offer in Hanoi and Ho Chi Minh City.
- The channel: custom audience versus lookalike. See From List to Ad Set.
Results only matter if someone reviews them. Read The Weekly Pipeline Review next.
