Beta
SEO split test planner: size and schedule a page-group test
Turn an SEO test idea into a plan: group sizes, duration, and a clear flag when it can't be tested yet.
credit cost
1
Overview
What it does
The split test planner takes your test type, a hypothesis, the number of eligible pages, their combined daily organic clicks, the smallest effect worth detecting, and a confidence level. It returns a plan for running the test.
The tool calculates the numbers itself:
Group sizes. Pages are split into control and variant as matched pairs. They're sorted by recent clicks, paired, and one of each pair is assigned to variant at random. An odd page goes to control.
Duration. How long the test needs to run, in whole weeks, between two and eight.
Underpowered flag. When the page count or traffic can't support your effect size within eight weeks, the plan says so first and shows the smallest effect you could detect in eight weeks instead. Fewer than 20 pages is always flagged.
The model then writes the plan around those numbers: how to split the pages, what to change and what to hold constant, a launch checklist, and stop rules and risks. It never calculates a number itself.
Assumptions, stated plainly. Two-sided test at 80% power. Clicks are assumed to be four times noisier than pure counting, and individual pages are assumed to vary by about 25% from each other. You'll want a pre-period at least as long as the test. Nothing adjusts for seasonality, algorithm updates, or crawl lag; the plan covers those as risks.
When the test has run, check the result with the SEO test results analyzer. Both tools use the same model, so a test the planner calls workable is one the analyzer can resolve.
The sample below continues the Northtrail example from Title bet.
Method
How it works
1. Describe the test
Choose the test type and write the hypothesis.
2. Enter pages and traffic
The number of eligible pages, their combined daily clicks, and the smallest effect worth detecting.
3. Get the plan
Group sizes, a duration, and a plan to run it, or a clear flag when the test is underpowered.
Example
Sample output
Input
Test type: Title / meta
Hypothesis: Adding the two most-used filter attributes to category page titles (for example, "Waterproof & Lightweight") will increase organic clicks to those pages.
Eligible pages: 120
Average daily organic clicks: 4,800
Minimum effect worth detecting: 10%
Confidence level: 95%
Result
Value | Result |
Control pages | 60 |
Variant pages | 60 |
Clicks a day per group | 2400 |
Expected daily swing | 7.4% |
Days per period | 10 |
Test length | 2 weeks |
Underpowered | no |
Detectable effect in 8 weeks | 4% |
The plan
This test splits 120 category pages into two groups of 60, runs for 2 weeks, and is sized to detect a 10% effect at 95% confidence. That's the smallest change in clicks worth calling a result — anything smaller won't reliably separate from noise at this size and duration.
How to split
Sort all 120 eligible pages by recent clicks, then pair adjacent pages down the sorted list.
For each pair, flip a coin to decide which page goes to variant and which stays control.
This keeps the traffic mix even across groups — the coin flip is what makes the assignment random, not the pairing itself.
Expect the two groups' daily click ratio to move around by about 7.4% on its own — that's normal noise, not a signal.
Exclude or flag pages that are seasonal, newly published, recently changed, or already being worked on by another team.
What to change and what to hold constant
Change: on variant pages only, append the two most-used filter attributes to the title tag.
Change: nothing else on the page — meta description, H1, body content and URL stay as they are.
Hold constant: internal linking, redirects and publishing schedule stay untouched across both groups.
Hold constant: no paid campaigns or promotional pushes pointed at either group's pages during the test window.
Launch checklist
Record the final control and variant page lists by URL, with the pairing and coin-flip result for each.
Pull and log pre-period daily clicks for both groups, covering at least the length of the test period.
Apply the title change to variant pages only, in one batch if possible.
Spot-check a sample of variant pages to confirm the new titles are live and the pages are still crawlable.
Log the launch date.
Set a check-in date at the end of the 2 weeks — not before.
Stop rules and risks
Don't read the test before the 2 weeks are up — early reads are noise.
The one legitimate reason to stop early is damage: a clear drop in indexing, rankings or clicks on variant pages.
Watch for seasonality inside the window, an algorithm update landing mid-test, and crawl lag.
With only 120 pages, a handful of drop-outs can meaningfully weaken the split.
Northtrail Outfitters is a fictional retailer. All numbers are sample data.
FAQ
Common questions
How much does the SEO split test planner cost?
More tools