An A/B test has one purpose: to reliably measure which variant leads to a better result. But when measurement loses conversions — and unevenly between variants — the test picks the wrong winner. Here's how to avoid that.
Why measurement matters doubly for tests
- The winner is decided from conversions. If some conversions are missing, the variant comparison is skewed.
- Uneven losses are worst. When one variant loses more conversions (e.g. due to different iOS behavior), the test can declare the worse one the winner.
- Small differences, big decisions. Tests often look for differences of a few percent — noise in the data easily drowns them.
How server-side helps
- More complete conversions. Server-side captures conversions even through ad-blockers and ITP, so both variants measure on equally complete data. See ITP and lost conversions.
- Consistent identification. A customer stays in their variant across visits, so the result isn't blurred.
- A clean basis. Reliable measurement is also a prerequisite for more advanced incrementality tests.
What to watch
- Make sure the variant is correctly recorded in measurement (you know who saw which).
- Measure the same conversion for both variants, the same way.
- Don't end the test early — you need enough data for statistical significance.
Summary
An A/B test is only as good as the data you measure the result with. Uneven conversion losses can declare the worse variant the winner — server-side tracking gives experiments a complete, consistent basis to decide on. More in the complete guide.