Read your test report and pick a winner

Interpret the Reports tab of a running test, understand SmartStats decisions, dig into segments, and deploy the winning variation.

#When to use this

Your test on the product page has been running for two weeks and you need to tell stakeholders whether the new layout should ship. The campaign's Reports tab shows how each variation performs against the baseline, whether the difference is statistically reliable, how segments react, and lets you roll out the winner.

#Before you start

• The campaign has been started. The Reports tab stays disabled while a campaign is in Draft.
• It has run for at least a week. Wingify's Experiment Vitals flag conclusions drawn before this minimum runtime.
• You know which statistical model the campaign uses (Bayesian or Frequentist); the columns differ. See Configure SmartStats and guardrails.

#Steps

#1. Open the Reports tab

Go to Experimentation > Web Experimentation, open your campaign and click Reports. The report opens on the primary metric. Click another metric in the metrics panel to view it, or use Compare to see several metrics side by side.

#2. Check the health of the test first

Click the Experiment Vitals (heart) icon in the report header. Checks include Data Tracking, Conversion Tracking, Minimum Runtime, Guardrails, Experimentation Conduct and Sample Ratio Mismatch (SRM). If a check is red, fix the cause before trusting the numbers. If you see Collecting minimum data for statistical calculations, the campaign is still in observatory mode.

#3. Read the report table

In the Statistics view, focus on three things for each variation:

• Expected conversion rate (or Expected value per visitor for revenue-type metrics): the most likely rate you'll get after deployment.
• Expected Improvement / Improvement %: how much better or worse the variation is than the baseline, with a range that narrows as data accumulates.
• Probability to be Better (Bayesian) or Significance Level (Frequentist): the main decision indicator.

When a variation crosses the winner threshold (typically 95%), Wingify declares it better than the baseline and shows a recommendation banner. Below the lower threshold (typically 5%), it recommends disabling the variation. The Decision Engine and campaign progress indicator summarize these decisions and when they happened.

#4. Break the results down by segment

Use the report filters:

• The date filter to look at a period (or quick ranges like Last 7 days).
• All Visitors to apply Standard Segments, My Segments or a Custom Segment, and compare segments side by side.
• Visitor Dimensions (up to two) to split rows by country, device, browser and similar attributes.
• The Opportunities view to spot positive and negative segments.

#5. Get a summary with Wingz

Click Summarize at the top of the Reports tab. Wingz Chat generates a summary with the date range, variations, unique visitors, expected conversion rates and improvement percentages. You can keep asking Wingz follow-up questions.

#6. Deploy the winner

For an A/B test, click Roll out instantly next to the winning variation in the report table (or Roll out / Roll out variation from a variation's ⋮ menu). Wingify creates a linked Rollout campaign; you can Preview it, then click Start Campaign Now. Starting the rollout automatically stops the original test. Rollouts use your Rollout quota, or your testing quota if you don't have one.

To make the change permanent, have your developers implement it on your site, then end the Wingify campaign.

#7. Close and document the test

Set the campaign status to In Analysis while you review with stakeholders, then Ended. Add a learning from the report header so the result is recorded for your team.

#Check that it worked

• The rollout campaign appears in Experimentation > Web Rollout and in a tile on the original campaign's report.
• Your live site shows the winning version to all targeted visitors.

#Common questions

The test showed a winner early, then the results evened out. Why? Early data comes from few visitors and novelty effects. Run tests for at least a full week, and look at the day-wise view if you see large swings.

Why is there no Roll out option? It's only available for A/B tests, and not when the Control is the winner.

What does "Collecting Data" mean in the probability column? Minimum thresholds (by default 500 visitors and at least one conversion) aren't met yet. They are set in Observatory Mode.

Can I use the winner as the new control for my next test? Yes. Clone the campaign, keep the winning variation, disable the original Control and set the winner as the baseline with Use this as baseline.

#Learn more

• Navigate the Campaign Reports Dashboard
• Wingify Campaign Reports: Statistical Models and Inference Levels
• Navigate and Customize Your Campaign Report
• Filter Your Report Data
• Instantly Roll Out a Variation in Wingify
• What is a Smart Decision?