One automatic upsell (10% off)

Before

After

Hypothesis

If we replace multiple manually curated checkout upsells with a single AI-selected upsell at a consistent 10% discount, then Revenue Per Visitor and AOV will increase, because reducing choice paralysis at the final conversion step will improve checkout completion rates while the AI-driven relevance ensures higher upsell attachment on the single offer presented, offsetting any loss from showing fewer total products and creating a cleaner path to purchase that reduces friction at the moment of highest purchase intent.

What we learned

Individual Widget Performance RPV Formula: Sessions × Conversion Rate × AOV Conversion Rate = 100% - Widget Checkout Abandonment Rate 1. Writing Products Control (Version A): Sessions 1,109, Widget checkout abandonment 33.63%, Conversion rate 66.37%, Conversions 736 orders, AOV $30.16, Total revenue $22,197.76, RPV per session $20.02 Variation (Version B): Sessions 1,146, Widget checkout abandonment 33.68%, Conversion rate 66.32%, Conversions 760 orders, AOV $35.91, Total revenue $27,291.60, RPV per session $23.81 Performance: RPV Lift +$3.79 per session (+18.9%), AOV Lift +$5.75 (+19.1%), CVR Change -0.05pp, Orders Change +24 orders (+3.3%), Revenue Lift +$5,093.84 (+22.9%) Revenue Impact: Total visitors over 60 days 2,255, Average monthly visitors 1,128, RPV lift $3.79, Add Monthly Rev $4,275, Add Yearly Rev $51,300 2. Alphabet Bundles - Beginning Sound 4 MP Control: Sessions 1,907, abandonment 43.00%, CVR 57.00%, Conversions 1,087, AOV $25.15, Revenue $27,338.05, RPV $14.33 Variation: Sessions 2,018, abandonment 43.36%, CVR 56.64%, Conversions 1,143, AOV $25.26, Revenue $28,872.18, RPV $14.31 Performance: RPV Lift -$0.02 (-0.1%), AOV Lift +$0.11 (+0.4%), CVR Change -0.36pp, Orders Change +56 (+5.2%), Revenue Lift +$1,534.13 (+5.6%) Revenue Impact: Total visitors 3,925, Average monthly visitors 1,963, RPV lift -$0.02, Add Monthly Rev -$39, Add Yearly Rev -$468 3. Math Generic Control: Sessions 3,722, abandonment 38.26%, CVR 61.74%, Conversions 2,298, AOV $25.98, Revenue $59,701.04, RPV $16.04 Variation: Sessions 3,801, abandonment 36.52%, CVR 63.48%, Conversions 2,413, AOV $26.15, Revenue $63,100.95, RPV $16.60 Performance: RPV Lift +$0.56 (+3.5%), AOV Lift +$0.17 (+0.7%), CVR Change +1.74pp (+2.8%), Orders Change +115 (+5.0%), Revenue Lift +$3,399.91 (+5.7%) Revenue Impact: Total visitors 7,523, Average monthly visitors 3,762, RPV lift $0.56, Add Monthly Rev $2,107, Add Yearly Rev $25,284 4. Catch All - MP Control: Sessions 4,064, abandonment 41.66%, CVR 58.34%, Conversions 2,371, AOV $21.93, Revenue $51,995.03, RPV $12.79 Variation: Sessions 4,186, abandonment 41.14%, CVR 58.86%, Conversions 2,464, AOV $23.58, Revenue $58,100.72, RPV $13.88 Performance: RPV Lift +$1.09 (+8.5%), AOV Lift +$1.65 (+7.5%), CVR Change +0.52pp (+0.9%), Orders Change +93 (+3.9%), Revenue Lift +$6,105.69 (+11.7%) Revenue Impact: Total visitors 8,250, Average monthly visitors 4,125, RPV lift $1.09, Add Monthly Rev $4,496, Add Yearly Rev $53,952 3 out of 4 widgets showed positive lifts Combined Revenue Impact Verification Individual Monthly Impacts Sum: Writing Products $4,275, Alphabet Bundles -$39, Math Generic $2,107, Catch All $4,496, Total $10,839/month Individual Yearly Impacts Sum: Writing Products $51,300, Alphabet Bundles -$468, Math Generic $25,284, Catch All $53,952, Total $130,068/year Alternative Combined Calculation (Verification): Combined RPV lift $0.98, Combined monthly visitors 10,977, Combined monthly impact $10,757, Combined yearly impact $129,084 Note: Small difference ($10,839 vs $10,757 = $82) is due to rounding in individual RPV calculations. Using sum of individual calculations is more precise. Key Findings ✅ Variation wins on revenue metrics: RPV +6.6% ($0.98 more per visitor), AOV +5.4% ($1.33 higher average order), CVR +0.72pp improvement, Revenue +10.0% over test period ✅ Reducing checkout friction works: Replacing multiple upsells with one AI-selected offer reduced decision paralysis. Consistent 10% discount simplified the value proposition. AI relevance maintained monetization efficiency despite fewer options ✅ Performance was consistent across widgets: 3 of 4 widgets showed meaningful lifts (3.5% to 18.9%). Only Alphabet Bundles was flat (essentially neutral). No negative results. Writing Products showed exceptional performance (+18.9% RPV lift) ✅ Higher AOV drove results: Conversion rates improved slightly (+0.72pp overall). Strong AOV gains across winning widgets (+0.7% to +19.1%). Suggests the single AI upsell is better at surfacing higher-value complementary products Recommendation Ship the variation immediately across all checkout upsell widgets. The single AI upsell approach delivered a clear win: Improved checkout conversion rates, significantly improved average order values, projected $130K in additional annual revenue with $10.8K monthly uplift. The test validates that simplifying checkout decisions, after users have already seen cart upsells, creates a cleaner path to purchase while actually improving monetization through better product recommendations.