Remove AI widget

Hypothesis

If we remove the AI shopping agent widget from the PDP, then conversion rate and revenue per visitor will increase, because clearing an interruptive overlay from the mobile buy zone removes distraction and decision friction for the 99% of visitors who never engage with it, and eliminates off script AI answers and discount nudges that undermine trust and anchor order value down.

What we learned

Winner: removing the widget lifted CVR ~+2.4% relative (3.96% to 4.05%) and RPV ~+4.2%, with AOV up ~+1.8% ($67.66 to $68.86). 1. The AOV lift aligns with the widget's own admin data, where agent assisted sessions had lower AOV than unassisted ($69.01 vs $70.98). The agent's discount nudges anchored orders down; removing them recovered both conversion and order value. 2. Vendor dashboards overstate value. The agent claimed $24K in "agent assisted sales", yet the controlled test shows the store earns more without it. Judge widgets by A/B tests, never by attributed revenue. 3. Default on app embeds are not neutral. This one shipped bundled with the Clarity analytics app and was never deliberately chosen. Audit app embeds after every app install or update. 4. The engagement telemetry predicted the outcome: under 1% engagement, 1.1 turns and 7 second conversations meant the widget was visual noise at the decision point. These signals can pre-qualify future removal tests. 5. Pattern for this account: the skeptical 40+ mobile buyer converts better with fewer interruptions in the buy zone. Next: switch the widget off sitewide (single app embed toggle) and prioritize the pre checkout upsell popup removal test, which follows the same principle.