Experimentation When the Agent Makes the Decision
As of 22 October 2026.
The experimentation and personalization industry faces a question that standards bodies, security teams and regulators are now putting to it: may a site treat an agent differently from a human, and can the difference be measured at all? The thesis of this article is simple: when an agent enters the loop, experimentation no longer has one obvious unit, one obvious variant or one obvious decision-maker, and that changes its trust model as well. The WebMCP draft of 20 October 2026 already lists price discrimination based on attributes an agent passes to a site among its risks, so agent-facing personalization is in the threat model before it is a product feature. Optimizely is a useful example, because it has clearly bet on both sides of that equation: on agents that optimize the site, and on tools the site exposes to agents.
The argument runs in four steps, from principle to architecture: parity (may a site tell an agent from a human), trust (may it optimize the instructions the agent reads), measurement (can an experimental unit still be defined) and ownership (who controls the agent-facing experience at all). Everything else is evidence.
Figure 1. The argument in four steps: parity, trust, measurement and ownership. Each section below takes one.
What Optimizely promised at Opticon 2026
At Opticon 2026 in New York (31 August–1 September), Optimizely previewed “Agent-Ready Experiences”: a protocol that lets a site expose a structured menu of actions agents are permitted to perform, such as booking or form completion, which is WebMCP. Alongside it the company showed “Autonomous Optimization” and 1:1 personalization, agents deciding which content and structure belong on a page, arguing that conventional experimentation leaves too much of a site untouched. Its own recap also contained the thesis of this article in the vendor’s own words: 60% of web visits are now AI referral traffic and crawlers (citing Cloudflare data), so “your primary audience is no longer human”.
The London edition (13 October, Barbican) did not change WebMCP’s status. According to a partner’s recap of the OMVP Summit and Opticon, WebMCP and autonomous personalization remained “preview”, with some announcements realistically landing only in 2027; the news was “Virtual Teammates” (seven roles, including a CRO Manager and a Personalization Strategist), CMS 13 and Commerce 15 on .NET 10, and the agent platform connecting to remote MCP servers, Slack and Microsoft Copilot. In spring 2026 the company had also launched a remote MCP server for Web and Feature Experimentation.
The vision is coherent: the company’s agents optimize the site for humans, and the site hands the user’s agents a menu of actions. There is a new symmetry in it, too: a virtual “CRO Manager” will optimize pages that other people’s agents increasingly visit, so the A/B test starts to look like agent versus agent. The problem is that both halves of that vision are being challenged right now on principle. Nobody asks whether they will be built. The question is whether the rules now being written will allow them to be built the way they were shown.
1. Parity: WebKit, the W3C and EU law
WebMCP, the standard proposed by Google and Microsoft, works today only in Chromium browsers. Chrome has run an origin trial since version 149. On 28 September 2026 it extended the trial to version 162 (30 March 2027, per the webmcp.com tracker), but the request itself contains no shipping milestone. Since 8 September 2026 Chrome ships a new version every two weeks, so a request to ship could come at any point in the trial. WebKit (Safari) filed a formal position of oppose in June 2026, Mozilla went neutral in August, and the draft specification of 20 October describes JavaScript tools only; the declarative form-based variant remains in Chrome’s documentation and in Google’s 28 September request, and per the same tracker it was set aside in the draft on 8 October.
Google’s browser direction is clear. The web-standard question remains unresolved. At I/O 2026 (19 May) Chrome announced that Gemini in Chrome will “soon” support WebMCP and showed nine brands experimenting with the API, including Shopify, Expedia, Booking.com, Etsy, Instacart and Target. In its 28 September request to extend the trial, Google states that the API is meant to augment browser add-ons, so it rates non-adoption by other engines as a low risk; TAG review will be requested only before shipping. The dispute, then, is no longer only whether WebMCP will exist. Two other questions now matter more: whether WebMCP becomes a web standard or a single-engine API, and whether a site may tell an agent from a human. On that second question Google’s decision settles nothing; it makes the question urgent at Chrome’s scale.
At its core, WebKit’s objection is philosophical. In its view an agent acting for a user is assistive technology: it should operate the site as the user would, and the site should not single it out for different treatment. Since some users will not or cannot use agents, the outcome must not privilege those who have one. WebMCP makes “an agent is driving” separately addressable. And whatever is addressable can be personalized. The W3C’s accessibility group (APA) raised a related worry in August 2026: authors may start stuffing the accessibility tree with content meant for agents.
Mozilla, formally neutral, describes the same risk in other words: sites may provide tools that do not match the user’s experience on the page, in order to tar-pit automated browsers, inject instructions invisible to people, or collect data through tool inputs. Its reviewer also proposes a direction that matters more to vendors than the objection itself: standard tool sets for classes of websites. If a shop, a bank and an airline are to expose the same interfaces to agents, interface differentiation stops being an advantage.
The draft specification of 20 October has its own section on privacy leakage through over-parameterization: a tool that asks for age, location or pregnancy status turns the agent’s personalization context into a profile, and the authors list price discrimination among the consequences. The spec’s example is a dress search that asks for size and budget in the benign version and for age, pregnancy, location, height, skin tone and previous purchases in the malicious one.
EU law does not yet provide a clear, agent-specific answer. A CERRE paper by Christoph Busch notes that it protects consumers from dark patterns but offers no equivalent protection against the manipulation of agents acting on their behalf. The Commission has scheduled the Digital Fairness Act for Q4 2026; as of 22 October the proposal had not been tabled, and the Commission’s indicative meeting plan points to 11 November. For a personalization vendor the question is simple: is a variant that only the agent sees an optimization, or a manipulation of the person that agent serves?
The hard deadlines that follow (Directive 2024/2853 on product liability by 9 December 2026, the AI Act after the Digital Omnibus from 2 December 2026 and 2 December 2027) are in the calendar at the end of this article.
2. The tool description as the new variant
In WebMCP a tool is a name, a natural-language description, an input schema and a function. The description is read by the agent’s model, so the description becomes part of the context that shapes the decision, and the CRO industry’s natural reflex is to test descriptions, because a better description means a higher call rate. Sounds reasonable enough. The same mechanism is what the security literature describes as an attack technique when a malicious site uses it.
The June 2026 paper “WebMCP Tool Surface Poisoning” calls it tool framing: influencing the agent’s perception of a tool’s role through metadata fields such as the tool name, description, readOnlyHint and inputSchema. Chrome’s agent security guidance of 9 June 2026 warns about tool definitions with hidden instructions in names, parameters or descriptions, and its documentation recommends budgets: 500 characters per tool description, 150 per parameter description, 1,500 per tool output.
The numbers in that paper show how strong the lever is. Tested on GPT-5.4, Claude Opus 4.6 and Gemini 2.5, a tool description alone was enough to make the agent pick a foreign tool in 93% of runs on average; the readOnlyHint annotation alone, in 87%; description plus annotation, in 100% on all three models. A tool name with corporate semantics (audit_logger instead of diagnostic_probe) scored 39% with no change to the description. One caveat: the experiments ran on a polyfill, without Chrome’s native implementation, so they show influence and leave production exploitability an open question. The exact percentage matters less than the direction: metadata alone can materially change tool selection.
Call it the CRO–security collision. Traditional CRO optimizes what humans see; CRO for agents (call it agentic CRO for now) would optimize what models read; security already treats that surface as input it does not trust. And the conclusion for CRO is uncomfortable: the fields it would want to optimize are exactly the ones attackers use to steer agents, and the ones agent frameworks already label as untrusted. The documentation of Vercel’s agent-browser and of Kernel’s cloud browsers puts it plainly: page-provided names, descriptions, schemas, annotations and results are data, not instructions, and are not to be trusted.
There is no generally accepted line today between optimizing a description and manipulating an agent; in practice, agent security teams draw it. A vendor that builds A/B testing for tool descriptions enters territory where Google and OpenAI write the rules, and where agent-side safety systems may reject or constrain a tool that reads as manipulative. This is already happening: since 25 August 2026 the ChatGPT desktop app has offered WebMCP “site tools” to eligible accounts and models with nothing to configure (and a switch to turn them off in the browser settings), does not support tools from embedded frames, and reviews the tool and the data it would receive before every call.
3. The experiment unit
Today’s experimentation implementations assume that exposure, identity and outcome can be reliably attributed to a stable unit: a user, a session, a device. The problem is not the mathematics but the attribution, and that is what WebMCP blurs. Tools run inside the logged-in user’s session, so from the site’s point of view an agent’s call may look like a human’s action. Chrome warns that browser extensions can query and execute WebMCP tools through content scripts, so “the agent” may be anything installed in the browser. Anything at all. For now that is only a possibility: per the webmcp.com tracker of 15 October 2026, none of the largest assistant extensions (Sider, Monica, HARPA AI, more than 8 million installs between them) uses page tools yet.
The fact that an agent is not a human breaks nothing by itself; an agent could simply be another channel. The problem starts when several conditions hold at once: the agent’s identity is unobservable, the same agent makes the same decision for many users, one task is many tool calls, the agent’s decision corresponds to no UI variant, and the outcome is attributed to a different unit than the one that received the treatment.
A simple thought experiment. An A/B test on 100,000 users, 10,000 of whom act through an agent. The agent sees the same tool description for each of them and chooses the same way every time. Suppose the experimentation platform treats the result as 10,000 independent observations. Are they really 10,000 independent observations, or one decision multiplied by 10,000? And what exactly is the treatment: the UI variant, the tool, its description, the payload, or the agent’s interpretation? A shared agent decision does not by itself invalidate the independence of individual users’ outcomes; the trouble comes only when the unit of randomization drifts from the unit of analysis or outcomes are correlated, and the platform counts replicated decisions as independent exposures.
In short: a traditional experiment is user → exposure → variant → action → conversion. An experiment with an agent is user → agent → tool metadata → agent decision → tool payload → task outcome. Five things change at once: identity, treatment, decision-maker, outcome and attribution. Traditional CRO controls the treatment. Agentic CRO may have to optimize a treatment that the model interprets before the user ever sees the result; that is what changes the decision-maker. The most important change, then, may not be the experimental unit at all. It may be the location of the decision.
Figure 2. The experiment chain with and without an agent. The decision moves from the user to the model, and the treatment moves with it.
| Element | Human web | Possible agent-mediated model |
|---|---|---|
| Unit | user | user, session or agent task (unresolved) |
| Identifier | cookie | agent and session context |
| Exposure | page view | tool discovery or invocation |
| Event | click | decision or tool call |
| Conversion | purchase | task completion |
| Variant | UI version | UI, tool metadata or payload (candidate) |
| Intent | human | agent interpretation |
The most developed answer to agent identity, Web Bot Auth (the IETF webbotauth working group, signatures built on RFC 9421), is intended to verify operator identity and covers HTTP requests from cloud agents. Calls made inside a browser tab are outside its scope. Cloudflare and AWS WAF verify those signatures in production. But as of August 2026 the working group had not adopted a single document of its own. The consequences for experimentation: one user generates dozens of calls through different agents, one agent makes the same decision for thousands of users, and an agent’s conversion never passes through a UI variant.
There is also a simpler data-pollution mechanism. Shopify’s changelog says the cart tools call the same standard storefront actions that apps use: if a theme opens a cart drawer on updates, agents trigger it too. An agent’s cart change therefore fires the same analytics events as a human click, and the storefront events themselves do not distinguish whether a human or an agent triggered them. The first agent-channel data also cools enthusiasm for “agentic readiness” as a KPI: Pickrate, one of the first agent-channel analytics tools, notes that across its corpus a site’s readiness barely predicts whether agents pick it.
There are more such products already: Pickrate (whether agents pick you over rivals and whether those visits convert) and Conscriba (agent conversions by provider and model). Optimizely has taken a first step here: the “Agent Visibility” shown at Opticon measures AI citations, share of voice and which pages receive requests from machines. That is visibility measurement, though, and no more than that. Experimentation platforms should close that gap first.
4. Who writes your site’s tools: the layer is being built above the DXP
Shopify announced and switched on, on 5 August 2026, ten WebMCP tools on every Liquid storefront: catalog, cart, checkout, orders and store policies, with nothing to install or configure. Cloudflare enables WebMCP from a dashboard toggle and uses HTMLRewriter to inject a module that registers tool packs on the page.
On 21 October 2026 Commerce.com (BigCommerce) and Cloudflare switched WebMCP on for nearly 100,000 store domains at once, through Cloudflare’s edge, with no action from merchants (none at all). The model is on by default with an opt-out in the merchant dashboard; the agent gets tools such as search_products and add_to_cart, and checkout stays with the human. Commerce.com stresses that its MCP server for outside agents and WebMCP in the browser expose the same set of shopping tools, so a common interface for a whole class of stores, of the kind Mozilla described, is starting to emerge at the platform and edge layer, above the DXP, with no W3C standard involved. Cloudflare says its next step is making a site’s tools easier for agents to discover.
It goes further than platforms and CDNs. Brave has had a change open since 7 October that turns WebMCP on by default for all users of its Leo assistant, held until a security review is complete, and Vercel’s agent-browser ships a separate “webmcp-gen” workflow for sites that do not expose WebMCP. Add the Commerce.com model, where the tools stay on until the merchant switches them off, and parity gains a new dimension: a tool you did not write can fire your analytics events, while responsibility for the resulting flow may not be obvious. The variant may no longer live where the experimentation platform expects to find it, because the decision moves, the treatment moves, and attribution has to follow both. It raises a question experimentation never had to ask: who owns the variant? The CMS, the merchant, the CDN, the browser, the agent, the extension or the MCP runtime?
Figure 3. Seven layers at which a site’s tools can be written today. Only the first is where an experimentation platform expects the variant to live.
As late as July 2026 an independent survey of 111,076 top domains, published on dev.to, found zero WebMCP deployments. After the Shopify and Commerce.com moves, the tool layer exists on every Liquid storefront and on nearly 100,000 BigCommerce domains, even though deployment happened without merchant action. For DXPs the implication is hard to hear: “agent-ready” may become plain infrastructure, and then it stops being a platform differentiator. Value may therefore move above that layer: toward deciding what to expose, how to describe it and how to measure it.
What this means for experimentation vendors in 2027
- Parity testing as a product: do the agent and the human receive the same offer. Probably a compliance requirement first, a feature later.
- Govern tool descriptions as content: versioning, localization and security review. A natural role for the CMS.
- If tool descriptions turn out to be too security-sensitive to optimize, the more controllable experiment surface may be the payload: what search_catalog returns, in what order and with what data is merchandising for agents. Ranking, availability, recommendations and response structure are better test material than tool descriptions, although payloads are not risk-free either: the specification and Chrome also warn about instructions hidden in tool outputs. And if tool sets become standardized, it is where most of the differentiation will be left.
- A new unit of analysis alongside the user: experiments where the user remains the randomization unit while a tool call or an agent task becomes the unit of analysis, with task completion as the metric.
- Agent-channel analytics: which agents call which tools, with what outcome, and how much of it is extension traffic. Today the most visible mainstream example of an agent using WebMCP with no user-side configuration is the ChatGPT desktop app; Gemini in Chrome is expected to join in early 2027.
- Standards watch: TPAC 2026 (26–30 October, Dublin), the Agentic Web Working Group charter proposed on 28 September and the end of Chrome’s trial on 30 March 2027 will decide whether WebMCP leaves Chromium.
Conclusion
The strongest objection to this article is that it extrapolates from an experimental specification, a polyfill-based security paper and early deployments. Agreed. Nobody here claims the future has arrived. The claim is more modest: some of the assumptions behind today’s experimentation systems stop holding once an agent makes the decision, and that is already happening in ChatGPT, on every Shopify storefront and in Chrome’s trial.
This is not an article against Optimizely; if anything, it is credit, because the company put both sides of this problem on stage earlier than most, and the questions in this article are the cost of going first. It is a question for everyone selling experimentation and personalization. When an agent enters the loop, three things today’s experiment stands on start to wobble: who was exposed, what the variant was, and which decision generated the outcome. Whether agents will become another channel is no longer the question. The question for platforms is whether the existing experiment model can measure that channel without confusing the user, the agent and the decision. What does an A/B test look like when the variant is a tool description, the visitor is still human, and the decision-maker is an agent?
Appendix: dates to watch (as of 22 October 2026)
| Date | What happens | Why it matters |
|---|---|---|
| 22–23 October 2026 | AGNTCon + MCPCon North America, San Jose (AAIF / Linux Foundation) | Happened: on 22 October AAIF announced four new projects (Agent Skills from Anthropic, Open Responses from OpenAI, OpenEnv from Hugging Face, OpenSandbox from Alibaba) and 305 members. The center of gravity for agent standards now sits at the Linux Foundation; a week later the W3C has to show at TPAC that the web is in that conversation |
| 26–30 October 2026 | W3C TPAC 2026, Dublin: agentic-web breakouts on 27, 28, 29 and 30 October; Agentic Web WG charter discussion on Friday 30 October | Decides whether WebMCP enters the W3C standards track or stays a single-engine API; Google intends to ship it either way |
| 14–20 November 2026 | IETF 127, San Francisco; the Web Bot Auth working group | Whether agent identity finally gets working-group documents |
| Q4 2026 (Commission plan; indicatively 11 November) | Digital Fairness Act proposal | Rules aimed at dark patterns and “unfair personalization”; per Commissioner McGrath the DFA is meant to close gaps rather than regulate specific technologies, so it could reach agents only indirectly |
| 2 December 2026 | AI Act after the Digital Omnibus (provisional agreement of 7 May 2026): new prohibitions and the duty to mark AI-generated content | The first hard AI Act deadline for vendors |
| 9 December 2026 | Transposition deadline for Directive 2024/2853 | The new product-liability regime explicitly covers software and may reach agent vendors, depending on their role |
| Early 2027 (webmcp.com tracker estimate; Google has said “soon” since May 2026) | Gemini in Chrome with WebMCP support | A browser-integrated agent with page-tool support, at Chrome’s scale |
| 30 March 2027 | End of the WebMCP origin trial in Chrome (through version 162) and in Edge, per the webmcp.com tracker | By this date Google must ask to ship or extend again |
| 28–29 April 2027 | AGNTCon + MCPCon North America 2027, San Francisco | Next checkpoint for MCP and agent standards |
| May 2027 (customary) | Google I/O 2027 | The natural moment to announce “WebMCP ships” and the Gemini in Chrome status |
| 15–16 September 2027 | AGNTCon + MCPCon Europe 2027, London | European edition |
| 2 December 2027 | AI Act: obligations for stand-alone high-risk systems (Annex III) | Applies to agents used in Annex III areas (employment, credit and insurance, essential services, among others), not to every decision-making agent |
Sources
- WebProNews: Commerce.com turns on WebMCP for nearly 100,000 stores through Cloudflare’s edge (22 October 2026)
- Commerce.com and Cloudflare: press release (21 October 2026)
- Cloudflare Blog: WebMCP for Commerce.com storefronts
- AAIF: four new projects, AGNTCon + MCPCon NA (22 October 2026)
- WebKit standards-positions #670: position on WebMCP (oppose)
- OpenAI Help Center: Using site tools in the ChatGPT desktop app
- Brave brave-core PR #40616: Enable WebMCP by default (7 October 2026)
- EVZ Deutschland: Digital Fairness Act - status as of 2 October 2026
- IETF: IETF 127 San Francisco, 14–20 November 2026
- OpenAI Developers: WebMCP support announced for the ChatGPT desktop app (25 August 2026, via Gigazine)
- Scott Reed (OMVP): WebMCP is coming to Optimizely - a strategic view, including Agent Visibility (8 September 2026)
- Privacy Laws & Business: Commissioner McGrath on the Digital Fairness Act - gaps, not technologies
- Optimizely: Camp Opticon 2026 - New York recap (Shafqat Islam, 1 September 2026)
- WebMCP specification, Draft Community Group Report, 20 October 2026
- Mozilla standards position on WebMCP, issue #1412 (neutral)
- Chrome for Developers: 15 updates from Google I/O 2026 (19 May 2026)
- blink-dev: Intent to Extend Experiment: WebMCP (28 September 2026)
- CMSWire: Opticon 2026 and WebMCP at Optimizely
- Opticon 2026 New York: dates
- CMSWire: Optimizely remote MCP server
- WebMCP ecosystem tracker (webmcp.com, 15 October 2026)
- Chrome: two-week release cycle from September 2026
- WebKit and Mozilla positions on WebMCP (documentation.ai)
- WebKit’s assistive-technology argument (Wislr)
- W3C APA minutes, 5 August 2026
- CERRE: Agentic AI and Consumer Protection
- European Parliament: Digital Fairness Act (legislative train)
- Gibson Dunn: Digital Omnibus agreement (AI Act)
- Directive 2024/2853 and agent vendors (ActuIA)
- arXiv: WebMCP Tool Surface Poisoning (Lee, Chang, Yu, Yeh, NYCU, 4 June 2026)
- Vercel agent-browser: WebMCP (page data as untrusted)
- Chrome: agent security guidance for WebMCP
- InfoQ: WebMCP character budgets
- Chrome for Developers: WebMCP (extension warning)
- IETF: Web Bot Auth working group
- NerdLevelTech: Web Bot Auth status, August 2026
- Pickrate: agent-channel analytics
- Conscriba (BetaList)
- Shopify changelog: WebMCP on storefronts
- Shopify rollout dates (Idukki)
- Canopy Management: Shopify tools and storefront events
- Cloudflare WebMCP bridge (Martech.zone)
- Brave: WebMCP in Leo, issue #55232
- Kernel: WebMCP in cloud browsers
- WebMCP adoption survey, July 2026 (dev.to)
- W3C strategy #576: Agentic Web WG charter
- TPAC 2026 breakout sessions
- Linux Foundation: AAIF 2026 events program
- AGNTCon + MCPCon 2027: dates