A11Y.md · the first formal move

The Remedy, the Placebo and the Magic Phrase

Or: when claiming accessibility without measuring stopped fitting inside A11Y.md.

I've spent the last few months telling people that AI-generated interfaces can be born accessible. All it takes, according to my own thesis, is giving the agent the right context before the first line of code. That's the premise of A11Y.md, and I believe in it.

If you're just arriving: A11Y.md is a plain markdown text file that, once plugged into a code-generation assistant or development environment, makes accessibility a premise of every AI-built interface. It has a core with the essential rules, a set of specialized guides the agent fetches only when the task calls for them (form, modal, table and so on), and a body of documentation and standards from the global entities that define and validate accessibility (WCAG 2.2, ISO 9241-171, WAI-ARIA APG, ADA, Section 508, EAA, EN 301 549, ABNT NBR 17225:2025 and others). On top of that, during and after generation, A11Y.md instructs the model to produce reports, record its decisions and document any exceptions.

But I've learned one thing from a lifetime of watching product:

A promise without a number is an opinion. And the author's own opinion is worth even less.

What was missing was me asking myself the uncomfortable question: what if I'm wrong?

(One honest warning before we go on: this was the project's first formal move, made with method borrowed from science, a budget of zero, and an enormous willingness to find out things I didn't want to hear. What follows is the account of it.)

How you test a remedy

When a lab wants to know whether a remedy works, it doesn't ask the inventor. It builds groups: one takes the remedy, another takes a flour pill identical on the outside — the placebo — and the results are compared. The recipe is registered with a notary before anything starts, so nobody can move the goalposts mid-game. And the doctor who scores the results doesn't know who took what, so that bias can't contaminate the diagnosis.

That is exactly what I did with A11Y.md.

Four hundred times, the same AI model (Gemini Flash Lite, small and free) received the same list of interface tasks (form, table, carousel, modal), designed around the most common error categories in the WebAIM Million.

Four generation conditions were tested:

• Nothing — the model got the task alone, with no extra instruction;
• The magic phrase — the task plus the request "make it accessible";
• The placebo — besides the task, a document with the same size, same structure and same mechanics as A11Y.md, except about performance, without a single word on accessibility;
• The real A11Y.md — the model received the verbatim invocation, "When developing the frontend, follow strictly the accessibility rules defined in A11Y.md", pointing at the file on disk. The exact invocation for each condition is in the protocol, and the full standard is published in the project repository.

The entire protocol was registered publicly before the first data point existed, at the OSF. Every generated page went through an automated accessibility auditor (axe-core, the most used checker in the world, frozen by cryptographic verification so nobody could swap the measuring stick along the way), and the part that required human judgment I judged blind: the pages were renamed to codes, and I only learned which condition each one came from after the last verdict. Including my own rejections.

Every piece of that apparatus (the placebo, the blinding, the analysis decided before seeing the data, the registration out of my reach) exists for a single purpose: to strip the author of the power to fool himself.

It's worth highlighting that the performance placebo document exists to test the hypothesis that any large document, organized exactly like A11Y.md and inserted into the model's context, would improve the output.

And a word about what is genuinely new in this study. The WebAIM Million census has been measuring the errors of the human-built web for years, and recent academic research has already measured AI-generated code: most pages are born with barriers, and asking for accessibility in the prompt fixes part of what a robot can detect. What nobody had tested was the layer where A11Y.md lives — the convention file read by agents. As far as I know, this is the first preregistered, blinded, placebo-controlled evaluation of a file of this kind.

Study 1 — isolated components, four hundred pages

The registered bet said that, with the right context in place, the violations a robot (axe-core) detects go down, and the verification process shows up without anyone asking. To measure it, the auditing robot went page by page, and blind human judgment handled the items only a person can decide.

Before the scoreboards, one term that will keep coming back: flagged element. It's each individual button, text or image the robot marks as a violation. If a page has a single type of error that hits twelve different elements, the robot counts twelve. Hold on to that distinction, because it will matter a lot in Study 2.

(You'll notice the human-judgment denominators differ across the four conditions. That's because the draw is blind: each condition enters the sample with however many pages the dice hand it. Evening them out by hand would be cheating.)

Against the placebo, A11Y.md won comfortably. In the registered statistical model, which compares task by task, the rate of serious violations dropped to one third. In the table's raw count, 61 elements against 122. And the big document alone cost dearly whoever received it: it was the condition with the fewest 100% clean pages (60%, against 70% for those who received nothing), it spread contrast problems across more pages than "nothing" itself (29 versus 22), and it was the only one to get the document itself wrong. What does the work in there is the content, not the volume.

(Two footnote honesties, both declared in the protocol before any data. First: A11Y.md has been public since April and may have entered the models' training data; the placebo was written for this study and therefore isn't in any AI training dataset. If that weighs anything, it weighs in A11Y.md's favor, and you've been warned. Second: in the matchup against "nothing", the two registered statistical accounts disagree. One sees a clear effect, the other sees a borderline result. I publish both, and no conclusion in this text depends on that matchup.)

The study's only 100%-against-0% scoreboard

Before any tie, the result no metric ordered — and, for anyone who uses the project, the main finding.

In 27 of 27 runs with real agents (Claude, Codex and Gemini), A11Y.md made the agent spontaneously produce the process documentation the standard mandates. The verification report came out in 26 of them. The decision log, in 21. At least one of the two, in all 27. Without A11Y.md, zero of 27 produced either. Mandating these documents is the very nature of the project and doesn't exist natively in AI models, which naturally explains a gap this wide.

Study 2, further ahead, closes the account still missing here, because there the magic phrase competed too. Across the journeys, the complete pair of documents came out in 10 of 10 with A11Y.md. With nothing, zero of 10. With the phrase, zero of 10. No other measurement in either study produced a hundred percent against zero. And it's worth saying what an audit can and cannot do: it can demand documentation after the code is done; what it cannot do is make that documentation be born together with the code. A11Y.md can — by induction, beforehand.

The magic phrase "make it accessible" begs the model for goodwill, while A11Y.md turns accessibility into premise and process.

When we compare A11Y.md against the magic phrase, on what axe-core sets out to measure, we get a tie. On the small model I used as my workbench, asking to "make it accessible" knocked down the automatic errors almost as much as the whole standard did: 68 elements against 61. In the registered statistical model, the ratio between the two conditions came out at 0.76, with an uncertainty margin running from 0.38 to 1.53. Since that margin crosses 1, the statistics don't license saying either condition is better. The number was born inside a registered protocol and goes in here exactly as it came out, because a study where the author wins at everything doesn't deserve your trust. This one wins where it wins and ties where it ties.

And there's a frame I only went looking for later, in the literature: earlier studies were already measuring exactly this. Asking for accessibility in the prompt fixes most of what a robot detects. The result was the expected one, and it is fully aligned with this project's vision of the future, which is to make accessibility simple — as banal as asking the AI for something accessible, or not even needing to ask. The question that remains, and that the rest of this text answers, is what the phrase does not buy.

What is the nature of these errors?

Three readings come out of that:

• Any mention of accessibility nearly zeroes out the basics of the web. From 40 elements with no instruction to 4 with the magic phrase and 2 with A11Y.md.
• The placebo charged in a different currency. It spread contrast problems across 29 pages, more than "nothing" itself (22), and it was the only one to ship pages with no title and no declared language. A large context about the wrong subject occupies the model's attention.
• The routine honesty. Saying "make it accessible" made the model get fewer contrast elements wrong (35 against A11Y.md's 46), even though A11Y.md wins on the number of pages affected (9 against 10).

The deep dive into contrast — and an honest hypothesis

I went after that contrast result. A missing rule it is not: the rule is in A11Y.md's core ("text MUST 4.5:1"), loaded in all 100 generations. Nor is it just the in-depth guide being left out — though it is, hanging on a trigger of words like "color" and "palette" that no prompt ever says, so only 13 of the 100 generations read it.

The decisive fact is another one: of the 9 generations that violated contrast, 2 had read the entire guide and violated it anyway.

The most likely explanation, which I record as a hypothesis, is that the model reads "4.5:1", agrees, and cannot calculate the luminance between two color codes by looking at them. Whether it chose to ignore the rule or tried and failed, no log reveals. What the data shows is that knowing the rule wasn't enough.

Alt, label and dialog are structural: to know is to do, and A11Y.md zeroed them. Contrast is computational: knowing is not being able to check.

The hypothesis is testable, and the roadmap at the end bets on it — with one methodological caveat: a ready-made palette fixes contrast under either explanation (missing arithmetic or a taste trained on elegant gray), so it corrects, but it doesn't decide. That will be tested.

(The recent literature points the same way. In the studies of LLM accessibility repair, contrast is the violation models most fail to fix — the best rates sit near 48% — and there are records of the classic symptom of someone who doesn't calculate: forcing black text on a white background to guarantee a pass. The reason described by AccessGuru is the same as my hypothesis: an element's real color only exists on the rendered page, after inheritance and style overlays, and the model works by reading code, not seeing screens. The answer that line of research proposes is delegating the calculation to a deterministic tool instead of trusting the model's reasoning — exactly the discussion in the first item of the roadmap, at the end.)

Contrast is an arithmetic problem, not a knowledge problem. And a language model doesn't do luminance arithmetic by eye.

Figure 1 · The same task, the same agent — and what the eye can't see

Where human judgment saw what the robot can't see

The robot (axe-core) only measures what it can reach. On the items that require a person (keyboard focus properly trapped inside a modal, the honest size of a touch target), blind judgment told a different story: A11Y.md's pages passed 5 of 5; the magic phrase's, 2 of 3. Every modal in the A11Y.md condition used the browser's correct native element. The failures were imitations: the look of accessible, the mechanics of a prop.

(The exact size of that human layer: I was the judge, not knowing which condition each page came from, evaluating two item types. It's too little for statistics, and I don't treat it as a scoreboard. What holds this section up is the pattern that kept repeating — a true modal on one side, an imitation on the other. Figure 3 shows both in code. For the study's formal version, a second independent judge is already in the plan.)

Figure 2 · Two open modals — and the difference the photo doesn't show

Failed blind<div class="modal"
     role="dialog"
     aria-modal="true">
  …
</div>
the promise of accessible in the attribute;
not one line of keyboard in the file.
Tab leaks, Esc doesn't exist, focus is lost

Passed blind<dialog aria-labelledby="titulo">
  …
</dialog>
dialog.showModal()
the browser's native element:
inert backdrop, Esc out of the box,
focus returned to the origin button

The real agents, page by page — no sampling

The blind judgment above covers a drawn sample; the real-agents arm is small enough to show everything. Fifty-four runs — Claude Code, Codex and Antigravity, official clients, each in a pristine folder, with A11Y.md delivered by the same one-line rule any adopter of this project is supposed to use in their setup.

Below, the serious violations of every page, page by page (these are the same 27 runs from the 100%-against-0% scoreboard). Each cell lists the agent's nine pages in that condition, cleanest to worst, and each number is one page's count of serious violations:

The asterisk is a finding. The 12 violations on Antigravity's only non-zero page come from a single rule: ARIA applied where it is prohibited, repeated 12 times. For someone who depends on a screen reader, this error is not abstract: the prohibited attribute makes the reader announce the wrong information, or go silent where it should speak. It's a control that lies to a blind person. The fix is one line of code; the harm, until it comes, repeats on every visit. Hold on to this pattern of systematic error, because it comes back in Study 2 with a name and a figure.

And two things happened in these runs that no violation counter captures.

The first is about where the work ended up: With A11Y.md present in the project folder (the folder where the site is being built, where the user expects to receive the files), all three agents built inside it, like workers on a job site. With nothing in the folder, more than once the agent treated the request as a chat and left the page in a temporary system directory, where the user wouldn't even know to look. The A11Y.md standard gives the agent the sense of being inside a project, and it uses the folder correctly.

The second is time: With A11Y.md, Claude Code took 3.8 times longer to deliver; Codex, 1.7; Antigravity, 2.8. It's the time of reading, verifying and documenting — the price of running the whole protocol. Now it's a number, not a guess, and it was one of the reasons for designing the second study, coming right up.

(A note of method about this real-agents test: it ran only the "nothing" and "A11Y.md" conditions, because 54 runs don't give statistical power for the active controls of the 400 generations. Nor does it prove the main study's numbers repeat here, since machine and model change together from one test to the other. What it demonstrates is simpler, and still valuable: the same effect, in the same direction, in real agents from three different vendors.)

Study 2 — a whole site, a single session

Then I wanted to know about the real world: not one component at a time, but a complete site in a single session, the way anyone uses an agent today.

I registered a second study (again, the measuring stick notarized before the data) and had the agents build a seven-screen bookstore, thirty times. Two agents did the work. Claude Code, running the most capable model in the study. And Antigravity, running a Gemini Flash — a direct relative, one step up, of the small model from Study 1's workbench. The registered plan had Codex in Antigravity's place, but it hit the subscription's usage limit right out of the gate, and the registration already provided for the substitute and the swap rule, documented with a date.

One warning that holds from here to the end: each agent is compared only with itself across the three conditions. Nothing here compares one agent to another. The registered bet said that, in real use, A11Y.md makes the screens more consistent with each other, and the chosen ruler counted how many different ways the same component was built across the site. The fewer variants, the better: that was the premise I froze before collection. Hold on to that sentence, because the study itself will correct it further down.

Figure 4 · One journey = seven screens, a single session

one of the study's 30 journeys: catalog, search, book, cart, orders, sell and dashboard

The number came out dry: A11Y.md did not win on the consistency ruler. It stood side by side with the magic phrase, and on one of the agents the "nothing" condition produced the most uniform pages of all. It gets published like this, because this is how the ruler measured it.

But what mechanism produced these results? So I went and read the pages. All of them. And the mechanism changed the reading without changing the score.

The rubber stamp: uniformity through poverty

The champion uniformity of "nothing" was the uniformity of barrenness: the same poor page, stamped out again and again. Whoever receives no criteria has no reason to vary. Nor to get it right.

Look at three screens from the consistency-champion journey, and then the same three screens from the flawless journey with A11Y.md. From a distance, they look like relatives — that visually similar. The difference lives where the eye can't reach, which is why each figure carries its own tallied "inside view": in one, zero accessibility attributes across three screens; in the other, forty-eight.

Figure 5 · The ruler's "perfect" journey — no instruction, zero variants

orders · cart · dashboard — the same minimal table three times; nothing varies because nothing tries
inside the 3 screens: 37 interactive elements · 0 ARIA attributes · headers without scope · no captions

Figure 6 · The flawless journey with A11Y.md — consistent where it should be, rich where it needs to be

the same model, the same tasks — zero violations and zero unjustified variants
inside the 3 screens: 60 interactive elements · 48 ARIA attributes, verified: zero violations · headers with scope · 2 captions

The adaptation: when varying is getting it right

And the "variation" the ruler docked from A11Y.md was, for the most part, correct adaptation. There's an invisible notice called aria-sort: it tells the screen reader "this column is sorted, in descending order". In a real journey from the A11Y.md condition, the orders table, which the user can sort, carries that notice in its headers; the cart table, which doesn't sort, doesn't carry it. That's how it should be: announcing sorting where none exists would be lying to whoever listens to the page. The ruler counted this as "2 variants" of the same component (table) and docked A11Y.md for doing exactly what accessibility mandates.

Figure 7 · The "2 variants" the ruler punished — same journey, A11Y.md condition

The lesson the ruler taught me

A frozen ruler takes no appeals after the game, so the number stands. But it taught me to correct the premise I myself had frozen.

For accessibility, consistency was never "being identical all the time". It's being predictable where predictability serves the user, and adapting where context demands it.

The variant-counting criterion mixes three behaviors that need separate names:

• The constancy of doing well, always.
• The uniformity of doing little, always.
• And the adaptation of doing differently where context demands.

Doing less, with less quality, also varies less. The ruler I froze for this study (v1) sees all three as the same thing. The instrument's next version (v2), built with this study's dated examples as its own test cases, will tell them apart as it should.

Where did the CSS (stylesheet) go?

The second block of findings was born from another mistake of mine — the third this process caught.

While assembling an illustration from the screens, I noticed they had no styling at all. I dug in and found that my verifier's server labeled every file as HTML, and a browser that receives a stylesheet with the wrong label discards it silently. Result: the robot had audited all 210 screens with no CSS applied. This matters because CSS is not decoration for accessibility — it's where sizes, margins, spacing and COLORS live. Without color there is no contrast to judge. I fixed the label and measured everything again. Both measurements stay published.

With the CSS in place, the story changed.

On the large model, "no instruction" is a contrast disaster: in a typical journey, 45 flagged elements, gray-on-gray pages.

With the magic phrase or with A11Y.md, zero.

And zero on both sides means the robot hit its own ceiling. When both conditions zero out, the instrument has nothing left to measure, and whatever difference remains between them lives where it cannot reach.

On the small model, a single ruler doesn't tell the story, because "counting violations" mixes different questions. Table 5 brings five rulers, each explained and labeled in its own row.

(How to read the table: Study 2 was registered as a descriptive study. It's five journeys per condition on each agent, and five is too few for statistical proof, so it estimates and describes rather than decrees. The consistency ruler is the registered primary one, and its result stands. The other rows are readings I built after seeing the pages, and they're labeled as such. None of them separates the conditions with statistical certainty. They are directions and mechanisms, not victories.)

The table's gradient exposes the error's mechanism — and mechanism, here, means the pattern of how the model errs, not a methodology term. The more a ruler weighs repetition, the worse A11Y.md looks. The more it counts decisions, the better. The explanation is in Figure 8: nearly all of A11Y.md's volume comes from a single wrong mold, replicated across the whole site.

The temptation here would be a subtraction of convenience. Take the 98 elements of the single attribute in Figure 8 away from A11Y.md and declare a tie with the phrase. Not allowed. The magic phrase also has its own multiplying single error — a journey with 16 images missing alternatives — and honest subtraction applies to every arm or to none. Applied to all, it already exists in the table and has a name: it's the wrong-decisions ruler, where each error counts once no matter how many times the mold repeats. On it, the score is 9 with nothing, 11 with the phrase, 8 with A11Y.md. But the basic errors of the web zeroed out on one side only.

The rulers say different things because they serve different people. The first reading is the one that hurts. For the navigator — the person with a disability — the free phrase produces fewer stumbles today than A11Y.md on this small model. That ruler comes before the others, and it's the first one the roadmap at the end attacks. That said, I take my position instead of ending up on the fence. For whoever adopts and maintains the project, the fixer's and the shipper's rulers are the ones that say the most, and on them the direction favors A11Y.md, with no statistical separation at this sample size. The one who decides isn't me. It's ruler v2, commissioned with its hierarchy and judge declared before the next round.

Table 6 goes deeper into the wrong-decisions row, answering what nature each one has:

What the model built (real journey)<ul role="menu" hidden>
  <li>
    <a role="menuitem">All Categories</a>
  </li>
  …
the menu pattern is almost all there,
but the bare <li> between menu and
menuitem breaks the ARIA hierarchy:
98 flagged elements across 7 screens

What was missing<ul role="menu" hidden>
  <li role="none">
    <a role="menuitem">All Categories</a>
  </li>
  …
one attribute per item.
the single bug that clears 98 elements
at once: the error of one who knew the
pattern and missed the step

And Figure 8 is the rule, not the exception. Across the study's 18 journeys that had any failure, in 12 of them a single rule concentrates three quarters or more of the flagged elements: agent error, when it comes, comes systematic — one wrong mold replicated across the whole site, not scattered chaos. That changes the price of the fix: it's one repair, not a hunt.

A11Y.md zeroed out two of the most persistent error classes of the last two decades — missing label and missing image alternative — across every journey.

The magic phrase let the basics leak. And the errors left over for A11Y.md are of a different species: the errors of someone building something sophisticated on a model that almost gets there.

(The WebAIM Million census reminds us the web's biggest accessibility debt is another one: color contrast, present on 83.9% of pages. It's exactly where A11Y.md still fails, and where the roadmap attacks first.)

And I record the uncomfortable hypothesis, because it deserves a name. The rival explanation for Figure 8 isn't "an attribute was missing". It's "ambition was left over". A navigation made of plain links, with no role at all, would have zero violations. The 98-element error only exists because the model attempted the sophisticated ARIA pattern and stopped mid-staircase — and no uninstructed journey ever attempts that staircase.

If A11Y.md pushes the small model toward structures it can't finish, the remedy has a side effect. The accessibility field has an old principle for this: no ARIA is better than bad ARIA. The answer went into the roadmap with a declared success criterion: this error class must disappear in the next rounds, not be patched case by case.

The magic phrase and A11Y.md fail on different floors of the building.

And the token bill?

Study 2 also existed to settle an account Study 1 had left open. Think of A11Y.md as the dose of a remedy. In the first study, by design, it was re-administered whole with every task: full dose, every time. That measures the ceiling of the cost, not the cost of use, because nobody takes the whole remedy again at every symptom. The patient takes it once and lets the effect work over time. In Study 2's whole site, the dosage is the real one: the agent reads A11Y.md once, at the start of the session, and the effect lasts the whole session, with the model's cache doing the rest.

(Both rows measure the same small model. "Per screen" is the session total divided by the journey's 7 screens. The placebo didn't enter Study 2 by registered decision: the question it answers had already been answered, and a journey costs hours.)

The number, without makeup: with A11Y.md, each screen costs more in fresh tokens. 2.1 times more on the table's small model. 1.4 times on the large model, which spends 46k per screen with no instruction and 62k with the standard. Far from the factor of ten that Study 1 suggested to anyone reading only the ceiling, but far from free.

Where the surcharge comes from

I broke the small model's bill into two line items:

• Reading — all the fresh context entering the session: documentation, the site's own files, the reasoning. Accounts for 128 of the 142 thousand extra tokens per journey.
• Writing — the code and documents going out. Accounts for 12 thousand.

And here lives an honest surprise. The standard's entire library is about 44 thousand tokens and is read at most once, so it explains at most a third of the surcharge. The rest is the cost of the fuller treatment the standard prescribes. The agent rereads its own screens, which got richer. It consults and updates the report and the decision log. It thinks more before delivering.

This corrects an expectation I myself had. I expected that the longer the journey, the more the cost would dilute, since the reading happens only once. The big dilution did happen, and Table 7 shows it: re-administering the dose per task cost 10 times "nothing"; the single dose per session costs 1.4 to 2. But from there the curve doesn't drop to 1. The part that dilutes — the single read — is a minority of the surcharge. Longer journeys tend toward a floor, generously estimated at 1.3 times on the large model and 1.7 on the small one, because what gets expensive is not reading the prescription, but following the treatment it prescribes. This study doesn't measure that curve, because every journey has 7 screens. Measuring it became a roadmap item.

The mechanism that holds the bill down

The economics of on-demand loading was measured, not assumed. The agent reads the core (~10k tokens) and fetches only the guide the task calls for, not the entire ~45k library. In Study 1:

• The right guide from the trigger table was loaded in 88% of generations. The placebo, with the same map, hit 96% — the index works regardless of subject.
• There were 3.3 file reads per generation with A11Y.md, against the placebo's 4.7.
• In the human judgment of the extra reads, 70% of A11Y.md's were justified by a rule written in the standard itself (77% for the placebo). The rest is the model's initiative — counted and published, not punished.

The economy is not the standard being ignored. It's the standard being well indexed. And it's what explains the placebo costing twice as much as A11Y.md (50 versus 26 thousand) in the table's top row.

What the tokens buy

In the regime of Study 2's bill, the price difference paid for:

• The verification report and the decision log in 10 of 10 journeys, against zero in 20 without the standard, the phrase's 10 included. These are the documents audits charge dearly to reconstruct afterwards.
• The basics of the web zeroed out. It sounds like the bare minimum, but, as we've seen, it's a big deal.

The earlier regimes, each in its own currency, had already shown the rest: human judgment at 5 of 5 on Study 1's workbench, and some governance artifact in 27 of 27 real-agent runs.

Honestly, these documents are only a process trail that counted presence, not quality. No report has had its content audited yet, and a report written by the AI itself is not an audit, nor a substitute for testing with assistive technology. Validating that layer is an open roadmap item. For the record: A11Y.md's own protocol requires human auditing (human in the loop), and without it, it claims no accessibility.

On what the robot measures, the magic phrase is accessibility's generic drug: it delivers the same active ingredient for the price of one line. What doesn't come in the box is the rest of the package. Whoever buys only the output pays less.

Whoever buys the output with a process trail, governance and decision memory pays between 40% and 100% more in fresh tokens (in total tokens, with cache billed at a reduced rate, between 1.5 and 2.6 times) but takes home a governance package the magic phrase doesn't deliver.

The A11Y.md user pays more tokens and receives more than tokens: they receive process, records and memory.

What these findings already changed in A11Y.md's roadmap

This study ends the way I wanted it to end: becoming an agenda. Six changes have already entered the roadmap for the next versions, each with the finding that commissioned it.

• Contrast leaves the reasoning and goes to the tool. If the hypothesis is right, asking the model to calculate contrast is asking it to fail. Two candidate routes, decision open: providing pre-verified color palettes to copy, or embedding a deterministic check in the protocol, called on demand the way the guides already are. The repair literature points to the second — delegate the calculation to a tool instead of trusting the model. And along with it goes the test that separates the two possible explanations, not knowing how to calculate versus preferring the pretty gray of its training: asking the model to judge ready-made color pairs, besides producing them. If it judges well and produces badly, it's trained taste. If it judges badly, it's arithmetic.
• ARIA composition with a handrail, and a preference for the simple. The 98-element error was one step (li without role="none" inside a menu). The guides will now show the complete hierarchies of composite patterns and, more importantly, mandate preferring native HTML when the composite pattern isn't required. Plain links have no steps to miss. The success criterion: the "near-miss composition" class must disappear from the next round.
• Rebalanced loading triggers. The visual-perception guide was read by 13 of 100 generations, because its triggers ("color, palette") appear in no prompt, even though every page has color. Triggers will now reflect what every task does, not only what the request says.
• Consistency ruler v2. Separate constancy, uniform poverty and correct adaptation, using this study's dated examples as the instrument's own test cases. And with the ruler hierarchy (which one rules, for whom) registered before the next round, not after it.
• The amortization curve as an outcome. Cost was measured at a single journey length, 7 screens. Knowing whether the surcharge dilutes on longer journeys requires journeys of 3, 7 and 14 screens, with cost per screen as a registered outcome. The bill's decomposition already offers the hunch: the cost of reading dilutes, the cost of care doesn't. The round will test it.
• The human eye as a protocol step. Four instrument defects were found in this project, all four by human contact with the object, never by a log — three by eye on the screens, the fourth by a simple question in front of a table. Future studies gain the rule: no measurement batch enters a result before a drawn sample passes a human eye.

What's still coming

Two registered fronts remain open. The first: the extension arms, the same frozen protocol running on the frontier models' APIs (Claude and GPT), which are the ones that answer the question the small model can't reach: does the magic phrase's tie survive where adopters actually live? The research-credit applications were submitted to Anthropic's and OpenAI's programs; if they come, the protocol runs as is, without changing a comma. The second: consistency ruler v2 and Study 2's next round, with the corrections this account documents. Nothing pauses waiting for anyone's answer.

The verdict, in one pass

• Against the placebo, confirmed: one third of the serious violations. A big document alone doesn't do the job; what works is the right document.
• Against the magic phrase, on what the robot detects: a tie on the small model, published without discount. On the large model both conditions zero out, which is the instrument's ceiling. The literature already predicted this result, and the standard's difference lives where the robot doesn't measure.
• In blind human judgment, A11Y.md never failed once (5 of 5, against the phrase's 2 of 3; I judged blind, on a small sample, so I treat it as a signal, not a scoreboard), and every one of its modals used the correct native element.
• On real agents, the standard dominated: 25 of 27 pages 100% clean against 10 of 27 without it, across three vendors — each against its own baseline.
• The basics of the web were zeroed out at both scales: labels, names and image alternatives, the twenty-year errors, practically disappeared with A11Y.md in context.
• Process emerges on its own: by design, 27 of 27 real-agent runs produced process documentation without anyone asking — the report in 26, the decision log in 21, at least one of the two in all. Zero of 27 without A11Y.md.
• Consistency tied, and it taught: part of A11Y.md's "variation" is correct adaptation that ruler v1 can't recognize. V2 is commissioned.
• The cost is known: 1.4 to 2 times more fresh tokens per screen, buying records, governance and decision memory.
• And the process caught its own mistakes: four instrument defects, all found, fixed and published with both measurements preserved. This is how I wanted the study to be judged.

The plan to become unnecessary

I end with the reflection this study gave me — perhaps the strongest claim in the entire text.

The tie with the magic phrase doesn't scare me. It's part of the plan.

Language models learn from the web that exists. If today they deliver inaccessible interfaces by default, it's because they were trained on a web that is inaccessible by default. Every product born accessible now — whether through A11Y.md or through the daily work of accessibility professionals — enters the mass of data that will train tomorrow's models. And those models will deliver accessible screens without anyone asking, because they'll have learned from a web where building that way was simply the way you build.

Academic research has already named this mechanism: a recent W4A paper — the web accessibility conference — asks in its very title how to break the cycle of generated code that perpetuates barriers. A11Y.md is my operational answer to that open question. And the bet has a test, because a thesis with no refutation criterion is faith: the same frozen tasks from this benchmark, run in the "nothing" condition on each new model generation, have to improve release after release — the protocol already registers that version delta as exploratory. If the curve doesn't climb, the thesis falls, and I'll publish the fall.

That's why I want A11Y.md to become an unnecessary quality product. I want its days to be numbered. Today's "make it accessible" already improves any vibe-coded product, and that's a win for all of us. Tomorrow's will work even better, fed by what A11Y.md and this community deliver now. And the day comes when not even the phrase will need saying, because the answer will already have been born ready.

The project's ambition is not to be eternal. It's to make accessibility banal in the doing — no special request, no ceremony, just the normal way interfaces get built. And non-negotiable in the why: accessibility is a right, and that is precisely why it must become routine.

In the end, it's this text's metaphor completing itself: every true remedy should work toward its own retirement. And, to make it plain, the disease in this metaphor was never anyone's disability. The disease is digital products that exclude, and the cure is the environment learning to build right by default. A11Y.md exists precisely so that one day it no longer needs to.

I want the cure. I don't want to sell remedies forever.

Check it — don't take my word

Everything I claimed here has a public address: the two protocols registered before any data, the complete dataset under a permanent identifier — the 664 generated pages (400 from the first study, 210 journey screens from the second and 54 pages from the real agents), every report, every screenshot, every mistake of mine documented with a date, because there were mistakes of mine, four, and all four were caught by the process working. And the code, open end to end. And this invitation carries what I consider the most important part of the whole work. Don't take my word for it. Check.

(And one question that deserves an answer before it's asked: running everything again does not produce the same bytes — language models offer no repetition seed, and the study doesn't promise that. It promises something stronger: the process repeats. Frozen prompts, the model version recorded on each of the 1,197 calls, and every file published — anyone can reanalyze without rerunning, and rerun from scratch if they want.)

This was a first move, from a small workbench, with its limits written inside the study itself, made by someone who learned the method while executing it. The two biggest limits, said without detours:

First, the ten tasks live inside the standard's own guide map (16 of the 29 guides were never exercised by any task), so these studies don't yet separate "improves accessibility" from "follows its own manual well" — off-map tasks are a future arm.

Second, the robot sees only a fraction. Published estimates give automated tools a reach of 30% to 57% of real-world barriers, so "zero violations" here means zero of what the robot reaches — never "accessible site", never lived experience. That bridge is Phase 2, registered: people with disabilities using real products, because nothing in this text substitutes for those who use assistive technology.

What comes next depends on a question I hand back to you, who use or distrust this project: what evidence would make a difference in your day-to-day? It has an address: open an issue in the repository with your question, and it enters the roadmap. The next round of tests starts exactly where your doubt is.