Does llms.txt actually work? An honest look at the evidence

Let's start with the uncomfortable part, because most articles about llms.txt skip it: as of today, no major AI vendor has publicly committed to reading the file. OpenAI, Anthropic and Perplexity document their crawlers' robots.txt behavior — none guarantees llms.txt is fetched or used. Anyone selling llms.txt as a magic ranking switch is overselling it. We build a tool that generates these files, so consider this us arguing against our own marketing.

So why publish one at all?

Three reasons. First, cost: it's one plain-text file, ten minutes, zero risk. Second, crawler traffic to /llms.txt is real and growing — server logs across the industry show AI crawlers fetching it even without official commitments. Third, and most importantly: writing one forces you to produce the exact content AI engines demonstrably do use — a concise, factual, quotable summary of your business.

What the evidence actually supports

What's well-established: assistants quote concrete facts from crawlable pages; structured data reduces ambiguity; server-rendered text gets read where JavaScript-only content doesn't. In our own before/after cases, engines went from "I don't have information about this product" to accurately describing it — after shipping llms.txt, JSON-LD and open robots.txt together. Attribution honesty: we can't isolate which signal did the work. The bundle worked; the file alone is unproven.

The right way to think about it

Treat llms.txt like an early-days sitemap.xml: a cheap, forward-compatible convention that concentrates your citable facts in one place, with the real gains coming from the discipline it imposes on your content. Publish it as part of the AI-readable layer — never instead of crawler access, structured data and server-rendered facts.

How to do it without wasting time

Facts only — founding year, services, prices, locations, key page links with one-line descriptions. No adjectives an assistant would skip anyway. Verify it serves as plain text with a 200 status (frameworks love routing unknown paths to HTML 404 pages). Re-read it quarterly the way you'd re-read your homepage.

AnswerRank's grader checks the whole layer at once — robots.txt rules, llms.txt, JSON-LD, FAQ schema, sitemap — and the fix pack drafts the files from your site's own content. Free to run, and it will tell you honestly when a signal is already fine.