A TXT file is plain text with no formatting. llms.txt has a .txt extension, but its content follows Markdown syntax: # for headings > for quotes text for links Markdown is itself plain text, so any browser or program can open the file.

What Is llms.txt? What It Does, How to Add It and Does It Work

Q: What Is the Difference Between llms.txt, robots.txt and sitemap.xml?

robots.txt sitemap.xml llms.txt --- --- --- --- Purpose Tell crawlers what they may and may not access List all URLs for search engines Present curated key pages with descriptions Audience Search and AI crawlers Search engines LLMs and AI agents Format Plain text (User-agent, Allow, Disallow) XML Markdown Binding? Voluntary, but major crawlers largely comply Not binding, a hint Not binding, no guarantee it is read Location /robots.txt Usually /sitemap.xml (can be declared in robots.txt) /llms.txt In short: robots.txt says “here is where you may go.” sitemap.xml says “here is everything we have

Q: llms.txt Format: What Sections Does It Have?

The proposal on llmstxt.org defines this order: H1 heading (required): # Site or project name. This is the only mandatory element. Blockquote summary (recommended): One or two sentences starting with > that explain what the site is. Additional notes (optional): Paragraphs or lists without headings, for example notes on how to read the file. H2 sections with link lists: Under each ## heading, add lines in the form - Page name: short description. Optional section: Links under ## Optional count as secondary. A tool can skip them when context is tight.

Q: Who should add it, and who shouldn't prioritize it?

Worth adding: documentation and API sites, SaaS products, developer tools and large knowledge bases. Not a priority: local businesses, small blogs and e-commerce category pages. For these sites, a Google Business Profile, reviews and structured data matter far more.

AI assistants are changing how B2B buyers research vendors, and a new file has landed on marketing teams' to-do lists: llms.txt. What is llms.txt, what does it do, and will adding it make your brand more visible in AI answers? Many guides describe it as a rulebook that “decides how AI crawls your site.” That is wrong. This guide covers the definition, the format, how it differs from robots.txt, what Google has said about it and how to measure its effect after you publish it.

Short answer: llms.txt is a plain-text file written in Markdown and placed at a website's root (example.com/llms.txt). Jeremy Howard proposed it in September 2024. It gives large language models such as ChatGPT and Claude a short summary of the site and clean links to its most important pages. It does not block access the way robots.txt can. It only offers guidance, and it is not an official standard.

What Is llms.txt?

llms.txt is a summary and link map for large language models (LLMs). It sits in a website's root directory. The file contains a short description of what the site is, followed by annotated links to its key pages. An AI tool can start from one clean file instead of crawling dozens of pages to understand your site.

Who proposed it, and is it an official standard?

Jeremy Howard, co-founder of Answer.AI, published the proposal in September 2024. The specification lives at llmstxt.org. llms.txt is not a W3C or IETF standard. It is a community proposal, so no search engine or AI provider is obliged to read it.

Why was it created?

LLMs can only process a limited amount of text at once. This limit is called the context window. A typical HTML page is full of navigation menus, cookie banners, ads and JavaScript, and the actual content gets buried in that noise.

Clean Markdown uses fewer tokens and is easier for a model to interpret correctly. That is why llms.txt was designed mainly for inference time: the moment a user asks a question or an AI agent carries out a task.

What is a TXT file?

A TXT file is plain text with no formatting. llms.txt has a .txt extension, but its content follows Markdown syntax:

# for headings
> for quotes
[text](URL) for links

Markdown is itself plain text, so any browser or program can open the file.

What llms.txt Does and Doesn't Do

Where it helps

AI agents and developer tools: Coding assistants and agents can reach a library's or API's documentation quickly through llms.txt. Many developers paste an llms.txt URL straight into their assistant to load up-to-date docs into context.
Documentation and knowledge bases: A help center can have hundreds of pages. Summarizing what each page covers in one file makes it easier for a model to find the right one.
Internal clarity: Writing the file forces your team to answer one question: which 15 to 20 pages on our site actually matter? That exercise is a useful content audit on its own.

The common misconception: llms.txt does not control access

llms.txt is not a permission or prohibition file. Writing “don't crawl this page” or “don't use my content for training” in it binds no bot.

If you want to block AI crawlers or keep your content out of model training, the right tool is user-agent rules in robots.txt.

What it does not guarantee

Adding the file will not raise your Google rankings. It also does not guarantee that AI answers will cite you or that you will appear in AI Overviews. Google's own statements on this are covered below.

What Is the Difference Between llms.txt, robots.txt and sitemap.xml?

	robots.txt	sitemap.xml	llms.txt
Purpose	Tell crawlers what they may and may not access	List all URLs for search engines	Present curated key pages with descriptions
Audience	Search and AI crawlers	Search engines	LLMs and AI agents
Format	Plain text (User-agent, Allow, Disallow)	XML	Markdown
Binding?	Voluntary, but major crawlers largely comply	Not binding, a hint	Not binding, no guarantee it is read
Location	/robots.txt	Usually /sitemap.xml (can be declared in robots.txt)	/llms.txt

In short:

robots.txt says “here is where you may go.”
sitemap.xml says “here is everything we have.”
llms.txt says “these pages matter most, and here is what each one covers.”

How do you block AI crawlers with robots.txt?

robots.txt is a plain-text file in your site's root. To block an AI crawler, name its user-agent and add a Disallow rule:

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: PerplexityBot
Disallow: /

Keep in mind:

Each crawler belongs to a different provider. GPTBot is OpenAI's crawler, ClaudeBot is Anthropic's and PerplexityBot is Perplexity's. Some providers run separate bots for training and for search, so blocking one may not affect the other. Check each provider's documentation for the current list.
Google-Extended is a control token, not a separate crawler. It limits how your content is used for Gemini models. It does not affect Google Search or AI Overviews.
Blocking has a cost. It can also reduce your chances of being cited in those assistants' answers, which is usually not what a B2B brand wants.

llms.txt Format: What Sections Does It Have?

The proposal on llmstxt.org defines this order:

H1 heading (required): # Site or project name. This is the only mandatory element.
Blockquote summary (recommended): One or two sentences starting with > that explain what the site is.
Additional notes (optional): Paragraphs or lists without headings, for example notes on how to read the file.
H2 sections with link lists: Under each ## heading, add lines in the form - [Page name](URL): short description.
Optional section: Links under ## Optional count as secondary. A tool can skip them when context is tight.

llms-full.txt and .md versions

llms-full.txt: Instead of linking out, this variant puts all key content into a single Markdown file. Documentation platforms helped popularize it. On large sites the file can grow very long, so it is mostly used for docs.
.md versions: The proposal also suggests serving a clean Markdown copy of each page at the same URL with .md appended. Examples are page.html.md, or index.html.md for URLs without a file name. Pointing your llms.txt links to these versions makes the model's job easier.

Example llms.txt File (for a Turkish Site)

Here is a complete example for a fictional Turkish SaaS company that you can copy and adapt. On a Turkish-language site, write the descriptions in Turkish.

# Example CRM

> Example CRM is cloud-based customer relationship management software built for SMEs in Türkiye. It offers quotes, e-invoice integration and sales pipeline tracking, sold as a monthly subscription priced in TRY.

This file helps AI assistants quickly find our product, pricing and support content. All pages are in Turkish.

## Products
- [Sales Pipeline](https://example.com/products/sales-pipeline.md): Track deals by stage, with automatic reminders and reporting.
- [Quote Management](https://example.com/products/quotes.md): Template-based quotes, approval flow and PDF export.
- [Pricing](https://example.com/pricing.md): Plans, per-user fees and the annual billing discount.

## Guides
- [Setup Guide](https://example.com/guide/setup.md): Creating an account, inviting your team and importing data.
- [e-Invoice Integration](https://example.com/guide/e-invoice.md): How to configure e-invoicing that complies with the Turkish Revenue Administration (GIB).
- [API Documentation](https://example.com/docs/api.md): REST endpoints, authentication and rate limits.

## Support
- [FAQ](https://example.com/support/faq.md): Short answers on billing, cancellation and data security.
- [KVKK and Data Security](https://example.com/kvkk.md): Where data is stored and obligations under Turkish data protection law (KVKK).
- [Contact](https://example.com/contact.md): Support hours, email and phone channels.

## Optional
- [Blog](https://example.com/blog.md): Current articles on sales and CRM.
- [Press Kit](https://example.com/press.md): Logo, boilerplate and images.

Bad example: don't do this

# Example CRM
- https://example.com/
- https://example.com/products
- https://example.com/products?page=2
- https://example.com/blog/tag/crm
- [Our Products](https://example.com/products): The best, most innovative solution in the industry!

What goes wrong here:

Copying the sitemap: A list of URLs with no descriptions gives the model no context, and you already have sitemap.xml for that.
Marketing language: Words like “best” or “leading” carry no information. A model looks for facts: what you do, for whom and under what terms.
No summary or structure: Without a blockquote summary and H2 sections, the file is just a pile of links.

A note on Turkish characters

Always save the file in UTF-8 encoding and make sure your server sends charset=utf-8. Otherwise the Turkish letters ç, ğ, ı, ö, ş and ü appear broken, and a model may misread the text.

How to Create llms.txt and Add It to Your Site

Step by step

Choose your key pages: Pick the 10 to 30 pages an AI needs to describe your brand accurately, such as product, pricing, docs, FAQ and contact.
Write the descriptions: Give each link one fact-focused sentence.
Save the file: Name it exactly llms.txt and use UTF-8 encoding.
Upload it to the root: Put it in the same folder as robots.txt.
Verify it: Open example.com/llms.txt in a browser.

Platform notes

WordPress: Some popular SEO plugins can generate llms.txt automatically. You can also upload the file to the public_html folder via FTP or your hosting panel's file manager.
Shopify, Webflow and similar platforms: Uploading files directly to the root may be restricted. Common workarounds are:
- uploading the file to the platform's file storage and redirecting /llms.txt to it
- using an app
- setting up a proxy at the CDN layer
Platform settings change, so check the current documentation.
Ticimax and other Turkish e-commerce platforms: The panel may not let you upload files to the root. Ask platform support to upload the file or set up a redirect.

Generator tools

Tools that crawl your site and generate llms.txt save time, but their output often looks like a copy of your sitemap. Always review the result by hand: keep only the pages that matter and fix the descriptions.

Technical checklist

The URL returns HTTP 200, not a 301 chain or a 404.
Content-Type is text/plain or text/markdown, with UTF-8 charset.
All links work, with no broken URLs.
The file does not point to pages behind a login or pages blocked in robots.txt.
Pricing, product and contact details are current, and the file is updated when content changes.

Does llms.txt Actually Work? What Google and AI Companies Say

Google's position

John Mueller (April 2025, Reddit r/TechSEO): He said that, as far as he knew, none of the AI services had said they use llms.txt. He added that server logs show their bots don't even check for it, and compared the file to the obsolete keywords meta tag.
Gary Illyes (July 2025, Search Central Live): He said Google does not support llms.txt and is not planning to. In his words, normal SEO is enough to appear in AI Overviews.

Where AI providers stand

As of this writing, none of the major LLM providers has officially committed that its crawlers routinely read llms.txt and use it when generating answers.

Companies such as Anthropic, Cloudflare and Stripe do publish llms.txt files for their own documentation. That is a different matter. It helps developers and agents use their docs, and it is not a statement of crawler support.

Who should add it, and who shouldn't prioritize it?

Worth adding: documentation and API sites, SaaS products, developer tools and large knowledge bases.
Not a priority: local businesses, small blogs and e-commerce category pages. For these sites, a Google Business Profile, reviews and structured data matter far more.

Cost and risk

A good llms.txt takes most sites 30 to 60 minutes to prepare, and it has no known downside. It does not replace a GEO (generative engine optimization) strategy, though. Treat it as low-cost hygiene, not a growth lever.

How to Measure the Impact After Adding llms.txt

1. Check your server logs

Search your access logs for /llms.txt requests and note which user-agents they come from:

grep '/llms.txt' /var/log/nginx/access.log

Sometimes there are no requests at all, or they come only from SEO tools. In that case, don't expect the file to have a direct effect on AI answers yet.

2. Run a before-and-after comparison

Before publishing the file, list 10 to 20 questions your buyers actually ask. One example: “What is the best CRM for SMEs in Istanbul?” Ask each question in ChatGPT, Gemini, Perplexity, Claude and Google AI Overviews, and record:

Is your brand mentioned?
In what position does it appear?
Is your site cited as a source?
Is your brand described accurately and in a positive tone?

Repeat the same measurement 4 to 8 weeks after adding the file. AI tools can give different answers to the same question at different times, so a one-off check is misleading. Run the measurement at regular intervals.

3. Automate the measurement

Tracking this by hand quickly becomes tedious. Jumpix AI Visibility asks your customers' questions to ChatGPT, Gemini, Perplexity, Claude and Google AI Overviews every week. It reports:

brand mention, position, citation, tone and accuracy
share of voice against up to 5 competitors
a source-site analysis showing which websites the AI reads when it answers

It then lists what you need to fix, for 9.99 USD a month per brand. To see your starting point, try the free AI visibility check, which needs no sign-up. You enter your industry and city, and it tests 5 buyer questions across 3 engines.

The real visibility levers

Source-site analysis often reveals that AI relies more on third-party sources than on your own site when it talks about your brand. That is why these factors matter more than llms.txt:

Being present in the third-party sources AI reads, such as industry comparisons, guides and review platforms.
Clear fact pages that state what you do, whom you serve, and your pricing and terms.
Structured data markup such as Organization, Product and FAQPage.
Content that is updated regularly, dated and verifiable.

Conclusion: Put llms.txt in Its Proper Place

llms.txt is a low-cost community proposal that presents your most important content to AI tools as a clean Markdown map. It does not control access; robots.txt does that. Google has said plainly that it doesn't use the file, and the major providers have not committed to supporting it.

If you run a documentation, API or SaaS site, it is worth an hour of your time. If you are a local business or a small blog, your priorities lie elsewhere.

Either way, the real question is not “did I add the file?” but “does AI mention my brand and describe it correctly?” Without measuring that, any GEO work is guesswork. Start with a free check to see where you stand, then track the effect of every change, llms.txt included, on a regular schedule.

Frequently Asked Questions

What is a TXT file, and why is llms.txt a .txt file written in Markdown?

A TXT file is unformatted plain text. Markdown is also a plain-text syntax, so it can be stored in a .txt file and opened by any program. Using Markdown for the heading, summary and links helps LLMs recognize the structure easily.

Are llms.txt and robots.txt the same? How do I create robots.txt and block AI crawlers?

No. robots.txt tells crawlers what they may and may not access, while llms.txt only offers guidance and binds no bot. To create robots.txt, place a plain-text file in your root and write a User-agent line and a Disallow rule for each bot. For example, adding Disallow: / for GPTBot, ClaudeBot, Google-Extended or PerplexityBot blocks those crawlers.

What is an LLM, and how does llms.txt affect how LLMs read my site?

An LLM (large language model) is the AI model behind assistants like ChatGPT, Claude and Gemini, trained on large amounts of text. When a model or agent wants to read your site, llms.txt gives it a clean summary and a list of key pages. Whether the model actually reads the file depends entirely on the provider and the tool.

Will adding llms.txt improve my Google rankings or AI Overviews visibility?

No. Google representatives have said they don't use llms.txt and that normal SEO is enough for AI Overviews. The file guarantees neither rankings nor visibility. Quality content, structured data and trusted third-party sources matter far more.

How can I measure my llms.txt file's impact on AI answers?

First, check your server logs for /llms.txt requests and their user-agents. Then ask your target questions in ChatGPT, Gemini, Perplexity, Claude and AI Overviews before adding the file and again a few weeks later. Compare changes in mentions, position and citations. An AI visibility tool can run this tracking weekly and automatically.