Is your site agent-ready? How to find out, and what to fix first

For thirty years, the web has had two audiences. Humans got design, copy, and UX. Search crawlers got sitemaps, clean URLs, and structured data. Every site you have ever worked on was built for those two readers.

There is now a third: AI agents. Not just the crawlers that feed ChatGPT and Perplexity their answers, but assistants that browse, compare, book, and buy on a person's behalf. They do not see your hero animation. They do not scroll. They read a handful of machine-readable files most sites have never published, and if those files are missing, the agent moves on to a competitor that has them.

Most sites are not ready. Run any agent-readiness audit on a typical domain and it scores below 30 out of 100. Not because the site is broken, but because nobody ever had a reason to make it machine-friendly beyond the basics. That reason now exists.

What does "agent-ready" actually mean?

An agent-ready site is one that AI systems can find, read, understand, and act on without human help. It allows AI crawlers in, declares how its content may be used, describes itself in machine-readable formats, and, for products with an API or MCP server, tells agents how to connect and authenticate. Agent-readiness is the technical layer of GEO: it does not replace content and authority work, it makes that work legible to machines.

Think of it this way: GEO content work determines whether AI has something good to say about you. Agent-readiness determines whether AI can read it at all.

Why this matters for your AI visibility

The economics are already visible. AI platforms generated over a billion referral visits per month by mid-2025, growing triple digits year over year, and that traffic converts at roughly five times the rate of Google organic. Every one of those visits starts the same way: an AI system retrieving and reading web content.

Retrieval is the gatekeeper. When ChatGPT or Perplexity assembles an answer, it fetches live pages, extracts what it can parse, and cites what it trusts. A site that blocks AI crawlers, hides content behind JavaScript, or sends no machine-readable signals simply is not in the running, regardless of how strong the content is.

And the third audience is growing more capable. Agents are beginning to do more than quote pages: they compare products, fill forms, book demos, and connect to services autonomously. The standards enabling this, such as MCP, Content Signals, and agent discovery records, are young, but the pattern is familiar. The brands that added sitemaps in 2005 and schema markup in 2015 did not see instant results either. They saw compounding ones.

The agent-readiness stack: four layers

Agent-readiness checks sort into four layers, from essential to forward-looking.

Layer	What it covers	Who needs it
1. Access	AI crawlers allowed, HTTPS, no accidental noindex	Every site
2. Discoverability	sitemap.xml, robots.txt, Link headers, canonical URLs	Every site
3. Content signals & structure	Content-Signal directives, llms.txt, schema markup, clean extractable HTML	Every site with content worth citing
4. Agent interfaces	MCP server card, API catalog, OAuth discovery, agent skills	Products with an API or MCP server

Layer 1: Let AI in, and check you haven't locked it out by accident

This is the layer where most visibility is silently lost. The checks are binary and take minutes:

AI crawler access. Your robots.txt must not block GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, or Google-Extended. Blocking Google-Extended, for example, removes you from Gemini and AI Overview grounding entirely.
HTTPS with a valid certificate. AI systems treat certificate errors as a dead end.
No stray noindex directives. A noindex meta tag left over from a staging deploy makes a page invisible to both search and AI retrieval.

One of our customers discovered through KIME that ChatGPT was blocked from their site entirely. Unblocking it made an immediate, measurable difference. The most expensive GEO problems are often one line in a text file.

Layer 2: Make yourself findable

Sitemaps and robots.txt are old news, but agents also check newer discovery signals. Link response headers (RFC 8288) let your server point agents to documentation and API catalogs in the very first response. DNS-based agent discovery (DNS-AID), a Linux Foundation standard from 2026, lets agents find your services through a DNS lookup, the same way email servers have been discovered for decades.

These take an hour to set up, and almost nobody has them yet. That is precisely the argument for doing it: early signals on an empty field get noticed.

Layer 3: Tell AI what it may do, and make content extractable

The Content Signals standard extends robots.txt with one line declaring your AI usage preferences:

Content-Signal: search=yes, ai-input=yes, ai-train=yes

Three switches: may your content appear in AI search, may it be quoted in AI answers, may it train future models. For most brands, all three are yes. Being absorbed into the next generation of models is free brand distribution. The point is that the choice is now yours to declare rather than each AI company's to assume.

The rest of this layer is classic GEO foundation work we have covered in depth before: clear heading structure, direct-answer paragraphs, FAQ and Article schema, llms.txt as a structured entry point. If your content is not extractable, nothing downstream matters.

Layer 4: Give agents a front door

This layer applies to products with an API or MCP server, and it is where the agentic web gets interesting. A set of well-known files (/.well-known/mcp/server-card.json, /.well-known/api-catalog, OAuth discovery metadata, an auth.md registration guide) lets an agent discover your service, understand what it offers, register itself as a client, and ask its human for one approval click. No documentation reading, no manual setup.

Today, connecting an AI assistant to a SaaS product is a human task. The standards above make it a machine task with human consent. When agents start shopping for tools on their users' behalf, sites without this plumbing are not candidates. They are invisible.

How to test your site in minutes

You cannot fix what you have not measured, and this is one audit you should not run on instinct.

KIME's new AI Readiness feature scans your domain and scores it across these checks: whether OpenAI, Claude, Perplexity, and Google's AI bots can access your site, whether HTTPS is valid, whether noindex directives are blocking key pages, and more, with page-level checks across your site and a run history so you can track improvements over time. It lives directly in the KIME platform next to your visibility, sentiment, and citation data, so the technical audit and the outcome it drives sit in one place.

The broader ecosystem is testing the same things, which tells you where the market is heading: Cloudflare offers an AI-readiness scanner covering the deeper protocol checks, and community tools like isitagentready.com audit the discoverability basics. Practitioners are already writing up their own audit-and-fix journeys, and this developer walkthrough is a good example of how fixable the findings usually are.

Most agent-readiness issues are fixable in an afternoon. The typical audit finding is not "rebuild your site". It is "add this file", "add this line", "unblock this bot".

Where to start: a priority order

This week: run an AI-readiness scan. Fix crawler access, HTTPS, and noindex issues the same day. These are the ones costing you visibility right now.
This month: add Content-Signal directives to robots.txt, verify your sitemap is complete and fresh, and add Article and FAQ schema to your key pages.
This quarter: if you have an API or MCP server, publish the discovery files (server card, API catalog, OAuth metadata). If you do not, invest the time in content structure instead.
Ongoing: re-scan monthly. Standards in this space move fast. New checks will appear, and early adoption is cheap.

The bigger picture

Agent-readiness will not change your ChatGPT rankings overnight, and anyone who claims otherwise is selling something. What it does is remove every technical reason for AI systems to skip you, today for retrieval and citations, tomorrow for autonomous agents acting on behalf of your buyers.

The web's third audience is arriving either way. The only question is whether your site is readable when it does.

Frequently asked questions

What does agent-ready mean?

Agent-ready means an AI system can find, read, understand, and act on your website without human help. In practice it covers four layers: AI crawler access, machine discoverability (sitemap, Link headers, DNS records), content signals and structured data, and, for products with APIs, agent-facing interfaces like MCP server cards and OAuth discovery metadata.

Is agent-readiness the same as GEO?

No, it is one layer of GEO. Generative Engine Optimisation covers everything that gets your brand into AI answers: content quality, authority, off-site signals, and technical readiness. Agent-readiness is the technical foundation. It determines whether AI systems can access and parse your content at all. Strong content behind a blocked crawler is invisible; an open, well-structured site with weak content gets read but not cited. You need both.

How do I check if my site is agent-ready?

Run an automated scan. KIME's AI Readiness feature checks your domain against the core signals, including AI bot access for OpenAI, Claude, Perplexity, and Google, valid HTTPS, and noindex issues, and tracks your score over time inside the platform. Cloudflare and community tools like isitagentready.com offer complementary audits of the newer protocol checks.

What is the Content-Signal directive in robots.txt?

Content Signals is a draft standard that extends robots.txt with explicit AI usage preferences. One line, for example Content-Signal: search=yes, ai-input=yes, ai-train=yes, declares whether your content may appear in AI search results, be used as input for AI answers, and be used to train future models. It costs one line to add and makes your preferences explicit rather than assumed.

Should I allow AI to train on my content?

For most brands, yes. Training exposure means future AI models know your brand natively, what it does and who it serves, without needing to retrieve it. That is free distribution into the tools your buyers increasingly use for discovery. The exception is businesses whose content is the product, such as publishers and stock media, where licensing considerations apply.

Do I need an MCP server to be agent-ready?

No. Layers one through three, covering crawler access, discoverability, and content signals, apply to every site and deliver most of the near-term value. Agent interface files like MCP server cards only make sense if you have a service agents can actually connect to. If you do have an API or MCP server, publishing the discovery files is a small effort with asymmetric upside.

How long does it take to make a site agent-ready?

Most sites can fix the majority of audit findings in a day. Crawler access and Content Signals are one-line changes. Sitemaps and schema are hours, not weeks. The agent-interface layer takes longer if you have an API, but a typical marketing site can move from a failing score to a strong one in a single working session.

Related guides

Find out if your site is agent-ready, in minutes

KIME's AI Readiness scan checks whether ChatGPT, Claude, Perplexity, and Google's AI can actually access and read your site, flags exactly what is blocking you, and tracks your score as you fix it, alongside daily tracking of how your brand actually performs across every major AI model.

Start a free trial of KIME → and run your first AI Readiness scan today.