Featured image for Your Website Isn't Ready for AI Agents (Here's How to Check in 30 Seconds)

Your Website Isn't Ready for AI Agents (Here's How to Check in 30 Seconds)

Published on:

Reading time: 15 min

Topic: Technology

Author: Leandro Valencia

#ai agents#seo#geo#agent readiness#cloudflare#mcp#llms.txt#claude cowork#superpowers

AI agents are already real visitors to your website, and most sites ignore them. Learn how to scan your domain with isitagentready.com, turn the report into an actionable improvement spec with a ready-to-copy prompt, and execute it without breaking anything using Superpowers' brainstorming workflow.

Table of Contents

The visitor with no eyes

Think about how a lead reaches your business today. Someone no longer types "best design agency in Bogotá" into a search engine and opens ten tabs. They ask their assistant: "find me three vendors, compare prices, and book a call with the best one."

That assistant does something very different from a human:

  • It doesn't see your carousel, your scroll animation, or your color palette.
  • It doesn't wait around for your JavaScript to hydrate if it can avoid it.
  • It consumes structured text, and the cleaner, the better.
  • It needs to know, without guessing, what it can actually do on your site: is there an API? An endpoint to book something? Can it pay?

If your website only responds with heavy HTML full of divs and no machine-readable signal of who you are or what you offer, the agent does the logical thing: it goes with the competitor who made it easy.

This isn't futurism. It's exactly the same shift we lived through when Google went mobile-first: whoever didn't adapt in time lost rankings for years. The difference is that the window to get ahead this time is much shorter.


What "agent readiness" is, and how it differs from SEO

Agent readiness is how prepared your site is for an autonomous agent to discover it, read it, understand its capabilities, and act on it.

It's a close cousin of SEO, but with different goals:

Classic SEO Agent readiness
Audience Search engine crawler + human Autonomous agent (and the LLM behind it)
Goal Appear in a results listing Be chosen and executed by the agent
Key format Semantic HTML, metadata, speed Clean markdown, manifests, protocols
Success User click Task completed without friction
Main signal Backlinks and content Discoverability and declared capabilities

You'll also see the term GEO (Generative Engine Optimization), which focuses on getting cited inside AI-generated answers. Agent readiness is the infrastructure layer that makes that GEO possible: if the agent can't read you properly, it's unlikely to cite you properly either.

And here's the strategic point: what's at stake isn't traffic, it's intent. An agent that lands on your site already carries a purchase, hiring, or booking decision behind it. It's the highest-intent visitor you'll ever get, and the one with the least patience.


What isitagentready.com checks

Is Your Site Agent-Ready? is a free tool from Cloudflare that scans your domain against the emerging standards of the agentic ecosystem. You give it a URL, hit Scan, and within seconds it returns a score with the detail of what you have and what's missing.

It groups its checks into five categories:

1. Discoverability (can they find you?)

robots.txt, sitemap, HTTP response Link headers, and DNS-AID (DNS for AI Discovery). This is the most basic layer: if an agent can't discover your resources, nothing else matters.

2. Content Accessibility (can they read you?)

Mainly Markdown content negotiation: whether your server can return a plain-text/Markdown version of your pages when the client asks for it with the right header. For an LLM, getting Markdown instead of HTML is the difference between reading a book and reading the book with the binding still glued shut.

3. Bot Access Control (what do you allow, and what don't you?)

Specific rules for AI bots in robots.txt, Content Signals, and Web Bot Auth. This is the category most people ignore, and the one that can hurt you the most: this is where you decide whether your content can be used for training, for search, or for real-time inference. Blocking everything is just as bad a business decision as saying nothing at all.

4. Protocol Discovery (what can you do for the agent?)

The most interesting category: MCP Server Card, Agent Skills, WebMCP, API Catalog, OAuth discovery, OAuth Protected Resource, Auth.md, A2A Agent Card, and the ARD manifest. This is where your website stops being a document and becomes a tool.

5. Commerce (can it transact?)

x402, MPP, UCP, and ACP: the agentic commerce protocols. If you sell something, this category determines whether an agent can complete a purchase on its own or has to give up and hand the task back to the human.

An honest note: many of these standards are still young, and some won't survive. You don't need to implement all of them. The value of the scan isn't scoring 100/100 — it's having, for the first time, a clear inventory of decisions you're currently making by default.


How to run your first scan, step by step

  1. Go to isitagentready.com.
  2. Paste your full domain, including https://.
  3. If you want to fine-tune it, open Customize scan and uncheck the categories that don't apply (if you don't sell online, you can leave Commerce out of your first pass).
  4. Click Scan and wait for the result.
  5. At the end of the report you'll find a block of generated instructions to paste into your coding agent, with a Copy all instructions button.

Copy them. But don't run them yet.

That block is a decent starting point and a terrible finish line: it's generic, AI-generated recommendations with no knowledge of your stack, your real traffic, or your business priorities. If you hand it as-is to a coding agent, you'll end up with an empty llms.txt, a manifest nobody consumes, and three new files nobody maintains.

What comes next is the step that turns that raw output into something that actually moves the needle.


From report to spec: the ready-to-copy prompt

The idea is simple: analysis first, then spec, then code. Never the other way around.

Prompt 1 — Audit and diagnosis

Paste this into Claude, ChatGPT, or whatever agent you use. It works better if it has web browsing access.

Act as a senior agent readiness and GEO (Generative Engine Optimization)
consultant. Your job is to audit a website from the perspective of an
autonomous AI agent, not from a human's perspective.

SITE TO ANALYZE: [PASTE YOUR URL HERE]
BUSINESS CONTEXT: [what you sell, to whom, and what the highest-value
action is that you want to happen on the site]
TECH STACK: [e.g. Next.js on Vercel, WordPress, Shopify, Astro on Cloudflare]
isitagentready.com SCAN RESULTS: [paste the full report here]

Do the following, in this order:

1. INDEPENDENT VERIFICATION
   Try to access these resources and report what you actually found
   (exists / doesn't exist / exists but is broken):
   - /robots.txt  (are there explicit rules for AI bots? which ones?)
   - /sitemap.xml
   - /llms.txt and /llms-full.txt
   - /.well-known/  (mcp, agent-card, oauth-authorization-server, ard)
   - HTTP headers on the homepage: Link, Content-Type, cache
   - Does it return Markdown when requested with Accept: text/markdown?

2. READ IT LIKE AN AGENT
   Read the homepage and the 3 most important pages. Answer with brutal
   honesty:
   - In under 50 words, what does this company do? Is it unambiguously clear?
   - What concrete actions could an agent take here, today, with no human
     intervention? List them.
   - What critical information (pricing, availability, coverage, contact,
     terms) is hidden behind JavaScript, images, or forms?
   - If a user asked their assistant "find me a vendor for X and compare it
     against two others," would this site win or lose the comparison? Why?

3. PRIORITIZED DIAGNOSIS
   Return a table with: Finding | Category | Impact (High/Medium/Low) |
   Effort (High/Medium/Low) | Why it matters for the business.
   Sort by impact/effort ratio, not alphabetically or by category.

4. WHAT NOT TO DO
   Explicitly list which recommendations from the scan do NOT make sense
   for this specific site, and why. Be selective: implementing everything
   is a sign you didn't understand the problem.

Don't write any code yet. Don't propose solutions yet. Just diagnose.
If there's something you couldn't verify, say so explicitly instead of
assuming it.

Prompt 2 — Generate the improvement spec

Once you have the diagnosis and agree with it, chain this second prompt in the same conversation:

Great. Now turn this into an executable IMPROVEMENT SPEC.

I don't want a loose list of tasks. I want a document that another
person — or a coding agent — can pick up and execute without having
to ask me anything else.

Exact structure:

# Spec: Agent Readiness — [site name]

## 1. Problem
What the business is losing today by not being agent-ready. Be concrete,
no fluff.

## 2. Goal and success metrics
Goal in one sentence. And 3-5 verifiable metrics. Each metric must be
checkable with a command, an HTTP request, or a re-scan. No
"improve visibility": that's not a metric.

## 3. Scope
### In scope
### Out of scope (and why)

## 4. Implementation phases
Split into 3 phases by impact/effort ratio:
- Phase 1 — Quick wins (< 1 day)
- Phase 2 — Structural (< 1 week)
- Phase 3 — Agentic capabilities (evaluate after measuring phases 1 and 2)

For EACH task include:
- ID and title
- Exact file or path to create/modify
- Proposed content or change (with a real example, not a generic
  placeholder)
- Verifiable acceptance criteria (curl command, concrete check, re-scan)
- Risk and how to roll it back

## 5. Specific content to generate
Write the actual content now for:
- /llms.txt tailored to this business
- AI bot rules for robots.txt, with a declared stance (what's allowed
  for search, what's allowed for training, what's blocked)
- The 3-sentence summary I want any AI to repeat about this brand

## 6. Risks and open decisions
What can go wrong, and what business decisions I need to make, not you.

## 7. How to measure in 30 days
What to re-verify and what to expect to see.

Rules: prioritize by real business impact, not checklist completeness.
If you can't justify a task with a concrete benefit, cut it.
Flag any assumption you make with [ASSUMPTION] so I can validate it.

The output of this second prompt is a document you can version in your repo, discuss with your team, and hand to a coding agent as a source of truth. That's the real deliverable, not the tool's score.


Before you execute: use Superpowers' brainstorming

This is where most people trip up. You have a beautiful spec, you hand it to your agent, you say "do it," and three hours later you have twelve new files, a Cloudflare configuration nobody understands, and no way of knowing if anything actually improved.

The fix is to insert a deliberate friction phase before any code gets written. That's what Superpowers is for: the open-source skills framework that turns your agent into a methodical engineer instead of an unpredictable code generator.

Its first phase is exactly what you need here:

/brainstorm I want to implement this agent readiness spec on my site [URL].
Here's the full spec: [paste the spec from prompt 2]

What does brainstorming do that a regular prompt doesn't? It interrogates you before obeying you. It challenges assumptions, catches contradictions between the spec and your real stack, forces you to define "done," and hands you back a refined scope before touching a single file.

From there you chain the rest of the Superpowers workflow:

  1. /brainstorm — refines the spec with uncomfortable questions.
  2. Git worktree — work on an isolated branch, no risk to production.
  3. /write-plan — breaks the spec into 2-5 minute, verifiable tasks.
  4. /execute-plan — delegates to subagents with built-in review.
  5. Verification — evidence before declaring anything "done."

If you don't have it installed yet, it takes a minute:

/plugin install superpowers

The full installation and usage guide is here: How to Install and Master the Superpowers Skill in Claude Cowork.

The rule that saved me the most time: don't let the agent write code until the spec has survived brainstorming. Twenty minutes of questions is a ridiculously cheap price compared to undoing a badly-aimed implementation.


Quick wins you can implement today

If you're only going to do three things this week, make it these:

A real llms.txt, not a decorative one. A Markdown file at the root of your domain stating what you are, what you offer, and links to your key pages. What matters isn't that it exists: it's that a model can read it and correctly describe your business. Test it: feed the content to an AI and ask what your company does. If you don't like the answer, the file is poorly written.

Explicit AI bot rules in your robots.txt. Silence isn't consent and it isn't prohibition either — it's ambiguity, and every bot resolves ambiguity however it wants. Declare your stance: what you allow for search, what you allow for training, what you block.

Critical content out from behind JavaScript. Pricing, availability, coverage, terms, and contact info in server-rendered HTML, not client-rendered. It's the improvement with the best effort-to-impact ratio on this whole list, and it helps your traditional SEO too.

After that, and only after measuring results, evaluate the more ambitious layers: Markdown negotiation, an MCP server for your services, or agentic commerce protocols if you sell online.


Common mistakes when making your site agent-ready

Chasing the score. The tool's score is a diagnostic, not a KPI. A site with a well-chosen 40/100 beats one with a carelessly implemented 90/100.

Copying the instructions block and running it blindly. It's AI-generated content with no knowledge of your business. Use it as diagnostic input, never as a work plan.

Blocking every AI bot out of fear. I get the impulse, but blocking indiscriminately takes you out of the generated answers where your customers are already looking for you. Separate training from search and inference: they're not the same decision.

Creating files nobody maintains. An outdated llms.txt with a year-old price list is worse than not having one at all: you're handing AI ammunition to misinform people about you.

Implementing protocols your business doesn't use. If you don't have an API, don't publish an API Catalog. If you don't sell online, x402 gets you nothing.


Frequently asked questions

Is isitagentready.com free?

Yes. It's a free tool from Cloudflare, requires no signup, and you can scan any public domain. The recommendations it generates at the end are AI-generated, so apply your own judgment before implementing them.

Does this replace traditional SEO?

No, it complements it. Many agent readiness improvements (server-rendered HTML content, clean semantic structure, a correct sitemap) directly benefit classic SEO too. Keep optimizing for search engines and humans; you're simply adding a third audience to the equation.

Does this work if my site runs on WordPress or Shopify?

Yes, though you'll have less fine-grained control. The quick wins — llms.txt, bot rules in robots.txt, critical content in HTML — are perfectly achievable on any CMS. The protocol layers, like an MCP server, need more technical flexibility or a plugin that handles it for you.

Do I need Superpowers to do this?

It's not required. You can execute the spec manually or with any agent. What you do need is the discipline that Superpowers automates: thinking before coding, planning in small tasks, and verifying with evidence. Superpowers gives you that for free by default, which is exactly what nobody does when they're in a hurry.

How often should I re-scan?

Monthly is reasonable while you're implementing changes, and quarterly after that. The agentic standards ecosystem is moving fast: what's optional today could be table stakes in six months.


Conclusion

The web we built over the last twenty years was designed for eyes. The one that's coming also has to work for agents that don't see, don't wait, and don't forgive.

The good news is the barrier to entry is still remarkably low: scanning your domain takes 30 seconds, a serious diagnosis takes an afternoon, and the quick wins are three text files. The competitive advantage isn't in technical difficulty — it's in doing it before it's obvious you had to.

My concrete recommendation for this week: scan your site at isitagentready.com, run Prompt 1 to diagnose, Prompt 2 to generate the spec, and put it through /brainstorm before executing a single line.

How did your site score on the scan? Tell me in the comments on creacosas.com.

Let's create amazing things, now also for those without eyes!


Related Posts

Keep exploring similar content that may interest you

Your Website Isn't Ready for AI Agents (Here's How to Check in 30 Seconds)