About the Agentic Readiness Scanner
The scanner behind 4all.digital/agentic-readiness/scan grades how ready a website is for AI agents. If you found this page from your server logs, a person asked us to scan the site — the scanner only runs on request; it does not crawl the web on its own.
What it fetches
One scan requests the site's homepage plus a handful of well-known public files:
robots.txt, the sitemap named there, llms.txt,
llms-full.txt, a few /.well-known/ discovery files, and the
homepage once more asking for a markdown version. It does not crawl inner pages, does
not submit forms, and does not execute JavaScript. A scan is roughly twenty small
requests, once, from a single client.
robots.txt
The scanner honours robots.txt groups addressed to 4AllAgentScan. If your
robots.txt disallows that user-agent for a path, the corresponding checks are skipped
and reported as "not applicable" rather than failures. The homepage and robots.txt
itself are always fetched — reading robots.txt is how your policy is discovered.
What it stores
We store the scan result: which checks passed or failed, short factual evidence lines (for example, "no Sitemap directive in robots.txt"), the detected platform, and the score. We do not store page content beyond what those evidence lines quote. Results are shown at a per-site results page that is excluded from search indexing.
Removal
Site owners can have their results removed and their domain excluded from future scans. Email support@4all.digital from an address at the domain in question (or with other evidence of ownership) and we will delete stored scans for the domain and add it to the scanner's blocklist — the apex domain and all its subdomains.