AI Product Index / Content Signals declared in robots.txt

Content Signals declared in robots.txt

It states how your content may be used — search, AI answers, training — in the one place a crawler already looks. Under 5% of sites declare it, so it is a differentiator rather than a baseline, and the common default was written for publishers protecting an archive rather than for anyone who wants to be read.

3
of 122 points
2.5%
of the total grade
2026
checklist generation

Does this apply to your site?

This check concerns site information and access. Consider your intended audience and publishing policy before making a change.

This score measures coverage of a published technical checklist. It does not measure AI visibility, referrals, content understanding or successful agent tasks. Optional service interfaces may not apply to your site; missing them still affects this fixed checklist.

How to address it when relevant

Add a Content-Signal line to robots.txt (contentsignals.org) stating how your content may be used: search, ai-input, ai-train. Roughly 4% of sites declare it, so it is a differentiator rather than a baseline — and pick the values deliberately, because the common default (ai-train=no) suits a publisher protecting an archive and not everyone is one.

Check your own site

Free, no signup, same endpoint that produced every grade on the leaderboard:

curl "https://index.percall.dev/api/score?url=https://your-site.example"

It returns the letter grade and every check by name with pass/fail. The $0.05 call at POST /api/audit adds why each one failed and a snippet with your origin already in it.

The rest of the checklist

6 more 2026 checks, and 13 from the other generation.