These pages live under /tests so crawlers can find and verify cache directives and robots.txt allow/disallow scenarios. Robots nocache + HTTP no-cache headers Robots noarchive Robots noarchive + nocache with HTTP no-cache headers General allow (allowed for all)

General disallow (blocked for all) GPTBot allowed (general blocked) GPTBot blocked (general allowed) Bingbot + GPTBot allowed Once indexing starts for /tests , crawlers will encounter these links, check robots.txt, and apply the directives on each target page.