Test and validate robots.txt directives for Googlebot, Bingbot, and AI crawlers (GPTBot, ClaudeBot, PerplexityBot) with QuackLabs.