Google SEO
Best practices — Google Search Central
Does Google respect crawl-delay?
Google does not respect the robots.txt crawl-delay directive. Other search engines and some AI bots may interpret it differently, so check each crawler's behaviour.
Further analysis
Use this entry as an audit checkpoint. The cited source establishes the context; the direct answer identifies what to check before choosing an action.
In SFT, this topic relates to Structure. Check whether the signal is visible on the page, connected to the rest of the website and supported by understandable evidence.
Related search terms include robots.txt crawl-delay, crawler directives. Use them to describe the issue in a brief, audit or answer page.
Applying SFT
Structure turns a rule into readable architecture: headings, URLs, structured data, visible content and clear hierarchy.