G06 crawler fixtures

Each page below exercises one crawler behaviour verified on 28 September 2026.

• HEAD 200 but GET 500 (F05)
• Real ISO-8859-1 bytes with accented letters (F10)
• Header says ISO-8859-1 but the body is UTF-8 (F10)
• Declared UTF-8 with an invalid byte (F10)
• Page with a base href pointing to another directory (F12)
• Blocked by a robots.txt wildcard rule (F08)
• Blocked by a robots.txt query wildcard (F08)
• Same page without the query, allowed (F08)
• HTML response over 10 MB (F11 max_size)
• Redirects to an external host (F04 external redirect, recorded without download)
• H1 of 12 words under 70 chars (008: allowed since 28 Sep)
• H1 of 13 words under 70 chars (008: too long by words)
• GTM container reference (altea public container) (037 credits, 28 Sep)
• Plain page

Back to the site index