Read
Reference
The checklist
Use it
Generators
Validators
Most sites ship whatever robots.txt their framework generated. Here is the crawler-by-crawler analysis, and why the AI-versus-search framing produces the wrong policy.
Product analytics are optional. On the skill page, X advertising measurement is also optional and shares your visit and ad identifiers with X. Both are off until you allow them. Privacy policy.