SEO for Solos
crawling

noindex

noindex is a robots directive, in a meta tag or an HTTP header, telling a search engine not to include a page in its index.

Why it matters

It is the only reliable way to remove a URL from an index. A robots.txt disallow prevents crawling, not indexing.

How it is measured

Fetch the page and read the robots meta tag or the X-Robots-Tag header.

Checks that involve noindex

Related terms

Questions

What is noindex?

noindex is a robots directive, in a meta tag or an HTTP header, telling a search engine not to include a page in its index.

Why does noindex matter?

It is the only reliable way to remove a URL from an index. A robots.txt disallow prevents crawling, not indexing.

How is noindex measured?

Fetch the page and read the robots meta tag or the X-Robots-Tag header.