Meta robots

The robots meta tag let we :

Utilize a granular,
Page-specific approach to controlling how an individual page should be indexed and served to users in search results.

Place the robots meta tag in the <head> section of a given page, like this:

<!DOCTYPE html>
<html><head>
<meta name="robots" content="noindex" />
(…)
</head>
<body>(…)</body>

</html>

The robots meta tag at the above, instructs search engine not to show the page in search results.

Robots-text attribute value, specifies that the directive applies to all crawlers.

To address a specific crawler, replace the robots value of the name attribute with the name of the crawler that you are addressing. Specific crawlers are also known as user-agents.

Google's standard web crawler has the user-agent name Googlebot. To prevent only Googlebot from crawling your page, update the tag as follows:

This tag now instructs Google (but no other search engines) not to show this page in its web search results.
Search engines may have different crawlers for different properties or purposes.

See the appendix for a complete list of Google's crawlers.

For example, to show a page in Google's web search results, but not in Google News, use the following meta tag: <meta name="googlebot-news" content="noindex" />
If you need to specify multiple crawlers individually, it's okay to use multiple robots meta tags: