Meta Robots Tag

crawl . index . preview
Indexing
Link following
Extra restrictions
Output
HTML meta tag

  
X-Robots-Tag HTTP header
The meta tag goes in the head of an HTML page. The X-Robots-Tag header does the same job for non-HTML files like PDFs and images, set at the server. noindex tells engines to drop the page from results; nofollow tells them not to pass signals through its links.