|
Allowing
or Disallowing Robots Using Meta Tags
If you choose not to use a
robots.txt file, you can also allow or disallow robots from indexing the
page or following the links on the page.
There are four different ways
you can use the meta tag code:
| Meta
tag |
What
it does |
| INDEX |
Index the page |
| NOINDEX |
Do not index the page |
| FOLLOW |
Follow/index all links
on the page |
| NOFOLLOW |
Do not follow/index any
links on the page |
To use the tag, you need to
insert the code into the HEAD content of your html page.
To index the page and follow
all links:
|
<META
NAME="ROBOTS" CONTENT="INDEX,FOLLOW">
|
To index the page but not follow
all links:
|
<META
NAME="ROBOTS" CONTENT="INDEX,NOFOLLOW">
|
To not index the page but follow all links:
|
<META
NAME="ROBOTS" CONTENT="NOINDEX,FOLLOW">
|
To not index the page and not
follow all links:
|
<META
NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW">
|
|
