|
A search engine robot is called a “spider”.
The reason being that when the robot/spider goes to a page, it
follows all the links on that page. In the imagination this action
resembles a many-legged spider. The robot follows the links to
see if the pages go where they say they go. To know what these
pages are, the spider reads their meta tags. Robots normally always
follow down at least one level when they index a page, and many
spiders return later to do a “deep crawl” and index
every page they find.
back to web marketing
|
