![]() |
|
||
|
|
|
||
|
|
|||
|
|
|||||||
| Notices |
| Legacy Archived Main Board Business chat and general industry chat. All participation is welcome. Dont post your fucking spam here. |

|
|
Thread Tools | Display Modes |
|
|
#1 |
|
Members
Join Date: Mar 2005
Location: Florida
Posts: 215
|
I was wondering about a couple meta tags...
name="revisit-after" content="7days" name="robots" content="index, follow" Do these tags tell bots and spiders to follow the links on the page they are crawling and to come back again in 7 days? I was also wondering if a spider or bot followed the links, does it include links outside of the site it is crawling... if so how do you control it? |
|
|
|
| Bookmarks |
|
|