Thread: The proper usage of the robot.txt file
View Single Post
Old 03-20-2005   #5
Titan
Members
 
Titan's Avatar
 
Join Date: Jan 2005
Posts: 492
Default
unless u want to keep something unlisted in the se's then i dont see why you would - everything they crawl is a potential listing somewhere in the se's
Titan is offline   Reply With Quote