Proper Robots.txt file required

**soulja90** · 01-19-2007, 04:47 PM

Hey guys and gals,

I think what we need is a proper robots.txt file for pligg. I'm noticing even the users are getting indexed?? Can anyone put a proper robots.txt file up please. Thanks!

JB

**silent1643** · 07-31-2007, 05:35 PM

bump i would also like to see a proper file

**cannibal** · 07-31-2007, 07:10 PM

Im using this as my robots.txt

Code:

User-Agent: * 
Disallow: /search 
Disallow: /submit 
Disallow: /login 
Disallow: /admin 
Disallow: /webalizer 
Disallow: /modlogan 
Disallow: /files 
Disallow: /template_c 
Disallow: /user 
Disallow: /3rdparty 
Disallow: /cache 
Disallow: /files 
Disallow: /backup 
Disallow: /avatars 
Disallow: /modules 
Disallow: /js  
Disallow: /libs 
Disallow: /internal 
Disallow: /evb 
Disallow: /faq-en.php

I added all the other folders just in case, since google index my 3rdparty folder and the modules, files, libs folders :S