Quote:
Originally Posted by juanmartinbravo  As you can see I am a beginner and I'm just learning, but maybe we should look at the meneame robots.txt (spanish famouse pligg)
They have disallow /index.php > http://meneame.net/robots.txt
Maybe we should do the same isn't? |
Google should only index pages it finds by following links so there should be no point in disallowing pages such as story.php because nowhere in pligg is there a need to link to these specific pages without them being passed story information etc. The spanish site you mention may be doing it due to modifications they made.