Stop Indexing Duplicate Content

[P1mpPanther]

Looks as if the Mod is keeping Google from indexing my "Upcoming" stories.

I'm just getting back on my box from a day at the doctors so I'll report back in more detail in a bit

[wanderingmind]

Guys,
I was hit quite nicely by the duplicate content penalty once. Took six months to identify everything and recover!
I came across one more issue http://forums.pligg.com/general-help...-new-post.html
Please check the last post.
The tag cloud is not working as a tag cloud should - if it was, from lessons I have learnt the hard way so far, there is no dup content issue.
However the tag cloud does a lot of guesswork - click on a tag, and it shows up stories whcih do not have that tag at all! In fact, trying to be like a guesswork search algo!
This is another potential problem area...

[P1mpPanther]

I passed out before getting back to this so I will after work tonite.

I did notice a page of mine indexed with the 'tags' as well - this definitely seems like an issue.

[Simon]

I just don't index tags for now. Easiest way to make sure it doesn't happen until the cloud is fixed.

Btw, you can also try this when you've made your changes. Faster re-indexing supposedly. I tried it on one of my sites last week, and it has seemed to make a difference. To be honest, my own pligg site has completely fallen off my radar for now, and without any link building on my part, most of my results will get stuck in supplemental in Google. But they won't be duplicates at least.

[P1mpPanther]

How do you keep from indexing tags? Is it:

OR $pagename eq "tags"

Thanks for the link, btw!

[Simon]

FYI, Google turned off supplemental markings last week. I would bet they still use the filtering methods behind supplemental results, but there is no way to tell anymore. Same rules apply as far as I'm concerned - keep duplicate content to a minimum, and produce quality content, combined with link building.

[sefs]

Simon I want to index not only the index and story, but the upcoming as well.

How can i do that.

Thanks.

UPDATE:

I seem to be talking foolishness...

What we really want is the index page to be indexed in google and simply follow all other pages and index the story pages when a story page is found by the google bot is that it?

So really we don't need to index the upcoming page just the stories on the upcomming page which will be indexed anyway when google follows the link to the upcoming page ect..?

Is that how it works?

[juanmartinbravo]

Hello,

I have pligg installed on Esto se sale 2.0 / Portada

This is my my robots.txt file that I uploaded to http://estosesale.com/news:

# All robots will spider the domain
User-agent: *
Disallow: /templates/
Disallow: /templates_c/
Disallow: /3rdparty/
Disallow: /libs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /internal/
Disallow: /backup/
Disallow: /thickbox/
Disallow: /api/
Disallow: /evb/
Disallow: /avatars/
Disallow: /admin_index.php
Disallow: /admin
Disallow: /story.php
Disallow: /login.php
Disallow: /user.php
Disallow: /js/
Disallow: /img/
Disallow: /story/
Disallow: /story/title/

#Disallow: /search
#Disallow: /submit
#Disallow: /webalizer
#Disallow: /modlogan
#Disallow: /files
#Disallow: /cache
#Disallow: /faq-en.php

(Some of them have # because I am not sure about if I also should disallow it but I was planning to do it.)
I took it from Stop Pligg Duplicate Content Hack : SEO Hacks

1. I think that my pligg doesn't have story pages (I mean with the word story on the url) because I'm using urlmethod2. Am I right?

2.I don't want to exclude from google upcoming, but I want to exclude pages as:
estosesale.com/news/upcoming/page/2/
estosesale.com/news/upcoming/page/3/
and
estosesale.com/news/published/page/2/
estosesale.com/news/published/page/3/

Should I add the lines detailed below to my robots.txt file?
Disallow: /upcoming/page/
Disallow: /published/page/

3.IMPORTANT QUESTIONS:
Do you think my robots.txt file will have an effect as good as what is being discussed on this thread? Or should I delete it and follow your instructions? I mean is better to edit the files as Simon explained or the robots.txt will produce same effect.

Hope my regular english has let me explain everything clear...

Thanks in advance,

Juan

[juanmartinbravo]

As you can see I am a beginner and I'm just learning, but maybe we should look at the meneame robots.txt (spanish famouse pligg)
They have disallow /index.php > http://meneame.net/robots.txt

Maybe we should do the same isn't?

[rossoonline]

Originally Posted by juanmartinbravo

As you can see I am a beginner and I'm just learning, but maybe we should look at the meneame robots.txt (spanish famouse pligg)
They have disallow /index.php > http://meneame.net/robots.txt

Maybe we should do the same isn't?

Google should only index pages it finds by following links so there should be no point in disallowing pages such as story.php because nowhere in pligg is there a need to link to these specific pages without them being passed story information etc. The spanish site you mention may be doing it due to modifications they made.