is "karma" idea flawed for niche pligg sites? please read...

[daveyc]

i think the karma idea is brilliant, so please don't prepare to flame me, and it reminds me of a drupal.org mod that handles weighting and scoring...however, there are some (in my opinion) fundamental flaws: without a 'negative karma' measure all users will eventually be high - this causes the caucus rules effect (from alice in wonderland, as in "everybody is a winner") - in this case, the idea is nice, but the site becomes more of a mutual admiration society and has the potential - like digg - to become a bit of an echo chamber when top 'volume' users from within the high karma subset are given more weight and tend to be in agreement with one another...

my point: i'd like to run my site with a clear 'power users' group - this would mean serious professional contributors from my associations (industry peers) but keep the site open to others forever - karma serves this need well, but not if all can eventually catch up simply by gaming the system (e.g. 3 friends join and vote each other ad infinitum)...

drupal mod devs tried to resolve this by creating a ridiculously complicated matrix that could be manipulated by the admin and it alienated users who could never understand the advanced weighting features, either because they were afraid of math, or didn't understand the implications of playing with numbers within the big grid...

slashdot also has some functional issues with karma - a strong contributor can be made invisible overnight - they've kinda gone the other way, giving almost too much weight to recency of contributions/actions versus historical value...

SO anyways, my point is this: karma might be the wrong name for it completely - perhaps rethink of it as 'contributor grade' in much the same way that forum software changes member sub-names based on volume of inputs...and then perhaps a separate 'rate this user' function a la ebay seller feedback so that people can either see 'nobody' or 'top rated users based on reputation with site users on a qualitative level and not volumes or numbers of submissions'

does that make any sense? for example, people on social bookmarking sites do in fact use the rss feeds for popular contributors who touch topics or subjects of interest, but if in fact the ENTIRE site is a subject (as many pligg sites appear to be quite niche), then really it's about: reliability and expertise, in which case a qualitative measure seems (imho) to complement the entirely qualitative democratic nature of voting up items to begin with, just abstracted out to the user base level to basically say, "hey, there are kazillions of you, but check out this dude - even though he's only submitted 2 items, he's a guru and everybody in the world knows him and hundreds of users have boosted his reliability and expertise, so toss him on your watch list"

does this make sense? for example: imagine a 'writer's pligg' where writers submit all sorts of articles about writing (tips, pitches, etc) - but then the Editors in Chief from the NY Times, WSJ and five major magazines and some giant publishing house all join and contact the site admin - the site admin might "manipulate the base expertise" (similar to that drupal mod) and in turn, the site users might view the profile (or some special page or static node with bios) and in turn 'rate up' the gurus as 'more authoritative' - this sounds a bit like what netscape is doing to attempt to improve upon the digg model, hiring professionals in their case, but the overall idea is that the sites eventually can get filled with crap (like digg) and sort and filter at this kind of level is a very distinct feature that might set pligg way far out ahead of digg and others like digg...

any thoughts? ideas? feedback?

[3dKiller]

Hey man,

You make some interesting points.

First, I call it 'credibility' instead of karma. I throw the word karma around here just because it's a word most of 'seem to' understand.

My idea is that users earn 'cred' for posting good quality stories that users like. The problem w/digg is when someone w/lots of friends posts a story, their fanboy gangbanger friends vote it up, regardless of quality.

So the problem seems to be w/the friend system. There's too much power there.

Or is there?

The question is, does it matter if a site is full of crap stories, if the trade-off is allot of eyeballs? I mean, isn't this really all about traffic, and click-through ads?

Something to think about.

~3d

[daveyc]

okay, so here's a simple remedy/idea: an attribute for users that might be managed by admins or a 'subset' within the community...for example, all remains the same but there is a new attribute - like a forum user title change - that can be used as a site filter on top of tags

for example, if a few major gurus are contributing, rather than simply manipulate the weight or whatever, they simply obtain an attribute like 'industry expert' at which point the attribute is almost like a container category, with the option to view submissions and items from such experts across all categories and tags (as a lens of sorts)..

this would allow for a broad topical site to have experts within multiple categories

in a sense, this would be like taking a site like about.com, combining with digg, and in turn allowing those "about guides" to have a separate 'seal of approval' for those who do not want to just meander about, but still leaves the wander option wide open for anybody else, in effect providing two entirely different ways to interact with the site...

does that make sense? it doesn't work well on digg because that's pure social circle gaming, but this idea appears to work (in theory) because it's not touching a weighting system at all, just an attribute assignment that serves as a meta-tag

[stephencarr]

I think we need to remember the importance of "gaming" what makes digg different from other link sites is that the user is involved with a "game", you play Digg and rise to the top if you are good. If you remove all of the gaming or simplify the game too much people will loose interest and you will be running just another link site.

I think the more complexity, or the more levels you have to the game the better. Flickr started life as an experiment in online games (remember game neverending?), read a really interesting interview with Eric Costello here where he explains what I mean:

http://www.adaptivepath.com/publicat...ves/000519.php

[3dKiller]

I think we need to remember the importance of "gaming" what makes digg different from other link sites is that the user is involved with a "game", you play Digg and rise to the top if you are good. If you remove all of the gaming or simplify the game too much people will loose interest and you will be running just another link site. I think the more complexity, or the more levels you have to the game the better.

I totally agree with this statement. It's a game, and everyone, young, old, rich, poor, tech, non-tekkie - like to play games. Mass-appeal.

Think digg+wow. I love it!

~3d

[daveyc]

i totally agree with what you're saying - what i'm talking about though is not a typical mass user site, but really a specialized implementation for a narrow audience, in which case newbies will be welcome but potentially ignored or valued less...but then again, that sort of is the point, discovering that some newbie is a diamond in the rough...yeah, so now i'm kinda turning around on this...

[stephencarr]

I think the scenario where people game the site with their friends and overthrow your preferred 'power users' group would be a perfect test for the Pligg game.

The Pligg rules state that only links that the majority of users deem as quality reach the top. This rule is more important than any other rule and the whole game is designed around it. If the users that have gamed the system to get to the top are posting rubbish links then they will soon find out the game is designed to shove them to the bottom.

Say on your niche site you have 1000 active users. Now one guy gets together with his friends, say there are 10 of them, and votes up a story which naturally gets to the top. If this link is really crap then its exposure at the top should be brief because it will be reported as lame by the majority (the remaining 989 of them) of users. The karma system would then weight the rubbish story poster accordingly so next time it is not quite as easy for him to post a rubbish story. If he keeps re-offending then he will find it almost impossible to get a front page story. This is scalable.

Where this could be perceived to fail is when you have a large inactive userbase who are just lurking passively, and a significantly smaller active set of users. This is where the gamers can win, if your users are not actively voting up and reporting down links then they will end up with a poor quality site. To be honest though, if people are really too lazy to get rid of bad links on the front page then maybe the site is now indeed aimed at the more interested gamer and they are winning for this reason. Done right the Pligg system is the perfect storm, always relevant to those that want it to be relevant.

If you want something with more control, where you are in command of the links, you should use a more conventional CMS.

[daveyc]

yeah, so here's an idea - assume here that i completely agree with the democratic idea but also fear a level of inactivity wiht those 1000 users - so in this case, there could be a detection rule - like if the same 5 or 10 users vote for the same 5 or 10 users on more than 75% of stories it's gaming, otherwise leaves it to chance...

does that make sense? like a collusion detection rule, more logic required to develop an effective rule...

[stephencarr]

Nah, that aint gonna work. How do you detect legitimate voters who just happen to always like the same user's posts because they are consistently good posts? By your new rules these voters get bad karma when all they are really doing is voting for good stories.

[stephencarr]

This is an interesting read about a Digg story that got modded down by slashdotters:

http://www.planetblur.org/beta/index...g_reply&ID=597

The forum in the link is pretty cool too, user moderated posts, not quite sure how their algorithm works but they seem to think it is the muts-nuts.