Last modified: 2011-03-13 17:46:07 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T2504, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 504 - Moderation of articles with slashdot-like karma; identification of experts in var. fields; "ask the expert" forums; distributed proofreading
Moderation of articles with slashdot-like karma; identification of experts in...
Product: MediaWiki
Classification: Unclassified
General/Unknown (Other open bugs)
All All
: Lowest enhancement with 1 vote (vote)
: ---
Assigned To: Nobody - You can work on this!
Depends on:
Blocks: 3741
  Show dependency treegraph
Reported: 2004-09-16 01:48 UTC by Ted Russ
Modified: 2011-03-13 17:46 UTC (History)
1 user (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Description Ted Russ 2004-09-16 01:48:05 UTC
A solution is to be a bit like Slashdot and do things by 
vote.  My schema (which I'm not particularly attached to and 
which can serve as a jumping-off point) involves ranking 
articles depending on how well "moderated" it is, and how 
good the people ranking the article have been in the past.

- Article WikiBlah Bug is entered by Person A.

- It is made available immendiately with a big 
red "UNVERIFIED" label across the top.

- Immediately, it is also put in the list for moderation

- Anyone who logs in and has indicated that they will 
moderate, will be informed that the article is available for 

- A formula should be used to select whom an article should 
be shown to.  For instance, the next "experts"  to log in 
after the article is posted should be asked to score the 
article and this stopped once the target of (say) six experts 
is reached.  Similarly, it should also be offered to six (or 
however many is found to work best) non-experts for scoring.

- If a person's indicated area of expertise matches the 
classification the article was placed under, their vote 
carries a slight weighting.  These are "experts" and their 
expertise is rated by how many articles they are in 
concordance with their peers in.

- a person also has the option of selecting a "don't know" 
button - their "expertise" in the particular subject is then 
reduced accordingly.

- If this person has a history of consensus with their peers 
in this subject, (i.e. they seem to be getting it right more 
than they're getting it wrong,) their vote carries an 
additional weighting and they are considered an expert, or a 
negative weighting in the case of repeated non-consensus.  
The more positive this rating the more expert a reviewer is.

- Both the "specialist rating" and "general rating" scores of 
the article are indicated and the "UNVERIFIED" label removed 
once the required number of scores have been received and 
averaged.  Until those review scores are reached, readers 
should be able to see both the label and the scores.

- Specialist scores (someone who knows that particular 
subject well as indicated by their positive weighting in that 
subject) are the major component of the article score.

- Should a reviewer feel the article has been miscategorised, 
they may suggest a different category.  The article can then 
be split into two articles, one in each category, and the 
fittest of the two, after a certain threshold is reached, 
will survive while the other is removed.

This way, you will collect statistics about your pool of 
experts, contributors, and articles.  

I think this would improve wikipedia tremendously, as you 
have human peer review.  After a while you will have a huge 
list of subjects that a reviewer is "expert" in, and if they 
submit an article on a subject of their expertise, it could 
get a provisional rating immediately.

Social animal homo erectus being what it is, you will soon 
have a pool of experts on most topics, and therefore a 
possibility of expanding into an "ask the experts" forum as 
well.  Apply the same rating system to forum responses and 
you have another fine resource of wikipedia articles, 
the "vox populi articles" perhaps... 

Articles could also be distributed for review and scoring by 
email to the required number of reviewers, or else made 
available much as the Distributed Proofreading Project is.

I for one would find this to be a great confidence booster, 
I'm sure others would too.  Also, it ensures that the 
practical grassroots experts are found in each subject.  A 
database of email addresses of experts in many fields will be 
the result.  Important to note:  You don't NEED to know if 
the person answering all the questions on radium is Mme 
Curie - what's important is that the person is right.  Their 
ranking in the wikipedia expert system is their correct title 
as long as they produce the right information...

Awww ain't that all starry-eyed and idealistic...  But 
honestly, you could become the largest knowledge exchange in 
the world by doing this.
Comment 1 Wikipedia:en:User:Paddu 2004-11-14 15:50:49 UTC
Changing summary to be more catch-phrase intensive and hence "search-friendly".

Ted, did you check out the article validation system of MediaWiki 1.4 that's
currently live on [[test:]]? That implements at least a tiny part of what you want.

BTW the phrase "database of email" makes me shudder. Any spammers around?
Comment 2 Guttorm Flatabø 2004-11-14 15:58:46 UTC
This kind of stuff has been thouroughly discussed on the different mailinglists
(wikipeda-l, wikitech-l) for a looong time. You can try using and
search for karma or perhaps "trust networks". In conclusion: its very un-wiki
and encourages gaming (note that I have not read this proposal), I'm afraid even
the current CVS (1.4) feature has far too much gaming potential.
Comment 3 Wikipedia:en:User:Paddu 2004-11-14 16:43:29 UTC
[[test:]] doesn't seem to work. Use this link:
Comment 4 Dan F 2005-05-03 15:27:48 UTC
Could someone please post a working link to a summary discussion of this type of
thing?  I'm sure you guys are sick of it, but I bet a lot of people would like
to see the FAQ-style version of what's there for review, and why or why not.

Comment 5 Mark A. Hershberger 2011-03-13 17:46:07 UTC
Changing all WONTFIX high priority bugs to lowest priority (no mail should be generated since I turned it off for this.)

Note You need to log in before you can comment on or make changes to this bug.