Last modified: 2011-03-11 20:16:49 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T29956, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 27956 - Large category not registering all files
Large category not registering all files
Status: RESOLVED FIXED
Product: Wikimedia
Classification: Unclassified
General/Unknown (Other open bugs)
unspecified
All All
: Normal major (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks: 27339
  Show dependency treegraph
 
Reported: 2011-03-09 07:59 UTC by Rob Lanphier
Modified: 2011-03-11 20:16 UTC (History)
6 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Rob Lanphier 2011-03-09 07:59:52 UTC
This category:
http://commons.wikimedia.org/wiki/Category:Images_from_the_State_Library_of_Queensland

...is supposed to contain ~13,000 images.  As of this writing, it's only registering 264 images.


Examples of files in the category that don't register as being in the category:
* http://commons.wikimedia.org/wiki/File:Laurence_Olivier_and_Vivien_Leigh_receive_a_pineapple,_Brisbane,_1948.jpg
*  http://commons.wikimedia.org/wiki/File:StateLibQld_1_66111_San_Pedrico_(ship).jpg

Seems likely related to category changes, but worth rechecking after category work is done.
Comment 1 Rob Lanphier 2011-03-09 16:08:39 UTC
This is part of a bigger problem:
http://commons.wikimedia.org/wiki/Commons:Village_pump#Strange_system_behaviour

It appears to be a temporary result of new category sorting deployment. There's a long running script that is still in the process of running on the largest wikis (including commons). After the script is complete (hopefully later today), much of the weirdness should clear up.
Comment 2 Antoine "hashar" Musso (WMF) 2011-03-09 18:02:19 UTC
Looks like the script finished on the S3 cluster:
#  12:31 RoanKattouw: updateCollation.php finished on s3 

Still, the wikimania2011 Category:Wikimania_submissions holds only six pages while there seems to be a bit more:

http://wikimania2011.wikimedia.org/wiki/Special:PrefixIndex/Submissions/

[wikimania2010wiki]> select count(*) from categorylinks where cl_to='Wikimania_submissions' \G
*************************** 1. row ***************************
count(*): 179
1 row in set (0.00 sec)
Comment 3 Antoine "hashar" Musso (WMF) 2011-03-09 18:04:25 UTC
Sorry, wrong database name, the count is actually six.

[wikimania2011wiki]> select count(*) from categorylinks where cl_to='Wikimania_submissions' \G
*************************** 1. row ***************************
count(*): 6
1 row in set (0.00 sec)
Comment 4 MZMcBride 2011-03-09 18:16:49 UTC
(In reply to comment #3)
> Sorry, wrong database name, the count is actually six.
> 
> [wikimania2011wiki]> select count(*) from categorylinks where
> cl_to='Wikimania_submissions' \G
> *************************** 1. row ***************************
> count(*): 6
> 1 row in set (0.00 sec)

http://wikimania2011.wikimedia.org/wiki/Category:Wikimania_submissions seems fine now. It showed only six category members until I logged in. Now it's showing 52. I imagine the category description page was simply cached.

Regarding Commons, these issues should mostly be resolved once the category maintenance script finishes on s4. The category member counts (stored in the category table) are going to be wrong until another maintenance script is written/run to fix those counts, as far as I'm aware. I spoke with Roan about this earlier.
Comment 5 Antoine "hashar" Musso (WMF) 2011-03-09 19:44:45 UTC
We have to purge the squids for Categories pages :(
Comment 6 Antoine "hashar" Musso (WMF) 2011-03-09 21:05:13 UTC
I wrote a script (r83404) to purge all pages of a given namespace 

Example usage:
 maintenance/purgeNamespace.php wikimania2011wiki --namespace 14

You probably do not want to run it on enwiki NS_MAIN though :)
Comment 7 Mark A. Hershberger 2011-03-11 01:40:06 UTC
Is this fixed? the category is reporting 14,000 images now.
Comment 8 Antoine "hashar" Musso (WMF) 2011-03-11 07:18:34 UTC
I have not run the script since it is not reviewed and I avoid "playing" with caches systems.
Comment 9 Rob Lanphier 2011-03-11 20:16:49 UTC
This looks fine now.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links