Last modified: 2011-01-25 00:37:05 UTC

Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 16497 - Paginate Special:AllMessages
Paginate Special:AllMessages
Status: RESOLVED FIXED
Product: MediaWiki
Classification: Unclassified
Special pages (Other open bugs)
1.16.x
All All
: Normal major with 1 vote (vote)
: ---
Assigned To: Nobody - You can work on this!
http://hmwiki.x10hosting.com/wiki/Spe...
: patch, patch-need-review
: 15389 17800 18439 18610 (view as bug list)
Depends on: 18620
Blocks: 7006 7008
  Show dependency treegraph
 
Reported: 2008-11-29 22:09 UTC by Charles Melbye
Modified: 2011-01-25 00:37 UTC (History)
13 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments
Test patch against r45241 adding some debug lines checking mem usage (1.20 KB, patch)
2008-12-31 20:26 UTC, Brion Vibber
Details
Paginate Special:Allmessages using TablePager, against r50159 (27.06 KB, patch)
2009-05-03 22:34 UTC, Happy-melon
Details
Revised patch, still against r50159 (27.56 KB, patch)
2009-05-04 17:51 UTC, Happy-melon
Details

Description Charles Melbye 2008-11-29 22:09:25 UTC
I'm getting a Fatal Error when viewing Special:AllMessages:

Fatal error: Allowed memory size of 52428800 bytes exhausted (tried to allocate 954416 bytes) in /home/yourwiki/htdocs/html/w/includes/specials/SpecialAllmessages.php on line 208

This should be fixed by adding pagination or something like that. This is reproducible if you have a lot of modified system messages.
Comment 1 Siebrand Mazeland 2008-12-04 10:39:58 UTC
Changed component to SpecialPages. Not an i18n issue.
Comment 2 Aaron Schulz 2008-12-28 21:31:28 UTC
(In reply to comment #0)
> I'm getting a Fatal Error when viewing Special:AllMessages:
> 
> Fatal error: Allowed memory size of 52428800 bytes exhausted (tried to allocate
> 954416 bytes) in
> /home/yourwiki/htdocs/html/w/includes/specials/SpecialAllmessages.php on line
> 208
> 
> This should be fixed by adding pagination or something like that. This is
> reproducible if you have a lot of modified system messages.
> 

Does r45141 work for you?
Comment 3 Radon210 2008-12-28 21:33:26 UTC
no same error still happening (and yes I did svn up)
Comment 4 Aaron Schulz 2008-12-28 22:00:49 UTC
(In reply to comment #3)
> no same error still happening (and yes I did svn up)
> 

Try setting memory_limit = 80M in php.ini
Comment 5 Aryeh Gregor (not reading bugmail, please e-mail directly) 2008-12-28 22:26:45 UTC
AllMessages *seriously* needs to be paginated.  It's way too big, not just from a memory usage standpoint.
Comment 6 FunPika 2008-12-30 00:02:10 UTC
Updated summary to reflect need for pagination. 
Comment 7 Brion Vibber 2008-12-31 20:26:11 UTC
Created attachment 5634 [details]
Test patch against r45241 adding some debug lines checking mem usage

Here's the memory_get_usage() results I get on my installation during various points of the rendering using this patch:

memory 1: 13744240 near start of special page
memory 2: 26570352 +13M after $wgMessageCache->loadAllMessages();
memory 3: 27037696
memory 4: 27037704 +0.5M after merging extensions, sorting to $sortedArray
memory 5: 31358432 +4M after creating $messages
memory 6: 33826000 +2.5M after rendering HTML output

By far the biggest memory-user is just loading all the message texts to begin with, eating 13 megabytes. Yeowch! This is potentially hard to trim, as it could require changing the localization infrastructure.

Building the array of sorted & transformed messages adds ~4 megabytes, and another ~2.5 megabytes gets used up making the HTML output. These could be reduced relatively easily by using a more compact rendering and/or doing some paging.

(Your sizes will vary depending on system architecture, installed extensions, and what's in your localized messages.)

If paging is done, we'll want a better integrated search, so it's still at least as easy as now to find a particular message based on name or content substring.
Comment 8 Chad H. 2009-03-05 13:44:04 UTC
*** Bug 17800 has been marked as a duplicate of this bug. ***
Comment 9 Chad H. 2009-03-05 13:44:50 UTC
*** Bug 15389 has been marked as a duplicate of this bug. ***
Comment 10 DaSch 2009-03-05 13:52:40 UTC
I think the problem is that with a rising number of extensions and personal changes the whole thing gets to big, I think the sugestion from Bug 15389 is a good beginning to solve this problem
Comment 11 DaSch 2009-03-05 15:32:33 UTC
I think there should be two things made

1. The messages should be grouped by extensions, so every extension and the mediawiki core messages should all get a single page
2. The messages should be paginated, maybe the way that there are only 50 or 100 messages per page

I think this is a very critical point so there should be created a solution very soon
Comment 12 Roan Kattouw 2009-04-12 18:19:15 UTC
*** Bug 18439 has been marked as a duplicate of this bug. ***
Comment 13 Alex Z. 2009-04-27 22:47:44 UTC
*** Bug 18610 has been marked as a duplicate of this bug. ***
Comment 14 Happy-melon 2009-05-03 22:34:30 UTC
Created attachment 6086 [details]
Paginate Special:Allmessages using TablePager, against r50159

I've been working on a complete solution to this and related bugs, using the TablePager class that is used in, for instance, CodeReview and the AbuseFilter.  This framework provides a nice way to paginate large data sets such as Allmessages.  Notable features include:

*Server-side filtering by customisation state (modified, unmodified or all), and  by prefix
*Integrated language selection
*No horizontal scrolling
*No javascript :D

The one aspect of the implementation that I'm not entirely happy with is the need to pretend, for the benefit of TablePager and its parent IndexPager, that the data was drawn entirely from the database.  To do this, I had to write a FakeResultWrapper class that could be loaded with the message array and that would then spit out the results in the manner IndexPager expects from a real database.  The alternative was to overload so many of IndexPager's core methods (extractResultInfo, getBody and formatRow, at least) as to rather defeat the object of using the framework in the first place.  With the FakeResultWrapper,  new features would have to be implemented to ResultWrapper itself, and utilised in TablePager, in order to cause breakages.

The only breaking change is the removal of the variable output formats; only HTML is now supported.  Both PHP and XML-formatted lists are now available through the API; this should certainly be the access point for scripts that need a machine-readable list of the messages.  The PHP output was added almost exactly five years ago (r3454), from the time when message defaults were still specified in the database, XML 18 months ago (r27389).  I'll CC both committers, but I think that this functionality should be accessed through the API. Admittedly there is currently no way to get the 'original' message through the API, but that's a separate bug.

comments/criticisms would be much appreciated. The patch only blanks phase3/skins/common/allmessages.js for some reason; that file is actually now unused and can be deleted.
Comment 15 Chad H. 2009-05-03 22:42:50 UTC
Applies cleanly and works well here. Have a strict standards bug though, on line 92 of the patched SpecialAllmessages.php:

"Non-static method TablePager::getHiddenFields() should not be called statically, assuming $this from incompatible context"
Comment 16 Happy-melon 2009-05-04 17:51:39 UTC
Created attachment 6089 [details]
Revised patch, still against r50159

Hmn, I guess this could be resolved by calling the method from the TablePager instance that's actually created? This revised patch should do this.  I've also added some replacement CSS; this resolves bug7006.
Comment 17 Happy-melon 2009-05-28 11:35:36 UTC
Anyone planning on doing anything with this? 
Comment 18 Dan Jacobson 2009-06-04 21:09:51 UTC
Glad something is going to be done soon, else it is the click of death
(well, 5 minute freeze) for my browser (emacs-w3m), so I was going to
say maybe make php the default output mode, instead of the giant HTML
table).
$ for i in php html xml; do echo -en $i:\\t
  GET -P ...title=Special:AllMessages\&ot=$i|Ac -c; done
php:	164635
html:	1586231
xml:	209453
Comment 19 Happy-melon 2009-06-04 22:34:43 UTC
PHP was once the default mode, but was replaced with the human readable form because, well, it's not the human-readable form :-D  It's a human-facing page, ergo it should have pretty human-readable content.  For a machine-readable form, go to the API.  

Even with the limit set to 5000, so all the entries display, the TablePager version is still significantly more compact: I make it 1.31Mb vs 2.27Mb; not sure how that meshes with your stats, but it would be a significant saving even if it were only 200kB.
Comment 20 Chad H. 2009-07-15 02:03:47 UTC
Done in r53286
Comment 21 folengo 2010-04-09 08:44:05 UTC
I had asked for this in duplicate bug 15389, and I must say that it works fine now. Thank you.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links