Last modified: 2010-01-13 22:24:58 UTC

Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 16451 - GIF scaling limit should be applied to animated GIFs only
GIF scaling limit should be applied to animated GIFs only
Status: RESOLVED FIXED
Product: MediaWiki
Classification: Unclassified
File management (Other open bugs)
unspecified
All All
: Normal normal with 3 votes (vote)
: ---
Assigned To: Nobody - You can work on this!
http://commons.wikimedia.org/wiki/Cat...
:
: 16892 19089 (view as bug list)
Depends on:
Blocks: gif 20312
  Show dependency treegraph
 
Reported: 2008-11-24 20:27 UTC by Brion Vibber
Modified: 2010-01-13 22:24 UTC (History)
14 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments
perl script to parse blocks from a GIF (1.49 KB, text/plain)
2009-03-23 22:57 UTC, Steve Sanbeg
Details
Mostly finished patch (8.68 KB, patch)
2009-03-24 04:14 UTC, Andrew Garrett
Details
pics/Spinning-Jimbo-head-of-doom.gif: FAILED: At position: 803, Unknown byte 135 (77.72 KB, image/gif)
2009-08-05 01:29 UTC, Brion Vibber
Details
unsorted/spider_wuv.gif: FAILED: At position: 803, Unknown byte 46 (76.22 KB, image/gif)
2009-08-05 01:30 UTC, Brion Vibber
Details

Description Brion Vibber 2008-11-24 20:27:33 UTC
Due to recurring problems with scaling of large animated GIFs taking down the image scaling servers, we put in a hack to prevent server-side scaling of very large, then *all* animated GIFs.

Unfortunately this currently applies to *all* GIFs, which can be very nasty for cases like this page:

http://commons.wikimedia.org/wiki/Category:Neue_Gedichte_(Heine_1852)

This category on Commons has a gallery with a large number of scanned pages... uploaded as large GIFs... which are not being thumbnailed... so they're all being loaded inline...

My Firefox process was using 2.31GB of real memory after loading the page. Not so bad when you've got a loaded 4-gig box, but pretty crappy if you're on a more typical machine which will keel over and die.

We need to distinguish between animated and non-animated GIFs so still images can continue to be scaled cleanly.
Comment 1 Chad H. 2008-11-25 06:06:54 UTC
Googling around for a solution, it seems the eZ Components framework guys have hit a solution (or they did? I can't find the same bit of code in their current trunk). In any case, check out the processGif() function at http://svn.ezcomponents.org/viewvc.cgi/packages/ImageAnalysis/trunk/src/analyzer.php?view=markup&pathrev=1474 (very bottom of page). Seems they count the frames based off the number of commas. Code seems to be under New BSD license.
Comment 2 Brion Vibber 2008-12-09 20:01:52 UTC
A quick test of that extracted code seems to show incorrect results; I get multiple frames listed for a lot of images in my downloads folder which are not animated.
Comment 3 Raimond Spekking 2009-01-05 17:16:58 UTC
*** Bug 16892 has been marked as a duplicate of this bug. ***
Comment 4 Dan Jacobson 2009-01-05 17:34:50 UTC
Bug 16892 is saying that you are not thumbnailing images whose filenames start
with numbers, nothing to do with animation...
Comment 5 Aryeh Gregor (not reading bugmail, please e-mail directly) 2009-01-05 17:37:55 UTC
(In reply to comment #4)
> Bug 16892 is saying that you are not thumbnailing images whose filenames start
> with numbers, nothing to do with animation...

Your description is wrong.  Plenty of files whose names start with numbers are being thumbnailed.  Note that all of the individual files you had problems with are GIFs -- this is a more plausible explanation for the lack of thumbnailing.
Comment 6 Robert Rohde 2009-02-28 06:35:45 UTC
The first comment in the user notes section at:

http://www.php.net/manual/en/function.imagecreatefromgif.php

provides instructions for determining whether a GIF is animated or not.

It looks for 6 specific bytes as part of a 10 byte sequence that denote the opening of a new frame header.  Strictly speaking, I believe this will sometimes misidentify an image as animated when it is not, since the GIF spec appears to allow the same control bytes to occur within compressed image data (etc.)  However, getting a 6 byte sequence by accident is a low probability event in random data.  More importantly, a false positive (that causes a static image not to be scaled) would appear to be fine from a WMF point of view provided that one never has false negatives, which I think is correct for the code sample given.

If the issue is simply that one wants to the detect and disable scaling on animated GIFs then I believe the code sample at the linked page is adequate for that purpose.
Comment 7 Brion Vibber 2009-03-06 01:47:37 UTC
This fix would make some people happy again. :)
Comment 8 Steve Sanbeg 2009-03-23 22:57:10 UTC
Created attachment 5955 [details]
perl script to parse blocks from a GIF

Those PHP solutions seem a bit kludgy.  A better solution would be parse the GIF, and count the frames; then only thumbnail single-frame images.

It needs < 100 lines of perl to do this, and that could be used as a wrapper around the thumbnailer, or called to check the value before thumbnailing.  Feel free to use the example & credit me.
Comment 9 Brion Vibber 2009-03-23 23:01:43 UTC
This would be pretty easy to implement in PHP. :) Thanks for the sample code!
Comment 10 Andrew Garrett 2009-03-24 04:14:51 UTC
Created attachment 5956 [details]
Mostly finished patch

This is a mostly-done patch which I need to polish off at some point. Putting it here so it doesn't get lost.
Comment 11 Steve Sanbeg 2009-03-24 21:09:25 UTC
Cool, glad it's useful.  BTW

+				## Skip dimensions (Why is this 8 bytes and not 4?)

Maybe I should've called that a bounding box, since the frame overlays a part of the background, so that's the upper left corner + width & height as 4 unsigned shorts.

Comment 12 Andrew Garrett 2009-04-01 06:54:31 UTC
(In reply to comment #11)
> Cool, glad it's useful.  BTW
> 
> +                               ## Skip dimensions (Why is this 8 bytes and not
> 4?)
> 
> Maybe I should've called that a bounding box, since the frame overlays a part
> of the background, so that's the upper left corner + width & height as 4
> unsigned shorts.
> 

I ended up pulling the information from the GIF spec.
Comment 13 Steve Sanbeg 2009-06-09 20:56:42 UTC
*** Bug 19089 has been marked as a duplicate of this bug. ***
Comment 14 Andrew Garrett 2009-07-16 17:04:34 UTC
(Batch change)

These are low-priority miniprojects that I can mop up at some point when I'm doing general code work as opposed to working on a specific projects.
Comment 15 Jools Wills 2009-07-16 17:42:29 UTC
Bug 19089 https://bugzilla.wikimedia.org/show_bug.cgi?id=19089 which was mine and marked as a duplicate is quite important though. What do you think ?
Comment 16 Robert Rohde 2009-07-16 18:34:00 UTC
I have to agree with Jools that this doesn't really feel like a low priority issue.  This bug may not be the end of the world, but certain GIF heavy categories and pages are highly burdensome because the client gets sent many megabytes of uncompressed GIFs rather than usable thumbnails.  This is also one of the most frequently complained about bugs on wiki (once every few weeks in my experience).
Comment 17 Steve Sanbeg 2009-07-17 18:52:44 UTC
Upgrading severity; I don't think "not bringing down the entire site" is really in the spirit of an enhancement.

It would be helpful to have some information on what still needs to be done, and possibly what time frame it's expected.  It's not clear to me whether there's still development work needed, or just testing.  If the current behavior is sufficiently broken, and the time frame is long enough, people may want to migrate from gif to png just to avoid this bug; so they should have something to base that decision on.

When I closed Jools, bug we already had the basic functionality implemented, so I expected a better resolution to be quickly forthcoming, but in the meantime it's only been de-prioritized, so I may have been mistaken.
Comment 18 Andrew Garrett 2009-07-17 19:14:11 UTC
Note that my change of the priority was merely a tool to try and make sense of my bug list, distinguishing bugs that need urgent attention, my ongoing priorities, side projects that I need to get to on my bug day (currently Friday), and bugs that are best assigned to me but which I have no intention of expending the effort to fix.

This bug is in the penultimate category — I will get to it, but it is not urgent nor one of my day to day priorities.

The functionality requested has been implemented in the attached patch, but I need to test and apply the patch when I have the time to.
Comment 19 Brion Vibber 2009-08-02 16:41:17 UTC
Bumping severity down and priority up; it's not a big deal but it's damned annoying and a fix has been a long time coming. :)
Comment 20 Brion Vibber 2009-08-02 16:47:39 UTC
Patch looks pretty nice, as long as the metadata extraction works. :D

Quick note on UI/localization...

+			$info[] = wfMsgNoTrans( 'file-info-gif-frames', $metadata['frameCount'] );
...
+'file-info-gif-frames' => '$1 frames',

For some languages such as a lot of the Slavic languages, we'll still want to be able to use {{PLURAL:}} here even if we're guaranteed to be more than 1, as different numbers can require different plural forms.
Comment 21 Andrew Garrett 2009-08-03 15:49:08 UTC
Fix has been committed, with modifications, in r54284.
Comment 22 Brion Vibber 2009-08-05 01:29:16 UTC
I've got some GIF files that throw an exception reading the metadata, all "At position: 803". Will attach samples...
Comment 23 Brion Vibber 2009-08-05 01:29:58 UTC
Created attachment 6419 [details]
pics/Spinning-Jimbo-head-of-doom.gif: FAILED: At position: 803, Unknown byte 135
Comment 24 Brion Vibber 2009-08-05 01:30:20 UTC
Created attachment 6420 [details]
unsorted/spider_wuv.gif: FAILED: At position: 803, Unknown byte 46
Comment 25 Brion Vibber 2009-08-05 01:33:17 UTC
And one more sample which is too big to upload to bugzilla:

http://leuksman.com/misc/giant.gif

evil/giant.gif: FAILED: At position: 803, Unknown byte 194

Warning -- this is a very large animated GIF which caused problems with our scalers last year. Your browser might not like it either. ;)
Comment 26 Andrew Garrett 2009-08-14 12:37:58 UTC
[Batch change]

Removing Dave McCabe from CC, who I somehow managed to add to the CC list of 12 bugs assigned to me.
Comment 27 Andrew Garrett 2009-08-14 13:20:39 UTC
Fixed bug with non-GCF, non-application extensions in r55015.
Comment 28 Jarek Tuszynski 2009-10-22 21:02:30 UTC
This bug is marked as "fixed", but when non animated GIF's on commons are still not being scaled. Today i was looking at [http://commons.wikimedia.org/wiki/File:Durer-vision-hires.gif] and the 100x100 thumbnail is being created out of 2000x1000 image. Is it fixed and not released yet, or the fix is not working?

Additional question about animated gif's: Are there any plans to thumbnail them? Current fix on commons to add __NOGALLERY__ to all categories that might have any gif's, in order to prevent browser crashes, makes those categories impossible to browse. Since scaling of animated GIF's was disabled "due to recurring problems with scaling of large animated GIFs taking down the image scaling servers", is there a way to outsource this job to some other machine or even allow community to upload scaled versions of GIF file?
Comment 29 Timeshifter 2009-10-23 02:04:03 UTC
Correctly resized animated GIF images are showing up on last.fm pages:

* http://userserve-ak.last.fm/serve/126/36538753.gif
* http://www.last.fm/user/Inci90 - largest size. 55 kilobytes.
* http://www.last.fm/user/Inci90/library - smallest size. 8 kilobytes.
* http://www.last.fm/music/Cluster/+shoutbox - medium size. 20 kilobytes. The Inci90 avatar is currently next to the third comment.

Maybe last.fm is using open source code for this? Can someone ask them? People can load static or animated GIF images as profile/avatar images at the last.fm site. The uploaded image shows up very quickly in the various sizes. See also:
http://commons.wikimedia.org/wiki/Commons:Graphics_village_pump#Animated_GIF_resizing._Last.fm_resizes_them._Why_not_MediaWiki.3F

I used a throwaway email address here at Bugzilla. I don't check it more than once a week. I suggest making email addresses private at Bugzilla. Bugzilla would be much better for it. 
Comment 30 Andrew Garrett 2009-11-03 15:09:51 UTC
(In reply to comment #29)
> Maybe last.fm is using open source code for this? Can someone ask them? People
> can load static or animated GIF images as profile/avatar images at the last.fm
> site. The uploaded image shows up very quickly in the various sizes. See also:
> http://commons.wikimedia.org/wiki/Commons:Graphics_village_pump#Animated_GIF_resizing._Last.fm_resizes_them._Why_not_MediaWiki.3F

We use open-source code to resize animated GIFs, but it is very difficult to do for very very big images.



(In reply to comment #28)
> This bug is marked as "fixed", but when non animated GIF's on commons are still
> not being scaled. Today i was looking at
> [http://commons.wikimedia.org/wiki/File:Durer-vision-hires.gif] and the 100x100
> thumbnail is being created out of 2000x1000 image. Is it fixed and not released
> yet, or the fix is not working?

Yes, for ordinary GIFs to render properly somebody needs to activate it on Wikimedia.

Comment 32 Andrew Garrett 2009-11-12 23:14:04 UTC
I'm reopening this bug.

Essentially, there's some configuration at Wikimedia that says that GIFs should not be scaled at all. Somebody needs to remove it.

Marking with shell keyword.
Comment 33 Robert Rohde 2009-11-13 00:54:38 UTC
(In reply to comment #32)
> I'm reopening this bug.
> 
> Essentially, there's some configuration at Wikimedia that says that GIFs should
> not be scaled at all. Somebody needs to remove it.

It's the lines:

// Disable server-side GIF scaling
$wgMediaHandlers['image/gif'] = 'BitmapHandler_ClientOnly';

in CommonSettings.php which need to be removed.
Comment 34 Larry Pieniazek 2009-11-14 05:24:47 UTC
Do you need another but against a different component entered in order to get this enabled at Commons or is this bug sufficient? What about at other wikis (in case they want it too?). Thanks!
Comment 35 Timeshifter 2009-11-14 17:08:53 UTC
The Commons thread says that system administrators can make the change in CommonSettings.php which is linked from here:
http://noc.wikimedia.org/conf

http://meta.wikimedia.org/wiki/System_administrators#List says "Do not contact random people on this list if you want something done.  Instead, go to the [[irc:wikimedia-tech|#wikimedia-tech]] channel on irc.freenode.net."

I see the part that needs to be removed: 

 // Disable server-side GIF scaling <br>
 $wgMediaHandlers['image/gif'] = 'BitmapHandler_ClientOnly';

It is found here: 
http://noc.wikimedia.org/conf/highlight.php?file=CommonSettings.php 
Comment 36 Guillaume Paumier 2009-12-31 19:13:15 UTC
I believe this is FIXED in MediaWiki now, and there's a dedicated request ( bug 20312 - Enable GIF thumbnail scaling on Wikimedia websites) to enable GIF scaling on Wikimedia websites.

Removing shell keyword and closing accordingly.
Comment 37 Platonides 2010-01-13 22:24:58 UTC
The fix was reverted, follow up on bug 22041

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links