Wikimedia Bugzilla is closed!

it's not a bug.  When you use thumb the image is resized to 180px.  The bloat occurs when it's transcoded from original format to 180px; no idea how all that works... maybe since you have the original rastor data you can render it optimally, whereas a resized animation that is deconstructed and reconstructed causes significant bloat.  The fix:  either render it at 180px, or change the type to frame (as I have now done).

I would say it's a bug when the size is increased 15 times.

It may be.  I thought that the act of deconstructing and reconstructing an animated .GIF would be massively more inefficient than re-rendering it (from the vector to the rastor - or maybe i have the terms backwards?) would cause the bloat.  I am not an image/graphics person, and so maybe this shouldn't be.

This problem can be fixed by telling ImageMagick to optimize the animation after scaling.  The only difficulty is that this won't work in very old versions of ImageMagick (where the definition of "very old" depends on which optimization method is chosen).  For more information, see <http://www.imagemagick.org/Usage/anim_opt/>.

An alternative would be to postprocess the scaled animations with a separate program designed specifically for optimizing GIF animations, such as GIFsicle (<http://www.lcdf.org/gifsicle/>).  This would probably produce the best results, but I think this would have to be an optional / Wikimedia-specific change, since we can't really count on all webhosts having GIFsicle installed.

In any case, there is, of course, a simple workaround: (re)upload your animations at the size they're meant to be shown at.  Of course that's not very elegant, but it's what _everyone_ did until about 1.5 years ago (back when the scaling of optimized GIF animation was simply broken and useless).

Anyway, I'll see what I can do to fix this.

It's a BUG.

Let's have a look into includes/media/Bitmap.php:

			$cmd  =  wfEscapeShellArg($wgImageMagickConvertCommand) .
				" {$quality} -background white -size {$physicalWidth} ".
				wfEscapeShellArg($srcPath) .
				// Coalesce is needed to scale animated GIFs properly (bug 1017).
				' -coalesce ' .
				// For the -resize option a "!" is needed to force exact size,
				// or ImageMagick may decide your ratio is wrong and slice off
				// a pixel.
				" -thumbnail " . wfEscapeShellArg( "{$physicalWidth}x{$physicalHeight}!" ) .
				" -depth 8 $sharpen " .
				wfEscapeShellArg($dstPath) . " 2>&1";


The ImageMagick parameter '-coalesce' de-optimizes the animated GIF. That's okay for the further thumbnail step. However, the result is NOT re-optimized, but saved UN-optimized. I can confirm that a downscale from 320px to 120px can yield a 10-fold increase in file size.

PROPOSED FIX:

			$cmd  =  wfEscapeShellArg($wgImageMagickConvertCommand) .
				" {$quality} -background white -size {$physicalWidth} ".
				wfEscapeShellArg($srcPath) .
				// Coalesce is needed to scale animated GIFs properly (bug 1017).
				' -coalesce ' .
				// For the -resize option a "!" is needed to force exact size,
				// or ImageMagick may decide your ratio is wrong and slice off
				// a pixel.
				" -thumbnail " . wfEscapeShellArg( "{$physicalWidth}x{$physicalHeight}!" ) .
				" -depth 8 $sharpen " .
				' -layers optimize ' .
				wfEscapeShellArg($dstPath) . " 2>&1";

The additional ImageMagick option '-layers optimize ' helps to keep the file size more reasonable. "More reasonable" means that the thumbnail may, in some cases, still have a larger filesize than the original, but at least not very much larger.

Like '-coalesce' it has no effect on other file formats (like PNG). Above code is currently in effect at http://www.c64-wiki.de and http://www.c64-wiki.com (which is still running V1.11.x). Note that similar scaling code can be found in includes/Image.php, which should also be fixed.

Currently, only about half of the animated GIFs at the C64-Wiki have been re-done with the 
'-layers optimize' option, so you may still find some excessivly sized thumbnail files at http://www.c64-wiki.de/index.php/Kategorie:Animation !

- Klaus

Created attachment 4865 [details]
Patch to resolve excessive size of thumbnails of animated GIfs when using ImageMagick

Fix has no effect on JPG, PNG and non-animated GIFs, verified at the C64-Wiki with Imagegaick 6.4.0.

It feels like the optimization of the animated GIF thumbnails requires more CPU power (that was to be excpected...). OTOH, thumbnails which require 10 or 15 times the storage space and bandwitdh than the original don't really make sense, so investing a little more CPU time into the occassional animated GIF appears to be the right thing to do :-)

> Note that similar scaling code can be found in includes/Image.php, which should also be fixed.

Forget it, the code in the runk has already been sanitized wrt to Image.php. Only media/Bitmap.php matters.

> Forget it, the code in the runk has already been sanitized wrt to Image.php.
"runk" = "trunk". Yup, 1337 skillz on da keyboard. I haz them! ;-)

Created attachment 4906 [details]
Updated patch

Rearranges some code to look nicer to me and only add the extra options for GIFs.

Some issues, however...

The -layers option seems to have come in semi-recently. I get about twice the filesize under ImageMagick 6.2.8 (on our Fedora boxes) as I do on my home box w/ 6.3.8, while the 6.2.4 on our Ubuntu boxes rejects the -layers option outright, failing to scale entirely!

Not good -- this would break every GIF on sites with old ImageMagick. :(

We can upgrade our own copies, but it would be good for MediaWiki to be more careful with these options. It could detect failure and back off, retrying without the 'fancy' options, or do a version check beforehand (potentially slower or less reliable).

Note our image scaler boxes are currently running ImageMagick 6.3.7... on the other hand we've disabled scaling of GIFs for now due to recurring problems handling GIFs. :P

A version check, or a fallback-on-failure could deal with the versioning issue in general (eg remove the -layers option if it failed to run the first time).

*** Bug 23398 has been marked as a duplicate of this bug. ***

I intend on fixing this issue. Do we still think we need backward compatibility with imagemagick before 6.3 ?

ImageMagick 6.3 was first released in October-2006.

as a matter of fact, 6.2.7 which is the first to support -layers optimizeplus was already released in May 2006. And 6.2.6 the first release with -layers was released in april of 2006.

Hmm... I see that Ubuntu 6.06 LTS (dapper) still uses IM 6.2.4, and the server version is officially supported until June 2011.  Then again, they also include MW 1.4.  Even Debian stable has IM 6.3.7 these days.

Personally, I think I'd be OK with requiring an ImageMagick version less than four years old.  Although that means I'll have call my webhost and ask them to upgrade -- it seems they're running a mix of 6.2.4 and 6.3.7 on their servers.  Oh well, I should do that anyway.  Or compile my own.

After having studied

http://www.imagemagick.org/Usage/video/#gif
http://www.imagemagick.org/Usage/anim_opt/

I have studied the options "-fuzz 10% -layers optimize". This provided reasonable optimization. I also tried adding +map to reduce to prevent per frame colortables, but this option doesn't deal very well with transparency unfortunately (because it does significantly reduce filesize).

I'm also considering splitting animated from non-animated behavior, using png thumbs for non-animated images.

For backwards compatibility i'm considering expanding the ImageHandler with a routine to check /opt/local/bin/convert -version  That info should probably be cached in some way, but I don't really have any ideas about that yet.

*** Bug 2604 has been marked as a duplicate of this bug. ***

Right this is annoying. Apparently using "-layers optimize" breaks transparency. I have discussed this with an ImageMagick developer, and "This is a know problem, and one that suddenly appeared, but the cause is not known. It appears to effect the first image in a sequence but does not always happen."

So we cannot optimize images and expect transparency to work, though it does seem that the "-layers OptimizeTransparency" option doesn't have this problem as much. I'll try to see if I can figure that out for 100%.

I'll also consider at least adding +map to the images, to keep the colortable reasonable.

Confirmed, just using -layer OptimizeTransparency doesn't exhibit this problem, but it was only added in IM v6.3.4-4).

So we could use that, but then we should probably have a sort of version check for imagemagick. I'm not sure how to do something like that in PHP (at least not with it being at all effecient).

Should be much improved with r71354.

I'm using OptimizeTransparency, hopefully at some time the normal -optimize is repaired and we can start using that.

The +map option appears to still damage at least some transparent animated GIFs; see bug 28631.

Removed the +map bit in r87099