Last modified: 2014-10-29 23:05:46 UTC

Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 61711 - out of memory for some animated GIF images on Commons
out of memory for some animated GIF images on Commons
Status: NEW
Product: Wikimedia
Classification: Unclassified
General/Unknown (Other open bugs)
unspecified
All All
: High normal (vote)
: ---
Assigned To: Nobody - You can work on this!
:
: 47409 70296 (view as bug list)
Depends on:
Blocks: gif 41371
  Show dependency treegraph
 
Reported: 2014-02-20 23:11 UTC by Alex Monk
Modified: 2014-10-29 23:05 UTC (History)
8 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments
mem limit values I tested image magick with (1.76 KB, patch)
2014-06-21 20:40 UTC, Bawolff (Brian Wolff)
Details

Comment 1 Bawolff (Brian Wolff) 2014-02-22 14:03:02 UTC
137 = OOM


In theory we are supposed to render only the first frame if image big enough to cause this. Maybe settings need to be tweaked.
Comment 2 PRO 2014-03-13 16:59:56 UTC
The joke is that a error message ("Note: Due to technical limitations, thumbnails of high resolution GIF images such as this one will not be animated." - maybe from Bug 23063) and a static thumb only appears if the GIF is over 50MP! But the limit is really 12,5MP! (It was 
a short time 50MP at 2013 https://commons.wikimedia.org/w/index.php?title=Special:Contributions&offset=201305261339&limit=500&tagfilter=&contribs=user&target=Slick)
Comment 3 PRO 2014-04-27 00:45:20 UTC
Sorry my correction, the limitation is not somehow 12.5MP it is above ~23 MP (and seems for sure as of 26 MP, dependence by the frame size I've not tested). I've tested some in [[Category:Animated GIF files affected by MediaWiki restrictions]].
Comment 4 Bawolff (Brian Wolff) 2014-05-24 21:00:44 UTC
(In reply to PRO from comment #3)
> Sorry my correction, the limitation is not somehow 12.5MP it is above ~23 MP
> (and seems for sure as of 26 MP, dependence by the frame size I've not
> tested). I've tested some in [[Category:Animated GIF files affected by
> MediaWiki restrictions]].

Note, the megapixel limit and the memory limit are separate limits.

The amount of memory a file takes does not precisely correspond to a specific number of megapixels.
Comment 5 Dominic 2014-05-29 20:21:43 UTC
This may be a separate issue, but that error message is particularly unhelpful. When you hit the MP limit, at least you get a message for that, in theory.
Comment 6 Bawolff (Brian Wolff) 2014-05-30 00:17:29 UTC
(In reply to Dominic from comment #5)
> This may be a separate issue, but that error message is particularly
> unhelpful. When you hit the MP limit, at least you get a message for that,
> in theory.

Would actually writing out "out of memory" actually be anymore helpful? I cant imagine that that means much to average user.

(For reference 137 comes from 9 + 128, which is bash's way of saying process recieved SIGKILL (signal number 9), which gets caused by exceeding the linux cgroup memory limit)
Comment 7 Dominic 2014-05-30 02:43:31 UTC
It may not help the average user experience (though if they are hitting this error, I am nit sure they can be helped), but I was just thinking how frustrated I get at vague error messages, which I can't look up to see what the cause is or if it is a known issue.
Comment 8 Marco 2014-06-01 16:10:46 UTC
(In reply to Dominic from comment #7)
> Would actually writing out "out of memory" actually be anymore helpful? I
> cant imagine that that means much to average user.
Maybe not the average user but the uploader may think his GIF file is broken in some way other than being too large.


The error message for TIF seems to be more helpful. C.f: https://upload.wikimedia.org/wikipedia/commons/thumb/2/2f/19-N-9283-1720-42-K-Launching_BB-62_waterborne.tiff/lossy-page1-599px-19-N-9283-1720-42-K-Launching_BB-62_waterborne.tiff.jpg
Comment 9 Bawolff (Brian Wolff) 2014-06-01 16:13:36 UTC
Different error: oom can be triggered by different but related conditions. Tiff files can also be oom
Comment 10 Bawolff (Brian Wolff) 2014-06-20 22:55:07 UTC
Hmm, I was recently testing image magick, and it seems to have better memory performance then older versions, with me being able to scale a bunch of images locally that don't work on cluster.
Comment 11 Bawolff (Brian Wolff) 2014-06-21 20:40:27 UTC
Created attachment 15694 [details]
mem limit values I tested image magick with

So in my tests, newer versions of image magick seem to sometimes be able to recover from ulimit -v, and use less memory if they run out (at least that's what I think is happening). That doesn't seem to happen with cgroups.

There are various environmental variables you can set to tell image magick its memory limits. When I tried setting them on newer image magick (6.8.9-3), it caused the animated gif I was testing to be rendered without exceeding memory limit (where previously cgroups killed it for OOM) [See attached patch for values I was testing with].

However, testing on 6.6.0-4, this didn't seem to have a lot of affect. WMF is currently using "ImageMagick 6.6.9-7 2014-03-06 Q16", so I'm unsure what affect it would have there (Although rumour has it they are updating some applications in the next 2 months).

The values I was testing with were mostly pulled out of my hat/trial and error, so also unclear if they are good values. Anyways, more research needs to be done to see if this is practical/a good idea [which is why I added my patch here, not in gerrit].

The gif file I was testing with is https://upload.wikimedia.org/wikipedia/commons/archive/6/64/20140614071837%21BULGARI_-_a_gentleman%27s_Carbongold_Hong_Kong_chronograph_wrist_watch._Fellows-1438-2-A.gif . I was using cgroups with $wgMaxShellMemory set to 400*1024*1024

see also: http://www.imagemagick.org/script/command-line-options.php#limit
Comment 12 Bawolff (Brian Wolff) 2014-09-03 04:59:43 UTC
*** Bug 70296 has been marked as a duplicate of this bug. ***
Comment 13 Bawolff (Brian Wolff) 2014-09-11 00:34:49 UTC
*** Bug 47409 has been marked as a duplicate of this bug. ***
Comment 14 Kelson [Emmanuel Engelhart] 2014-10-29 23:05:46 UTC
New example of problematic image (although the image is really not so big!):
https://commons.wikimedia.org/wiki/File:DNA_orbit_animated.gif

Try to get for example:
http://upload.wikimedia.org/wikipedia/commons/thumb/1/16/DNA_orbit_animated.gif/185px-DNA_orbit_animated.gif

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links