Last modified: 2014-05-22 18:29:13 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T27403, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 25403 - OOM when rendering SVGs
OOM when rendering SVGs
Status: RESOLVED INVALID
Product: MediaWiki
Classification: Unclassified
File management (Other open bugs)
unspecified
All Linux
: Normal normal (vote)
: ---
Assigned To: Nobody - You can work on this!
https://www.hackerspace.lu/w/thumb.ph...
:
Depends on:
Blocks: svg
  Show dependency treegraph
 
Reported: 2010-10-03 17:08 UTC by David
Modified: 2014-05-22 18:29 UTC (History)
5 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description David 2010-10-03 17:08:26 UTC
On our wiki, as well as on many others ([0],[1],[2],..) I've come across this error report:


SvgHandler::rasterize: convert -background white -thumbnail 64x64\! '/var/www/hackerspace.lu/www/w/images/4/4f/LightOff.svg' PNG:'/var/www/hackerspace.lu/www/w/images/thumb/4/4f/LightOff.svg/64px-LightOff.svg.png' 2>&1
wfShellExec: '/var/www/hackerspace.lu/www/w/bin/ulimit4.sh' 180 102400 102400 'convert -background white -thumbnail 64x64\! '\''/var/www/hackerspace.lu/www/w/images/4/4f/LightOff.svg'\'' PNG:'\''/var/www/hackerspace.lu/www/w/images/thumb/4/4f/LightOff.svg/64px-LightOff.svg.png'\'' 2>&1'
thumbnail failed on syndi: error 1 "sh: wmf2eps: command not found
convert: Delegate failed `"wmf2eps" -o "%o" "%i"'.
convert: unable to open image `/tmp/magick-XXjbtgDb': No such file or directory.
convert: unable to load module `/usr/lib/ImageMagick-6.3.7/modules-Q16/coders/svg.la': file not found.
convert: UnableToOpenBlob `/tmp/magick-XXjbtgDb': No such file or directory.
convert: missing an image filename `PNG:/var/www/hackerspace.lu/www/w/images/thumb/4/4f/LightOff.svg/64px-LightOff.svg.png'." from "convert -background white -thumbnail 64x64\! '/var/www/hackerspace.lu/www/w/images/4/4f/LightOff.svg' PNG:'/var/www/hackerspace.lu/www/w/images/thumb/4/4f/LightOff.svg/64px-LightOff.svg.png' 2>&1"

Note that this is independent of the wiki version. The real problem is the last message coming from convert. The "unable to load module" message is incorrect, because the svg.la file is present and the path is correct.

I've tried executing the same command on the shell and it works flawlessly when removing all of the single quotes. I think that convert is somehow interpreting both arguments, source file and output file as a single argument due to some erroneous escaping. (Removing one pair of "\'" from every argument works as well)

I've not dug into the mw source for his though.
Comment 1 David 2010-10-03 17:11:21 UTC
Sorry, forgot the sources

[0] http://www.drcarlscamp.com/wiki/index.php?title=File:Logo_drcarl.svg&limit=50
[1] http://www.scotlug.org.uk/wiki/Image:Scottux.svg
[2] http://wiki.freifunk.net/Datei:Freifunk-logo.svg

There is more if you search the interwebs for the error message
Comment 2 Bryan Tong Minh 2010-12-12 15:37:37 UTC
Can you try to see if the (unreleased) 1.16.1 from SVN solves your problem? 1.16.1 has quite a lot fixes regarding IM escaping.
Comment 3 David 2011-02-12 17:16:09 UTC
I'm trying out 1.16.2 from svn now. I'm having issues getting MW to use my $wgCustomConvertCommand right now. I'll have to look into this more thoroughly and will get back to you asap.
Comment 4 David 2011-02-12 17:33:21 UTC
Well I can confirm that the $wgCustomConvertCommand = "gm convert %s -resize %wx%h %d"; is being ignored on r80886 when uploading svg files. I.e. svghandler::reasterize() does not respect that variable:

"SvgHandler::rasterize: convert -background white -thumbnail 800x427\! '/var/www/hackerspace.lu/www/w/images/8/84/Crowd.svg' PNG:'/var/www/hackerspace.lu/www/w/images/thumb/8/84/Crowd.svg/800px-Crowd.svg.png' 2>&1"

this fails (although graphicsmagick-imagemagick-compat is installed) with the following error message:

"thumbnail failed on syndi: error 1 "convert: Unrecognized option (-thumbnail)." from "convert -background white -thumbnail 800x427\! '/var/www/hackerspace.lu/www/w/images/8/84/Crowd.svg' PNG:'/var/www/hackerspace.lu/www/w/images/thumb/8/84/Crowd.svg/800px-Crowd.svg.png' 2>&1""

It is NOT being ignored by the BitmapHandler though:

"BitmapHandler::doTransform: Running custom convert command gm convert '/var/www/hackerspace.lu/www/w/images/d/df/Defcon-8.jpg' -resize 800x600 '/var/www/hackerspace.lu/www/w/images/thumb/d/df/Defcon-8.jpg/800px-Defcon-8.jpg'"

This works just fine.
Comment 5 Bryan Tong Minh 2011-02-12 18:07:52 UTC
That is more or less intended, $wgCustomConvertCommand is only meant to work with bitmaps. For SVGs you need to set $wgSVGConverter and add your custom convertor to $wgSVGConverters. See includes/DefaultSettings.php for examples.

However, I think the real problem is that SVGHandler::rasterize does not use the IM specific escaping that is defined in BitmapHandler. (We should really split our MediaHandler and MediaTransformer code one day)
Comment 6 David 2011-02-12 18:18:25 UTC
Oh, sorry, I wasn't aware of the $wgSVGConverter variable. I'll add it right away.
Comment 7 David 2011-02-12 18:30:46 UTC
(In reply to comment #6)
> Oh, sorry, I wasn't aware of the $wgSVGConverter variable. I'll add it right
> away.

Just for completeness' sake I'll add the debugging output below. It seems odd though that it 496 bytes would not be allocatable.

SvgHandler::rasterize: rsvg -w120 -h78 '/var/www/hackerspace.lu/www/w/images/c/c7/Speaker_badge_rear.svg' '/var/www/hackerspace.lu/www/w/images/thumb/c/c7/Speaker_badge_rear.svg/120px-Speaker_badge_rear.svg.png' 2>&1
wfShellExec: '/var/www/hackerspace.lu/www/w/bin/ulimit4.sh' 180 102400 102400 'rsvg -w120 -h78 '\''/var/www/hackerspace.lu/www/w/images/c/c7/Speaker_badge_rear.svg'\'' '\''/var/www/hackerspace.lu/www/w/images/thumb/c/c7/Speaker_badge_rear.svg/120px-Speaker_badge_rear.svg.png'\'' 2>&1'
Removing bad 0-byte thumbnail "/var/www/hackerspace.lu/www/w/images/thumb/c/c7/Speaker_badge_rear.svg/120px-Speaker_badge_rear.svg.png"
thumbnail failed on syndi: error 0 "***MEMORY-ERROR***: rsvg-convert[16683]: GSlice: failed to allocate 496 bytes (alignment: 512): Cannot allocate memory" from "rsvg -w120 -h78 '/var/www/hackerspace.lu/www/w/images/c/c7/Speaker_badge_rear.svg' '/var/www/hackerspace.lu/www/w/images/thumb/c/c7/Speaker_badge_rear.svg/120px-Speaker_badge_rear.svg.png' 2>&1"
Comment 8 Mark A. Hershberger 2011-04-01 19:53:46 UTC
We're about to release 1.17RC1 ... could you update this after trying it?  (Sorry I just found your bug... it is on my radar now.)
Comment 9 Bryan Tong Minh 2011-04-01 20:00:37 UTC
Tweaking summary.
Comment 10 Bawolff (Brian Wolff) 2011-04-02 18:21:52 UTC
Works for me on both trunk (1.18 - r85088) as well as on 1.15.5.

The command used (according to debug messages) is:
/bin/bash '/var/www/w/phase3/bin/ulimit4.sh' 180 102400 102400 'convert -background white -thumbnail 128x128\! '\''/var/www/w/phase3/images/0/0e/Inkscape_logo_2.svg'\'' PNG:'\''/var/www/w/phase3/images/thumb/0/0e/Inkscape_logo_2.svg/128px-Inkscape_logo_2.svg.png'\'' 2>&1'

Version of inkscape is

bawolff@Bawolff-L:/var/www/w/phase3$ convert --version
Version: ImageMagick 6.3.7 08/07/09 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2008 ImageMagick Studio LLC

Version of wmf2eps is:

bawolff@Bawolff-L:/var/www/w/phase3$ wmf2eps --version
libwmf: version 0.2.8


----
>I've tried executing the same command on the shell and it works flawlessly when
>removing all of the single quotes. I think that convert is somehow interpreting
>both arguments, source file and output file as a single argument due to some
>erroneous escaping. (Removing one pair of "\'" from every argument works as
>well)

When trying it on shell, did you include the ulimit.sh part to? Did you try at as the same user as apache, did increasing $wgMaxShellMemory do anything for making it work on mediawiki, etc.
Comment 11 Bawolff (Brian Wolff) 2014-05-22 18:28:52 UTC
(In reply to David from comment #7)
> (In reply to comment #6)
> > Oh, sorry, I wasn't aware of the $wgSVGConverter variable. I'll add it right
> > away.
> 
> Just for completeness' sake I'll add the debugging output below. It seems
> odd though that it 496 bytes would not be allocatable.

Its probably the last 496 bytes that give a problem (e.g. The limit was already almost over the limit, and the last allocation of 496 bytes is what brought it over).

Please increase $wgMaxShellMemory in your LocalSettings.php file

-----

Closing invalid. This appears to be a configuration error. The default for $wgMaxShellMemory has been increased since this bug was filed.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links