Last modified: 2013-09-21 15:35:18 UTC
At some point, things just become insane, but we should probably see which resource is the limiting factor for "huge" (>140MP) files. Example files: [[:file:Agriculture_characterisation_map,_Flanders.png]], [[:file:Basemap_for_Afil.png]], [[:file:Drapeau_hillion.png]]
Testing locally (Important, locally I'm not using cgroups, along with probably a different version of vips (vips-7.34.2), so these local tests might not mean much) > [[:file:Agriculture_characterisation_map,_Flanders.png]], This one worked fine locally (It did require more than 100mb of memory, but still less than 400mb). > [[:file:Basemap_for_Afil.png]] Seems to need a bigger temporary file (575 mb seemed to do it) >, [[:file:Drapeau_hillion.png]] Seems to need a 738 mb temp file. --- So it would appear increasing $wgMaxShellFileSize to say 1 GB would decrease the number of images we fail to resize by quite a bit. Not sure if that's something viable to do. At the very least, probably not until bug 52203 is fixed.
Doesn't block bug 9497 as that's closed.
(In reply to comment #1) > At the very least, probably not until bug 52203 is > fixed. Well, if your bug 52203 comment 4 is correct, raising the limit to fix this bug would be a workaround for that bug too, because if uninterrupted the extension cleans up its garbage by itself. The existing workaround now in place seems to make disk storage not a problem, but I'm sure that's not the only consideration.
(In reply to comment #3) > (In reply to comment #1) > > At the very least, probably not until bug 52203 is > > fixed. > > Well, if your bug 52203 comment 4 is correct, raising the limit to fix this > bug > would be a workaround for that bug too, because if uninterrupted the > extension > cleans up its garbage by itself. > The existing workaround now in place seems to make disk storage not a > problem, > but I'm sure that's not the only consideration. J had a tweak to the vips command line that should significantly reduce size of temp files (which should be deployed next week). That may stop many of the failures. Proper solution to the temp file issue would maybe be to create temporary working directory for vips and delete everything in directory when done.
In regards to j's tweak - http://test.wikipedia.org/wiki/File:Huge.png is 160 mp and scaling fine now. True test will come when changes hit commons next week.
Ok, these are the biggest pngs (all pngs bigger than 170 MP) on commons: MariaDB [commonswiki_p]> select img_name, img_width*img_height from image where img_media_type = 'BITMAP' and img_minor_mime = 'png' and img_major_mime = 'image' and img_width *img_height > 170000000 order by img_width*img_height desc; +-----------------------------------------------------------+----------------------+ | img_name | img_width*img_height | +-----------------------------------------------------------+----------------------+ | Iowa_County,_Wisconsin,_Courthouse_-_Icon_-_3000-dpi.png | 4161055734 | | Mandelbrot_fractal_large.png | 715776516 | | Mandelbrot_High_Resolution.png | 600000000 | | Iowa_County,_Wisconsin,_Courthouse_-_Icon_-1000-dpi.png | 462349287 | | Gonders_flats.png | 375955914 | | U-undStraßenbahnGleisplanBerlinundUmgebung2013.png | 293840000 | | Union_Stock_Yards,_Omaha,_1899.png | 260564824 | | Union_Stock_Yards,_Omaha,_1925.png | 259465592 | | Union_Stock_Yards,_Omaha,_1958.png | 258749920 | | Union_Stock_Yards,_Omaha,_1916.png | 258264072 | | Union_Stock_Yards,_Omaha,_1999.png | 257947776 | | Union_Stock_Yards,_Omaha,_1887.png | 257896800 | | Redstone_Rocket_Test_Stand_HAER_AL-129-A_sheet_1_of_7.png | 241231872 | | World4-isometric_north.png | 200950784 | | Rensselaerswyck_Original_Map_Large.png | 195843032 | | Drapeau_hillion.png | 192853712 | | Income_inequality_in_the_United_States,_1979-2007.png | 174420000 | +-----------------------------------------------------------+----------------------+ enwiki has no pngs bigger than 170 MP I tested the top 5, they all seem to render. I will note, that [[commons:file:Iowa_County,_Wisconsin,_Courthouse_-_Icon_-_3000-dpi.png]] doesn't seem to render the courthouse part of the image (compare with [[commons:file:Iowa_County,_Wisconsin,_Courthouse_-_Icon_-1000-dpi.png]]). I'm not sure if that's a problem with our rendering, or if the original image is just broken. I'm going to go with the original image is just broken unless someone comes up with evidence to the contrary. In any case, marking this bug fixed
