Last modified: 2014-05-23 00:43:51 UTC
04:40 <+logmsgbot> !log andrew synchronized php-1.5/wmf-config/CommonSettings.php 'GIF scaling working correctly on testwiki, pushing to other wikis again' Working for normal gifs, and small animated gifs. Large animated gifs all create the error: Error creating thumbnail: Invalid thumbnail parameters or PNG file with more than 12.5 million pixels We should really have a better method to deal with this, at least a more descriptive error. Preferably we should thumb the first frame, render a "play" icon on top and always load the full GIF in the image description page, or something like that. This is a followup to 22041
*** Bug 23070 has been marked as a duplicate of this bug. ***
Self-running animations have worked for years and worked fine only yesterday. For instance, an animation on en.Wikipedia “Thermodynamic temperature” self-ran since it was first put there. The animation here on NURBS: http://en.wikipedia.org/wiki/Non-uniform_rational_B-spline …ran fine until only hours ago.
I note that Derk wrote “Preferably we should thumb the first frame, render a "play" icon on top and always load the full GIF in the image description page, or something like that.” Am I reading correctly that a change to the way self-running animations play when one goes to a page containing them is being contemplated here?
(In reply to comment #0) > 04:40 <+logmsgbot> !log andrew synchronized > php-1.5/wmf-config/CommonSettings.php 'GIF scaling working correctly on > testwiki, pushing to other wikis again' > > Working for normal gifs, and small animated gifs. Large animated gifs all > create the error: > Error creating thumbnail: Invalid thumbnail parameters or PNG file with more > than 12.5 million pixels > > We should really have a better method to deal with this, at least a more > descriptive error. > > Preferably we should thumb the first frame, render a "play" icon on top and > always load the full GIF in the image description page, or something like that. One of the problems is that GIFs with large numbers of frames require very large amounts of memory to thumbnail under the current process. For example this simple 400 KB image with 370 frames: http://upload.wikimedia.org/wikipedia/commons/6/6d/Translational_motion.gif Requires >85 MB to thumbnail. For items like that, where the full file size is actually very small, we should probably just send the whole image to the browser and let the browser handle it as is.
(In reply to comment #2) > Self-running animations have worked for years and worked fine only yesterday. > For instance, an animation on en.Wikipedia “Thermodynamic temperature” self-ran > since it was first put there. The animation here on NURBS: > http://en.wikipedia.org/wiki/Non-uniform_rational_B-spline > > …ran fine until only hours ago. They ran fine because they were served unscaled however large or small they were. Now they are being scaled, and so some of them are causing problems. Well, problems more noticeable than filesize and image quality. (In reply to comment #3) > I note that Derk wrote “Preferably we should thumb the first frame, render a > "play" icon on top and > always load the full GIF in the image description page, or something like > that.” > > Am I reading correctly that a change to the way self-running animations play > when one goes to a page containing them is being contemplated here? We're considering options for how to handle the instance where a file is large enough that scaling it will consume a dangerous amount of memory. Currently we just show a useless error; options suggested have included "show a more helpful error"; serving a static image which links to the full file at full file*size*; and serving the whole file unscaled and letting the browser hack at it. Whichever method is chosen, the scaling limit should be checked at upload so that these issues are not encountered in new files.
I have created a Javascript that replaces the error with a "play button" image, and links to the file description page. The file description page will then load the fullsize animated GIF. It does the same for PNGs btw. http://en.wikipedia.org/wiki/User:TheDJ/largeanimatedgifs.js
Derk, is that supposed to be a permanent fix? Quoting Robert: “For items like that, where the full file size is actually very small, we should probably just send the whole image to the browser and let the browser handle it as is.” Indeed. Let’s look at that. If Explorer, Firefox, Chrome, and Safari—which has a bug of its own with Wikipedia’s animated GIFS (I’ve alerted Apple about it)—can scale a 398 KB GIF, let the browser handle it. That’s just my 2¢ as a non-developer. I might also openly wonder just how many animated GIFs there are that fall in this category. If the thumbnailing only requires a couple of gigabytes ($100) of server space, is it worth the fuss? Is there a compelling reason to not restore the previous functioning until a wall-to-wall technical solution is identified and implemented in one fell-swoop?
Greg, No, i'm not a core software developer. It's just an idea I had of something that might be implemented in the core, and this is a quick demo of the idea using Javascript. GregL, further, you make a lot of good points, but they have all been considered WELL before. GIFS are not as easy as most people think they are, Wikipedia is not as simple as some people think it is, and 200 animated gifs between 20kb and 10MB (think category page) is not something you want to throw at a browser.
(In reply to comment #7) > Derk, is that supposed to be a permanent fix? No; it's a temporary implementation of the proposed solution; which will presumably be replaced by a server-side implementation. > I might also openly wonder just how many animated GIFs there are that fall in > this category. If the thumbnailing only requires a couple of gigabytes ($100) > of server space, is it worth the fuss? The process consumes large amounts of RAM, not disk space. There would be precious little point in a rescaling process which produced an output file two hundred times the size of the input. Allowing a process to exist which is known to consume measurable quantities of a resource is a DoS vector: saving a page which requests the image above in every size between 1 and 5000px would not exceed the maximum page size, but could easily send an image scaler into swap trying to produce all the necessary thumbnails. > Is there a compelling reason to not restore the previous functioning until a > wall-to-wall technical solution is identified and implemented in one > fell-swoop? > 200 animated gifs between 20kb and 10MB (think category page) is not > something you want to throw at a browser. That.
I would only make the point that category pages tend to be venues that registered Wikipedians visit—not so much our regular I.P. editors, who constitute some 99.95% of our readership and who are really supposed to be the ones we are creating content for. Mindful of that point, I suggest turning off any outrageously large category pages so that animated GIFs will auto-play without a “Play” button. Category pages can then be re-enabled after a wall-to-wall solution can be implemented that doesn’t pile up the RAM requirement for browsers.
(In reply to comment #10) > I would only make the point that category pages tend to be venues that > registered Wikipedians visit—not so much our regular I.P. editors, who > constitute some 99.95% of our readership and who are really supposed to be the > ones we are creating content for. False premise, like nearly everything else Category pages (especially on Commons) are seen by more readers than editors.
Very well. Having multiple articles with broken animated GIFs seems a poorer state of affairs than what we had before with RAM-requirement problems. I’ll stay out of your hair so you can speedily solve this. If it is going to take more than 24 hours for a complete solution, I would suggest you consider restoring things to as they long were until you have a complete solution.
Additional info from personal inspection of the code. The particular error is generated by the thumb transform failing on normaliseParams() it does this in BitmapHandler, defined in includes/media/Bitmap.php This function matches wgMaxImageArea (12 million pixels) against getImageArea(). For GIFs getImageArea() (from includes/media/GIF.php) is a multiplication with the amount of frames. So as far as I can tell, everything is behaving now as designed and intended. So this bug can focus on someone developing a better method to deal with large GIF images.
Please allow MediaWiki to continue scaling GIF images. MediaWiki seems to be taking a long time to finish up creating animated GIF thumbnails. This is seen by the experiences some have had with individual animated GIFs that have had their image pages purged. See http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29#Category:Animations_of_geometry http://en.wikipedia.org/wiki/User_talk:Werdna#Non-megapixel_animation_doesn.27t_display_as_animated_thumbnail If there are a few animated GIFs that can't be thumbnailed, it is better that GIF scaling be kept turned on. People can upload resized animated GIFs for the animated GIFs that are causing problems. This is better than turning off GIF scaling for all static and animated GIFs. Static GIF scaling is working fine. Also, MediaWiki resizing creates sharper static GIF thumbnails than browser resizing of static GIFs. And turning off MediaWiki scaling of animated GIFs puts a big load on dialup users viewing pages with animated GIFs. Many animated GIFs use hundreds of thousands, or millions, of kilobytes.
Derk-Jan Hartman. So the number of frames times the bytes per frame equals the total bytes. I have noticed that many thumbnails of animated GIFs use more bytes than the full-size animated GIFs that use hundreds of thousands, or millions, of bytes. Maybe, at least for GIFs that get error messages, MediaWiki could use only every other frame, or every third frame, when scaling. This ways thumbnails will use less bytes than the full-size GIFs. Thumbnails can never be as good as the full-size images anyway. People can click the thumbnail image to get to the larger, more detailed GIF.
I just committed r64691 which should help a little -- it allows the unscaled image to be shown on pages even if it exceeds $wgMaxImageArea. It would be nice to have this speedily deployed on WMF wikis. However, it seems to me that the handling of $wgMaxImageArea and $wgMaxAnimatedGifArea is really kind of backwards; it's the latter that should be checked against the total size of all frames, not the former. As it is, it's basically impossible for any GIF with more than 12 frames to trigger the "only thumbnail first frame" case, which surely cannot be intent. Unless someone can point out a reason for doing it this way, I'll fix it in a follow-up commit soon.
...and done: r64692. One thing we could still do about this issue would be to make the error message given when $wgMaxImageArea is exceeded more helpful, and make it link to the image description page if it appears in an article. I'll leave that to someone else, though.
Pardon this suggestion from someone who knows enough about developer coding to make himself somewhat dangerous. But I DO know my way around animated GIFs. If you need to create animated thumbnails, can’t you build in the functionality of a GIF-building application to generate thumbnails? A true GIF with frame optimization would generate fantastically small thumbnails as far as file sizes goes. And, if my understanding of how computers and browsers function is sufficiently on target, the RAM requirements for thumbnail-size GIFs ought to be quite small too. It seems there would be some open-source Linux app to draw from.
Probably someone could do that if they were willing to devote the time, sure. I don't know if any developers are interested enough in pursuing this to put a lot of work into it, though.
The patches have been deployed last night. If an animation is more than 12.5MP total, you get a single frame If an animation is more than 12.5MP per frame, you get the thumberror. Old thumberrors are cached, so you might have to purge the filepage. (Remember to purge on Commons for Commons files)
I assume that “TheDJ” who posted at http://en.wikipedia.org/wiki/Wikipedia:Featured_picture_candidates/delist/File:CSA_states_evolution.gif#Violating_image:_CSA_states_evolution_delisting is Derk-Jan Hartman? To read Happy-melon’s post on Village Pump http://en.wikipedia.org/w/index.php?title=Wikipedia%3AVillage_pump_%28technical%29&action=historysubmit&diff=360916959&oldid=360909356 …it seems the intention of the developers is to eventually get >12.5 MP animated GIFs to work when thumbnailed or scaled from native size. Will someone please explain if there is a difference of opinion in the developer community on this issue an what is going on?
Current status: If an animation is more than 12.5MP total, you get the (Error creating thumbnail: Error code: 137) and no thumb - Bug 61711. If an animation is more than 50MP total, you get a single frame If an animation is more than 50MP total, you get an note (in each language) in the file description ("Due to technical limitations, thumbnails of high resolution GIF images such as this one will not be animated.")
