Last modified: 2014-09-23 22:41:22 UTC
When an OGV video file is uploaded, the requested preview is supposed to be available at a URL like this (the convention is "mid-(filename).jpg", which I believe corresponds to the middle keyframe, extracted without any scaling). http://commons.wikimedia.org/wiki/Special:UploadStash/thumb/d4zwo6jo2olm526ap1h6d2v1hmphn91.ogx/mid-d4zwo6jo2olm526ap1h6d2v1hmphn91.ogx.jpg This file is not available where UploadStash expects it, even after several minutes, so this isn't a thumbnailer-flakiness thing.
However, this seems to work just fine on commons.prototype, with the exact same file, and in fact a nearly identical URL, except for the hostname. http://commons.prototype.wikimedia.org/wiki/Special:UploadStash/thumb/d4zwo6jo2olm526ap1h6d2v1hmphn91.ogx/mid-d4zwo6jo2olm526ap1h6d2v1hmphn91.ogx.jpg (Note that InstantCommons is not an issue since I abandoned both files before publishing).
Since Ian is taking this over, I figured I should brain-dump everything I know. So this is what we know about thumbnails for video - ogg thumbnails do work in Special:Upload, and after the file has been uploaded. - ogg thumbnails work on commons.prototype.wikimedia.org (and properly configured local machines) for everything, even Special:UploadWizard So, what are the differences? - different ffmpeg config on the cluster? (unsure) - files are mounted in NFS on the cluster (definitely a big difference) And what's different between what UploadWizard does and what everything else does: - we are doing something slightly unusual in that we are attempting to get a thumbnail for a file that is in the "temp" zone. This goes back to the definition of zones I told you about. Basically, a zone defines a mapping between a filesystem directory and a base URL. Also, there are some security characteristics associated with zones -- deleted and temp files are not normally served to most users. the standard zones are - 'public' -- the usual thing - 'temp' -- files in the stash are here - 'deleted' -- files that have been "archived" - 'thumb' -- thumbnails Note: the temp zone, is, if I recall correctly, not the same as the temporary directory that files are written into as PHP receives the upload. The 'temp' zone exists in NFS. I had to do a lot of trickery to make thumbnails work for files in the "temp" zone and also be secure. We don't want people to be able to see thumbnails unless the file is published (which is the moment when presumably it has a good license). So, all thumbnails are public, but I hid the actual thumbnail URL from the user by streaming it through PHP. This is the real purpose of Special:UploadStash -- to stream files while hiding their scaler URL. Listing files is just a bit of functionality I threw in there for fun and debugging. (I discovered later that img_auth.php (in the root of MediaWiki) does almost *EXACTLY* the same thing, for wikis which define some forms of security.)
Peter is going to upgrade all the scalers either this week or next. He recommends that we see if moving them to the same OS release as all our dev environments fixes it before digging further.
pinging peter in IRC (and on this bug) to see if the image scalars are upgraded.
To confirm -- is the prototype configuration using on-demand thumbnailing, a separate image scaler subcluster, and the $wgUploadStashScalerBaseUrl configuration option? Things seem to go through a wildly different code path in this configuration, where it performs some sort of manually-proxied HTTP request instead of loading the file from the filesystem. IIRC img_auth.php is handled by having the proxy configuration send those requests to the image scalers directly, so in the MW code nothing very clever has to happen.
(In reply to comment #5) > To confirm -- is the prototype configuration using on-demand thumbnailing, yes > a separate image scaler subcluster no >, and the $wgUploadStashScalerBaseUrl > configuration option? Okay, live, we use this in CommonSettings.php: $wgUploadStashScalerBaseUrl = "$urlprotocol//upload.wikimedia.org/$site/$lang/thumb/temp"; We could use this with commons.prototype.wikimedia.org but that would just point to the same server again. > Things seem to go through a wildly different code path in this configuration, Yup. > where it performs some sort of manually-proxied HTTP request instead of loading > the file from the filesystem. Yes, if I understand the code, img_auth.php cannot be used with scaled images. It chops off any /\d+-px/ prefix. So you get the original no matter what you ask for, via the StreamFile interface. The whole point here was to get thumbnails, in a way that nobody but the uploading user could access (since it wasn't published yet, hasn't been looked at by anybody). So, we assume that the scaler has NTP access to the temporary file in the temp zone. Then we request a scaled version via HTTP, and then pass the data to the client. (Note that the client theoretically could have done this request directly from their browser, but they don't know the tmpnam the server gave their file. That's what makes it reasonably secure.) What could be simpler? :( > IIRC img_auth.php is handled by having the proxy configuration send those > requests to the image scalers directly, so in the MW code nothing very clever > has to happen. I don't understand what you mean here.
all image scalers are now on ubuntu 10.04
So is this bug fixed?
(In reply to comment #8) > So is this bug fixed? It doesn't look like this has been fixed. When I tried to upload [[File:Folgers.ogv]], I got a broken thumbnail. But, really, I'm not sure if that is a good test.
Reassign to default assignee per bug 37789
Chris, can you reproduce this on 1.21wmf2 or git master?
I believe this can be closed, but I would like to confirm that I truly understand what the root issue is. Using a version of UploadWizard from Oct 22 2012 on beta commons, I uploaded an ogv file http://commons.wikimedia.beta.wmflabs.org/wiki/File:Intelligent_risk_taking.ogv and saw correct thumbs in UW itself and on the resulting page in commons. Screen shots attached.
Created attachment 11234 [details] thumbs
Created attachment 11235 [details] thumbs
Created attachment 11236 [details] thumbs
Created attachment 11237 [details] thumbs
Created attachment 11240 [details] UploadWizard with ogv and black box I tried again. At least in UploadWizard it's the same black box as before. See screenshot. Might be specific to the file.
It's browser-specific. The thumb is black for me in FF but normal in Chromium. I tried with both an ogv already on beta commons and also one new to beta commons.
*** This bug has been marked as a duplicate of bug 41403 ***
