Last modified: 2011-06-14 18:21:10 UTC
Transcoding a local copy of the 1920x1088 test file from http://prototype.wikimedia.org/tmh/images/b/b2/File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv Ubuntu 11.04 x86_64 system with ffmpeg stock 0.6.2-4:0.6.2-1ubuntu1. The '480p' WebM file renders fine, but the '720p' WebM version fails. The .queue.webm file got written out to about 37 of the 42 seconds of input data, and is missing index etc. The recorded 'error' looks like regular output: *************************** 5. row *************************** transcode_id: 20 transcode_image_name: File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv transcode_key: 480_900kbs.webm transcode_error: transcode_time_addjob: 2011-05-28 00:03:06 transcode_time_startwork: 2011-05-28 00:09:05 transcode_time_success: 2011-05-28 00:10:48 transcode_time_error: NULL transcode_final_bitrate: 888048 *************************** 6. row *************************** transcode_id: 21 transcode_image_name: File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv transcode_key: 720_VBR.webm transcode_error: FFmpeg version 0.6.2-4:0.6.2-1ubuntu1, Copyright (c) 2000-2010 the Libav developers built on Mar 22 2011 15:55:04 with gcc 4.5.2 configuration: --extra-version=4:0.6.2-1ubuntu1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --enable-libvpx --disable-stripping --enable-runtime-cpudetect --enable-vaapi --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --enable-shared --disable-static libavutil 50.15. 1 / 50.15. 1 libavcodec 52.72. 2 / 52.72. 2 libavformat 52.64. 2 / 52.64. 2 libavdevice 52. 2. 0 / 52. 2. 0 libavfilter 1.19. 0 / 1.19. 0 libswscale 0.11. 0 / 0.11. 0 libpostproc 51. 2. 0 / 51. 2. 0 Input #0, ogg, from '/var/www/trunk/images/b/b2/File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv': Duration: 00:00:42.20, start: 0.000000, bitrate: 12632 kb/s Stream #0.0: Data: skeleton Stream #0.1: Video: theora, yuv420p, 1920x1088, 25 fps, 25 tbr, 25 tbn, 25 tbc Stream #0.2: Audio: vorbis, 48000 Hz, stereo, s16, 192 kb/s Metadata: ENCODER : ffmpeg2theora-0.26+svn16924 SOURCE_OSHASH : cd67a9923275c17c [libvpx @ 0x15ae000]v0.9.6 Output #0, webm, to '/var/www/trunk/images/thumb/b/b2/File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv/File-Arborophila_brunneopectus_pair_feeding_-_Kaeng_Krachan.ogv.720_VBR.webm.queue.webm': Metadata: encoder : Lavf52.64.2 Stream #0.0: Video: libvpx, yuv420p, 1280x725, q=19-19, 200 kb/s, 1k tbn, 25 tbc Stream #0.1: Audio: libvorbis, 48000 Hz, stereo, s16, 64 kb/s Metadata: ENCODER : ffmpeg2theora-0.26+svn16924 SOURCE_OSHASH : cd67a9923275c17c Stream mapping: Stream #0.1 -> #0.0 Stream #0.2 -> #0.1 Press [q] to stop encoding frame= 945 fps= 10 q=0.0 size= 22121kB time=37.59 bitrate=4821.1kbits/s transcode_time_addjob: 2011-05-28 00:03:06 transcode_time_startwork: 2011-05-28 00:07:31 transcode_time_success: NULL transcode_time_error: 2011-05-28 00:09:05 transcode_final_bitrate: 0 Using these shell memory settings to override the defaults: $wgMaxShellMemory = 1024000; $wgMaxShellFileSize = 1024000; Possibly these are not enough and ffmpeg is dying; README recommends 102400*64 for $wgMaxShellMemory, though there are no recommendations on the wiki page. Is there any way we can detect a failed process specifically?
it does look like a good encode from the error... It does have a high bitrate ... Maybe we need to recommend more memory for ffmpeg? Can you upload or put this asset somewhere so Jan and I can take a look at it? Jan is there a good target limit for memory that ffmpeg can consume when dealing with normal range of HD content? The 64Megs recommendation is not based in science of any kind.
I have added a new features help with this: 1) We list transcode status and error on the image page, that would show why the transcode failed. 2) We use the --maxtime argument when calling the runJobs.php script and with r90068 we enforce a maxtime for transcodes. 3) In r90068 I have made transcode calls run in the background and added a check for growing file function that checks if the target file is growing. If the target file stops growing we kill the process and flag the transcode as an error. This should make it a lot harder for frozen transcodes to stay around for any substantial amount of time.
