Last modified: 2014-11-19 20:45:48 UTC
We currently treat video and audio content like images, which is not ideal for Flow, mobile and offline dumps. We need to - get information about something being an audio or video file from the API - develop a spec that makes sense in HTML5 (likely using video/audio tags instead of img) To make this directly useful for page views we might also need to integrate with https://www.mediawiki.org/wiki/Extension:TimedMediaHandler, ideally using client-side JS code.
There is a videoinfo API end point that provides a mediatype and a link to the full-size resource, which we can set in a <source> tag inside <video> or <audio>. It also seems to support everything we currently get from imageinfo, so I guess we can just switch to using that. http://en.wikipedia.org/w/api.php?action=query&prop=videoinfo&titles=File:Wiki.png&format=json&viprop=url|mediatype There is also prop=transcodestatus, which returns the list of transcodes for a video [1]. This should be used by the client as the list of transcodes changes over time (after upload, when new sizes are added). It currently also only seems to work locally, so the API request for a video on commons needs to go to commons [2]. I guess that could be fixed in the API in the longer term. The current resource attribute is always a relative link, which makes it somewhat hard for a client to discover that they'll need to contact commons. Maybe this can be worked around client-side though by a) sending the API request to both commons and the local wikis, or b) fixing prop=transcodestatus to always work in the local wiki. In a perfect world all our media was on commons anyway so we could just use a single API all the time. [1]: http://commons.wikimedia.org/w/api.php?action=query&prop=transcodestatus&titles=File:Sparkler_359-fps_Slow-Motion.webm&format=json [2]: http://en.wikipedia.org/w/api.php?action=query&prop=transcodestatus&titles=File:Sparkler_359-fps_Slow-Motion.webm&format=json
Note you can get the transcodes from prop=videoinfo on the local wiki using the 'derivatives' key: http://en.wikipedia.org/w/api.php?action=query&prop=videoinfo&titles=File:Sparkler_359-fps_Slow-Motion.webm&format=json&viprop=url|mediatype|derivatives http://en.wikipedia.org/w/api.php?action=query&prop=videoinfo&titles=File:Zh-xi%C3%A8xie.ogg&format=json&viprop=url|mediatype|derivatives Be warned there is an inconsistency which causes some numeric entries to appear as strings in the JSON output (bug 55742 -- I have an old patch for it but nobody's got round to reviewing it, we should probably poke it).
See also some old notes I made: https://www.mediawiki.org/wiki/User:Brion_VIBBER/Media_rendering_encapsulation
(In reply to Brion Vibber from comment #2) > Note you can get the transcodes from prop=videoinfo on the local wiki using > the 'derivatives' key: Ahh, nice! That solves the resource issue, so we can continue to use relative links in there.
Flow would like this to work, embedding video and audio in Flow results in https://www.mediawiki.org/wiki/Topic:S52i2un1zblzd8s8#flow-post-s5fley5jhc8a8pub <img> links to the raw video and audio files that the browser renders as a broken image, hence raising the severity of this. In regular pages ogg and webm appear as <audio> or <video> tags within div class="mediaContainer", even with JavaScript disabled.
S -- this wasn't on the current quarter's roadmap. How high of a priority is it? We should try to coordinate a schedule between our teams. Full audio/video support will probably take a bit of effort to nail down. Is flow interested in starting to work with a rough first draft, with the caveat that our DOM spec could change and break things as we nail down more details of obscure audio/video parameters (there are a lot of them, IIRC)? Or would you rather wait until we've got a reasonable test suite and decent options coverage and are more confident our DOM spec is on the right track?
currently flow is storing the html content and not migrating it forward(that is an independent bug we need to fix). Personally due to this i would rather wait for a more stable DOM spec.
