Last modified: 2014-10-19 17:44:26 UTC
From bug 69445 comment 13: --- However it would be great to have automated tests running after changes in common.js and gadgets. Such general tests can at least validate no external resource is loaded, or no invalid syntax is added. The simplest option is to run it outside MW (such as a script in labs that runs X times a day), but an extension to do it while editing would be even better... BTW I just runned a simple phantomjs script on all wikipedia languages to catch simple errors: testing als ERROR: ReferenceError: Can't find variable: ta TRACE: -> https://als.wikipedia.org/w/index.php?title=MediaWiki:If-pt-login.js&action=raw&ctype=text/javascript&dontcountme=s: 9 testing kk ERROR: TypeError: 'null' is not an object (evaluating 'document.getElementById('mw-vector-current-variant').innerHTML=wgULS("Кирил","Latın","توتە")') TRACE: -> https://bits.wikimedia.org/kk.wikipedia.org/load.php?debug=false&lang=kk&modules=site&only=scripts&skin=vector&*: 9 anyone wants to fix them? :) --- From bug 69445 comment 14: --- Created attachment 16185 [details] Example for errors across wikis Maybe it is a bit off-topic but here are some cases of errors across wikis in simple read/edit transaction using phantomJS ---
The idea here is to use a tool such as PhantomJS (a headless WebKit browser) to hit a bunch of pre-defined URLs and report JavaScript errors. In addition to strictly checking errors, checking fetches of content on domains outside of Wikimedia's control (e.g., Google Analytics) would also be nice.
<http://tools.wmflabs.org/legobot/wmf_js_scanner.js> checks for any JS errors being thrown on the main page (as an anonymous user) as well as the inclusion of any external resources. Ideally such a tool would be able to login, visit a few pages (including ?action=edit), enable and disable individual gadgets and change skins.
(In reply to MZMcBride from comment #0) > ... > testing als > ERROR: ReferenceError: Can't find variable: ta > TRACE: > -> > https://als.wikipedia.org/w/index.php?title=MediaWiki:If-pt-login. > js&action=raw&ctype=text/javascript&dontcountme=s: 9 > ... FIXED: https://als.wikipedia.org/wiki/MediaWiki:If-pt-login.js?diff=584752
here is my version used for error checking: https://github.com/eranroz/wiki-js-error-log/blob/master/testwiki.js
Seems we have two versions :-] If you guys could unify the effort and have the resulting code hosted in Gerrit, I would be more than happy to write a Jenkins job that runs the script once per hour and spam some place (mail/IRC...).
There's also the VE production browser test that ensures that all wikis with VisualEditor installed actually let you get to VE successfully: https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-VisualEditor-production-linux-firefox-sauce/ (The fy.wikipedia.org one is just a regular test suite failure; the kk.wikipedia.org one looks like a local sysop has broken JS there.)
