Last modified: 2011-05-17 00:18:40 UTC
We could move to a test system similar to the php parser tests. Main advantage being no need to escape everything in addTest( '' ); <pre> # hash lines ignored !! test Title here !! input javascript = 'being'; evaluated.with = the.EvalFunction(); return evaluated.with; !! output hello world !! type (typeof output) string !! end </pre> Or we could use an existing framework (jQuery QUnit[1]) or even an off-site environment (eg. TestSwarm) [1] jQuery QUnit: http://docs.jquery.com/Qunit Example: http://view.jquery.com/tags/1.3.2/test/ [2] TestSwarm: https://github.com/jeresig/testswarm/wiki
It seems like this could fit somewhere in with the current Selenium Test Suite. No?
*** Bug 26800 has been marked as a duplicate of this bug. ***
Just for the record, if we use this, it's not QUnit OR TestSwarm, but both, QUnit provides the testing, TestSwarm to distribute to different browsers. Using Selenium could work as well, although I dont know if TestSwarm supports that.
Linkfix: https://github.com/jquery/testswarm/wiki
TestSwarm looks like a light-weight selenium, ie selenium without the weight of the protocol and a http based controller instead of the selenium 'grid' system. TestSwarm looks easier to setup "new browsers" ( like mobile devices ) and would integrate better with arbitrary jQuery unit tests system ( like jasmin that the upload wizard is using and QUnit or whatever other folks use ). And it appears TestSwarm has support for selenium tests. What is the status of continues integration testing with selenium? The goal is clear, per commit automated testing and reporting. Maybe the last mile of dashboard like status with per-commit hooks, that has not materialized for selenium grid, would be more practical to realize with TestSwarm ?
I agree it appears to be a good choise for us. However I've never used TestSwarm from a management point of view (I only run it as a client from time to time for the jQuery team, they use it themselfs here: http://swarm.jquery.org/ ). I'm planning to experiment with it on my local test server. If that goes well I might show it to some people next week in Berlin [1] and perhaps we can set up something within WMF shortly after that (either on a WMF server of perhaps on the Toolserver first). [1] http://www.mediawiki.org/wiki/Berlin_Hackathon_2011
Krinkle > since I am interested by setting up a testswarm, I will come to you with a couple beers :-)
Setting to assigned, I'm actively working on this. Here's a few points (likely in this order, but not all per se) == Init QUnit == Point 1: Get QUnit in core and prepare some core modules to be more easily testable (ie. return values) Point 2: Create a basic (neutral) environment where QUnit will be running Point 3: Go through core modules created/maintained by Wikimedia develoeprs, line by line, and make sure there's suffecient test suits. == Extend / Integrate QUnit == Point 4: Create a nice way for extensions to state that a module should be loaded on Unit-test page and register a js file with qunit tests which should be executed. Point 5: Figure out what needs to be done to make a QUnit test page into a TestSwarm-able thing through SVN sending periodic tests to clients – without requiring users to have a TestSwarm - a vanilla QUnit page should stay available. == Stuff for TestSwarm == Point 6: Set up a local TestSwarm Point 7: Set up at Toolserver (if possible). Point 8: IRC Bot for reporting breakage ? Point 9: Set up in WMF (probably a wmf-like wiki running trunk, perhaps multiple (one on trunk, one of wmfbranch) see [1]) Point 10: Whatever I forgot ? -- Krinkle [1] http://www.mediawiki.org/wiki/WMF_Projects/Virtualization_cluster
Part of point 1: r87830.
r87830 breaks configuration settings and whatnot by making Map.set() exit early for no apparent reason. Breaks scripts using various config settings, causes scripts to be loaded in debug mode when non-debug mode is requested, and probably breaks other stuff. Needs immediate revert.
Reverted in r87846.
Per IRC this is now fixed in Krinkle's work tree, so it'll be back (fixed) with the next commit on this bug. Yay!
Further implementation of a QUnit-backend test suite is tracked/discussed under bug 28915.
(In reply to comment #12) > Per IRC this is now fixed in Krinkle's work tree, so it'll be back (fixed) with > the next commit on this bug. Yay! Yep! r88144/r88149 removed the old test suite, and the new QUnit module is now available in resource loader. Gadgets can access qunit by wrapper their test unit initialization in: mw.loader.using( 'jquery.qunit', function(){ /* qunit test suite here */ }); As with all Qunit testing, the following elements must be on the page. Create them in a wiki article or insert them dynamically: <h1 id="qunit-header">Name of my Test Suite</h1> <h2 id="qunit-banner"></h2> <div id="qunit-testrunner-toolbar"></div> <h2 id="qunit-userAgent"></h2> <ol id="qunit-tests"></ol>
(In reply to comment #14) > (In reply to comment #12) > Yep! r88144/r88149 removed the old test suite, and the new QUnit module is now > available in resource loader. Do this mean that Bug 28873 can be closed since the code doesn't exists anymore?
