Last modified: 2013-12-02 12:58:01 UTC
This includes both extensions tested with PHPUnit and parser tests. We probably should test all unitrusive extensions in our source control, but WMF-run stuff is definitely needed.
(In reply to comment #0) > This includes both extensions tested with PHPUnit and parser tests. > > We probably should test all unitrusive extensions in our source control, but > WMF-run stuff is definitely needed. I believe parser tests for things like ParserFunctions and Cite are already run, but Antoine can confirm. We'd just need a list of the extra extensions that have tests and they can be included (the full extensions directory is checked out)
In 1.19wmf1: PHPUnit: CentralNotice CodeReview Contest ContributionTracking DonationInterface FlaggedRevs Gadgets MobileFrontend Translate Parser tests: Cite DynamicPageList LabeledSectionTransclusion Math Poem wikihiero
It would also be nice to have Jenkins at least run a PHP lint for all extensions in git, not just for core.
(In reply to comment #3) > It would also be nice to have Jenkins at least run a PHP lint for all > extensions in git, not just for core. See bug 35585.
What's the status of this?
I heard "maybe in July" a week or two ago.
I am currently working on the beta labs. One could duplicate a job from integration/jenkins.git . It would need a specific PHP files to include all the extensions and set them up properly. Then make that job to trigger whenever a change is made to a branch matching 'wmf'.
(In reply to comment #7) > I am currently working on the beta labs. > > One could duplicate a job from integration/jenkins.git . It would need a > specific PHP files to include all the extensions and set them up properly. Then > make that job to trigger whenever a change is made to a branch matching 'wmf'. This wway, only submodule updates on WMF branches will result in tests being run, while I requested that _any_ change to extensions should trigger Jenkins.
Also, all extensions should be tested upon _core_ changes, to see if these changes break them.
Ideally we'd extend the test we already have instead of creating a new test. Instead of just checking out the target branch, applying the change and installing MediaWiki, it would also check out a selection of extensions. It's a little tricky though. The Jenkins job is monitoring all branches of mediawiki/core.git if a change is made to REL1_19, it checks out REL1_19, applies the change and runs the suite. If a change is made on master, it checkouts out master first etc. How do we deal with extensions? The script can quickly become very complex. We could make it so that it loops through the selected extensions, and if there is no branch in the extension by the same name, skip it. Also note that extensions have their own repository. So changes to extensions are not run through the test suite (yet). There's a few ways to go about this. Maybe it's obvious, but I'm not sure. == Re-use Jenkins "MediaWiki GIT Fetcher" job == * The job has an array of selected extensions that will be tested for (for now, we start with extensions deployed on WMF). Whenever the a build is triggered it starts as usual by updating the remote and switching to the correct branch and applying the change. We'd add to this that it will loop through the set of extensions and if it has an equally named branch, check that out into ./extensions and add it to a list for later (when creating LocalSettings.php). This way the installer will also take extension tables into account * Then we change the job configuration to also trigger on changes for any branches in the mediawiki/extensions/{in array}.git repositories (in addition to in mediawiki/core.git). If the extension change is on a branch that has no equally named branch in mediawiki/core.git then change is ignored and can't be tested. == Use wmf-branch == Alternatively we could take the simpler approach and keep the "MediaWiki GIT Fetcher" job exactly as it is. I don't know if it is running currently for wmf-branches or not (afaik it runs on ** all branches), but if not we'd make it so that it does. Maybe we need to add a call for it to initialize/update submodules. After that it should automatically install the extensions and run them as part of the test suite. Although this is a lot simpler and doesn't involve any branch magic-matching. The downside that it would only test things once we intend to update the submodule. I'd say we do both: * Make sure that any submodules are expanded. So that whenever a change is made to a wmf-branch, the test suite will run including extensions. Thus verifying that the to-be-merged change will work as expected on that wmf branch (i.e. not break core - and not break any deployed extensions). * Whenever a change is made to non-wmf branch: Include all selected extensions by installing them from the equally named branch in the relevant repositories. What'dya think?
The MW Git Fetcher is meant just for core. We want that one to be as fast as possible regardless of extensions being tested. I don't want a crapy test in an extension to slow down the publishing of core tests back to Gerrit. That being said, I would really want to setup a Jenkins job template, then have a script generating one Jenkins job for each extension. All the extensions could use a "known to work" copy of core to test. So my approach would be 1 extension = 1 job Also, I definitely need to setup some kind of job to test the wmf branches together with all their extensions. Not sure how to do that yet.
Just as a note (and also to CC myself...): The LabeledSectionTransclusion parser tests are not being run, and I'm also not sure whether the ParserFunctions parser tests are being run - I think the log [1] only shows phpunit tests. In addition, the LST testextensions-job is constantly failing [2]; probably because of a lack of phpunit tests. However, I cannot reproduce it locally, I just get a 'no tests' message. [1] https://integration.mediawiki.org/ci/job/mwext-ParserFunctions-testextensions/lastCompletedBuild/console [2] https://integration.mediawiki.org/ci/job/mwext-LabeledSectionTransclusion-testextensions/ or https://integration.mediawiki.org/ci/job/mwext-LabeledSectionTransclusion-testextensions/lastBuild/console
Being tired of copy pasting and manually updating the project definition, this now depends on bug 44954 "zuul should have a template system to define projects".
Jenkins jobs: https://gerrit.wikimedia.org/r/54356 Zuul triggers: https://gerrit.wikimedia.org/r/54357
Adding back bug 46262 as a dependency. This bug has not been fixed yet, reopening.
(In reply to comment #14) > Zuul triggers: https://gerrit.wikimedia.org/r/54357 This change seems to have broken testing for the TemplateSandbox extension, and possibly others. According to Krinkle, every project that is going to set jshint to voting must have a jshintrc file. Apparently whatever default settings jshint uses are broken and whine about spacing that is specifically called for in [[mw:CC/JS]].
Most of the extensions have been added, there might be some pending but they are being added as they are created. There is no point in keeping this bug open forever so just closing it.
