Last modified: 2014-11-13 14:46:24 UTC
We have two bugs requesting to adjust the Doxygen Jenkins jobs: Bug 46771 - Update doxygen from 1.7.x to 1.8.x on gallium Bug 72837 - Jenkins: migrate mediawiki-core-doxygen-publish to zuul cloner I crafted a Jenkins job to run Doxygen on labs Trusty Jenkins slave which have doxygen 1.8.6 https://gerrit.wikimedia.org/r/#/c/166034/ . The build ends up taking several minutes to complete (15 - 20 minutes) which is rather annoying: https://integration.wikimedia.org/ci/job/hashar-mediawiki-core-doxygen-publish/buildTimeTrend The job running on baremetal (gallium) only takes 2-3 minutes https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-publish/buildTimeTrend The slow down is one of: * a performance regression with Doxygen 1.8.x compared to 1.7.x * some new 1.8.x features that should be disabled * Doxygen not playing well on labs instance (could be confirmed by running the job on a Precise instance, if it is super slow, the issue is most probably with labs)
I have manually reproduced the documentation generation on Precise and Trusty slaves by doing: sudo su - jenkins-deploy cd /mnt/jenkins-workspace/ git clone --depth=1 https://gerrit.wikimedia.org/r/p/mediawiki/core.git mediawiki-core cd mediawiki/core git clone https://gerrit.wikimedia.org/r/p/mediawiki/vendor.git touch LocalSettings.php time php maintenance/mwdocgen.php On Precise: ... symbol cache used 23212/65536 hits=9268265 misses=23212 lookup cache used 131072/131072 hits=2485103 misses=138497 ... real 2m9.434s user 1m27.621s sys 0m30.214s On Trusty more than 10 minutes (aborted it eventually).
Turns out that on Trusty we end up using hhvm as the php interpreter: $ php --version HipHop VM 3.3.0 (rel) Compiler: heads/master-0-g9e9af5b2709d173e3a07b3bdb75f9b5af05b4d29 Repo schema: 45d9feee51e194627c004583325c607a563cdcde Extension API: 20140829
Our doxygen configuration runs a filter on each php files to tweak the comment block, the resulting processing is parsed by doxygen. Our Doxyfile has a placeholder for INPUT_FILTER which is expanded by mwdocgen.php as: INPUT_FILTER "php maintenance/mwdoc-filter.php" On Trusty, "php" is managed using Debian alternative system and points to HHVM. The first invocation of mwdoc-filter.php with HHVM takes up to 2 seconds. Once the byte code cache is warmed up, the next runs takes 200ms compared to 40ms for the Zend version: Zend: $ time (echo |php5 mwdoc-filter.php mwdoc-filter.php> /dev/null) real 0m0.046s user 0m0.028s sys 0m0.018s HHVM (warmed up byte code cache): $ time (echo |php mwdoc-filter.php mwdoc-filter.php> /dev/null) real 0m0.208s user 0m0.148s sys 0m0.056s strace shows HHVM read/write to /mnt/home/jenkins-deploy/.hhvm.hhbc which is the sqlite database holding the bytecode cache. Might be the slower. The alternatives system shows: $ update-alternatives --display php php - auto mode link currently points to /usr/bin/hhvm /usr/bin/hhvm - priority 60 /usr/bin/php5 - priority 50 slave php.1.gz: /usr/share/man/man1/php5.1.gz Current 'best' version is '/usr/bin/hhvm'. $ On continuous integration slaves we could stick to the Zend version (albeit v5.5.9) and explicitly use /usr/bin/hhvm when it is actually needed. Another possibility is to adjust the slave script wrapper (integration/jenkins.git tools/mwcore-docgen.sh) to adjust the INPUT_FILTER to /usr/bin/php5 instead of 'php'. Which would switch us to Zend. We are using MediaWiki wfShellWikiCmd() to determine the php script to use, which default to $wgPhpCli = '/usr/bin/php' Thus the mwcore-docgen.sh can be adjusted to set: $wgPhpCli = '/usr/bin/php5' which is available on both Precise and Trusty.
Change 172756 had a related patch set uploaded by Hashar: Point Doxygen input filter to Zend PHP https://gerrit.wikimedia.org/r/172756
I gave the patch a try on a Trusty instance. Using Zend we are down to 6 minutes which is an improvement. The run is still twice slower than on Precise, maybe because more files are being processed.
Change 172756 merged by jenkins-bot: Point Doxygen input filter to Zend PHP https://gerrit.wikimedia.org/r/172756
I have deployed the change to set $wgPhpCli = /usr/bin/php5 On a Trusty instance, the doxygen filter is now using the Zend PHP which make it faster, though doxygen is still twice slower than Precise, so keeping the bug open.
No more blocking: bug 46771 - Update doxygen from 1.7.x to 1.8.x on gallium bug 72837 - Jenkins: migrate mediawiki-core-doxygen-publish to zuul cloner The slowdown is acceptable so I am going to migrate the job to run on labs Trusty instances that have doxygen 1.8.x and switch the job to Zuul cloner. https://gerrit.wikimedia.org/r/#/c/166034/
