Last modified: 2014-07-21 11:48:31 UTC
In terms of complexity, jobs-loop.sh has become complex enough that its shell script implementation is awkward. Porting it to a proper language like PHP would increase the number of people who could meaningfully contribute to it. The following features would be desirable: * The ability to wrap runJobs.php in a per-section PoolCounter pool. It has recently become apparent (Ie5bb11b0) that if the CPU power available to the job runners is going to be efficiently utilised, then the number of job runners running on any given DB master needs to be limited. * A configuration file. Currently Puppet configures jobs-loop.sh in two different ways simultaneously: by changing the command line parameters via JR_EXTRA_ARGS in mw-job-runner.default, and by altering the code itself by making the shell script be a template. I don't think either is an especially elegant configuration method. * The ability to check for the exit status from runJobs.php, somehow avoiding a tight loop of respawns. * An error log might be nice too.
Based on previous comments by Ryan Lane that we should prefer PHP or Python for production scripts, tools, etc. (and I agree with him for a number of reasons), I'm tweaking the bug summary from "Rewrite jobs-loop.sh in a proper programming language" to "Rewrite jobs-loop.sh in PHP or Python".
It could also cooperate with the init script a bit more, say, by forking and writing a PID file. (In reply to comment #1) > Based on previous comments by Ryan Lane that we should prefer PHP or Python > for > production scripts, tools, etc. (and I agree with him for a number of > reasons), > I'm tweaking the bug summary from "Rewrite jobs-loop.sh in a proper > programming > language" to "Rewrite jobs-loop.sh in PHP or Python". The best tool for the job should be used.
Any rewrite should also try to absorb nextJobDB.php to avoid the overhead of shelling out to it all the time.
If we ever go for python, count me in.
Aaron re-wrote it in PHP: https://github.com/wikimedia/mediawiki-services-jobrunner
I have filled Bug 68318 - Bugzilla component for mediawiki/services/jobrunner with appropriate default CC Note the repository is missing tests :/
