Last modified: 2014-05-12 20:53:14 UTC
• allow the user the ability to control how many media file jobs per minute are added to the MediaWiki job queue. • minimum of 1 • default 10 • maximum of 20 chris steipp is concerned about someone setting up files that are near the max upload size, approximately 1GB on Commons. At 60/min, that’s 8 Gbps .... which would likely dos any other server, and be a significant chunk of MediaWiki bandwidth. chris would like us to keep the max at 20. if anyone wants to increase that value, they should add that request to this ticket and get ops approval and have them monitor server activity while they carry out their bulk upload.
Change 101008 had a related patch set uploaded by Dan-nl: user-job-throttle https://gerrit.wikimedia.org/r/101008
Change 101008 merged by jenkins-bot: user-job-throttle https://gerrit.wikimedia.org/r/101008
Regarding the number of uploads: 20/min per batch upload, for, say, 10 batch uploads at the same time, should be fine for Swift. It's not really that big of a number, even with large files. Regarding bandwidth: it's unlikely that we can do more than 60-70MB/s right now on the Swift cluster. I doubt that we can have enough third-party servers able to push us in aggregate files at that rate, though.
This has been deployed. Change status to verified?
dan: Is this ticket RESOLVED FIXED or is there more work to do?
this is supposed to be a "tracking" bug that keeps track of anyone who requests a higher throttle for GWToolset because it may be too slow in retrieving mediafiles from their server. maybe i haven't set-up the "tracking" bug properly. is there another way to do it?