Last modified: 2014-05-12 20:53:14 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T60417, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 58417 - User-controlled job throttle for GWToolset (tracking)
User-controlled job throttle for GWToolset (tracking)
Status: RESOLVED FIXED
Product: MediaWiki extensions
Classification: Unclassified
GWToolset (Other open bugs)
unspecified
All All
: Unprioritized normal (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks: tracking
  Show dependency treegraph
 
Reported: 2013-12-12 22:49 UTC by dan
Modified: 2014-05-12 20:53 UTC (History)
7 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description dan 2013-12-12 22:49:52 UTC
• allow the user the ability to control how many media file jobs per minute are added to the MediaWiki job queue.
• minimum of 1
• default 10
• maximum of 20

chris steipp is concerned about someone setting up files that are near the max upload size, approximately 1GB on Commons. At 60/min, that’s 8 Gbps .... which would likely dos any other server, and be a significant chunk of MediaWiki bandwidth. chris would like us to keep the max at 20. if anyone wants to increase that value, they should add that request to this ticket and get ops approval and have them monitor server activity while they carry out their bulk upload.
Comment 1 Gerrit Notification Bot 2013-12-12 22:50:36 UTC
Change 101008 had a related patch set uploaded by Dan-nl:
user-job-throttle

https://gerrit.wikimedia.org/r/101008
Comment 2 Gerrit Notification Bot 2013-12-13 15:38:36 UTC
Change 101008 merged by jenkins-bot:
user-job-throttle

https://gerrit.wikimedia.org/r/101008
Comment 3 Faidon Liambotis 2013-12-13 15:41:41 UTC
Regarding the number of uploads: 20/min per batch upload, for, say, 10 batch uploads at the same time, should be fine for Swift. It's not really that big of a number, even with large files.

Regarding bandwidth: it's unlikely that we can do more than 60-70MB/s right now on the Swift cluster. I doubt that we can have enough third-party servers able to push us in aggregate files at that rate, though.
Comment 4 David Haskiya 2014-04-14 06:31:37 UTC
This has been deployed. Change status to verified?
Comment 5 Andre Klapper 2014-04-25 05:20:47 UTC
dan: Is this ticket RESOLVED FIXED or is there more work to do?
Comment 6 dan 2014-04-25 08:48:12 UTC
this is supposed to be a "tracking" bug that keeps track of anyone who requests a higher throttle for GWToolset because it may be too slow in retrieving mediafiles from their server. maybe i haven't set-up the "tracking" bug properly. is there another way to do it?

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links