Last modified: 2014-11-17 10:36:14 UTC

Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 35497 - Implement a way to bring GitHub pull requests into Phabricator
Implement a way to bring GitHub pull requests into Phabricator
Status: REOPENED
Product: Wikimedia
Classification: Unclassified
Git/Gerrit (Other open bugs)
unspecified
All All
: High enhancement with 1 vote (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on: 35429 38196
Blocks: 22596
  Show dependency treegraph
 
Reported: 2012-03-26 18:36 UTC by Sumana Harihareswara
Modified: 2014-11-17 10:36 UTC (History)
19 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Sumana Harihareswara 2012-03-26 18:36:36 UTC
First: Find the various GitHub mirrors of MediaWiki. Turn one into an up-to-date clone of our codebase.

* https://github.com/mediawiki 
* https://github.com/mediawiki/mediawiki-svn
* https://github.com/mediawiki/mediawiki-trunk-phase3


Next, harder step: Make an interface to accept GitHub pull requests as merge requests in Gerrit, or do two-way syncing automatically via a bot.

Erik: "although I'm guessing that falls into the "hard" category, it could give us huge wins in terms of casual contribution.

Chad: "Yeah, that's definitely not straightforward--would need some careful thought to make sure we're doing it in a way that makes sense on our end too."
Comment 1 Sumana Harihareswara 2012-03-26 18:38:53 UTC
Chad, assigning this to you since you're the migration lead, but if you do not think you can start on it in the next month, please de-assign from yourself.  Thanks.
Comment 2 Chad H. 2012-03-26 18:42:02 UTC
Let's make this bug about pulling things back from outside repos into our ecosystem. Mirroring our repo to GitHub/Gitorious is much easier and already has bug 35429.
Comment 3 Ryan Lane 2012-03-27 03:21:49 UTC
Please make sure to be careful when setting up replication to github. We have private repos, and it would suck to have it replicated to github accidentally.
Comment 4 Antoine "hashar" Musso (WMF) 2012-03-27 19:07:43 UTC
What is the point in replicating to github if we are not going to use it to handle pull requests / issue tracking.  I think it will confuse people that will submit patches there that we will probably never take care of.

I would prefer we setup a Gitorious instance instead carefully restricted to hide / disallow pulls and issues tracking.
Comment 5 Sumana Harihareswara 2012-03-27 19:10:12 UTC
We can probably do something to indicate "sorry, please don't submit issues here", but we do want people to submit pull requests to GitHub *and then actually pull them into Gerrit*.  Lots of casual developers are on GitHub and are accustomed to working there -- we should be there too to take advantage of that.
Comment 6 Chad H. 2012-03-27 22:45:13 UTC
(In reply to comment #4)
> What is the point in replicating to github if we are not going to use it to
> handle pull requests / issue tracking.  I think it will confuse people that
> will submit patches there that we will probably never take care of.
> 

You can disable issue tracking on GitHub, iirc.

In any case, turning pull requests from GitHub/Gitorious into merge requests for Gerrit is not the simplest task and I don't see it happening in the very immediate future.
Comment 7 Mark A. Hershberger 2012-03-28 19:39:48 UTC
(In reply to comment #6)
> In any case, turning pull requests from GitHub/Gitorious into merge requests
> for Gerrit is not the simplest task and I don't see it happening in the very
> immediate future.

If I can find an api to list merge requests on GitHub and Gitorious, this shouldn't be a problem.
Comment 8 Erik Moeller 2012-03-28 19:43:54 UTC
I agree it's probably tricky, but it would be awesome. GitHub has a huge existing developer community, lots of people are familiar with it, and our current account creation process is a higher barrier to entry. So let's please look into this when we have time.
Comment 9 Krinkle 2012-03-29 04:02:58 UTC
Making this bug more specific about brining Github pull requests into Gerrit.

The mirroring in general would be one way (master) and is bug 22596.
Comment 10 Sumana Harihareswara 2012-04-03 23:03:22 UTC
Krinkle said:

"PHP has migrated to git at March 19 http://www.mail-archive.com/internals@lists.php.net/msg57098.html contains some links and to their wiki page of gitworkflow...

"They don't use github as primary source, as a one-way mirror only. however they do take pull-requests from there (since github is just git underneath, the php devs can merge from a github pull request into their repo, just like we can merge a github PR into gerrit for example."
Comment 11 Erik Moeller 2012-04-23 21:52:52 UTC
Doesn't solve this particular problem, but Jonathan Leto, who's explored GitHub/Gerrit integration before, sent me a link to an automatic pull request closer which might come in handy as reference code for anyone wanting to work on  actual pull request integration.

https://github.com/cloudfoundry/oss-tools/tree/master/pr-bounce
Comment 13 Krinkle 2012-07-05 10:32:24 UTC
So, some ideas:

* Don't use github.com/mediawiki/core as name
 - means we duplicate user groups/rights with the org "Wikimania" at github
 - means we're going to promote ourselves as "core". e.g. github.com/Krinkle/core.
 - any advantages?
  hashar mentioned we want to be able to allow volunteers to help out with the handling of pull-request and that to grant them rights we'd want to have it outside the @wikimedia organization. However this is not needed because:
   * github allows collaboration without any rights at all. You can leave inline comments, and stuff on any pull request anywhere
   * as in comment 12, they are pull-able in many different formats including plain "git pull" so anyone can pull it locally, and if they have a labs ldap account they can also push straight to gerrit for review.
   * github supports auto-closing of pull-requests when the commit hash is pushed into the repo, so no maintenance there either. And if the commit has to be amended, including "fixes GH-123" or "closes GH-123" will also close the relevant PR as soon as it is merged. And if all else fails, a repo collab in @wikimedia (of which there are many[1]) can just push "Close" manually on github
     So volunteers have complete access without needing to be manually added to anything, this is what made GitHub works. And if its only about closing some exceptional ones, then I'm sure we'll manage that.

* Use github.com/wikimedia/mediawiki-core
 - ideally we'd have some kind of auto-push from gerrit or jenkins, otherwise just set up a 30 minute cron somewhere to `pull gerrit -f` and `push github -f`.
 - admin settings: pull-request: true, issues: false, wiki: false

[1] https://github.com/wikimedia
Comment 14 Krinkle 2012-07-05 10:36:18 UTC
Note that setting up of the mirror in the first place is bug 38196.
Comment 15 Andre Klapper 2012-10-15 15:29:31 UTC
(In reply to comment #4)
> What is the point in replicating to github if we are not going to use it to
> handle pull requests / issue tracking.  I think it will confuse people

(In reply to comment #5)
> We can probably do something to indicate "sorry, please don't submit issues
> here", but we do want people to submit pull requests to GitHub

FYI, according to https://github.com/blog/1184-contributing-guidelines you can clarify contribution guidelines for people who submit pull requests or issues.
Comment 16 Chad H. 2012-10-15 15:44:43 UTC
(In reply to comment #15)
> (In reply to comment #4)
> > What is the point in replicating to github if we are not going to use it to
> > handle pull requests / issue tracking.  I think it will confuse people
> 
> (In reply to comment #5)
> > We can probably do something to indicate "sorry, please don't submit issues
> > here", but we do want people to submit pull requests to GitHub
> 
> FYI, according to https://github.com/blog/1184-contributing-guidelines you can
> clarify contribution guidelines for people who submit pull requests or issues.

Oh, didn't know that. We could easily add a CONTRIBUTING file to mediawiki core :)
Comment 17 Krinkle 2013-03-05 20:01:08 UTC
The Issues and Wiki features in GitHub repositories can be disabled easily. There is no need for any textual guideline to enforce.

We might want to have contribution guidelines for other reasons, but not this.
Comment 18 Krinkle 2013-03-05 20:04:39 UTC
(In reply to comment #5)
> We can probably do something to indicate "sorry, please don't submit issues
> here", but we do want people to submit pull requests to GitHub

(In reply to comment #6)
> You can disable issue tracking on GitHub, iirc.

(In reply to comment #13)
> - admin settings: pull-request: true, issues: false, wiki: false

(In reply to comment #17)
> The Issues and Wiki features in GitHub repositories can be disabled easily.

bug 38196 has been fixed. We have replication now, and the issue tracker and wiki are disabled:

https://github.com/wikimedia/mediawiki-core
Comment 19 Quim Gil 2013-04-09 17:53:32 UTC
In case it helps, I have listed this task at

https://www.mediawiki.org/wiki/Project:New_contributors#Merging_GitHub_pull_requests

What help do you need to get it done?

Also a question: is this about bringing pull requests to Gerrit as soon as they are submitted or about merging the ones that the maintainers accept in GitHub? The difference is where is the discussion supposed to happen.
Comment 20 Sumana Harihareswara 2013-04-09 18:01:38 UTC
http://lists.wikimedia.org/pipermail/wikitech-l/2013-March/067665.html ("Moving a GitHub Pull Request to Gerrit Changeset manually") and the subsequent thread may be useful.
Comment 21 Yuvi Panda 2013-04-15 07:33:52 UTC
I'm working on automating this and should have something in a day or so. See https://mediawiki.org/wiki/User:Yuvipanda/G2G and https://github.com/yuvipanda/SuchABot
Comment 22 Quim Gil 2013-05-23 18:05:05 UTC
Yuvi's tools are now bridging GitHub and Gerrit for a number of repostories that have requested it. More details at https://www.mediawiki.org/wiki/User:Yuvipanda/G2G

Seems to work. FIXED?
Comment 23 Yuvi Panda 2013-05-24 06:45:52 UTC
It is currently enabled for the following repositories, after consulting with people responsible for the repos:

qa/browsertests
extensions/MobileFrontend
extensions/PostEdit
extensions/GuidedTour
extensions/EventLogging
extensions/GettingStarted

Any pull requests to the appropriate repo in GitHub will automatically (and instantly!) have a Gerrit patchset created, and a comment will be left on the GitHub Pull Request informing people of the appropriate Gerrit URL. This is currently running on Wikimedia Tools Labs, and should be considered experimental. If anyone else wants to have their repo added to the 'watched' list, do let me know :)
Comment 24 Quim Gil 2013-05-24 19:08:09 UTC
Resolving as FIXED. There is a functional proof of concept and some precedents. There is a process to request new repos to be added. And there might be bugs, but we have the usual Bugzilla process for this.

Thank you to everybody involved! This is just the beginning.
Comment 25 Yuvi Panda 2013-06-16 16:35:49 UTC
It now syncs Comments / Changeset Abandon / Merging / Restore on Gerrit on GitHub! See https://github.com/wikimedia/apps-android-commons/pull/6 for an example (though usually it closes the PR automatically).


It doesn't sync inline comments yet, so I'll need to figure that out. There is also a Redis based reliable queue running on Tools-labs, so future tools that depend on Gerrit can be easily built too. Ideas for such welcome :)
Comment 26 C. Scott Ananian 2013-08-14 14:50:52 UTC
Yuvi, what's the process for adding new repos?  Maybe some docs can be added to http://www.mediawiki.org/wiki/Gerrit (which links to this bug).
Comment 27 Alex Monk 2014-03-07 20:05:02 UTC
I guess it's time to reopen this. The bot has been down for a long while and Yuvi doesn't know when he's going to get time to fix it.
Comment 28 Sumana Harihareswara 2014-09-23 19:30:08 UTC
If Yuvi does not have time to do this (and, presumably, to get GitHub->Phabricator syndication going), does anyone else?
Comment 29 Quim Gil 2014-09-23 19:53:31 UTC
If Phabricator is the focus of this report now, then we will have a duplicate after the migration: 

GitHub->Phabricator bridge for new contributors
https://phabricator.wikimedia.org/T173

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links