Last modified: 2014-11-17 09:21:20 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T39946, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 37946 - Add support for git branches to ExtensionDistributor
Add support for git branches to ExtensionDistributor
Status: RESOLVED FIXED
Product: MediaWiki extensions
Classification: Unclassified
ExtensionDistributor (Other open bugs)
unspecified
All All
: High major with 1 vote (vote)
: Future release
Assigned To: Chad H.
: code-update-regression
: 42365 (view as bug list)
Depends on: 38012
Blocks: 36802 43495
  Show dependency treegraph
 
Reported: 2012-06-26 02:29 UTC by Krinkle
Modified: 2014-11-17 09:21 UTC (History)
7 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Krinkle 2012-06-26 02:29:45 UTC
Right now there is general git support (bug 35574), but not for branches yet.

The only place Git is supported is as replacement for 'trunk'. I'm not sure that would even work, actually, since then
* extensions that are only in SVN would no longer be downloadable
* branches for Git-only extensions not visible
* if an extension is in both, the branches from the outdated dir in SVN would be shown together with an option for 'master' form Git
.. which is probably why it is disabled by default and we're not using it on mediawiki.org yet.

Instead we should use the local branches of the extensions. If we want to we could have a configuration option that only shows branch names that are in a whitelisted array.

Also, we likely need to get rid of the hardcoded path requirement that everything has to follow [mediawiki]/branches/$version/$extensionName.
Comment 1 Krinkle 2012-06-26 02:42:57 UTC
Here's an idea, let me know what you think:

array(
    'Cite' => array(
        'dir' => '/var/extdist-workingcopies/gerrit/mediawiki/extensions/Cite',
        'branches' => true,
    ),
    'CreateAPage' array(
        'branchDirs' => array(
            'trunk' => '/var/mw-snapshot/wikimedia-svn/trunk/extensions/Cite',
            'REL1_18' => '/var/mw-snapshot/wikimedia-svn/branches/REL1_18/extensions/Cite',
    ),  
);

* Adapters each have their own validator. Git supports any of the three properties, but requires at least one of 'dir' or 'branchDir' to be set. SVN only supports only 'dir' (as short cut to `'branchDir' => { trunk: dir }`) and 'branchDir'.

Where 'branches' can be either:
* - true (default): all branches it can automatically "find"
* - array: whitelist of branches from the auto-find

And 'branchDirs' for using different directories for branches (mostly svn, though a git extension could use this too, e.g.  branches from a fork in another repository clone directory).

Since this array would be huge and unlikely to be maintained manually, perhaps do this from a Hook (that would also save memory when viewing unrelated pages). For the Wikimedia install we'd have the working copy organized the same as now (with the addition of git):

/var/mw-snapshot/
- wikimedia-svn/
  - trunk/
    - extensions/*
  - branches/
    - RELx_yy/
      - extensions/*
- wikimedia-gerrit/
  - mediawiki/extensions/*

The hook would traverse these directories and format return array (which the hook caller should at least cache within the request). This would basically move the logic we currently have inside ExtensionDistributor to wmf-config.

Aside from that, something that the Git adapter still needs (either way) is to switch branches before making the tarball (and properly git-clean/git-reset as well, since switching branches has the tendency of leaving loose ends in Git).
Comment 2 Krinkle 2012-06-26 02:45:23 UTC
Some of the things one will encounter in implementing something like that I recently faced and fixed for mediawiki-core:
* https://toolserver.org/~krinkle/mwSnapshots/#!/mediawiki-core/master
* https://github.com/Krinkle/ts-krinkle-mwSnapshots

(except that that script updates and tarballs from cron for all branches, whereas this should do that on-demand).
Comment 3 Sam Reed (reedy) 2012-06-27 15:19:54 UTC
This should have a dependency on actually doing the branching on the git side, then this be for ED implementation
Comment 4 Krinkle 2012-07-01 20:40:28 UTC
(In reply to comment #3)
> This should have a dependency on actually doing the branching on the git side,
> then this be for ED implementation

Added bug 38012 as a dependency.
Comment 5 Krinkle 2012-11-13 02:51:40 UTC
All extensions bundled with 1.20.0 have a REL1_20 branch. And various others as well. Lets get this set up.
Comment 6 Krinkle 2012-11-23 19:56:40 UTC
*** Bug 42365 has been marked as a duplicate of this bug. ***
Comment 7 Krinkle 2012-11-23 19:57:58 UTC
Raising priority and tagging as regression. This is effectively a regression from the git migration until equaled.
Comment 8 Rob Lanphier 2012-12-28 23:16:04 UTC
Chad is working on this now.  See:
https://www.mediawiki.org/wiki/Requests_for_comment/Replacing_extension_distributor

...as well as Gerrit change #37478.
Comment 9 Chad H. 2013-01-08 20:38:32 UTC
Gerrit change #37478 was merged, so marking this fixed.

Will be deployed as part of the 1.21wmf8 cycle (Jan 16th).

Discussion ongoing on wikitech-l regarding branch/tagging strategies.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links