Last modified: 2014-06-11 18:05:01 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T60801, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 58801 - Copy contents of https://svn.toolserver.org/ to Wikimedia git
Copy contents of https://svn.toolserver.org/ to Wikimedia git
Status: RESOLVED FIXED
Product: Wikimedia Labs
Classification: Unclassified
tools (Other open bugs)
unspecified
All All
: Unprioritized enhancement
: ---
Assigned To: Marc A. Pelletier
:
Depends on:
Blocks: tool-missing-ts-feat
  Show dependency treegraph
 
Reported: 2013-12-21 15:48 UTC by Maarten Dammers
Modified: 2014-06-11 18:05 UTC (History)
8 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Maarten Dammers 2013-12-21 15:48:12 UTC
A lot of people have their code stored in the Toolserver svn. A copy should be made to the wikimedia git server so that no code or history is lost.
Comment 1 Ryan Lane 2013-12-21 20:34:28 UTC
How about a read only copy of the svn repo hosted as a tool in tool labs? It seems premature to copy all the code into gerrit repos.
Comment 2 Tim Landscheidt 2013-12-22 11:39:18 UTC
(In reply to comment #1)
> How about a read only copy of the svn repo hosted as a tool in tool labs? It
> seems premature to copy all the code into gerrit repos.

ACK.  There are also open requests to remove repos from svn.toolserver.org because they have moved to other locations (cf. https://jira.toolserver.org/browse/TS-1445, https://jira.toolserver.org/browse/TS-1573).

valhallasw also pointed out some time ago on toolserver-l (cf. http://article.gmane.org/gmane.org.wikimedia.toolserver/6370) that *any* user can copy their or other public repos off the Toolserver.
Comment 3 Silke Meyer (WMDE) 2014-02-04 16:58:51 UTC
Can we agree upon Ryan's proposal to add a tool for this in Tool Labs? Who would be responsible for this tool?
Comment 4 Marc A. Pelletier 2014-02-04 18:03:07 UTC
That seems indeed like a reasonable approach.  Given that this is "one shot", any of the current admins could create and set that tool up.  I'm going to be travelling for the next week or so, but if nobody has had the opportunity to make that project by then I'll put it on my list for my return.
Comment 5 Silke Meyer (WMDE) 2014-05-09 11:27:37 UTC
Proposal (from discussion at the Zürich Hackathon with Marc-André Pelletier):

Labs will host a backup of the svn repos on Toolserver. If somebody wants to create a live project out of it, it's possible. Labs staff won't do this though. Volunteers are welcome!

We'll ask Nosy/Marlen to create this backup and copy it to Labs.
Comment 6 Silke Meyer (WMDE) 2014-05-09 11:29:34 UTC
(In reply to Silke Meyer (WMDE) from comment #5)
> Proposal (from discussion at the Zürich Hackathon with Marc-André Pelletier):
> 
> Labs will host a backup of the svn repos on Toolserver. If somebody wants to
> create a live project out of it, it's possible. Labs staff won't do this
> though. Volunteers are welcome!
> 
> We'll ask Nosy/Marlen to create this backup and copy it to Labs.

Nosy has write access to /public/backups/toolserver.
Comment 7 Maarten Dammers 2014-05-10 11:13:17 UTC
What would you think of this approach:
* We create a toolserver_svn_backup repository (let the bike shedding about the name start!). Request at https://www.mediawiki.org/wiki/Git/New_repositories/Requests
* We import all non-empty repositories at https://svn.toolserver.org/repolist to the new git repository
* We write a piece of documentation on how to switch from svn to git 
* We kill of svn.toolserver.org and have it redirect to the piece of documentation on how to switch it.

ccing Merlijn. I hope he wants to help with the actual svn -> git import :-)
Comment 8 Merlijn van Deen (test) 2014-05-10 11:38:29 UTC
svn->git migration is not completely trivial, due to the free-form nature of SVN repos: some of the might be in the typical 'branches/', 'trunk/', 'tags/' format, but others might not.

In that aspect, just providing a read-only svn server would be the easiest option. Maybe we can run svnserve as webgrid task? I'm not sure about its security track record, though.
Comment 9 Merlijn van Deen (test) 2014-05-10 11:47:01 UTC
The list of used repositories (rev > 0) is here: http://tools.wmflabs.org/jira-bugimport/svn/000_USED_REPOSITORIES/ , including their root directory contents.

Indeed some of them are using the standard trunk/branch/tag layout, but many of them are not.
Comment 10 Tim Landscheidt 2014-05-10 17:04:06 UTC
(In reply to Merlijn van Deen from comment #8)
> [...]

> In that aspect, just providing a read-only svn server would be the easiest
> option. Maybe we can run svnserve as webgrid task? I'm not sure about its
> security track record, though.

AFAIUI, that would require opening a fixed port from the outside with masking the IP for privacy reasons, etc.  Setting up a lighttpd -> Apache -> mod_dav_svn chain isn't that easy either (one more advantage of Git: You can just dump a repository onto a webserver and everybody can clone it -- that's all).

However I'm still very certain that noone actually needs that :-).  If the code is in active use, its authors will have moved it.  If not, it's probably unusable by now.  So I think having a read-only copy (+ backup) in a shared directory will satisfy the demand.
Comment 11 Merlijn van Deen (test) 2014-05-10 17:27:07 UTC
Three of them have had commits in the last month (not counting nosy); 16 have had commits in the last year:

 9984 May  7  2013 magnus.txt
 1241 May 11  2013 geohack.txt
  889 May 25  2013 multichill.txt
  884 May 29  2013 qicvic.txt
 2440 Jul 16  2013 mazder.txt
  882 Aug 13  2013 p_wppb.txt
  950 Aug 25  2013 drtrigon.txt
  839 Aug 29  2013 jitse.txt
 1491 Sep  2  2013 chris.txt
 1086 Oct 10  2013 alexz.txt
 1294 Oct 14  2013 p_erfgoed.txt
  884 Oct 22  2013 p_dewpmp.txt
  851 Feb 11 21:50 p_locator.txt
 1365 Mar  2 16:07 gribeco.txt
  830 Apr 16 07:04 nosy.txt
  868 Apr 20 10:34 seth.txt
  834 May  1 15:31 eccenux.txt
  835 May  2 14:16 phe.txt

I think it would be a good plan if someone contacts those maintainers to see if they need any help moving. Silke, could you maybe do that?
Comment 12 nosy 2014-05-11 21:07:30 UTC
Tried to copy the svn to /public/backups/toolserver - the mount is read-only.
Comment 13 nosy 2014-05-18 20:51:35 UTC
Ok the stuff is still sitting in my home - where should I put it next?
Comment 14 nosy 2014-05-20 09:23:08 UTC
I guess I will put all the stuff I backup to Labs first in my home. If this gets unhandy just let me know.
Comment 15 nosy 2014-06-04 09:12:37 UTC
Ping
Comment 16 nosy 2014-06-11 17:29:11 UTC
The backup can be found in tools-login:/home/nosy/svn
Comment 17 Marc A. Pelletier 2014-06-11 18:05:01 UTC
This has been moved to /public/backups/Toolserver/svn/

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links