Last modified: 2014-10-01 14:46:16 UTC
wiki.toolserver.org should be preserved as a live, editable, no-new-users wiki at the WMF cluster. Steps to that goal: 1. Decouple wiki from the Toolserver SSO 2. Dump users and data 3. Set up a wiki on the WMF cluster without CentralAuth (wmgUseCentralAuth = false IIRC) 4. Load users and data 5. Reset the/mail out new users' passwords 6. Provide Toolserver SSL certificate to WMF 7. Set up wiki.toolserver.org as a CNAME for text-lb NB: The individual steps are not part of the Tools project, but there will be separate bugs filed for them in the appropriate components.
IIRC the TSWiki isn't setup on MySQL which will make it a tad bit harder to do.
(In reply to comment #0) > wiki.toolserver.org should be preserved as a live, editable, no-new-users > wiki at the WMF cluster. Why not move the content to the wiki at wikitech.wikimedia.org?
I'm not sure we want to move this wiki, per se. Looking at <https://wiki.toolserver.org/view/Special:Statistics>, there are currently about 320 content pages and a little over 1500 pages total on the wiki. We could just merge this content into Wikitech (or mediawiki.org or Meta-Wiki or ...) and take a full snapshot of wiki.toolserver.org for posterity. This would be a lot less work, I think. If we also wanted to set up redirects for legacy wiki.toolserver.org/view/ links, we could maybe do that, but before that... Who owns the toolserver.org domain? Will the toolserver.org domain ultimately belong to the Wikimedia Foundation?
(In reply to comment #2) > (In reply to comment #0) > > wiki.toolserver.org should be preserved as a live, editable, no-new-users > > wiki at the WMF cluster. > > Why not move the content to the wiki at wikitech.wikimedia.org? (In reply to comment #3) > I'm not sure we want to move this wiki, per se. Looking at > <https://wiki.toolserver.org/view/Special:Statistics>, there are currently > about 320 content pages and a little over 1500 pages total on the wiki. We > could just merge this content into Wikitech (or mediawiki.org or Meta-Wiki or > ...) and take a full snapshot of wiki.toolserver.org for posterity. This > would > be a lot less work, I think. I see three problems with that: 1. Moving a whole wiki is a more or less standardized process. If we only move parts, we probably need to develop some scripts for that. If the migration fails at some point, when moving a whole wiki you can just delete it and start anew, when moving to a live, already existing wiki rolling back is a *big* pain in the ass. 2. After a partial merge from one wiki to a namespace or something similar *no* user attribution will be correct, *no* link will be correct, links outside of the parts merged won't work at all or point to different content, the skin will be totally different. Moving a whole wiki doesn't pose these challenges nor corrupt information. 3. The Toolserver wiki will have no useful (apart from a historic view) information that would enrich wikitech because it relates to a totally different system. On the other hand, users would refer to this obsolete information so that the support channels would get pestered not only with the typical "I read that this should work somewhere on the InterNet!", but also "I read it on wikitech!" So I would like to avoid any possible confusion between current, up-to-date documentation and information kept for historic value as far as possible. > [...] > Who owns the toolserver.org domain? Will the toolserver.org domain ultimately > belong to the Wikimedia Foundation? toolserver.org is currently registered by WMDE; I have no opinion if it should be registered to WMF at some point because the differences will be negligible.
(In reply to comment #1) > IIRC the TSWiki isn't setup on MySQL which will make it a tad bit harder to > do. https://wiki.toolserver.org/view/Special:Version Indeed. There's very little (read, almost impossible) chance that Postgres will be setup on the cluster to just host the toolserver wiki
Fantastic, this gives us a chance to test the very reassuring and complete documentation available: https://www.mediawiki.org/wiki/Manual:PostgreSQL#From_PostgreSQL_to_MySQL
(In reply to comment #5) > [...] > There's very little (read, almost impossible) chance that Postgres will be > setup on the cluster to just host the toolserver wiki Eh, nobody has suggested that PostgreSQL should be used as a backend for this wiki. (In reply to comment #6) > Fantastic, this gives us a chance to test the very reassuring and complete > documentation available: > https://www.mediawiki.org/wiki/Manual:PostgreSQL#From_PostgreSQL_to_MySQL There's also [[mw:Manual:Backing up a wiki#XML dump]] that has seen a *lot* more use. As we are primarily interested in preserving the wiki's content, a table-for-table conversion of the database is not needed.
(In reply to comment #7) > (In reply to comment #6) > > Fantastic, this gives us a chance to test the very reassuring and complete > > documentation available: > > https://www.mediawiki.org/wiki/Manual:PostgreSQL#From_PostgreSQL_to_MySQL > > There's also [[mw:Manual:Backing up a wiki#XML dump]] that has seen a *lot* > more use. I've saved XML dumps for about 50k wikis ;) but the import part is another story: way trickier and not very well documented. Last time it was tried on a WMF wiki (wikitech -> labsconsole merge) the results have not been outstanding, in particular all logs got lost. Main documentation is at [[m:Data dumps/Tools for importing]]. > As we are primarily interested in preserving the wiki's content, a > table-for-table conversion of the database is not needed. I was mostly kidding.
Just mentioning that there are other needs for postgresql than just this, see #48896. Do you reckon I am rather wondering why the wiki should stay writeable, with wikitech being the new wiki for tools. Couldn't we just keep a read-only toolserver wiki as an archive (which is really good to have) and keep it in postgresql for simplicity? (If not, why?)
(In reply to comment #9) > Just mentioning that there are other needs for postgresql than just this, see > #48896. Do you reckon Yes, but a) we don't need PostgreSQL for a wiki and b) that would probably be a WONTFIX at WMF :-). wiki.toolserver.org is very plain and no frills, and there are no problems migrating it to MySQL (or any other backend). > I am rather wondering why the wiki should stay writeable, with wikitech being > the new wiki for tools. Couldn't we just keep a read-only toolserver wiki as > an > archive (which is really good to have) and keep it in postgresql for > simplicity? > (If not, why?) At some point in the future it should certainly be turned read-only. But if we migrate wiki.toolserver.org for example next month, people might want to add information where specific tools went to, update links to their user pages, etc. I think it's easier if we keep it writeable for users with a justification, rather than locking it totally up and having to handle every change by an admin.
Sure, if we migrate the wiki this early it should be writeable, I agree. Or else, we do it very late and set it to read-only the moment it moves. This brings me to the question *who* is going to move the wiki. That person(s) should do so when they have the time and thus should decide if it stays readable e.g. until toolserver's shutdown.
(In reply to silke.meyer from comment #11) > [...] > This brings me to the question *who* is going to move the wiki. That > person(s) should do so when they have the time and thus should decide if it > stays readable e.g. until toolserver's shutdown. The move of the content (bug #60415) is a two part process: nosy (or amette) switches the WMDE Toolserver wiki to read-only and creates an XML dump. Someone with import rights on the WMF Toolserver wiki then imports this and tests that it worked. Unfortunately, it's not up to them to determine the time to move: First, the WMF Toolserver wiki needs to have been set up.
Don't forget the logs, files, user rights etc.
(In reply to Nemo from comment #13) > Don't forget the logs, files, user rights etc. I don't intend to preserve logs or user rights. I assume that WMDE has plans in place to retain data for potential criminal or civil proceedings, but the scope of this bug is to keep the wiki's content in a (re-)usable form.
(In reply to Tim Landscheidt from comment #14) > (In reply to Nemo from comment #13) > > Don't forget the logs, files, user rights etc. > > I don't intend to preserve logs or user rights. I assume that WMDE has > plans in place to retain data for potential criminal or civil proceedings, > but the scope of this bug is to keep the wiki's content in a (re-)usable > form. You?
(In reply to Nemo from comment #15) > > > Don't forget the logs, files, user rights etc. > > I don't intend to preserve logs or user rights. I assume that WMDE has > > plans in place to retain data for potential criminal or civil proceedings, > > but the scope of this bug is to keep the wiki's content in a (re-)usable > > form. > You? I am quite certain that "I" is the correct translation of "ich".
Coming back to this open question... Where is the toolserver wiki supposed to live on? We'd need a mediawiki hosted somewhere that is non-SUL and not part of production. According to Coren (bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20140219.txt) the best way would be: Someone is willing to turn the toolserver wiki into a "tool" in Tool Labs and maintain it there in the long run. Do we have a volunteer for this?
Ok, I see we don't have a volunteer for this. So - same as for svn - a backup copy / xml dump will be kept in Labs. Whoever wants to resurrect the toolserver wiki as a "tool" in Tool Labs is welcome to do so. Silke will poke Nosy / Marlen to create the backup. Before, we'll announce a tswiki "freeze".
The point of moving wiki.toolserver.org to WMF *is* to have it be part of production so that (security) updates are done once and neither waste multiple people's time nor leave the wiki open for attack because no volunteer will have much enthusiasm about that wiki in a year. That Coren thinks the best way would be to turn it into a tool is understandable given that he is the Tools administrator. But WMF not only pays people to manage wikis, they also do it *every* *fucking* *day* so they're quite proficient at it, and they can even make use of the patches I (as a volunteer) submitted three months ago.
Putting it on production as a "standalone" wiki (like the private wikis etc) would be fairly easy. The "most difficult" part of it at would presumably be postgres -> mysql. Of course, depending on what we want to actually transfer. Special:Export might suffice? Shouldn't be much work to do... (ie I can do it this weekend)
Yay, cool, Reedy! I can poke Nosy to help you.
(In reply to Silke Meyer (WMDE) from comment #21) > Yay, cool, Reedy! I can poke Nosy to help you. I've got root ;)
(In reply to Tim Landscheidt from comment #19) > The point of moving wiki.toolserver.org to WMF *is* to have it be part of > production so that (security) updates are done once and neither waste > multiple people's time nor leave the wiki open for attack because no > volunteer will have much enthusiasm about that wiki in a year. > > That Coren thinks the best way would be to turn it into a tool is > understandable given that he is the Tools administrator. > > But WMF not only pays people to manage wikis, they also do it *every* > *fucking* *day* so they're quite proficient at it, and they can even make > use of the patches I (as a volunteer) submitted three months ago. The security argument is pretty amusing considering it's on 1.18 now. Doesn't it make more sense to be on labs rather than the production cluster? What benefits are there for it to be a "production" wiki?
(In reply to Sam Reed (reedy) from comment #20) > [...] > The "most difficult" part of it at would presumably be postgres -> mysql. Of > course, depending on what we want to actually transfer. Special:Export might > suffice? > [...] Yes, that's what I suggested for bug #60415. Nemo_bis wants to keep some logs in bug #61539. (In reply to Sam Reed (reedy) from comment #23) > (In reply to Tim Landscheidt from comment #19) > > The point of moving wiki.toolserver.org to WMF *is* to have it be part of > > production so that (security) updates are done once and neither waste > > multiple people's time nor leave the wiki open for attack because no > > volunteer will have much enthusiasm about that wiki in a year. > > [...] > The security argument is pretty amusing considering it's on 1.18 now. > Doesn't it make more sense to be on labs rather than the production cluster? > What benefits are there for it to be a "production" wiki? That's the reason I want the wiki to be set up in production. At Toolserver, one extremely lauded volunteer and two paid admins didn't update it, and I'm very certain that the sustained interest in Labs will be the same level. If on the other hand wiki.toolserver.org can be managed as one of 800+ (?) wikis, the extra time needed will be negligible in the long run.
Is here anything I can help with?
Reedy, I would like to communicate a "freeze" date for the wiki before it can't be edited any more. When will you be bale to migrate it and so when could that deadline be? End of May? Later?
Ping Reedy!
Summary of office hour: * We haven't heard from Reedy. * We questioned the importance of keeping the wiki again if no one has the resources to move it somewhere. * If no one does, we'll keep a backup dump for later ressurrection. https://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20140611.txt ([17:41:50])
(In reply to Silke Meyer (WMDE) from comment #28) > * If no one does, we'll keep a backup dump for later ressurrection. Full SQL dump?
Ahem, not SQL. * Full database dump + images?
(In reply to Silke Meyer (WMDE) from comment #28) > * We questioned the importance of keeping the wiki again if no one has the > resources to move it somewhere. > * If no one does, we'll keep a backup dump for later ressurrection. Someone should just write and run a bot to transwiki (export and import, maybe) all the content from wiki.toolserver.org to Meta-Wiki. Then we can go through it on Meta-Wiki at a leisurely pace. If we wait for the wiki to only be available as an SQL dump, it will be significantly more annoying and painful to retrieve the contents and its history. I might be able to help write such a bot if it's not trivial. I assume some of the larger MediaWiki bot code libraries such as pywikibot have built-in functions to do imports and exports, but even doing it with requests to the MediaWiki API directly shouldn't be too difficult. As I recall, a potential blocker here might be that user auth seemed to have broken for the wiki at some point for scripts. I ran a script under my main account for years, but at some point it developed an issue where it always had trouble logging in, so it would just update the wiki page as an IP address (cf. <https://wiki.toolserver.org/history/Wiki_server_assignments>). I assumed it was something to do with the shared auth system in place there.
(In reply to MZMcBride from comment #31) > I might be able to help write such a bot if it's not trivial. As I already said multiple times, it is: dumpgenenerator.py --api=http://wiki.toolserver.org/w/api.php --xml and then Special:Import on the target wiki, optionally after fancy templates/titles/category/user rewrites. That's without logs and images of course.
I too would recommend against setting up a new wiki. A wiki named "Toolserver" certainly wouldn't make sense as that as of late is no longer going to be operational. Perhaps set up its content as a public read-only archive (like nostalgia). But I think we don't even need that. The little relevant/valuable pages it has can be imported to Meta-Wiki, Wikitech or in user's namespace elsewhere. Most of the pages will only describe obsolete infrastructure.
Eh, the intention was always to set it up as a public, (almost) read-only archive. Of course it would describe obsolete infrastructure, just as https://wikimania2005.wikimedia.org/ is only "useful" for time travellers. But some people find history really interesting, and some societies even build museums at high costs. Or revision histories to see years later how a Wikipedia article evolved over time.
I have a dumpBackup.php --full copied off the server if you need it.
Moving to Site requests as this is a request for a WMF-hosted wiki.
Cool, thanks for the backup, Nosy!
