Last modified: 2014-04-04 20:05:48 UTC

Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 57876 - Provide commonswiki, wikidatawiki and centralauth on all clusters and document their usage
Provide commonswiki, wikidatawiki and centralauth on all clusters and documen...
Status: RESOLVED FIXED
Product: Wikimedia Labs
Classification: Unclassified
tools (Other open bugs)
unspecified
All All
: High blocker
: ---
Assigned To: Marc A. Pelletier
:
: 52559 58802 (view as bug list)
Depends on: 52559
Blocks: tool-missing-ts-feat
  Show dependency treegraph
 
Reported: 2013-12-02 19:53 UTC by Tim Landscheidt
Modified: 2014-04-04 20:05 UTC (History)
14 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Tim Landscheidt 2013-12-02 19:53:25 UTC
Currently, commonswiki_p.page is accessible and JOINable as commonswiki_f_p.page from within - for example - enwiki_p and dewiki_p.  This should be extended to:

1. all target databases (for example, zhwiki_p has no link commonswiki_f_p),
2. all source databases (not just Commons), and
3. all source tables in each of the source databases.

Once this is set up and automated, its usage and limitations (i. e. JOIN only on indexes of the source table, etc.) need to be documented.
Comment 1 Marc A. Pelletier 2013-12-02 20:37:22 UTC
There is no plan to support federation of all databases; only commons and wikidata are planned.

Is there a use case for other combinations that are not simply hypothetical and not better served by application logic?
Comment 2 Tim Landscheidt 2013-12-16 07:49:35 UTC
(In reply to comment #1)
> There is no plan to support federation of all databases; only commons and
> wikidata are planned.

> Is there a use case for other combinations that are not simply hypothetical
> and
> not better served by application logic?

Answering the first question: I'm not aware of any; Commons and Wikidata was what was available at the Toolserver and in use there.
Comment 3 Tim Landscheidt 2013-12-22 11:02:46 UTC
*** Bug 58802 has been marked as a duplicate of this bug. ***
Comment 4 merl 2014-01-27 13:24:53 UTC
Joining of a foreign databases with a wiki database is highly needed if this data is also used by mediawiki installation.

These databases are currently commonswiki, wikidatawiki and centralauth. The last one was only a requested feature on toolserver but never realized. But it would be useful to have it on labs.

commonswiki: needed because of included images

wikidatawiki: needed because of languagelinks and included properties

centralauth: getting all rights of a user on a local wiki (combining of local and global user groups)
Comment 5 Silke Meyer (WMDE) 2014-02-04 16:15:11 UTC
*** Bug 52559 has been marked as a duplicate of this bug. ***
Comment 6 Silke Meyer (WMDE) 2014-02-04 16:18:41 UTC
Closed bug #52559 as a duplicate of this one, gave higher priority.

Talked to Tim: I'll rename this bug to reflect the 3 databases commonswiki, wikidatawiki and centralauth.
Comment 7 Silke Meyer (WMDE) 2014-02-19 18:36:57 UTC
According to Marc-André Pelletier, this will take maximum two weeks after the migration of Labs from pmtpa to eqiad is done. (Not because it lasts that long, but because of the general work load.)
Comment 8 Nemo 2014-02-24 10:37:30 UTC
(In reply to Tim Landscheidt from comment #0)
> Currently, commonswiki_p.page is accessible and JOINable as
> commonswiki_f_p.page from within - for example - enwiki_p and dewiki_p.

Is this documented anywhere? I can't find any info whatsoever on federated tables, joins of commonswiki/centralauth, commonswiki_f_p or any other keyword I could think of.
Comment 9 Tim Landscheidt 2014-02-24 14:59:23 UTC
(In reply to Nemo from comment #8)
> (In reply to Tim Landscheidt from comment #0)
> > Currently, commonswiki_p.page is accessible and JOINable as
> > commonswiki_f_p.page from within - for example - enwiki_p and dewiki_p.

> Is this documented anywhere? I can't find any info whatsoever on federated
> tables, joins of commonswiki/centralauth, commonswiki_f_p or any other
> keyword I could think of.

The original summary of this bug was "Automate creation of federated tables and document their usage".  I've added it to the summary again.
Comment 10 Silke Meyer (WMDE) 2014-04-01 17:47:06 UTC
ETA according to IRC office hour: April 6th.
Comment 11 Gerrit Notification Bot 2014-04-04 19:51:43 UTC
Change 123916 had a related patch set uploaded by coren:
Labs: automate federated table maintenance

https://gerrit.wikimedia.org/r/123916
Comment 12 Gerrit Notification Bot 2014-04-04 19:52:59 UTC
Change 123916 merged by coren:
Labs: automate federated table maintenance

https://gerrit.wikimedia.org/r/123916
Comment 13 Marc A. Pelletier 2014-04-04 20:05:48 UTC
There are now a centralauth_f_p, wikidatawiki_f_p and commonswiki_f_p databases provided on every shard where the real corresponding database does not exist, and those databases now federate all the public tables by default.

Basic documentation (including the caveats) are at:

https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help#Joins_between_commons.2C_centralauth_and_wikidata_and_other_project_databases

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links