Last modified: 2014-11-17 10:36:07 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T11890, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 9890 - Reasonably efficient interwiki transclusion
Reasonably efficient interwiki transclusion
Status: NEW
Product: MediaWiki
Classification: Unclassified
Templates (Other open bugs)
unspecified
All All
: High enhancement with 20 votes (vote)
: ---
Assigned To: Nobody - You can work on this!
: crosswiki
Depends on: 28673
Blocks: 4547 12306 14724 Wikisource 10681 16137 16575
  Show dependency treegraph
 
Reported: 2007-05-13 01:22 UTC by Aryeh Gregor (not reading bugmail, please e-mail directly)
Modified: 2014-11-17 10:36 UTC (History)
29 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Aryeh Gregor (not reading bugmail, please e-mail directly) 2007-05-13 01:22:29 UTC
Currently $wgEnableScaryTranscluding seems to just pull templates from an HTTP 
request and cache them for some arbitrary period (probably, not having looked 
closely, indefinitely if the page isn't edited, since I bet nothing invalidates 
the parser cache).  It goes without saying that this is relatively slow and 
unreliable; therefore the feature is not enabled on Wikimedia sites.  A shared 
page database is necessary, and for timely invalidation of caches, the central 
repository must also know about the places that are sharing with it.
Comment 1 Jeff G. 2007-07-02 10:24:04 UTC
Do you mean to write that caching isn't down to an art form yet? :)
Comment 2 Daniel Friesen 2008-05-07 23:40:05 UTC
A shared page database would void the point of 'interwiki' transclusions, which basically mean "This data isn't mine, but I want to use it" so that's why it's defined as "Scary" transclusion and evilly cached.

A good solution for local wiki sharing content in the way you want would probably be an extension meant for transwiking content. That's actually something I'm working on, though I need a hook in the parser or preprocessor that will let me modify data grabbed from a template before it's parsed. ParserBeforeStrip is good for doing that to content of the current page, but there is no method for doing that with the start of transcluded text.
Comment 3 Zach Dennison 2009-04-26 03:26:32 UTC
Would it be possible to allow sending variables to the templates, like you can do with local templates? Also, I didn't know if the API would be a more efficient way to do this, assuming that the functionality hasn't already been modified to do it that way. Maybe there could be something in the api that allowed variable input for templates, then it returns the appropriate content. That way, when trying to transclude a template from Wikipedia that requires variables from a third-party wiki, even when using the correct syntax sending the variable information, you wouldn't just get the documentation page returned, telling you that you need variables to correctly use the template.
Comment 4 Roan Kattouw 2009-04-27 13:34:41 UTC
(In reply to comment #3)
> Would it be possible to allow sending variables to the templates, like you can
> do with local templates? Also, I didn't know if the API would be a more
> efficient way to do this, assuming that the functionality hasn't already been
> modified to do it that way. Maybe there could be something in the api that
> allowed variable input for templates, then it returns the appropriate content.
I think what you're looking for is:

api.php?action=parse&text={{templatename|param1|param2|etc.}}

Bug 14024 is related, and probably a dependency if the API is used this way.
Comment 5 Rich Farmbrough 2009-09-02 00:15:11 UTC
This isn't really an enhancement it's a fix to bug 1126 which was an enhancement.
Comment 6 Peter Potrowl 2010-03-03 13:48:56 UTC
Could somebody sum up the difficulties to solve this problem? If I understand well, the only need is to create a dynamic list of shared templates and store it in a database.

What are the underlying difficulties? What are the differences between including templates and including images? Does it require a big amount of code? As I really want this feature enabled on WMF, I would like to know if I can be of any help, but I need more information for this...
Comment 7 Aryeh Gregor (not reading bugmail, please e-mail directly) 2010-03-03 17:49:58 UTC
I don't think there are any overwhelming difficulties.  It should be pretty much the same as including images.  You'd need to coordinate templatelinks across wikis, but I assume that this is already done for imagelinks on Commons.  Someone does need to sit down and write the code, and get it reviewed -- it will probably be a substantial amount of code.
Comment 8 Peter Potrowl 2010-03-09 12:14:03 UTC
I have proposed this task as my project for Google Summer of Code 2010 (see http://www.mediawiki.org/wiki/User:Peter17#Project_1)
Comment 9 Peter Potrowl 2010-04-19 15:59:12 UTC
I have filled a student proposal on time, but nobody has reviewed it yet :-( and I got no mentoring proposal...

It is available here: http://socghop.appspot.com/gsoc/student_proposal/private/google/gsoc2010/peter17/t127055739388 for those who have a GSoC account.

I think this would be a good opportunity for this bug to be solved... If anyone is interested in mentoring me for this project, I would be very pleased!
Comment 10 Rich Farmbrough 2010-12-01 14:18:43 UTC
Peter has completed his work, a small amount of additional work is needed. It would be great if this could be accomplished as a reasonable priority. The project recently mentioned on Signpost would work well with this, as would:

* sharing Persondata
* sharing interwiki link lists on articles (saving tens of millions of edits by interwiki bots)
* sharing common templatology
Comment 11 Mark A. Hershberger 2011-03-13 16:59:43 UTC
I've talked to Peter and Roan about getting this merged.   They will have time in the next few weeks.
Comment 12 Mark A. Hershberger 2011-12-15 16:17:44 UTC
Don't know why I marked this "Low" since it is something that we (were?) planning to merge. re-raising to normal.
Comment 13 Jan Kucera (Kozuch) 2011-12-15 16:55:55 UTC
Setting high importance. Looks much better.
Comment 14 MZMcBride 2012-11-01 01:03:18 UTC
(In reply to comment #11)
> I've talked to Peter and Roan about getting this merged.   They will have time
> in the next few weeks.

What happened here?
Comment 15 Andre Klapper 2013-03-25 13:59:31 UTC
Mark: Do you remember? (comment 11)
Comment 16 Tyler Romeo 2013-04-23 05:36:34 UTC
I'm a little confused. This bug seems to imply that retrieving foreign wiki pages directly from the database is a *bad* thing. However, on http://www.mediawiki.org/wiki/User:Peter17/Reasonably_efficient_interwiki_transclusion it talks about fetching the wikitext from a shared database. Is it one or the other?

Second question: why the difference between {{wiki:test}} and {{raw:wiki:test}}? Is there some sort of parsing difference that I'm not aware of? Would there be a reason to prefer one over the other?

And finally, a third question based on the second question: if using {{raw:wiki:test}}, isn't all the parsing and template parameter whatnot done locally, thus eliminating the problems talked about in the aforementioned link (with the exception of the caching issue)?

I might pick up this issue depending on the answer to these questions.
Comment 17 Andre Klapper 2013-06-25 13:14:28 UTC
Aryeh / Peter / Rich: Can any of you answer the questions in comment 16?
Comment 18 Peter Potrowl 2013-06-25 15:42:11 UTC
1. IIRC, the code I worked on provided both ways to fetch the templates (API request and shared DB). A shared DB it the most efficient way.

2. I don't remember. Where have you seen {{raw:wiki:test}}?

3. I don't know.

Links to the code I developed:
* http://www.mediawiki.org/wiki/Special:Code/MediaWiki/author/peter17
* http://code.google.com/p/google-summer-of-code-2010-wikimedia/downloads/detail?name=Peter_Potrowl.tar.gz

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links