Last modified: 2011-09-06 15:11:25 UTC
Say I have Template:Custom with content: <mycustomtag>Param1 = {{{1}}}</mycustomtag> I call it on another page using {{Custom|parameter}} This results in the parser giving the hook's function "Param1 = {{{1}}}" as input. Since extension output isn't parsed further at all, and the extension itself has no way of recovering it (the recursion stack is inaccessible), the parameter is lost. Possible solutions: * Substitute parameters in parser hook input * Give extensions access to the stack containing the parameters I imagine the second would be a huge mess, so I would greatly prefer the first alternative.
Created attachment 576 [details] Fix for REL1_4 Before starting recursive parsing of templates, stores the parameters of that template. Uses these parameters to parse variables in extension input. Does not change non-extension functionality at all, so should be safe.
Created attachment 577 [details] Fix for HEAD As above.
See also bug 684.
Created attachment 753 [details] Adds methods to Parser for parsing of text Takes another approach on this. Instead of having the parser automactically parse template arguments, have the extension call a function for the purpose of expanding templates and parameters. Also includes a stub for doing a full parse. (Currently only calls htmlentities() on the input, since any attempt to parse was disasterous in my testing.) (Maybe this should be moved to a new bug?)
It is my opinion that extensions should be substitution-agnostic. If someone wants to put {{PAGENAME}} into an extension, that should work without the extension author having to predict it and add variable parsing to his extension. Furthermore, it's much easier to handle it once in Parser.php than in all kinds of different extensions.
*** Bug 5435 has been marked as a duplicate of this bug. ***
*** Bug 7591 has been marked as a duplicate of this bug. ***
*** Bug 4529 has been marked as a duplicate of this bug. ***
Created attachment 2536 [details] Parser.php method for extensions to interpolate template vars (1.9-svn) I've written a patch for this against 1.9-svn (rev. 17200). It defines a method called replaceTemplateVariables that extension authors can use. Sample usage: function myExtension_Render($source, $argv, &$parser) { $source = $parser->replaceTemplateVariables($source); ... } Making this function available but not mandating it I think preserves the greatest flexibility and backward-compatibility. This functionality is something that users of my extension have been asking for, and I think a lot of people will appreciate it. Thanks for reviewing it!
*** Bug 7673 has been marked as a duplicate of this bug. ***
I had some problems with patch 2536 when using <includeonly> and </includeonly>, getting the error: <b>Warning</b>: array_key_exists() [<a href='function.array-key-exists'>function.array-key-exists</a>]: The second argument should be either an array or an object in <b>/home/www/public_html/w/includes/Parser.php</b> on line <b>3277</b> The extension is using the $parser->replaceTemplateVariables() rather generically, so I suspect it's an issue in Parser.php. The error appears when viewing/editing the Template, and the Template essentially reduces to something likeso: {{{1}}} <includeonly> <extension> argument={{{1}}} </extension> </includeonly> Calling the template from another page works just fine, mind you.
Created attachment 2550 [details] Attempt to work with <includeonly> (1.9-svn) I could not reproduce the <includeonly> incompatibility, but given the symptoms you describe I think this will fix it.
I would prefer to entirely separate the preprocessing stage from the HTML generation stage. Then we should introduce a flags parameter to Parser::setHook(), giving three possible preprocessing styles: * PP_TRANSPARENT: Expand templates and template arguments inside the extension tag, as if it wasn't there. This is the desired behaviour for many extensions. * PP_OPAQUE: Treat text inside extension tags as opaque. This is the current behaviour. * PP_EXPAND: Expand the tag during preprocessing, via the callback. This allows extensions which are syntactically equivalent to tags, but functionally equivalent to parser functions. One obvious application for this would be a <template> tag, which provides an XML-like interface to template expansion, thereby providing rigorous argument escaping for applications which need it. Recursive parsing in extensions is slow, complicated and prone to failure. The fact that we do strip() before replaceVariables() produces counterintuitive behaviour which is mostly hidden at the moment by the lack of parser interaction features such as this one. For example, <ref> tags exposed via template expansion will be ordered after <ref> tags coming directly from the article text, in the <references/> list. Counterintuitive behaviour such as this (no doubt there are many other similar problems) can be avoided by doing extension expansion after a complete extension-tag-aware preprocessing phase.
(In reply to comment #12) > Created an attachment (id=2550) [edit] > Attempt to work with <includeonly> (1.9-svn) > > I could not reproduce the <includeonly> incompatibility, but given the symptoms > you describe I think this will fix it. Yes, this is essentially the same solution I came up with, but thank you for submitting it. Incidentally, the problem was broader than <includeonly>, applying to any variable reference (i.e. {{{1}}}) being parsed; it was simply coincidental that it first occurred with me while using <includeonly>.
In my opinion, template parameters should always be substituted before calling the hook, like the way it works for parser functions.
*** Bug 8600 has been marked as a duplicate of this bug. ***
*** Bug 8828 has been marked as a duplicate of this bug. ***
See Bug 8828 to get a working patch
*** Bug 9005 has been marked as a duplicate of this bug. ***
*** Bug 9645 has been marked as a duplicate of this bug. ***
Created attachment 3574 [details] Patch for REL1_9 The major issue, as stated above, is the strip() call before replaceVariables() in brace_substitution(). This prevents template arguments from being added to the stack which are therefore unavailable to an extension tags. As a relatively easy fix, strip() could be called from replaceVariables() as determined by the parser output type. This way the template arguments could be added to the stack and accessed by any extension tags. Tested against ParserTests.php for 1.9.3, against same items as head parser. The same patch should work for 1.10, but I haven't tested it.
Can we have patches against current trunk, HEAD revision? Non-critical fixes and new features don't go into stable branches.
Created attachment 3602 [details] Patch for parser.php Same as previous "Patch for REL1_9" but against the trunk (r21994). Produces identical ParserTests.php output as current revision. Makes template variables available to parser tags via the $parser->mArgStack attribute.
I strongly suspect [http://bugzilla.wikimedia.org/show_bug.cgi?id=8835|bug 8835] will be fixed at the same time this bug (2257) is. They seem related.
*** Bug 9930 has been marked as a duplicate of this bug. ***
Templates not working in MediaWiki 1.9.3 when used {{TemplateName|argument}}, there is {{{1}}} showing in page instead of argument. Using of {{TemplateName|1=argument}} works fine.
(In reply to comment #26) > Templates not working in MediaWiki 1.9.3 when used {{TemplateName|argument}}, > there is {{{1}}} showing in page instead of argument. > > Using of {{TemplateName|1=argument}} works fine. > Both REL 1_9 patch and the head patch passed the "Template unnamed parameter" parser test. I can't repoduce the bug. Do you see the bug in the current trunk, head revision?
To avoid clutter, please open new bug reports to discuss different problems. Thank you.
*** Bug 10712 has been marked as a duplicate of this bug. ***
*** Bug 10766 has been marked as a duplicate of this bug. ***
It has been about 2 years since this bug was submited, when is a fix going to to be put on Wikipedia?
Bug is still present in MediaWiki 1.11-svn, so I changed version from 1.9-svn to 1.11-svn.
Hi, I finally found some time for test with trunk version. I downloaded version 24597 from SVN and I had no problem to simulate the problem. At http://wikibug.appoge.net/ I created wiki page from trunk and also created two templates: 1. template "body" with code: <div style="font-weight: bold;"> {{{1}}} </div> 2. template "capital" with code: <span style="color: green;">{{{1}}}</span> Result of testing of these two templates is at http://wikibug.appoge.net/index.php/Template_bug_test btw: Why there are so old snapshots at http://download.wikimedia.org/mediawiki/snapshot/ ?
*** Bug 10860 has been marked as a duplicate of this bug. ***
*** Bug 10961 has been marked as a duplicate of this bug. ***
Does the patch in Comment #23 indeed resolve this issue? Could we double-check it and incorporate if possible? Bug 8693 might be helped by this fix, as well.
Re comment #36: > Does the patch in Comment #23 indeed resolve this issue? The patch in comment #23 solved the issue for me on MediaWiki 1.9.3, but with a few minor updates (added one line to initialize $this->mArgStack, and added a few lines to replace template args in the parameters & content for custom parser tags). Whether these updates are truly needed, or can be done better, I did not investigate - it was literally a 3-minute hack-job modification of Sean J's patch to solve a specific problem that was annoying me (i.e. that I wanted a custom parser tag to never have to care about templates, and to only ever see the expanded content, never the raw content). If it you're keen, the diff is here, but it's probably inefficient crap: http://files.nickj.org/MediaWiki/possibly-yucky-2257-parser-patch.txt
What does the patch do? From my testing (on 1.11) a parser function can already go through $wgParser->mArgStack to get arguments. I'd think removing a call to strip before internalParse would just break stuff. I don't quite see what we want to do that's not there already, or how this patch would do that; maybe things changed significantly since it was written.
(In reply to comment #38) > What does the patch do? From my testing (on 1.11) a parser function can > already go through $wgParser->mArgStack to get arguments. I'd think removing a > call to strip before internalParse would just break stuff. I don't quite see > what we want to do that's not there already, or how this patch would do that; > maybe things changed significantly since it was written. > OK, I think I understand now. So you can't currently access args from XML-style tags, since they're parsed before the call stack is set up. This patch tries to fix that by reordering the parse in such a way that the stack will be set up, but not breaking any core functionality; although it looks like extensions would get hosed by this (per my earlier comment).
So the patch is a no go then? Sounds like we need to make ALL the data available first, from all sources... then parse it all in a fully seperate, second step, no?
(In reply to comment #40) > So the patch is a no go then? > Sounds like we need to make ALL the data available first, from all sources... > then parse it all in a fully seperate, second step, no? > It looks to me like it has the potential for too many side affects. This could be as simple as pushing the current args on to the stack where the tag could get them, then setting a flag to indicate that was done early, so the normal processing can see that and clear the flag instead of pushing again. That should work without reordering the parsing, and should allow adding sanity checks, to keep things safe.
*** Bug 11684 has been marked as a duplicate of this bug. ***
Is it just me, or has this issue been resolved on Wikipedia:en? See, for example, http://en.wikipedia.org/wiki/Template:Citation/core. In fact, this kind of template use (templates within XML-style parser tags) is fairly frequent on the English-language Wikipedia site. Does anyone know why this works, and whether there is a reliable solution/patch available for REL1_11?
(In reply to comment #43) > Is it just me, or has this issue been resolved on Wikipedia:en? See, for > example, http://en.wikipedia.org/wiki/Template:Citation/core. In fact, this > kind of template use (templates within XML-style parser tags) is fairly > frequent on the English-language Wikipedia site. Does anyone know why this > works, and whether there is a reliable solution/patch available for REL1_11? > Which tag on http://en.wikipedia.org/wiki/Template:Citation/core do you refer to?
(In reply to comment #44) > (In reply to comment #43) > > Is it just me, or has this issue been resolved on Wikipedia:en? See, for > > example, http://en.wikipedia.org/wiki/Template:Citation/core. In fact, this > > kind of template use (templates within XML-style parser tags) is fairly > > frequent on the English-language Wikipedia site. Does anyone know why this > > works, and whether there is a reliable solution/patch available for REL1_11? > > > > Which tag on http://en.wikipedia.org/wiki/Template:Citation/core do you refer > to? > Hi and thanks for your response. I was referring to the <cite> tag -- The relevant section of Template:Citation/core is as follows: <cite {{ #if:{{{Ref|}}} |{{#ifeq:{{{Ref|}}}|none||id="{{{Ref|}}}"}} |id="CITEREF{{#if:{{{Surname1|}}} |{{{Surname1}}}{{{Surname2|}}}{{{Surname3|}}}{{{Surname4|}}} |{{{EditorSurname1|}}}{{{EditorSurname2|}}}{{{EditorSurname3|}}}{{{EditorSurname4|}}} }}{{{Year|{{{Date|}}}}}}" }}>{{ <!--============ .... {omitted for sake of readability} ... ============--> }}</cite> If I put this same code into a Special:ExpandTemplate page (REL1_11, Cite and ParserFunctions installed) it is rendered like: [1,1,1,1,1,1,1,... ] This occurs both with and without the html comment, because the templates are not expanded. Pasting the same into the Special:ExpandTemplates page on Wikipedia:en will, instead, give a valid <cite ...> ... </cite> tag. A little surprising is the fact that the following <span> tag code also within Template:Citation/core does appear to be correctly parsed on both versions: <span <!-- This is a COinS tag (http://ocoins.info), which allows automated tools to parse the citation information: --> class="Z3988" title="ctx_ver=Z39.88-2004&rft_val_fmt={{urlencode:info:ofi/fmt:kev:mtx:}}book{{ #if: {{{Journal|}}} |&rft.genre=article&rft.atitle={{urlencode:{{{Title|}}}}}&rft.title={{urlencode:{{{Periodical|}}}}} |{{ #if: {{{IncludedWorkTitle|}}} |&rft.genre=bookitem&rft.atitle={{urlencode:{{{IncludedWorkTitle|}}}}} |&rft.genre=book }}&rft.title={{urlencode:{{{Title|}}}}} }}{{ <!--============ .... {omitted for sake of readability} ... ============--> }}> FYI I tested REL1_11 behavior on http://www.wikidoc.org; see http://www.wikidoc.org/index.php/Special:Version for specifics. Hope this helps ... Jacki
Actually, <span> behavior is not surprising since it's not a parser tag, but it seems that you're correct about cite, although I can't find a relevant change in SVN. Can developers comment on this?
Actually, it seems that <cite> is not a parser tag on Wikipedia:en - see http://en.wikipedia.org/wiki/Special:Version
<cite> is a HTML tag, nothing more. http://www.w3.org/TR/html401/struct/text.html#edef-CITE
(In reply to comment #48) > <cite> is a HTML tag, nothing more. > http://www.w3.org/TR/html401/struct/text.html#edef-CITE <cite> was defined as parser tag on http://www.wikidoc.org/index.php/Special:Version along with </nocite> when I checked, so I got confused, it's not anymore.
Thanks, I was unaware of this and the information is tremendously helpful. We had installed the Biblio.php extension which has a <cite> XML-style parser tag, over-riding its HTML version. Apologies for the distraction ...
Is the bug 12016 related?
(In reply to comment #13) > I would prefer to entirely separate the preprocessing stage from the HTML > generation stage. Then we should introduce a flags parameter to > Parser::setHook(), giving three possible preprocessing styles: > > * PP_TRANSPARENT: Expand templates and template arguments inside the extension > tag, as if it wasn't there. This is the desired behaviour for many extensions. > * PP_OPAQUE: Treat text inside extension tags as opaque. This is the current > behaviour. > * PP_EXPAND: Expand the tag during preprocessing, via the callback. This allows > extensions which are syntactically equivalent to tags, but functionally > equivalent to parser functions. One obvious application for this would be a > <template> tag, which provides an XML-like interface to template expansion, > thereby providing rigorous argument escaping for applications which need it. To return to this idea a bit, I looked at Parser.php in trunk. setHook behavior hasn't changed as far as I could see, so this is still PP_OPAQUE. There's a new (1.10) function setTransparentTagHook with no documentation and commit comment "New Parser::setTransparentTagHook for parser extension and template compatibility". From what I see, it adds the custom tag to the "legal HTML tags" list, meaning they survive through parsing and get their contents parsed, and then does callback with the parsed attributes and contents. This seems close to PP_TRANSPARENT, but parses _all_ wikitext instead of just curly-brace constructs. Since we already have recursiveTagParse, this might not be optimal. I didn't find anything resembling PP_EXPAND.
*** Bug 12236 has been marked as a duplicate of this bug. ***
I found the following simple diff against revision 28900 of Parser.php was sufficient for my needs: --- includes/Parser.php.orig Fri Dec 28 13:31:20 2007 +++ includes/Parser.php Sat Dec 29 15:30:28 2007 @@ -5376,6 +5376,8 @@ } else { $s = $root->textContent; } + } elseif ( $root->nodeName == 'attr' ) { + $s = $this->parser->replaceVariables($root->nodeValue, $this, 1); } elseif ( $root->nodeName == 'ext' ) { # Extension tag $xpath = new DOMXPath( $root->ownerDocument );
Nathanael's diff solved my problem as well. I believe PP_TRANSPARENT for attributes would be a sane default, it's unlikely to break existing extensions (as attrubutes are usually just simple strings), it's easy do implement, document and remember, and it doesn't rule out the possibility of adding the preprocessing style flags later.
Created attachment 4519 [details] Proposed solution: Extension TagParser In the German Wikipedia we have hacked a solution to this bug. It makes every XML tag available through a parser function {{#tag}}. E.g. {{#tag:math|e = m c^2}} will become <math>e = m c^2</math> and {{#tag:math| {{{1}}} }} will evaluate {{{1}}} as a tex-expression. Any amount of attributes is available: {{#tag:source|lang="bash"|echo "test"}}. The last parameter is the content, all others are attributes. You may find further explanation on: - http://www.mediawiki.org/wiki/Extension:TagParser
The TagParser extension has inspired the addition of an approximately identical feature to core, in r29482. This fixes the bug for most practical purposes, i.e., complicated templates. A clean fix is still desirable, so the bug should probably be left open.
(In reply to comment #57) > The TagParser extension has inspired the addition of an approximately identical > feature to core, in r29482. This fixes the bug for most practical purposes, > i.e., complicated templates. A clean fix is still desirable, so the bug should > probably be left open. > So you're saying that parser tags that are in want of nested templates should use {{#tag:}} now? Agreed that a clean solution is available. Is it possible to let extension tags parse first perhaps, so returned wikimarkup can be parsed by MediaWiki?
I'm not clear what you're asking. Currently, extension tags like <ref> *are* parsed before curly-brace syntax. This means that when you do <myextension>{{{1}}}</myextension>, your extension receives the text "{{{1}}}", not the text of the first parameter, because template parameters and other curly-brace syntax have not been parsed yet. What needs to happen ideally is that extension tags and curly brace constructs should be parsed at the same time, not one after the other. r29482 allows this to happen, if I understand correctly, since you can use {{#tag:myextension|{{{1}}} }}, which behaves as expected.
Oh, whoops, I wasn't thinking properly. Yes, parsing templates/curly braces before or at the same times as extension tags would be ideal; I see that now. At least it's possible to do stuff like put templates in things like <imagemap> now, yes? It's just that nesting parser functions get confusing and tedious to deal with...
What's about <gallery width={{{1}}}>...</gallery>? Was this fixed?
(In reply to comment #61) > What's about <gallery width={{{1}}}>...</gallery>? Was this fixed? > No, but {{#tag:gallery|width={{{1}}}|...}} works.
That {{#tag:}} thing looks pretty dreadful, and I'd say we definitely don't want it. Just let tags accept parameters by default (or as option).
Can we nevertheless have it for the next year or two, until a solution with a nicer look has been developed?
In version 1.0Beta have new tag {{#ask: }} But how to use and where described syntax I don't know.
When is {{#tag:}} going to work in Wikipedia and other Wikimedia projects? -Timothy
As soon as the new Preprocessor is activated, which should be RSN. (You can already test it by adding &timtest=newpp to an URL, e.g. http://de.wikipedia.org/wiki/Benutzer:Tbleher/Test?timtest=newpp )
The new parser is active on all Wikimedia wikis (AFAIK) and MW 1.12 also contains the #tag:-feature, so I think we can consider this bug fixed.
Reopening, the bug is not fixed as stated and a real fix is still desirable. See previous comments: (comment #57, me) > The TagParser extension has inspired the addition of an approximately identical > feature to core, in r29482. This fixes the bug for most practical purposes, > i.e., complicated templates. A clean fix is still desirable, so the bug should > probably be left open. (comment #63, Brion) > That {{#tag:}} thing looks pretty dreadful, and I'd say we definitely don't > want it. Just let tags accept parameters by default (or as option).
*** Bug 14389 has been marked as a duplicate of this bug. ***
Hi, I am trying to do this (in a template) : <inputbox> type=create default={{{1}}} break=no buttonlabel=Create </inputbox> But the default value is {{{1}}} and not the value of my first parameter... Instead, is this supposed to work : {{#tag:inputbox|default={{{1}}}|type=create|buttonlabel=Create3}} The result is : "You have not specified the type of input box to create." If y try this : {{#tag:inputbox|type=create|default={{{1}}}|buttonlabel=Create3}} it seems that parameters others than "type" are ignored. Is there a solution ? Thanks
(In reply to comment #71) This should work: {{#tag:inputbox| type=create default={{{1}}} break=no buttonlabel=Create }}
(In reply to comment #72) > (In reply to comment #71) > > This should work: > > {{#tag:inputbox| > type=create > default={{{1}}} > break=no > buttonlabel=Create > }} > Yes ! Thanks a lot :)
*** Bug 15307 has been marked as a duplicate of this bug. ***
*** Bug 16010 has been marked as a duplicate of this bug. ***
What can I do to get ImageMap tags on version 1.13.3 to work? I need them to accept brackets....
*** Bug 16825 has been marked as a duplicate of this bug. ***
Use case: {{#tag:nowiki|{{{url}}}}} This tag is the only way I've found to receive a template parameter and do a <nowiki> on its expanded value. Such as preventing auto-linking of a URL parameter. Please ensure this ability doesn't go away when this solution is improved.
*** Bug 18417 has been marked as a duplicate of this bug. ***
Created attachment 6056 [details] Allow PPFrames to be used to truly make template variables available With the introduction of Preprocessor frames to the parsing process, a completely different approach to fixing this issue becomes available -- one which truly fixes the problem, instead of simply bypassing the problem through parser hooks (which do not provide identical functionality to extension tags). The attached patch works by: 1) Passing the current $frame to the extension tag-calling hook ($frame is also passed to the getVariableValue hook, too, in case extension writers have a need for $frame. there. too) $frame is added as an extra parameter, and therefore this change is transparent to existing extensions (which simply won't realize that there's an extra parameter available). 2) Passing the $frame back to the parser through recursiveTagParse Again, $frame is an extra, optional parameter, so there is no effect of this patch on any existing uses of recursiveTagParse; it simply makes additional functionality available to anyone who wishes to use it. 3) Assorted other code tweaks to pass the $frame variable as an optional argument to all necessary parser functions. In terms of usage, it means that all existing tag extensions will continue to work they way they always have. However, tag extensions that wish to expand template variables now have the option to do so, simply by 1) changing the argument list of the extension's implementation function, e.g. change: function efSampleRender( $input, $args, $parser ) to: function efSampleRender( $input, $args, $parser, $frame ) 2) add $frame to any recursiveTagParse calls, e.g., change: $output = $parser->recursiveTagParse( $text ); to: $output = $parser->recursiveTagParse( $text, $frame );
Following up on my previous post, I've run parserTests on the submitted patch (id=6056) and confirmed that the patch passes the parser tests (or at least, it does as well as the current HEAD (r49912) -- HEAD currently fails 14 tests; after adding this patch, the code still fails those same 14 tests).
*** Bug 591 has been marked as a duplicate of this bug. ***
*** Bug 14792 has been marked as a duplicate of this bug. ***
Patched in r55682 (see comment #80 for example of how to modify an extension).
(In reply to comment #0) > Possible solutions: > * Substitute parameters in parser hook input > * Give extensions access to the stack containing the parameters > > I imagine the second would be a huge mess, so I would greatly prefer the first alternative. Well, here we are, four years later. Turns out I was dead wrong. :) Much thanks for finally fixing this one.
*** Bug 23572 has been marked as a duplicate of this bug. ***
Sorry, when will Wikimedia Commons be updated to the fixed code?
*** Bug 24774 has been marked as a duplicate of this bug. ***
*** Bug 30782 has been marked as a duplicate of this bug. ***
