Last modified: 2014-07-20 21:59:49 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T37628, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 35628 - #switch or #ifeq: checks should be HTML escaped
#switch or #ifeq: checks should be HTML escaped
Status: RESOLVED FIXED
Product: MediaWiki
Classification: Unclassified
General/Unknown (Other open bugs)
1.19
PC Windows XP
: Low enhancement (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on: 35746
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-31 16:02 UTC by Bultro
Modified: 2014-07-20 21:59 UTC (History)
6 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Bultro 2012-03-31 16:02:40 UTC
When doing a #switch or #ifeq on a {{PAGENAME}} argument, and page title contains an apostrophe (for example, [[L'Aquila]]), it doesn't match correctly. For example:

{{#switch:{{PAGENAME}}
|L'Aquila = OK
}}

doesn't return OK. I tested on en.wikipedia and it.wikipedia.

Note that the following correctly returns OK:
{{#switch:L'Aquila
|L'Aquila = OK
}}
And {{PAGENAME}} alone correctly returns L'Aquila, but together they don't work...
Comment 1 Mark A. Hershberger 2012-04-02 18:06:39 UTC
Note that the following shows "39" -- so that gives you a work around.

{{#switch:{{PAGENAME}}
|L'Aquila = OK
|L = not ok
|L&rapos;Aquila = rapos
|L'Aquila = apos
|L'Aquila = 39
}}
Comment 2 Bultro 2012-04-04 12:05:47 UTC
Thank you, the same happens with titles containing & or ", which are converted to & and ". Many other symbols work normally.
The problem is in PAGENAME, looking at http://www.mediawiki.org/wiki/Help:Magic_words this behavior does not seem to be intentional
Comment 3 Mark A. Hershberger 2012-04-04 14:20:53 UTC
Thinking about this, I'm not sure if it would make sense to fix this -- it might cause problems for others.
Comment 4 Mark A. Hershberger 2012-04-04 15:10:11 UTC
See also bug 16474
Comment 5 Beta16 2012-04-06 07:26:16 UTC
See also bug 35746
Comment 6 Mark A. Hershberger 2012-04-06 14:30:43 UTC
*** Bug 35746 has been marked as a duplicate of this bug. ***
Comment 7 Philippe Verdy 2012-04-06 19:25:39 UTC
a swtich should not make the difference between a character that is represented by a numeric character reference of natively.

so if a templace is encoded like this:

{{#switch:{{{1|}}}|@=yes|#default=no}}

or like this:

{{#switch:{{{1|}}}|@=yes|#default=no}}

this should work equally when passing it the parameter 1=@ or 1=@ or 1=@

All numeric character references (plus some wellknown named character references that are warrantied to be suppoorted everywhere in XML and HTML; i.e. the 5 standard ones: & < > " &pos;) should be treated everywhere as counting for 1 Unicode character (excactly like the UTF-8 sequences of bytes represening this character). All valid syntaxes for numeric character references should be accepted (decimal and hexadecimal), as long as they designate a valid Unicode code point (in the valid numeric range from U+0000 to U+10FFFF), and that code point is assigned to a valid character (excluding codepoints assigned to surrogates, and codepoints assigned to non-characters like U+FFFE), and that character can be part of a valid HTML document (so, excluding most C0 and C1 controls, and converting all the few acceptable controls only as SPACE U+0020 or LINEFEED U+000A after unification of CR+LF into a single linefeed).

This should be a simple way to escape every character, deprecating the use of "nowiki", ecept as an esay way that avoids using character references in the source.

But character references should be usable EVERYWHERE a valid UTF-8 sequence representing a single character is usable and not absolutely needed by the syntaxic lexer/parser (so including in the name of parser functions and magic keywords, meaning that "{{#Kf:x|y}}" will be treated equivalently to "{{#if:x|y}}". This would make the wiki syntax more compatible with various character encodings, including via imports/exports to external files.

This also means that only a few characters should NOT be representable as character references, these are:
  { }
only where they are used as separators for the recognized wiki template call and parameters syntax, and:
  | =
only within template (or parserfunction) parameters in the wiki syntax, and:
  : ; *
only where they are recognized at the begining of lines for lists in the wiki syntax, and:
  | !
where they are recognized within wiki tables for delimiting cells/rows, and:
  < " ' >
where they are used as separators for the recognized markup syntax of HTML elements or special elements like "<nowiki ... />", "<includeonly ... />" and "<gallery ... />".

In this later case, character entities should be usable as the universal way of escaping the special handling given by the wiki syntax parser.

To make things simple, the lexer used in MEdiaWiki should uniformize all input characters (whever they are encoded as UTF-8 sequences or as numeric or named character entities) into a single format, even before staring to parse the content: only the special characters needed for one step should be treated specially, and kept in their syntaxic format, all others will be uniformized by NOT using any of these special characters (if they remain present in the source, the uniformized format should be the smallest decimal numeric character reference). This would also avoid the unnecessary complexity caused by "nowiki". All parser functions should be revisited to make sure they use this "character uniformizer"...
Comment 8 Gerrit Notification Bot 2014-02-15 02:41:47 UTC
Change 113518 had a related patch set uploaded by Brian Wolff:
Decode html entities before comparing strings in #ifeq: and #switch

https://gerrit.wikimedia.org/r/113518
Comment 9 Gerrit Notification Bot 2014-06-26 15:46:43 UTC
Change 113518 merged by jenkins-bot:
Decode html entities before comparing strings in #ifeq: and #switch

https://gerrit.wikimedia.org/r/113518
Comment 10 Jesús Martínez Novo (Ciencia Al Poder) 2014-06-27 14:06:53 UTC
How convenient :) Someone just reported this issue with PAGESINCATEGORY (now filed as bug 67196)

The problem is really {{PAGENAME}}, although I'm thinking it was done to prevent breaking HTML output when using {{PAGENAME}} inside HTML attributes (for example, title="Explanation of {{PAGENAME}}")

I'm wondering if this entity decoding should be done case by case or could be done for all parser functions parameters?
Comment 11 Philippe Verdy 2014-06-27 22:09:01 UTC
The problem affects templates trying to map a subpagename as a language code.
Currently {{#language:code1|code2}} causes a fatal server error (HTTP error 500) and all pages using that that template whose subpage name may contain an ASCII single or quote quote or an ampersand: {{SUBPAGENAME}} HTML encodes these characters with entities, and when this is used in the value of "code2" above, this will break.
To avoid this issue, we need a way to test if a subpagename can be a valid language code before trying to use {{#language:}}.

One way to test it includes comparing the (SUB)PAGENAME with the result of #titleparts, using a "#ifeq:" parser function call.

But if #ifeq: is HTML-decoding its compared items, it will alway reply that the (SUB)PAGENAME and #titleparts are equal, so it will no longer be alble to detect invalid language codes. As a result we'll get HTTP error 500 at amny random pages using some templates when viewing a subpage including that template and whose subpagename contains an apostrophe-quote, or double quote, or a few other characters.

An alternative would require using a Lua module for testing the validity of language codes. But in my opinion "#language:" MUST be urgently fixed to not crash when there are HTML entities in its second parameter (if this occurs, it should handle the case gracefull as if we specified an unknown/unsupported target language code.

Note that this critical bug of #language occurs in very important pages, notably many "Main pages" of wiki ˆprojects", or one of their subpages that are transcluding a page trying to display a list of alternate languages, using the content language of the current page (which may be translated).

As long as this critical bug of "#language:", the fix for "#ifeq:" or "#switch:" should be delayed (or be prapared to see lots of HTTP error 500 in server logs and many pages not rendered at all.
Comment 12 Jesús Martínez Novo (Ciencia Al Poder) 2014-06-28 09:02:41 UTC
Is there a bug open about {{#language:}} causing HTTP 500 errors? because the true error is #language, not what has been fixed here. Under any circumstances should a parser function throw an unhandled exception based on user input.
Comment 13 Jesús Martínez Novo (Ciencia Al Poder) 2014-06-28 09:49:07 UTC
I've filed bug 67241 about the {{#language:}} issue, which I was unable to reproduce locally.
Comment 14 Krinkle 2014-07-20 21:59:49 UTC
https://gerrit.wikimedia.org/r/#/c/113518/

(Mormegil Jul 11 18:04)
> Patch Set 8:
> This change broke all inline coordinates on cswiki (until I fixed the
> template) because of a small wikitext interpretation change. Formerly,
> “{{#switch:x|y=z|#default}}” would render empty, while currently, it
> renders as “#default”. The input wikitext is arguably wrong (an equal sign
> is missing there, it should be “...|#default=}}”), and it is debatable
> what is _better_ behavior in that case. However, forgetting an equal sign is
> an easy error to make, especially when it used to work fine.
> The original behavior was more or less a random byproduct, I’d say. (Keeping
> $test from “$mwDefault->matchStartAndRemove( $test )” to be used in the
> final “return $test;”.) The current behavior is arguably more logical, but
> in the name of backwards (bug-for-bug?) compatibility, we might want to do
> “$lastItem = $decodedTest” next to “$defaultFound = true;”... Dunno.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links