Last modified: 2014-11-17 10:36:34 UTC

Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 35038 - Make MathJax menus translatable [upstream]
Make MathJax menus translatable [upstream]
Status: RESOLVED FIXED
Product: MediaWiki extensions
Classification: Unclassified
Math (Other open bugs)
unspecified
All All
: High blocker with 2 votes (vote)
: ---
Assigned To: Nobody - You can work on this!
: i18n, upstream
Depends on:
Blocks: 31406 make-mathjax-default 48036 49267
  Show dependency treegraph
 
Reported: 2012-03-07 21:21 UTC by Helder
Modified: 2014-11-17 10:36 UTC (History)
18 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Helder 2012-03-07 21:21:30 UTC
As mentioned on r113203#c31944, currently the menus are always in English.
They need to be in user language.
Comment 1 Brion Vibber 2012-03-07 21:25:37 UTC
While it would be nice to make them translatable, I'm going to take this out of the blockers for initial experimental enabling (bug 31406).
Comment 2 Brion Vibber 2012-03-07 21:43:45 UTC
(Started a thread on mathjax-users asking if people would be interested in menu localization)
Comment 4 Siebrand Mazeland 2012-04-18 09:27:06 UTC
Updating priority and importance. We shouldn't push un-i18n-ed components to non-English language wikis -- or even consider adding it when it has no i18n.
Comment 5 Liangent 2012-04-18 09:37:58 UTC
(In reply to comment #4)
> Updating priority and importance. We shouldn't push un-i18n-ed components to
> non-English language wikis -- or even consider adding it when it has no i18n.

Maybe it can be disabled before it's i18n-ed...
Comment 6 Niklas Laxström 2012-05-03 17:49:18 UTC
It's now enabled site-wide :/
Comment 7 Siebrand Mazeland 2012-05-03 17:58:25 UTC
Updated priority. Appears to be a major oversight that un-i18n-able software was deployed side wide. Also added a few CCs.
Comment 8 Brion Vibber 2012-05-03 18:00:55 UTC
I'd say this blocks a nice full roll-out to default rendering (still a ways off), but it obviously doesn't block experimental opt-in usage as we have already turned it on.
Comment 9 Erik Moeller 2012-05-04 05:10:10 UTC
Yep, agree w/ Brion. No reason to not let people test it as a pref. (The menu structure is also fairly opaque in normal usage; I hadn't discovered it until seeing this bug.)
Comment 10 Rob Lanphier 2012-05-10 22:08:56 UTC
Dropping priority to "high".  "Highest" priority should be reserved for bugs that someone needs to drop what they're doing and address immediately (possibly on the weekend if necessary).
Comment 11 Siebrand Mazeland 2012-06-13 18:18:19 UTC
Any progress?
Comment 12 Siebrand Mazeland 2012-10-22 09:03:16 UTC
Any progress? Looks like the upstream thread is very dead.

Maybe MathJax can use https://github.com/wikimedia/jquery.i18n
Comment 13 Amir E. Aharoni 2012-11-10 13:29:08 UTC
Just to note that Aditya (added to CC) started to try doing this at the Bangalore DevCamp using the jquery.i18n that Wikimedia's Language Engineering team developed. He already has some results.
Comment 14 Peter Krautzberger 2012-11-10 20:45:54 UTC
Hi everyone. 

Just to keep you updated from the MathJax side. The next MathJax release (planned for the winter) will include proper UI localization.
Comment 15 Nemo 2012-11-10 21:07:39 UTC
(In reply to comment #14)
> Just to keep you updated from the MathJax side. The next MathJax release
> (planned for the winter) will include proper UI localization.

Great!
Can you link the instructions or the process you're going to follow?
I hope it doesn't involve emailing po files or similar pains.
Comment 16 Peter Krautzberger 2012-11-10 22:07:54 UTC
First we have to work on our code a bit. Once localization is possible, we'd love input on how to crowd source. But no, definitely not emailing po files.
Comment 17 Nemo 2012-11-10 22:31:58 UTC
(In reply to comment #16)
> First we have to work on our code a bit. Once localization is possible, we'd
> love input on how to crowd source. But no, definitely not emailing po files.

Good, so it's much better for you to plan in advance rather than to spend tons of time fixing stuff later.
In particular, if you follow i18n good practices and use a format supported by [[mw:Extension:Translate]] (there are several) then it will be easy to add it to translatewiki.net.
Comment 18 Siebrand Mazeland 2012-11-11 04:47:09 UTC
(In reply to comment #14)
> Hi everyone. 
> 
> Just to keep you updated from the MathJax side. The next MathJax release
> (planned for the winter) will include proper UI localization.

(In reply to comment #13)
> Just to note that Aditya (added to CC) started to try doing this at the
> Bangalore DevCamp using the jquery.i18n that Wikimedia's Language Engineering
> team developed. He already has some results.

Are these two messages at almost the same time referring to the same efforts, or different efforts at i18n-ing the code?
Comment 19 Peter Krautzberger 2012-11-11 05:07:01 UTC
Different efforts. 

I'm speaking for the MathJax team. I only heard about attempts to localize on twitter and here so I thought I should comment and clarify our own plans.

As you (Siebrand) pointed out, the thread on the MathJax dev-list has been cold since we've been focusing on a bug fix release over the last few months.
Comment 20 Siebrand Mazeland 2012-11-11 09:16:25 UTC
(In reply to comment #19)
> Different efforts. 
> 
> I'm speaking for the MathJax team. I only heard about attempts to localize on
> twitter and here so I thought I should comment and clarify our own plans.
> 
> As you (Siebrand) pointed out, the thread on the MathJax dev-list has been cold
> since we've been focusing on a bug fix release over the last few months.

AFAIK the DevCamp Bangalore participants are planning on creating a pull request with an i18n solution for MathJax today (India time). You're of course free to discard that in favour of something still to be developed.
Comment 21 Peter Krautzberger 2012-11-12 02:33:47 UTC
We'll definitely take a look but I can't make any promises.
Comment 22 Amir E. Aharoni 2012-11-13 14:47:05 UTC
So, here's the update:

Apparently, it's already possible to use menu-i18n.js as it is to localize the menu. We translated the menu to a few languages and it works. To achieve this we just need to change our code, but we would be happy to send you the translations that we made to a few languages already. I'm just unsure to which folder in GitHub to add them.

However, translating the menu alone is not enough. Messages like "Typesetting math: 45%" cannot be properly localized with the current code for at least two reasons:

1. Most importantly, they are concatenated on the fly. This will not work in languages with a different word order.

2. The digits in the percentage are not localized. This is relevant for Arabic, Persian, Marathi and some other languages.

Our jquery.i18n library has a ready-made solution for both problems - it has a flexible parameter system that allows any word order and it has automatic digit localization. So to have complete localization you should either use jquery.i18n or come up with your own parameter functions.
Comment 23 Frédéric Wang 2012-11-13 15:14:43 UTC
menu-i18n is an experimental extension made by Thomas Leathrum when we started to talk about localization. It was useful to give a prototype to base the discussion on. However, this extension has limits and we found many issues to address including your point 1). I don't think we thought about 2), although we also mentioned use of commas/dots in numbers. I'm sure Wikipedia folks has a good experience with localization and we certainly want to take benefits from your efforts and make the interface as convenient as possible for localization folks.

As Peter said, we are going to have a developer meeting today to plan MathJax's future developments now that MathJax 2.1 is released. Localization will be one of the topic. Personally, I would be in favor of using your extension or at least relying on it as much as possible. There is one thing that I would like to ask. Your extension is called "jquery.i18n" so I guess it requires jquery at the moment. Does that mean that MathJax has to include jquery or call a jquery installation on another server? Perhaps it does not rely so heavily on jquery and we could replace the use of jquery by MathJax's own API?

Thanks.
Comment 24 Frédéric Wang 2012-11-13 15:37:24 UTC
For the record, a summary of our previous discussions is available on our dev wiki:

https://github.com/mathjax/MathJax/wiki/Mathjax-v2.x-potential-feature%3A-localization
Comment 25 Frédéric Wang 2012-11-14 13:19:41 UTC
First I'd like to thank you again for providing another solution to localization in MathJax. I just read the code of jquery.i18n and speaking for myself I don't think it is really worth trying to integrate it to MathJax. The source code is small and there are many things that I think could be simplified if we implement something similar in MathJax. For example, I'm not sure whether redefining the split function is essential? Or whether you really need jquery to implement such a library? It seems that you only use it to load the file asynchronously but MathJax already has an API to do that and for this and the callback function in i18n.properties, we would rather use our own API to synchronize with MathJax's code, put feedback to our UI and author's listeners etc. Also, I think only the "map" mode is really important? We want to allow users to easily share the translation files and so the possibility to add Javascript code in the translation files with the "vars" mode does not sound robust nor secure. There are other weird things like why using parameters[i].split('='), keeping the first part and putting back the other parts together, instead of just parameters[i].split('=', 1)? And for multi-line values, why are these parts from the first line are only put back *after* the subsequent lines? Similarly in i18n.prop, it seems you are really giving you a lot of pain to implement a parser from the value to an array... just to finish with a typo "value.lengh == 1" that fortunately is without consequences. Mozilla uses a different format for *.properties files [1] where the parameters are just given by %1$S, %2$S etc and the percent can be represented by %S%%. That really just seems enough to me as the important point is to be able to pass the arguments and handle different orders in different languages, as you mentioned in comment 22. Replacing these placeholders will simply be a matter of calling the string's "replace" method with the right regexp expression and substitution function. I guess the Javascript's native regexp system will be more reliable and efficient (perhaps even more than just constructing the return value from the computed array).

I suspect we will just use JSON files to do the mapping from a key to a value and so parseData could just be replaced by our asynchronous file loading API while i18n.prop could just be replaced by a call to the standard "replace" method with appropriate parameters as I indicate above. I was not able to find the MathJax extension/changeset to use the jquery.i18n library. Is it public, still work in progress? I can't see the automatic digit localization in jquery.i18n, so perhaps it is implemented somewhere else? Is there anything important point to consider you'd like to mention besides what you indicate in comment 22?

Of course, we'd definitely want your translation files and I agree with comment 17 that it is important to have a format supported by Wikimedia and that follows i18n good practices. It seems that you have a JsonFFS script which "implements a message format where messages are encoded as key-value pairs in JSON objects". Would that make sense to you if we use this format? Or do you prefer to provide *.properties files and let us do the conversion?

Thanks.

[1] https://developer.mozilla.org/en-US/docs/XUL_Tutorial/Property_Files
[2] https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/extensions/Translate.git;a=tree;f=ffs;hb=HEAD
Comment 26 Frédéric Wang 2013-02-08 10:18:46 UTC
Hi, some questions for the i18n experts:

1) Does the translatewiki interface provide localization notes (like comments from the original translation file) to help translators to figure out the context of a strings to translate (e.g. to indicate whether "Post" is a noun or a verb, or explain technical terms)? And what is the best way to insert such localization notes in the translation files?

2) We would like MathJax to support plural forms. It seems that among the formats supported by Mediawiki, only YAML and PO have this feature implemented (and the way plural form is handled in YAML seems slightly more restrictive than in PO files). In particular that does not seem available in the json format used in the jquery.i18n extension mentioned and which is also the closest to MathJax's internal structure. What has been the experience with translators on that issue and what would you recommend?

3) Key-based format seems more robust to avoid breaking the translation when something like a small typo is fixed or when one has to export the translation to other formats. Most formats seem to take this approach but not PO files. Again, what do you recommend/prefer here?

Thanks.
Comment 27 Niklas Laxström 2013-02-08 10:34:33 UTC
1) Yes it does. We usually store them under "qqq" language code. This can be exported alongside other translations (but not displayed in the software) or can be edited directly from translatewiki.net

2) It does. It uses the inline syntax like some of our other formats. See https://github.com/wikimedia/jquery.i18n/wiki/Specification#wiki-Plural_Gender_Grammar_support

3) I recommend key based formats. I have a blog posts coming up on my take on Gettext.
Comment 28 Niklas Laxström 2013-02-08 10:37:56 UTC
https://github.com/wikimedia/jquery.i18n/wiki/Specification#wiki-API is perhaps a better link, gives an example how inline plural syntax looks. Internally it uses the plural logic from CLDR - CLDR plural rules are also used in the Ruby on Rails YAML format.
Comment 29 Frédéric Wang 2013-02-08 12:31:09 UTC
(Mmmh, either the jquery.i18n code has changed a lot or I looked into the wrong repository in comment 25. I remember there were only a 2 or 3 files that had not been changed for one year...)

I was reading the FFS list ( https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/extensions/Translate.git;a=tree;f=ffs;hb=HEAD ) and a grep for "plural" only returned YAML and PO. But I realize that the plural rules and gender are just indicated in the localizable string itself for the JSON format used in jquery.i18n. Is there a precise spec for this format, at least for the basic features? If not, perhaps examples that can be used to "guess" the format? In any case, having some translation files for the languages you translated (as indicated in comment 22) would be very helpful.
Comment 30 Niklas Laxström 2013-02-08 12:44:36 UTC
What kind of spec are you looking for? For developers or translators? As a dev you pass the info (number for plural) when calling the message. As a translator you use {{PLURAL:$1|uga|chaga}} with as many forms as it supported for given language.

Should we move this discussion to irc/email/wiki?
Comment 31 Frédéric Wang 2013-02-08 13:41:49 UTC
Thanks, yes I understood that but I'm looking for a spec that will gives exact syntax (like a grammar, regexp, or even informal description) if we are to implement a similar interface, so that we are sure that the files are compatible. For example, the wiki pages uses lowercase "plural" but comment 30 uses uppercase "PLURAL", so I guess this is case-insensitive? I suppose there are other exceptions of this kind.

I'll continue to discuss with the MathJax team and perhaps we could reopen the discussion on the MathJax dev mailing list later if Bugzilla is not the right place.
Comment 32 Niklas Laxström 2013-02-08 15:24:17 UTC
In MediaWiki and jquery.i18n the magic words are indeed case sensitive. Also, rather than using explicit keywords from cldr, we just rely on the order of the forms. If there are fewer forms than specified by the rules, we use the last provided form to fill the missing forms.

Maybe looking at the tests https://github.com/wikimedia/jquery.i18n/blob/master/test/jquery.i18n.test.js helps.
Comment 33 Matthew Flaschen 2013-05-03 20:16:38 UTC
The keywords are case *in*sensitive (I think it was just a typo above).
Comment 34 Peter Krautzberger 2013-05-03 20:53:47 UTC
This will be fixed upstream in the upcoming MathJax v2.2 release. The beta version has been released today cf. http://www.mathjax.org/mathjax-v2-2-beta-now-available/. 

We've asked to be hosted on TranslateWiki, otherwise we'll put something on Transifex. We're not using jquery.i18n for several technical reasons but our format is very close to that and we're happy to provide multiple file formats.
Comment 35 Peter Krautzberger 2013-05-17 23:03:59 UTC
This is now fixed upstream in MathJax v2.2 http://www.mathjax.org/mathjax-v2-2-now-available/
Comment 36 Matthew Flaschen 2013-05-17 23:21:01 UTC
Peter, you might want to more info on the translations:

* How should people use the two new localizations (German, French)?
* Where should people go to work on adding more translations?
* What technology is used (most relevant for this bug).
Comment 37 Derk-Jan Hartman 2013-05-19 15:39:00 UTC
MathJax 2.2: https://gerrit.wikimedia.org/r/64522

* You can right click a formula and select a translation from the menu. I haven't been able to find a config based way to 'preselect' a language, based on the information that we already have about users. Peter, can you clarify if I missed something ?

* I've heard they are talking to translatewiki.net https://groups.google.com/forum/?fromgroups#!topic/mathjax-users/FA5BNgw7AuM

* https://github.com/mathjax/MathJax/tree/v2.2-beta/unpacked/localization/en
Comment 38 Peter Krautzberger 2013-05-19 16:02:17 UTC
Hm, sorry about that. I had replied, but it didn't make it through. This is what I wrote two days ago.



Matthew,

Thanks, I mostly wanted to let everyone know that the v2.2 is out of beta.

Our documentation at docs.mathjax.org is usually the right place to
find answers but since you asked, let me try.

1) Not sure what you're after. Page authors set a preferred language
via their MathJax configuration, page visitors can override this via
the MathJax menu, page authors can prevent them from doing so by
removing the corresponding menu entry. Again, our documentation has
more details. The mediawiki math-extension people probably want to
take a look.
2) We're still working on getting MathJax hosted at TranslateWiki and
Transifex. Our file format is very similar to the jquery.i18n format
and we'll convert it to other formats where needed. Again, see our
documentation (which for now will point to our developer wiki
https://github.com/mathjax/MathJax/wiki/Quick-guide-to-translating-MathJax).
3) Again, not sure what's meant. I'm guessing you're asking if we're
using jquery.i18n? In that case, the answer is no. It was not powerful
enough. In addition, we don't use jquery, so using it would have
significantly added to the MathJax load. Maybe I should mention that
MathJax is very modular with its own extension system, so technically
the answer would be: it's a MathJax extension. But I'm guessing that
won't help.

Hope this helps,
Peter.
Comment 39 Matthew Flaschen 2013-05-19 19:13:07 UTC
(In reply to comment #38)
> 1) Not sure what you're after. Page authors set a preferred language
> via their MathJax configuration, page visitors can override this via
> the MathJax menu, page authors can prevent them from doing so by
> removing the corresponding menu entry.

Yes, that is what I was trying to find (either directly or through a link) in the announcement.  It seems from http://docs.mathjax.org/en/latest/localization.html it is passed in the URL.

> 3) Again, not sure what's meant. I'm guessing you're asking if we're
> using jquery.i18n? In that case, the answer is no. It was not powerful
> enough.

Yes, this was before I looked at the implementation, so I was wondering if you were using a pre-existing JavaScript localization library.
Comment 40 Gerrit Notification Bot 2013-06-04 19:32:56 UTC
Related URL: https://gerrit.wikimedia.org/r/67001 (Gerrit Change Ibf43a3e44ea2bfdbd9fddc01c71c72ba097ff1b7)
Comment 41 Derk-Jan Hartman 2013-06-04 19:34:15 UTC
After a bit of grepping I found the relevant entry points to configure the language I think, MathJax.Localization.locale and MathJax.Hub.config.menuSettings.locale
Comment 42 Gerrit Notification Bot 2013-06-04 20:49:11 UTC
Related URL: https://gerrit.wikimedia.org/r/67009 (Gerrit Change Ie7530d3a904fb39945ba8b8d261852c2b8cb666b)
Comment 43 Gerrit Notification Bot 2013-06-04 21:29:26 UTC
Related URL: https://gerrit.wikimedia.org/r/67014 (Gerrit Change I4e21cc5cdcb0d7a31a97b3b136d725f67a53f75c)
Comment 44 Matthew Flaschen 2013-06-04 21:39:59 UTC
(In reply to comment #41)
> After a bit of grepping I found the relevant entry points to configure the
> language I think, MathJax.Localization.locale and
> MathJax.Hub.config.menuSettings.locale

menuSettings.locale is actually the key one.  The rest follow from there, but if menuSettings.locale is omitted, it won't work.  See the last change above.
Comment 45 Matthew Flaschen 2013-06-05 05:04:47 UTC
This is fixed.

The language selection is still limited.  In the meantime, I encourage people to submit pull requests upstream (https://github.com/mathjax/MathJax/pulls) to add languages.  Currently, they have German, English, and French.

As noted above, they are looking at using a translation site like TranslateWiki in the future.

Either way, when a new translation hits upstream master, you can file a bug to make sure our extension gets updated.
Comment 46 Nemo 2013-06-05 06:44:16 UTC
Until the project is added to translatewiki.net and pull requests are needed from translators, this can't be considered fixed.
All the discussion is in this report, which has always been marked upstream, so I don't see why split it.
Comment 47 Derk-Jan Hartman 2013-06-05 22:35:54 UTC
Well sort of depends on your interpretation of 'translatable' I guess. Anyways, it definitely where my 'assign' stops :D
Comment 48 Peter Krautzberger 2013-06-06 12:04:00 UTC
I'd second Derk-Jan. The "upstream" tag seems wrong to me (although technically correct).

I do understand that the real bug is "MathJax doesn't have enough translations", but that's something that the MediaWiki community can actually help fix -- and in a real sense we (MathJax) cannot fix this on our own since we don't have the resources to hire an army of translators. 

So perhaps it should be closed and another bug opened that calls for translations.
Comment 49 Nemo 2013-06-06 12:23:51 UTC
(In reply to comment #48)
> So perhaps it should be closed and another bug opened that calls for
> translations.

No. This is a MediaWiki extensions and MediaWiki extensions are translated at translatewiki.net. You won't get help by MediaWiki translators outside of it and from a Wikimedia perspective the issue is not solved until the translation doesn't live there. I don't see a reason to separate the two things now.
Comment 50 Peter Krautzberger 2013-06-06 12:55:26 UTC
Just to clarify: MathJax is not a MediaWiki extension, it's a third party library used by the math-extension. Either way, the MathJax team has no power over getting MathJax accepted into TranslateWiki, so help is needed on that end. We'd be thrilled to be there, but since it's by invitation only, all we can do is ask.
Comment 51 Matthew Flaschen 2013-06-07 02:00:20 UTC
I've created an upstream issue, https://github.com/mathjax/MathJax/issues/493, specifically about coordinating what needs to be done to get MathJax on TranslateWiki.
Comment 52 Matthew Flaschen 2013-09-26 01:40:00 UTC
MathJax is now on TranslateWiki.  Translators are encouraged to participate at https://translatewiki.net/wiki/Special:Translate/out-mathjax-0-all
Comment 53 Peter Krautzberger 2013-09-26 04:06:22 UTC
Thanks to everyone at WMF, TranslateWiki and MathJax for making this happen!
Comment 54 Siebrand Mazeland 2013-09-26 07:42:08 UTC
I hope we can contribute to the project's success with translatewiki.net.

Please use plain "translatewiki.net" in your communication (TWN as short form if you have to). Other forms more easily confused with other online translation platforms and wikis.
Comment 55 Helder 2013-09-26 11:38:15 UTC
It seems part of my original pt-BR translations were not imported to translatewiki.net, since it shows "61% completion" at
https://translatewiki.net/w/i.php?title=Special:MessageGroupStats&language=en&group=out-mathjax-0-all#sortable:3=desc

For example, I had to create
https://translatewiki.net/w/i.php?title=MathJax:Mathmenu-OriginalMathML/pt-br&diff=5016014
again, but this was already in the MathMenu.js attachment from bug 49267.
Comment 56 Siebrand Mazeland 2013-09-26 11:50:41 UTC
(In reply to comment #55)
> It seems part of my original pt-BR translations were not imported to
> translatewiki.net, since it shows "61% completion" at
> https://translatewiki.net/w/i.php?title=Special:
> MessageGroupStats&language=en&group=out-mathjax-0-all#sortable:3=desc
> 
> For example, I had to create
> https://translatewiki.net/w/i.php?title=MathJax:Mathmenu-OriginalMathML/pt-
> br&diff=5016014
> again, but this was already in the MathMenu.js attachment from bug 49267.

Should be resolved. I ran "php sync-group.php --lang=* --group=out-math* --noask"
Comment 57 Helder 2013-09-26 12:09:14 UTC
(In reply to comment #56)
> Should be resolved. I ran "php sync-group.php --lang=* --group=out-math*
> --noask"

Thanks!

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links