Last modified: 2014-02-12 23:39:54 UTC
Please see Bug 22371. Normalization is enabled on All Malayalam Language wikies. But now Malayalam grew outside those wikies (mainly to commons, wiki data etc) and old defacto characters are not supported by many applications including various webkit browsers (Chrome, Chromium), and many mobile applications. Please Enable normalization in All Wikimedia Wikies. (Probably Arabic is also need this.)
What? And where is the consensus that this should be done?
Longer version of last comment: For any configuration change, we require a local consensus. As this request is cross-wiki, this requires discussing the matter on Meta, probably https://meta.wikimedia.org/w/index.php?title=Wikimedia_Forum (somebody correct me if this is the wrong place), in order to confirm that this change is wanted by the community. For more information about how to request these kinds of changes, please see https://meta.wikimedia.org/wiki/Requesting_wiki_configuration_changes . Thanks!
Here the consensus for original bug: http://ml.wikipedia.org/wiki/WP:Panchayath_(Technical)/Unicode_5.1.0 But at that time fix was limited to Malayalam Language wikis because of some performance issues (If I remember correctly). And as you can see in Mediawiki, If your wiki language is Malayalam, the normalization is enabled by default. And also joiner based combinations are always problematic (Bug 45111).
Normalization works only when content language is ml, or ar. It does not get triggered based on user interface language. includes/WebRequest.php normalizeUnicode method calls normalize on $wgContLang.
However, normalization in non-ml/non-ar wikis is also possible. Check translatewiki.net.
$wgAllUnicodeFixes = true; is the setting required to get this normalization irrespective of the content language. http://www.mediawiki.org/wiki/Manual:$wgAllUnicodeFixes Enabling this means, every normalize call will do Unicode normalization fix for Malayalam and Arabic. Documentation hints performance impact, but I don't know how much.
meta request: [[meta:General_requests#Unicode_normalization_for_Malayalam_text_in_all_Wikimedia_projects]] local notification: [[:ml:വിക്കിപീഡിയ:പഞ്ചായത്ത് (സാങ്കേതികം)#നോർമലൈസേഷൻ]]
ml.wiki link: http://ml.wikipedia.org/wiki/%E0%B4%B5%E0%B4%BF%E0%B4%95%E0%B5%8D%E0%B4%95%E0%B4%BF%E0%B4%AA%E0%B5%80%E0%B4%A1%E0%B4%BF%E0%B4%AF:%E0%B4%AA%E0%B4%9E%E0%B5%8D%E0%B4%9A%E0%B4%BE%E0%B4%AF%E0%B4%A4%E0%B5%8D%E0%B4%A4%E0%B5%8D_%28%E0%B4%B8%E0%B4%BE%E0%B4%99%E0%B5%8D%E0%B4%95%E0%B5%87%E0%B4%A4%E0%B4%BF%E0%B4%95%E0%B4%82%29#.E0.B4.A8.E0.B5.8B.E0.B5.BC.E0.B4.AE.E0.B4.B2.E0.B5.88.E0.B4.B8.E0.B5.87.E0.B4.B7.E0.B5.BB
Andre, Sam, Ori, what's going on with this issue?
