Last modified: 2014-09-19 22:40:31 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T58439, the corresponding Phabricator task for complete and up-to-date bug report information.
First Last Prev Next    This bug is not in your last search results.
Bug 56439 - Import of Commons partial export fails with error about plurals-mediawiki.xml
Import of Commons partial export fails with error about plurals-mediawiki.xml
Status: PATCH_TO_REVIEW
Product: MediaWiki
Classification: Unclassified
Export/Import (Other open bugs)
1.23.0
All All
: Normal normal (vote)
: 1.24.0 release
Assigned To: This, that and the other (TTO)
: i18n
: 60435 65116 (view as bug list)
Depends on:
Blocks:
  Show dependency tree / graph
 
Reported: 2013-10-31 23:27 UTC by Matthew Flaschen
Modified: 2014-09-19 22:40 UTC (History)
7 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---

Attachments
Dump that shows problem (190.80 KB, text/xml)
2013-10-31 23:28 UTC, Matthew Flaschen 		Details
Comparison trivial import that works (2.16 KB, text/xml)
2013-10-31 23:37 UTC, Matthew Flaschen 		Details
Add an attachment (proposed patch, testcase, etc.)

Description Matthew Flaschen 2013-10-31 23:27:53 UTC 
I exported the latest version of all of the pages starting with MediaWiki:Lang/ (MediaWiki:Lang/de, etc.), to test Template:LangSwitch.  However, on import I get repeating warnings like the below, I believe for every revision:

Warning: DOMDocument::load(): I/O warning : failed to load external entity "/vagrant/mediawiki/languages/data/plurals-mediawiki.xml" in /vagrant/mediawiki/includes/cache/LocalisationCache.php on line 588

Call Stack:
    0.0045     778856   1. {main}() /vagrant/mediawiki/maintenance/importDump.php:0
    0.0121    1396960   2. require_once('/vagrant/mediawiki/maintenance/doMaintenance.php') /vagrant/mediawiki/maintenance/importDump.php:291
    0.3200   17065176   3. BackupReader->execute() /vagrant/mediawiki/maintenance/doMaintenance.php:113
    0.3202   17065624   4. BackupReader->importFromStdin() /vagrant/mediawiki/maintenance/importDump.php:97
    0.3206   17066456   5. BackupReader->importFromHandle() /vagrant/mediawiki/maintenance/importDump.php:253
    0.3279   17824496   6. WikiImporter->doImport() /vagrant/mediawiki/maintenance/importDump.php:286
   98.5509  207738984   7. WikiImporter->handlePage() /vagrant/mediawiki/includes/Import.php:473
   98.5575  207748280   8. WikiImporter->handleRevision() /vagrant/mediawiki/includes/Import.php:609
   98.5603  207754536   9. WikiImporter->processRevision() /vagrant/mediawiki/includes/Import.php:658
   98.5607  207758064  10. WikiImporter->revisionCallback() /vagrant/mediawiki/includes/Import.php:706
   98.5607  207758696  11. call_user_func_array() /vagrant/mediawiki/includes/Import.php:351
   98.5608  207758840  12. BackupReader->handleRevision() /vagrant/mediawiki/includes/Import.php:0
   98.5608  207758920  13. call_user_func() /vagrant/mediawiki/maintenance/importDump.php:165
   98.5608  207758976  14. WikiImporter->importRevision() /vagrant/mediawiki/maintenance/importDump.php:0
   98.5610  207759760  15. DatabaseBase->deadlockLoop() /vagrant/mediawiki/includes/Import.php:257
   98.5623  207760720  16. call_user_func_array() /vagrant/mediawiki/includes/db/Database.php:3004
   98.5623  207761088  17. WikiRevision->importOldRevision() /vagrant/mediawiki/includes/db/Database.php:0
   98.5918  207781560  18. WikiPage->doEditUpdates() /vagrant/mediawiki/includes/Import.php:1434
   98.5927  207782424  19. WikiPage->prepareContentForEdit() /vagrant/mediawiki/includes/WikiPage.php:2069
   98.6048  207799064  20. WikitextContent->getParserOutput() /vagrant/mediawiki/includes/WikiPage.php:2026
   98.6048  207799384  21. Parser->parse() /vagrant/mediawiki/includes/content/WikitextContent.php:300
   98.6058  207812752  22. Parser->internalParse() /vagrant/mediawiki/includes/parser/Parser.php:395
   98.6079  207813128  23. Parser->replaceInternalLinks() /vagrant/mediawiki/includes/parser/Parser.php:1230
   98.6079  207813264  24. Parser->replaceInternalLinks2() /vagrant/mediawiki/includes/parser/Parser.php:1825
   98.6080  207815560  25. Parser->getTargetLanguage() /vagrant/mediawiki/includes/parser/Parser.php:1861
   98.6081  207815560  26. Title->getPageLanguage() /vagrant/mediawiki/includes/parser/Parser.php:837
   98.6086  207815624  27. ContentHandler->getPageLanguage() /vagrant/mediawiki/includes/Title.php:4824
   98.6097  207815920  28. wfGetLangObj() /vagrant/mediawiki/includes/content/ContentHandler.php:620
   98.6107  207922328  29. Language::factory() /vagrant/mediawiki/includes/GlobalFunctions.php:1304
   98.6107  207922328  30. Language::newFromCode() /vagrant/mediawiki/languages/Language.php:196
   98.6122  207922688  31. Language::getFallbacksFor() /vagrant/mediawiki/languages/Language.php:237
   98.6123  207922928  32. LocalisationCache->getItem() /vagrant/mediawiki/languages/Language.php:4120
   98.6123  207922928  33. LocalisationCache->loadItem() /vagrant/mediawiki/includes/cache/LocalisationCache.php:259
   98.6124  207922928  34. LocalisationCache->initLanguage() /vagrant/mediawiki/includes/cache/LocalisationCache.php:324
   98.6203  207923208  35. LocalisationCache->recache() /vagrant/mediawiki/includes/cache/LocalisationCache.php:442
   98.6667  208613368  36. LocalisationCache->readSourceFilesAndRegisterDeps() /vagrant/mediawiki/includes/cache/LocalisationCache.php:772
   98.7115  210162200  37. LocalisationCache->getPluralRuleTypes() /vagrant/mediawiki/includes/cache/LocalisationCache.php:630
   98.7115  210162200  38. LocalisationCache->loadPluralFiles() /vagrant/mediawiki/includes/cache/LocalisationCache.php:558
   98.7188  210162576  39. LocalisationCache->loadPluralFile() /vagrant/mediawiki/includes/cache/LocalisationCache.php:578
   98.7188  210163240  40. DOMDocument->load() /vagrant/mediawiki/includes/cache/LocalisationCache.php:588
Comment 1 Matthew Flaschen 2013-10-31 23:28:50 UTC 
Created attachment 13641 [details]
Dump that shows problem
Comment 2 Matthew Flaschen 2013-10-31 23:37:42 UTC 
Created attachment 13643 [details]
Comparison trivial import that works

The import with the warning also does not seem to work (though I have ctrl-Ced it every time) even partially.

A trivial one-page import (attached) works fine.
Comment 3 Yuvi Panda 2013-11-08 00:29:11 UTC 
Also affects testwikidatawiki
Comment 4 Yuvi Panda 2013-11-08 00:30:02 UTC 
Only the full history version of it, works fine for the current versions only dump
Comment 5 Nemo 2014-03-07 12:20:34 UTC 
*** Bug 60435 has been marked as a duplicate of this bug. ***
Comment 6 Nemo 2014-03-07 14:10:18 UTC 
I wonder if this has to do with insufficient caching: if we're hammering the localisation cache in MySQL for hundreds revisions per second, I wouldn't be surprised that something goes wrong.
In an instance set up by role::mediawiki-install::labs , I now uncommented
  $wgCacheDirectory = "$IP/cache";
and
  php maintenance/rebuildLocalisationCache.php --force
took only few seconds instead of minutes because it uses CDB files (I first checked write permissions in cache/ directory, owned by www-data rather than root).
From a cursory view of top, it seems that now the php process running importDump is as busy as before, but mysqld is one order of magnitude less loaded.

(In reply to Yuvi Panda from comment #3)
> Also affects testwikidatawiki

What caching system did it use?
Comment 7 Ori Livneh 2014-05-10 13:06:54 UTC 
Adding libxml_disable_entity_loader(false); to the top of LocalisationCache::loadPluralFile makes the warning go away. We disable the entity loader for a reason (it's insecure), so we can't use that as a solution. But it does finger entity references as most likely being the underlying cause.
Comment 8 Andrew Bogott 2014-05-10 16:48:39 UTC 
I have a fairly trivial import that fails in the same way.

https://github.com/wikimedia/operations-puppet/blob/production/modules/mediawiki_singlenode/files/privacy-policy.xml
Comment 9 This, that and the other (TTO) 2014-07-26 07:40:24 UTC 
Apparently caused by the fact that libxml_disable_entity_loader is not thread-safe. Great one, PHP!

See https://bugs.php.net/bug.php?id=64938 and http://pyd.io/f/topic/failed-to-load-external-entity-boot-confmanifest-xml
Comment 10 Gerrit Notification Bot 2014-07-26 07:41:33 UTC 
Change 149507 had a related patch set uploaded by TTO:
Use file_get_contents to load plurals XML data

https://gerrit.wikimedia.org/r/149507
Comment 11 This, that and the other (TTO) 2014-07-26 07:43:10 UTC 
*** Bug 65116 has been marked as a duplicate of this bug. ***
Comment 12 Bartosz Dziewoński 2014-09-19 15:34:43 UTC 
According to comments on the patch, it is not clear if it actually fixes the issue or not.
Comment 13 This, that and the other (TTO) 2014-09-19 22:40:31 UTC 
Not sure if any of those commenters actually tested it...
Note You need to log in before you can comment on or make changes to this bug.
First Last Prev Next    This bug is not in your last search results.

Navigation
Links