Last modified: 2014-09-19 22:40:31 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T58439, the corresponding Phabricator task for complete and up-to-date bug report information.

Bug 56439 - Import of Commons partial export fails with error about plurals-mediawiki.xml


Summary:	Import of Commons partial export fails with error about plurals-mediawiki.xml

Status:	PATCH_TO_REVIEW

Product:	MediaWiki
Classification:	Unclassified
Component:	Export/Import (Other open bugs)
Version:	1.23.0
Hardware:	All All

Importance:	Normal normal (vote)
Target Milestone:	1.24.0 release
Assigned To:	This, that and the other (TTO)

URL:
Whiteboard:
Keywords:	i18n

Duplicates:	60435 65116 (view as bug list)
Depends on:
Blocks:
	Show dependency tree / graph

Reported:	2013-10-31 23:27 UTC by Matthew Flaschen
Modified:	2014-09-19 22:40 UTC (History)
CC List:	7 users (show)

See Also:	62368
Web browser:	---
Mobile Platform:	---
Assignee Huggle Beta Tester:	---

Attachments
Dump that shows problem (190.80 KB, text/xml) 2013-10-31 23:28 UTC, Matthew Flaschen	Details
Comparison trivial import that works (2.16 KB, text/xml) 2013-10-31 23:37 UTC, Matthew Flaschen	Details
Add an attachment (proposed patch, testcase, etc.)

Description Matthew Flaschen 2013-10-31 23:27:53 UTC

I exported the latest version of all of the pages starting with MediaWiki:Lang/ (MediaWiki:Lang/de, etc.), to test Template:LangSwitch.  However, on import I get repeating warnings like the below, I believe for every revision:

Warning: DOMDocument::load(): I/O warning : failed to load external entity "/vagrant/mediawiki/languages/data/plurals-mediawiki.xml" in /vagrant/mediawiki/includes/cache/LocalisationCache.php on line 588

Call Stack:
    0.0045     778856   1. {main}() /vagrant/mediawiki/maintenance/importDump.php:0
    0.0121    1396960   2. require_once('/vagrant/mediawiki/maintenance/doMaintenance.php') /vagrant/mediawiki/maintenance/importDump.php:291
    0.3200   17065176   3. BackupReader->execute() /vagrant/mediawiki/maintenance/doMaintenance.php:113
    0.3202   17065624   4. BackupReader->importFromStdin() /vagrant/mediawiki/maintenance/importDump.php:97
    0.3206   17066456   5. BackupReader->importFromHandle() /vagrant/mediawiki/maintenance/importDump.php:253
    0.3279   17824496   6. WikiImporter->doImport() /vagrant/mediawiki/maintenance/importDump.php:286
   98.5509  207738984   7. WikiImporter->handlePage() /vagrant/mediawiki/includes/Import.php:473
   98.5575  207748280   8. WikiImporter->handleRevision() /vagrant/mediawiki/includes/Import.php:609
   98.5603  207754536   9. WikiImporter->processRevision() /vagrant/mediawiki/includes/Import.php:658
   98.5607  207758064  10. WikiImporter->revisionCallback() /vagrant/mediawiki/includes/Import.php:706
   98.5607  207758696  11. call_user_func_array() /vagrant/mediawiki/includes/Import.php:351
   98.5608  207758840  12. BackupReader->handleRevision() /vagrant/mediawiki/includes/Import.php:0
   98.5608  207758920  13. call_user_func() /vagrant/mediawiki/maintenance/importDump.php:165
   98.5608  207758976  14. WikiImporter->importRevision() /vagrant/mediawiki/maintenance/importDump.php:0
   98.5610  207759760  15. DatabaseBase->deadlockLoop() /vagrant/mediawiki/includes/Import.php:257
   98.5623  207760720  16. call_user_func_array() /vagrant/mediawiki/includes/db/Database.php:3004
   98.5623  207761088  17. WikiRevision->importOldRevision() /vagrant/mediawiki/includes/db/Database.php:0
   98.5918  207781560  18. WikiPage->doEditUpdates() /vagrant/mediawiki/includes/Import.php:1434
   98.5927  207782424  19. WikiPage->prepareContentForEdit() /vagrant/mediawiki/includes/WikiPage.php:2069
   98.6048  207799064  20. WikitextContent->getParserOutput() /vagrant/mediawiki/includes/WikiPage.php:2026
   98.6048  207799384  21. Parser->parse() /vagrant/mediawiki/includes/content/WikitextContent.php:300
   98.6058  207812752  22. Parser->internalParse() /vagrant/mediawiki/includes/parser/Parser.php:395
   98.6079  207813128  23. Parser->replaceInternalLinks() /vagrant/mediawiki/includes/parser/Parser.php:1230
   98.6079  207813264  24. Parser->replaceInternalLinks2() /vagrant/mediawiki/includes/parser/Parser.php:1825
   98.6080  207815560  25. Parser->getTargetLanguage() /vagrant/mediawiki/includes/parser/Parser.php:1861
   98.6081  207815560  26. Title->getPageLanguage() /vagrant/mediawiki/includes/parser/Parser.php:837
   98.6086  207815624  27. ContentHandler->getPageLanguage() /vagrant/mediawiki/includes/Title.php:4824
   98.6097  207815920  28. wfGetLangObj() /vagrant/mediawiki/includes/content/ContentHandler.php:620
   98.6107  207922328  29. Language::factory() /vagrant/mediawiki/includes/GlobalFunctions.php:1304
   98.6107  207922328  30. Language::newFromCode() /vagrant/mediawiki/languages/Language.php:196
   98.6122  207922688  31. Language::getFallbacksFor() /vagrant/mediawiki/languages/Language.php:237
   98.6123  207922928  32. LocalisationCache->getItem() /vagrant/mediawiki/languages/Language.php:4120
   98.6123  207922928  33. LocalisationCache->loadItem() /vagrant/mediawiki/includes/cache/LocalisationCache.php:259
   98.6124  207922928  34. LocalisationCache->initLanguage() /vagrant/mediawiki/includes/cache/LocalisationCache.php:324
   98.6203  207923208  35. LocalisationCache->recache() /vagrant/mediawiki/includes/cache/LocalisationCache.php:442
   98.6667  208613368  36. LocalisationCache->readSourceFilesAndRegisterDeps() /vagrant/mediawiki/includes/cache/LocalisationCache.php:772
   98.7115  210162200  37. LocalisationCache->getPluralRuleTypes() /vagrant/mediawiki/includes/cache/LocalisationCache.php:630
   98.7115  210162200  38. LocalisationCache->loadPluralFiles() /vagrant/mediawiki/includes/cache/LocalisationCache.php:558
   98.7188  210162576  39. LocalisationCache->loadPluralFile() /vagrant/mediawiki/includes/cache/LocalisationCache.php:578
   98.7188  210163240  40. DOMDocument->load() /vagrant/mediawiki/includes/cache/LocalisationCache.php:588

Comment 1 Matthew Flaschen 2013-10-31 23:28:50 UTC

Created attachment 13641 [details]
Dump that shows problem

Comment 2 Matthew Flaschen 2013-10-31 23:37:42 UTC

Created attachment 13643 [details]
Comparison trivial import that works

The import with the warning also does not seem to work (though I have ctrl-Ced it every time) even partially.

A trivial one-page import (attached) works fine.

Comment 3 Yuvi Panda 2013-11-08 00:29:11 UTC

Also affects testwikidatawiki

Comment 4 Yuvi Panda 2013-11-08 00:30:02 UTC

Only the full history version of it, works fine for the current versions only dump

Comment 5 Nemo 2014-03-07 12:20:34 UTC

*** Bug 60435 has been marked as a duplicate of this bug. ***

Comment 6 Nemo 2014-03-07 14:10:18 UTC

I wonder if this has to do with insufficient caching: if we're hammering the localisation cache in MySQL for hundreds revisions per second, I wouldn't be surprised that something goes wrong.
In an instance set up by role::mediawiki-install::labs , I now uncommented
  $wgCacheDirectory = "$IP/cache";
and
  php maintenance/rebuildLocalisationCache.php --force
took only few seconds instead of minutes because it uses CDB files (I first checked write permissions in cache/ directory, owned by www-data rather than root).
From a cursory view of top, it seems that now the php process running importDump is as busy as before, but mysqld is one order of magnitude less loaded.

(In reply to Yuvi Panda from comment #3)
> Also affects testwikidatawiki

What caching system did it use?

Comment 7 Ori Livneh 2014-05-10 13:06:54 UTC

Adding libxml_disable_entity_loader(false); to the top of LocalisationCache::loadPluralFile makes the warning go away. We disable the entity loader for a reason (it's insecure), so we can't use that as a solution. But it does finger entity references as most likely being the underlying cause.

Comment 8 Andrew Bogott 2014-05-10 16:48:39 UTC

I have a fairly trivial import that fails in the same way.

https://github.com/wikimedia/operations-puppet/blob/production/modules/mediawiki_singlenode/files/privacy-policy.xml

Comment 9 This, that and the other (TTO) 2014-07-26 07:40:24 UTC

Apparently caused by the fact that libxml_disable_entity_loader is not thread-safe. Great one, PHP!

See https://bugs.php.net/bug.php?id=64938 and http://pyd.io/f/topic/failed-to-load-external-entity-boot-confmanifest-xml

Comment 10 Gerrit Notification Bot 2014-07-26 07:41:33 UTC

Change 149507 had a related patch set uploaded by TTO:
Use file_get_contents to load plurals XML data

https://gerrit.wikimedia.org/r/149507

Comment 11 This, that and the other (TTO) 2014-07-26 07:43:10 UTC

*** Bug 65116 has been marked as a duplicate of this bug. ***

Comment 12 Bartosz Dziewoński 2014-09-19 15:34:43 UTC

According to comments on the patch, it is not clear if it actually fixes the issue or not.

Comment 13 This, that and the other (TTO) 2014-09-19 22:40:31 UTC

Not sure if any of those commenters actually tested it...

Note You need to log in before you can comment on or make changes to this bug.

Wikimedia Bugzilla is closed!

Search

Personal tools

Navigation

Links