Last modified: 2012-08-04 20:48:41 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T13080, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 11080 - Importing some pages fails because of deadlockLoop (using postgres)
Importing some pages fails because of deadlockLoop (using postgres)
Product: MediaWiki
Classification: Unclassified
Special pages (Other open bugs)
All All
: Normal normal (vote)
: ---
Assigned To: Nobody - You can work on this!
Depends on:
Blocks: postgres
  Show dependency treegraph
Reported: 2007-08-27 07:59 UTC by Ben Jencks
Modified: 2012-08-04 20:48 UTC (History)
1 user (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Description Ben Jencks 2007-08-27 07:59:34 UTC
Short version: certain pages from an XML dump (I haven't figured out any distinguishing characteristics) fail to import. Removing the call to deadlockLoop in SpecialImport.php fixes the problem.

Background: I'm attempting to migrate a wiki from UseMod wiki to MediaWiki. Luckily, MediaWiki includes a script to dump a UseMod wiki into MediaWiki's import/export format. I used this script, it worked without problems, and I now have a dump.

Now the problem: importing the dump. Running importDump.php with the dump file gives an error, so I split the dump into individual pages in order to try to isolate the problem. About a quarter of the pages imported successfully, the rest failed with the same error:

Warning: pg_query(): Query failed: ERROR:  current transaction is aborted, commands ignored until end of transaction block in /mnt/home/brj/public_html/fw/includes/DatabasePostgres.php on line 499
SQL error: ERROR:  current transaction is aborted, commands ignored until end of transaction block
#0 /mnt/home/brj/public_html/fw/includes/LinksUpdate.php(542): DatabasePostgres->fetchObject(false)
#1 /mnt/home/brj/public_html/fw/includes/LinksUpdate.php(91): LinksUpdate->getExistingImages()
#2 /mnt/home/brj/public_html/fw/includes/LinksUpdate.php(77): LinksUpdate->doIncrementalUpdate()
#3 /mnt/home/brj/public_html/fw/includes/Article.php(2297): LinksUpdate->doUpdate()
#4 /mnt/home/brj/public_html/fw/includes/Article.php(2366): Article->editUpdates('''The time spen...', 'link fix', true, 1188199043, '2110', true)
#5 /mnt/home/brj/public_html/fw/includes/SpecialImport.php(366): Article->createUpdates(Object(Revision))
#6 [internal function]: WikiRevision->importOldRevision()
#7 /mnt/home/brj/public_html/fw/includes/Database.php(1861): call_user_func_array(Array, Array)
#8 /mnt/home/brj/public_html/fw/includes/SpecialImport.php(511): Database->deadlockLoop(Array)
#9 [internal function]: WikiImporter->importRevision(Object(WikiRevision))
#10 /mnt/home/brj/public_html/fw/maintenance/importDump.php(58): call_user_func(Array, Object(WikiRevision))
#11 [internal function]: BackupReader->handleRevision(Object(WikiRevision), Object(WikiImporter))
#12 /mnt/home/brj/public_html/fw/includes/SpecialImport.php(762): call_user_func_array(Array, Array)
#13 [internal function]: WikiImporter->out_revision(Resource id #31, 'revision')
#14 /mnt/home/brj/public_html/fw/includes/SpecialImport.php(426): xml_parse(Resource id #31, '<?xml version="...', 0)
#15 /mnt/home/brj/public_html/fw/maintenance/importDump.php(109): WikiImporter->doImport()
#16 /mnt/home/brj/public_html/fw/maintenance/importDump.php(96): BackupReader->importFromHandle(Resource id #30)
#17 /mnt/home/brj/public_html/fw/maintenance/importDump.php(131): BackupReader->importFromStdin()

I dug into SpecialImport.php to figure out why this was happening. Eventually I guessed that deadlockLoop might have been written with some MySQL assumptions, and this hadn't been noticed because importing is relatively rare. Since I'm doing a migration, the site I'm importing to isn't live, so I didn't think deadlocks would be an issue. So, I made the following change:

--- SpecialImport.php.orig      2007-04-21 05:13:47.000000000 -0700
+++ SpecialImport.php   2007-08-27 00:55:33.000000000 -0700
@@ -506,8 +506,9 @@
         * @private
        function importRevision( &$revision ) {
-               $dbw = wfGetDB( DB_MASTER );
-               return $dbw->deadlockLoop( array( &$revision, 'importOldRevision' ) );
+               #$dbw = wfGetDB( DB_MASTER );
+               #return $dbw->deadlockLoop( array( &$revision, 'importOldRevision' ) );
+               return $revision->importOldRevision();

And then importing worked! I'm sure this isn't the actual fix for the problem, as there was probably a reason for the deadlock avoidance, but there's a bug in there somewhere. I can provide sample pages that succeed and fail, if needed. I believe, though I haven't confirmed for sure, that it is consistent which pages succeed and which fail. The proportion has definitely been the same (1/4 succeed, 3/4 fail, out of about 2000 pages) the times I've tried. Also, when attempting to import the failed pages again, all of them fail.
Comment 1 Greg Sabino Mullane 2007-09-02 18:39:23 UTC
You probably don't want to comment that section out, as it is doing some needed work. The error returned is the error *after* the error that failed the transaction. Can you check the database logs and see what the original error was that aborted the transaction?
Comment 2 Ben Jencks 2007-09-04 02:52:39 UTC
Sep  3 19:49:59 kay postgres[32058]: [7-1] ERROR:  syntax error at or near "," at character 129
Sep  3 19:49:59 kay postgres[32058]: [7-2] STATEMENT:  INSERT /* LinksUpdate::incrTableUpdate */ INTO pagelinks (pl_from,pl_namespace,pl_title) VALUES
Sep  3 19:49:59 kay postgres[32058]: [7-3]  ('4','0','BagOfChips'),('4','0','ShowOff')
Sep  3 19:49:59 kay postgres[32058]: [8-1] ERROR:  current transaction is aborted, commands ignored until end of transaction block
Sep  3 19:49:59 kay postgres[32058]: [8-2] STATEMENT:  SELECT /* LinksUpdate::getExistingImages */  il_to  FROM imagelinks  WHERE il_from = '4'   FOR UPDATE
Comment 3 Greg Sabino Mullane 2007-09-04 03:18:02 UTC
I think this is a side effect of a recently fixed bug (#24702). Try changing 8.1 to 8.2 around line 700 of DatabasePostgres.php and see if the import works with the deadlock stuff un-uncommented.
Comment 4 Ben Jencks 2007-09-04 06:16:46 UTC
Indeed, that seems to have fixed it.

Note You need to log in before you can comment on or make changes to this bug.