Last modified: 2014-04-04 15:55:28 UTC

Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 57724 - PostgreSQL / PHPUnit tests / MW 1.21.0 / $wikiPage->doDeleteArticle causes DatabasePostgres::reportQueryError: No transaction to rollback, something got out of sync
PostgreSQL / PHPUnit tests / MW 1.21.0 / $wikiPage->doDeleteArticle causes Da...
Status: NEW
Product: MediaWiki
Classification: Unclassified
Database (Other open bugs)
1.21.x
All All
: Normal normal (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on: 47055
Blocks: postgres
  Show dependency treegraph
 
Reported: 2013-11-28 21:47 UTC by MWJames
Modified: 2014-04-04 15:55 UTC (History)
3 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description MWJames 2013-11-28 21:47:42 UTC
Running unit tests [1] on Travis-CI [1] with PostgreSQL 9.1.x and MW 1.21.x is failing with

## Error
DatabasePostgres::reportQueryError: No transaction to rollback, something got out of sync! [Called from DatabaseBase::rollback in /home/travis/build/SemanticMediaWiki/phase3/includes/db/Database.php at line 3107]

## Cause
The doDeleteArticle() statement in the test is causing the test to break and fail with the above message.

if ( $wikiPage->exists() ) {
	$wikiPage->doDeleteArticle( "testing done on " . $on );
}

[1] https://github.com/SemanticMediaWiki/SemanticMediaWiki/pull/37

[2] https://travis-ci.org/SemanticMediaWiki/SemanticMediaWiki/builds/14663900

MySQL and SQLite tests which run the exact same tests do not fail which indicates an issues during the execution of doDeleteArticle().

## Trace
1) SMW\Test\MediaWikiFunctionHookIntegrationTest::testOnArticlePurgeOnDatabase
DatabasePostgres::reportQueryError: No transaction to rollback, something got out of sync! [Called from DatabaseBase::rollback in /home/travis/build/SemanticMediaWiki/phase3/includes/db/Database.php at line 3107]

/home/travis/build/SemanticMediaWiki/phase3/includes/debug/Debug.php:283
/home/travis/build/SemanticMediaWiki/phase3/includes/debug/Debug.php:144
/home/travis/build/SemanticMediaWiki/phase3/includes/GlobalFunctions.php:1105
/home/travis/build/SemanticMediaWiki/phase3/includes/db/Database.php:3107
/home/travis/build/SemanticMediaWiki/phase3/includes/db/DatabasePostgres.php:481
/home/travis/build/SemanticMediaWiki/phase3/includes/db/Database.php:983
/home/travis/build/SemanticMediaWiki/phase3/includes/db/Database.php:1434
/home/travis/build/SemanticMediaWiki/phase3/includes/Revision.php:372
/home/travis/build/SemanticMediaWiki/phase3/includes/Revision.php:314
/home/travis/build/SemanticMediaWiki/phase3/includes/Revision.php:294
/home/travis/build/SemanticMediaWiki/phase3/includes/Revision.php:143
/home/travis/build/SemanticMediaWiki/phase3/includes/WikiPage.php:575
/home/travis/build/SemanticMediaWiki/phase3/includes/WikiPage.php:615
/home/travis/build/SemanticMediaWiki/phase3/includes/WikiPage.php:2543
/home/travis/build/SemanticMediaWiki/phase3/includes/WikiPage.php:2477
/home/travis/build/SemanticMediaWiki/phase3/extensions/SemanticMediaWiki/tests/phpunit/integration/MediaWikiFunctionHookIntegrationTest.php:256
/home/travis/build/SemanticMediaWiki/phase3/extensions/SemanticMediaWiki/tests/phpunit/integration/MediaWikiFunctionHookIntegrationTest.php:140
/home/travis/build/SemanticMediaWiki/phase3/tests/phpunit/MediaWikiTestCase.php:116
/home/travis/build/SemanticMediaWiki/phase3/tests/phpunit/MediaWikiPHPUnitCommand.php:64
/home/travis/build/SemanticMediaWiki/phase3/tests/phpunit/MediaWikiPHPUnitCommand.php:48
Comment 1 José Antonio 2013-11-30 00:21:38 UTC
23:07:05Fri 29 Nov 13 23:07:07



The the fields to be use in outer joins need to be defined as not null. Then  PostgreSQL can enforce data consistency without falling the outer left join transaction. 

The root of the problem  can be found in the the PostgreSQL log.

The test from where the fail arises is: public function testOnArticlePurgeOnDatabase()
in line 121 of  
SemanticMediaWiki/tests/phpunit/integration/MediaWikiFunctionHookIntegrationTest.php.


GMT ERROR:  
SELECT FOR UPDATE/SHARE cannot be applied to the nullable side of an outer join.

This means in this context that the data fields used in the left join must be not nullable:
{
LEFT JOIN "unittest_mwuser" ON ((rev_user != 0) AND (user_id = rev_user))  WHERE page_id = '612' AND rev_id = '892'  LIMIT 1   FOR UPDATE
}

ref: http://www.postgresql.org/message-id/flat/98882B9F-5C37-4C5B-9B15-5E51D8767CF2@socialserve.com
http://postgresql.1045698.n5.nabble.com/outer-joins-and-for-update-td1937029.html


GMT ERROR:  
SELECT FOR UPDATE/SHARE cannot be applied to the nullable side of an outer join.

GMT STATEMENT:  
SELECT 

rev_id,rev_page,rev_text_id,rev_timestamp,rev_comment,rev_user_text,rev_user,rev_minor_edit,rev_deleted,rev_len,rev_parent_id,rev_sha1,rev_content_format,rev_content_model,page_namespace,page_title,page_id,page_latest,page_is_redirect,page_len,user_name  

FROM "unittest_revision" INNER JOIN "unittest_page" ON ((page_id = rev_page)) 



It seems that unit test reporting hides the real root of the problem that can be identified in the PostgreSQL log .Then in future It seems that it would very useful to include this log.
On a debian system the standard location is in:  /var/log/postgresql/postgresql-9.1-main.log. Thus perhaps including the output of 
tail -20  /var/log/postgresql/postgresql-9.1-main.log  with unit test fails report could be something easy to achieve.
Comment 2 Marcin Cieślak 2013-11-30 12:37:33 UTC
From above analysis it looks to be a dupe of bug 47055, adding dependency for now.

Be also aware of a quite nasty bug 37702, but I suppose it affects SMW to the lesser extent.
Comment 3 José Antonio 2013-11-30 14:21:51 UTC
Thanks for the link to 47055 and pointing to 37702.

It seems that all the problems related to postgresql are rooted on   creating  data sets that are not compatible with the operations that smw wants to excute on them.

Indeed in this case mysql seems to enable outer joins on nullables.

Here It seems that the most  more straight approach would be set “not null”  for fields that are used in outer joins.

Regards.

J.A.
Comment 4 José Antonio 2013-12-04 13:44:39 UTC
Hi,

According to my understanding the root of this bug is in
is in function selectSQLText( $table, $vars, $conds, $fname, $options, $join_conds ); starting in line 1453 of file  includes/db/Database.php.

In my understanding this function should  check if  an update on a select with outer joint is going to be requested . Thus avoiding the” select for update/share cannot be applied to the nullable side of an outer join.”  exception  thrown by postgresql.

This the secuence of calls that it seems to happen from 
/
SemanticMediaWiki/tests/phpunit/integration/MediaWikiFunctionHookIntegrationTest.php
 testOnArticlePurgeOnDatabase();

$this->deletePage( $wikiPage, __METHOD__ )  starts an “forupdate” process.
$wikiPage->doDeleteArticle( 
doDeleteArticleReal

[file includes/Hooks.php]
wfRunHooks( 'ArticleDelete',    array( &$this wikipage,
                 &$user, &$reason, &$error, &$status ) ) 
$hooks = self::getHandlers( ArticleDelete );

[file includes/wikipage.php]
loadPageData( 'forupdate' );
pageDataFromTitle( wfGetDB( DB_MASTER ), $this->mTitle, array( 'FOR UPDATE' ) );

[includes/db/LoadBalancer.php]
connection -->db->last query lef joint → reused for update

[file includes/wikipage.php]

 pageDataFromTitle
 pageData

[file includes/db/Database.php]
selectRow( 'page', $fields, $conditions, __METHOD__, $options );
selectSQLText( $table, $vars, $conds, $fname, $options, $join_conds );

Regards.

J.A.
Comment 5 José Antonio 2013-12-04 13:49:30 UTC
(In reply to comment #3)
> Thanks for the link to 47055 and pointing to 37702.
> 
> It seems that all the problems related to postgresql are rooted on  
> creating 
> data sets that are not compatible with the operations that smw wants to
> excute
> on them.
> 
> Indeed in this case mysql seems to enable outer joins on nullables.
> 
> Here It seems that the most  more straight approach would be set “not null” 
> for fields that are used in outer joins.
> 
> Regards.
> 
> J.A.


Setting a non null  constraint for all the fields of the tables  involved in the outer joint does not solve the problem.
Comment 6 José Antonio 2013-12-04 13:53:01 UTC
(In reply to comment #4)
> Hi,
> 
> According to my understanding the root of this bug is in
> is in function selectSQLText( $table, $vars, $conds, $fname, $options,
> $join_conds ); starting in line 1453 of file  includes/db/Database.php.
> 
> In my understanding this function should  check if  an update on a select
> with
> outer joint is going to be requested . Thus avoiding the” select for
> update/share cannot be applied to the nullable side of an outer join.” 
> exception  thrown by postgresql.
> 
> This the secuence of calls that it seems to happen from 
> /
> SemanticMediaWiki/tests/phpunit/integration/
> MediaWikiFunctionHookIntegrationTest.php
>  testOnArticlePurgeOnDatabase();
> 
> $this->deletePage( $wikiPage, __METHOD__ )  starts an “forupdate” process.
> $wikiPage->doDeleteArticle( 
> doDeleteArticleReal
> 
> [file includes/Hooks.php]
> wfRunHooks( 'ArticleDelete',    array( &$this wikipage,
>                  &$user, &$reason, &$error, &$status ) ) 
> $hooks = self::getHandlers( ArticleDelete );
> 
> [file includes/wikipage.php]
> loadPageData( 'forupdate' );
> pageDataFromTitle( wfGetDB( DB_MASTER ), $this->mTitle, array( 'FOR UPDATE' )
> );
> 
> [includes/db/LoadBalancer.php]
> connection -->db->last query lef joint → reused for update
> 
> [file includes/wikipage.php]
> 
>  pageDataFromTitle
>  pageData
> 
> [file includes/db/Database.php]
> selectRow( 'page', $fields, $conditions, __METHOD__, $options );
> selectSQLText( $table, $vars, $conds, $fname, $options, $join_conds );
> 
> Regards.
> 
> J.A.

If the real problem is in function  selectSQLText then perhaps a bug should be raised   in mediawiki...
Comment 7 Marcin Cieślak 2013-12-05 10:02:54 UTC
Looks like related to bug 36759, bug 37600 and unfortunately bug 37702 to me.
Comment 8 Marcin Cieślak 2013-12-06 14:25:26 UTC
Can you run the test again with 4b291909e0e91ad4e8ed98030c1312a872ca3bd4 reverted and post the error message again? (see bug 58095)
Comment 9 MWJames 2013-12-06 18:18:20 UTC
I can't because the current MW master is even more broken for PostgerSQL because of [1]. 

[1] Could not insert main page: A database error has occurred. Did you forget to run maintenance/update.php after upgrading?  See: https://www.mediawiki.org/wiki/Manual:Upgrading#Run_the_update_script
Query: INSERT INTO "recentchanges" (rc_timestamp,rc_namespace,rc_title,rc_type,rc_source,rc_minor,rc_cur_id,rc_user,rc_user_text,rc_comment,rc_this_oldid,rc_last_oldid,rc_bot,rc_ip,rc_patrolled,rc_new,rc_old_len,rc_new_len,rc_deleted,rc_logid,rc_log_type,rc_log_action,rc_params,rc_id) VALUES ('2013-12-05 20:33:02 GMT','0','Main_Page','1','mw.new','0','1','0','MediaWiki default',,'1','0','0','127.0.0.1','0','1','0','524','0','0',NULL,,,'1')
Function: RecentChange::save
Error: 23502 ERROR:  null value in column "rc_cur_time" violates not-null constraint

[2] https://s3.amazonaws.com/archive.travis-ci.org/jobs/15003959/log.txt

PS: I do wonder if someone on the WMF site really runs tests (it's the second time this week Travis fails because of deployed core incompatibilities).
Comment 10 Marcin Cieślak 2013-12-07 16:27:37 UTC
Yes, this is bug 40744 gone bad. The fix is in gerrit change I66034fad8a1cf5485b5daf0421378a28ca58beab but somehow having pains in review :)


I have some more interesting PostgreSQL fixes in the pipeline:

Gerrit change #99349
Gerrit change #99640
Gerrit change #99648
Gerrit change #99676  
Gerrit change #100141
Gerrit change #100154

feel free to try them as all of them are still in the works.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links