Last modified: 2012-08-04 20:48:46 UTC

Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 33663 - failure of PHPUnit tests for revision 108474 under postgres
failure of PHPUnit tests for revision 108474 under postgres
Status: RESOLVED FIXED
Product: MediaWiki
Classification: Unclassified
Unit tests (Other open bugs)
1.20.x
All All
: Normal normal (vote)
: ---
Assigned To: Antoine "hashar" Musso (WMF)
:
Depends on:
Blocks: postgres
  Show dependency treegraph
 
Reported: 2012-01-11 17:53 UTC by Dan Nessett
Modified: 2012-08-04 20:48 UTC (History)
3 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments
Output of make safe (35.06 KB, text/plain)
2012-01-11 17:54 UTC, Dan Nessett
Details
Output of 2nd make safe run (35.18 KB, text/plain)
2012-01-12 17:55 UTC, Dan Nessett
Details
make safe failures for MW r108821 (34.87 KB, text/plain)
2012-01-13 18:19 UTC, Dan Nessett
Details
TestSuite.php with embedded debug statements (27.71 KB, application/octet-stream)
2012-01-18 21:35 UTC, Dan Nessett
Details
Make Safe Output from r109424 (36.41 KB, text/plain)
2012-01-18 21:36 UTC, Dan Nessett
Details
Make safe run with debug information created by TestSuite.php.modified (147.11 KB, text/plain)
2012-01-18 21:37 UTC, Dan Nessett
Details
Parser Tests only with debug information from TestSuite.php.modified (26.24 KB, text/plain)
2012-01-18 21:38 UTC, Dan Nessett
Details
Run safe with strict equals false in suite.xml (11.10 KB, text/plain)
2012-01-19 21:48 UTC, Dan Nessett
Details

Description Dan Nessett 2012-01-11 17:53:08 UTC
I installed PHPUnit and ran the safe PHPUnit tests (i.e., executed make safe in tests). This produced 27 failures, 14 incomplete tests and 13 skipped tests. I will attach the output in a subsequent comment.

The environment for the run was:

OS: CentOS 5.7
MW revision: 108474
PHP: 5.3.3
PHPUnit: 3.6.7
DB: Postgres 8.3.9
Comment 1 Dan Nessett 2012-01-11 17:54:14 UTC
Created attachment 9842 [details]
Output of make safe

The output from the make safe run
Comment 2 Sam Reed (reedy) 2012-01-11 18:07:52 UTC
I'm guessing most of this is going to be postgres related issues

http://integration.mediawiki.org/ci/job/MediaWiki-phpunit/ is fine with SQLite, and MySQL locally for me is also fine
Comment 3 Chad H. 2012-01-11 19:05:19 UTC
(In reply to comment #2)
> I'm guessing most of this is going to be postgres related issues
> 

Worth fixing, at least :) The skipped tests all look fine (skipping Sqlite tests due to missing support, etc). I'm kind of concerned about the "Aborted execution after 1 second" errors failures though.
Comment 4 Dan Nessett 2012-01-12 17:54:28 UTC
Acoording to http://www.gossamer-threads.com/lists/wiki/wikitech/267086, there are now only 2 tests failing in latest trunk for postgres based runs. However, I just ran make safe locally and get 1 error, 25 failures and 12 incomplete tests. I am attaching the output in a separate comment.

Same environment, except the MW revision is 108734
Comment 5 Dan Nessett 2012-01-12 17:55:01 UTC
Created attachment 9850 [details]
Output of 2nd make safe run
Comment 6 Dan Nessett 2012-01-12 18:07:21 UTC
(In reply to comment #4)
> Acoording to http://www.gossamer-threads.com/lists/wiki/wikitech/267086, there
> are now only 2 tests failing in latest trunk for postgres based runs. However,
> I just ran make safe locally and get 1 error, 25 failures and 12 incomplete
> tests. I am attaching the output in a separate comment.
> 
> Same environment, except the MW revision is 108734

Ignore this. I forgot to run update. After doing so, I only get 2 failures.
Comment 7 Antoine "hashar" Musso (WMF) 2012-01-12 21:27:10 UTC
I am considering this bug fixed. Jenkins is now running tests against a postgreSQL backend so we get IRC notification like :

Project MediaWiki-postgres-phpunit build #23: STILL FAILING in 3 min 43 sec: http://integration.mediawiki.org/ci/job/MediaWiki-postgres-phpunit/23/

That is enough to remember everyone to have a look at it :-b


Thanks for the bug report. You made me add the Jenkins job!
Comment 8 Dan Nessett 2012-01-13 18:18:27 UTC
I thought I would look at the one remaining failure in the make safe phpunit tests for postgres. I did a svn up and then ran update. When I ran make safe, 24 failures showed up. I am attaching the make safe output in a separate comment. The revision is: 108821
Comment 9 Dan Nessett 2012-01-13 18:19:12 UTC
Created attachment 9853 [details]
make safe failures for MW r108821
Comment 10 Dan Nessett 2012-01-13 20:17:34 UTC
I found the problem causing ApiQueryTest::testTitlesGetNormalized to fail, but I don't know exactly how to fix it.

ApiQuery:execute() calls:
$this->mPageSet->execute() (line 254) calls:
$this->initFromTitles( $params['titles'] ) (line 303) calls:
$linkBatch = $this->processTitlesArray( $titles ) (line 414) calls:
$unconvertedTitle = $titleObj->getPrefixedText() (line 678) calls:
str_replace( '_', ' ', $s ) (line 1127)

In the test case, $wgMetaNamespace is left at its default set in DefaultSettings.php (false) and so is set to the value of $wgSitename, which in the test case is "MW_latest". The underscore is being replaced by a blank character, so the test fails when comparing the 'to' array entry expected and that returned.

One way to fix this is to change the test by replacing all spaces in the returned 'to' entry namespace prefix  with underscores, but I suspect this just covers over a bug. So, somewhere in the code, there is an assumption made that $wgMetaNamespace is a string without spaces/underscores. This assumption should be removed.

Just for the record, I checked and underscores are legal in namespace prefixes (http://www.mediawiki.org/wiki/Manual:Title.php).
Comment 11 Dan Nessett 2012-01-18 21:34:49 UTC
Things are getting worse. The make safe tests now throw an MW exception (attached Make Safe r109424.txt). Surprisingly, this actually gives a hint to what may be going wrong.

I instrumented TestSuit.php (attached TestSuite.php.modified) to print out $wgLanguageCode on each test run. When I run make safe, its value changes to "zh" on the 429th ParserTest (attached Full Safe Tests Run.txt) and stays that way for the rest of the run. I couldn't figure out why until I looked at the incomplete tests. The 20th incomplete test changes the language to "zh" and then fails after 1 second. So, my guess is there is no proper unwinding of incomplete tests and they leave globals in a non-initialized state.

I then looked at the first incomplete test. It is the first test run (ArticleTablesTest::testbug14404). It times out after 1 second. So, there is the distinct possibility that globals are being corrupted by the first test and this problem propagates through the whole run.

This is semi-confirmed when I ran only the parser tests, i.e., ./phpunit.php --group Parser (attached Parser Tests Only Run.txt). All tests pass and $wgLanguageCode remains at "en" through out the run.

I don't how to address the problem of tests aborting and corrupting globals, but I seem to recall phpunit has an option that saves and restores there values. Perhaps that option isn't being used because it would lengthen the time it takes to run the tests.
Comment 12 Dan Nessett 2012-01-18 21:35:34 UTC
Created attachment 9863 [details]
TestSuite.php with embedded debug statements
Comment 13 Dan Nessett 2012-01-18 21:36:22 UTC
Created attachment 9864 [details]
Make Safe Output from r109424
Comment 14 Dan Nessett 2012-01-18 21:37:00 UTC
Created attachment 9865 [details]
Make safe run with debug information created by TestSuite.php.modified
Comment 15 Dan Nessett 2012-01-18 21:38:09 UTC
Created attachment 9866 [details]
Parser Tests only with debug information from TestSuite.php.modified
Comment 16 Platonides 2012-01-18 23:55:59 UTC
(In reply to comment #11)
> I instrumented TestSuit.php (attached TestSuite.php.modified) to print out
> $wgLanguageCode on each test run. When I run make safe, its value changes to
> "zh" on the 429th ParserTest (attached Full Safe Tests Run.txt) and stays that
> way for the rest of the run. I couldn't figure out why until I looked at the
> incomplete tests. The 20th incomplete test changes the language to "zh"...

Tests changing the language should extend MediaWikiLangTestCase (which saves & restores it), not MediaWikiTestCase.


> and then fails after 1 second. So, my guess is there is no proper unwinding of
> incomplete tests and they leave globals in a non-initialized state.

What I wonder is, what makes your tests to abort after 1 second?
I have never seen that myself.
Comment 17 Dan Nessett 2012-01-19 21:00:49 UTC
(In reply to comment #16)
> (In reply to comment #11)
> > I instrumented TestSuit.php (attached TestSuite.php.modified) to print out
> > $wgLanguageCode on each test run. When I run make safe, its value changes to
> > "zh" on the 429th ParserTest (attached Full Safe Tests Run.txt) and stays that
> > way for the rest of the run. I couldn't figure out why until I looked at the
> > incomplete tests. The 20th incomplete test changes the language to "zh"...
> 
> Tests changing the language should extend MediaWikiLangTestCase (which saves &
> restores it), not MediaWikiTestCase.
> 

The test that changes the language is the ParserTest, which runs a whole bunch of sub-tests of the parser functionality. Most of these do not change the language. I still don't quite understand how the parser tests are run within the PHPUnit framework, so I don't know if one PHPUnit test runs all parser tests or each test runs under a separate instance of MediaWikiTestCase. In the latter case, it makes sense to run those tests that change the language out of MediaWikiLangTestCase. In the former case, it may or may not make sense

> 
> > and then fails after 1 second. So, my guess is there is no proper unwinding of
> > incomplete tests and they leave globals in a non-initialized state.
> 
> What I wonder is, what makes your tests to abort after 1 second?
> I have never seen that myself.

I was hoping someone would indicate what might cause such timeouts and suggest a way to fix the problem. I have no idea why they are aborting after 1 second.
Comment 18 Antoine "hashar" Musso (WMF) 2012-01-19 21:22:11 UTC
The timeout messages comes from PHPUnit. It is most probably caused because PHP_Invoker is installed ( https://github.com/sebastianbergmann/php-invoker  ) which make PHPUnit think our tests are small and should run in less than a second.

By looking at the code of PHPUnit, this behavior only happens while in strict mode (by looking at PHPUnit/Framework/TestResult.php. Can you try editing MediaWiki file "tests/phpunit/suite.xml", in the very first few lines you should have something looking like:
 strict="true"

replace it by:
  strict="false"

That should disable strict node and the whole timeout system. If that solve the issue, we would have to find a way to disable that system entirely.
Comment 19 Antoine "hashar" Musso (WMF) 2012-01-19 21:23:58 UTC
TIP: to show tests names, you can run the suite using --tap :

cd tests/phpunit
php phpunit.php --tap
TAP version 13

ok 1 - ArticleTablesTest::testbug14404
ok 2 - ArticleTest::testImplementsGetMagic
ok 3 - ArticleTest::testImplementsSetMagic
ok 4 - ArticleTest::testImplementsCallMagic
ok 5 - ArticleTest::testGetOrSetOnNewProperty
ok 6 - ArticleTest::testStaticFunctions
ok 7 - ArticleTest::testWikiPageFactory
ok 8 - BlockTest::testInitializerFunctionsReturnCorrectBlock
ok 9 - BlockTest::testBug26425BlockTimestampDefaultsToTime
ok 10 - BlockTest::testBug29116LoadWithEmptyIp with data set #0 (NULL)
ok 11 - BlockTest::testBug29116LoadWithEmptyIp with data set #1 ('')
ok 12 - BlockTest::testBug29116LoadWithEmptyIp with data set #2 (false)
ok 13 - BlockTest::testBug29116NewFromTargetWithEmptyIp with data set #0 (NULL)
ok 14 - BlockTest::testBug29116NewFromTargetWithEmptyIp with data set #1 ('')
Comment 20 Dan Nessett 2012-01-19 21:48:26 UTC
Created attachment 9873 [details]
Run safe with strict equals false in suite.xml
Comment 21 Dan Nessett 2012-01-19 21:50:46 UTC
(In reply to comment #18)
> The timeout messages comes from PHPUnit. It is most probably caused because
> PHP_Invoker is installed ( https://github.com/sebastianbergmann/php-invoker  )
> which make PHPUnit think our tests are small and should run in less than a
> second.
> 
> By looking at the code of PHPUnit, this behavior only happens while in strict
> mode (by looking at PHPUnit/Framework/TestResult.php. Can you try editing
> MediaWiki file "tests/phpunit/suite.xml", in the very first few lines you
> should have something looking like:
>  strict="true"
> 
> replace it by:
>   strict="false"
> 
> That should disable strict node and the whole timeout system. If that solve the
> issue, we would have to find a way to disable that system entirely.

That did the trick. There is now only 1 failure. That is easily fixed if the correct tactic is to normalize the expected output on the assert (see Comment 10). If so, someone speak up and I will make the necessary change.

I have attached the output of make safe with strict="false" in suite.mxl.
Comment 22 Antoine "hashar" Musso (WMF) 2012-01-30 13:03:56 UTC
Opened bug 34037 to make our suite works when PHP_Invoker is installed.
Comment 23 Antoine "hashar" Musso (WMF) 2012-01-30 15:57:09 UTC
 Array (
     'from' => 'Project:articleA'
-    'to' => 'MW_latest:ArticleA'
+    'to' => 'MW latest:ArticleA'
 )

This test use $wgMetaNamespace which contains space/underscore on your project. The ApiQueryTest::testTitlesGetNormalized() test assumed it to be normalized and thus failed whenever that is no the case.

r110298 fix that issue (which has nothing to do with postgres anyway).


Thanks Dan for all the bug reporting you did so far on this bug. Since this last issue is solved, I am marking it as fixed. You probably want to open a new bug report if you have anymore issues.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links