Last modified: 2008-06-08 16:16:41 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T14946, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 12946 - getUnusedPropertiesSpecial has serious performance problems
getUnusedPropertiesSpecial has serious performance problems
Status: RESOLVED FIXED
Product: MediaWiki extensions
Classification: Unclassified
Semantic MediaWiki (Other open bugs)
unspecified
All All
: Normal normal (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2008-02-06 22:01 UTC by Harold Solbrig
Modified: 2008-06-08 16:16 UTC (History)
1 user (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments
Patch for revision 30649 of SMW_SQLStore.php (3.57 KB, patch)
2008-02-06 22:01 UTC, Harold Solbrig
Details

Description Harold Solbrig 2008-02-06 22:01:49 UTC
Created attachment 4626 [details]
Patch for revision 30649 of SMW_SQLStore.php

getUnusedPropertiesSpecial in SMW_SQLStore.php exhibits horrible performance when the number of attributes and/or relations is large (>200,000).  The outer joins result in several million tuples needing to be examined to select the non-matching fields and this can take many (many!) minutes, even on large processors.

The attached patch has a proposed alternative - instead of outer joins, we create a temporary table which contains all of the properties and then delete rows from the table that occur in the relations, attributes, ... tables instead.
Comment 1 Markus Krötzsch 2008-06-08 16:16:41 UTC
For large wikis, this special page should probably be disabled or cached (as all slow Special pages are on large wikis). I have applied the above patch now, and SMW's new storage implementation (to be released soon) will also use the above method with temporary queries. 

I still note that, in principle, this can also fail if there are really a lot of properties. One certainly would not wish a temporary table with 10000s of entries (with VARCHAR(255) one can expect about 64000 entries to be the absolute maximum for a temporary table in standard MySQL settings).

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links