Last modified: 2014-11-20 22:31:44 UTC
Created attachment 12471 [details] SMWQueryProcessor without $params['offset']['upperbound']. Running an ask query with an offset greater than 5000 returns the results from the query with offset=0 instead. Because of this, the only way to reliably fetch all results for large queries is to set offset=0, limit=<very_large_number> and fetch all results at once. This is less desirable for the user as well as the site host than "paging through" large queries by specifying a limit and incrementing the offset by the limit size for every page queried. But limiting the offset in any way makes this preferred scenario impossible. The code responsible is the 'upperbound' setting for $params['offset'] in SemanticMediaWiki/includes/SMW_QueryProcessor.php at line 601: $params['offset'] = array( 'type' => 'integer', 'default' => 0, 'negatives' => false, 'upperbound' => 5000 // TODO: make setting ); My guess is that the upperbound was intended for $params['limit'], but was added to $params['offset'] by mistake. The solution is not to make 'upperbound' configurable with a global variable as the comment implies, but simply to remove this line altogether (see attachment).
I've now hit this issue a few times, but just now did the debugging to figure out what was going on. I've been recently trying Yaron's new Miga [1] project and have been trying to use it with the Wikinosh [2] dataset. Wikinosh has 21,000+ food items and I hit this bug trying to page through results and export everything to Miga [3]. [1] http://migadv.com [2] http://wikinosh.com/wiki/Main_Page [3] https://github.com/yaronkoren/miga/issues/4
This bug is now recorded on GitHub as issue 242 [1] [1] https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/242
This bug was fixed with pull request 637 [1]. Thus closing as RESOLVED FIXED. [1] https://github.com/SemanticMediaWiki/SemanticMediaWiki/pull/637