Last modified: 2013-08-22 14:42:00 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T48378, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 46378 - rebuildTermSearchKey script on wikidata.org
rebuildTermSearchKey script on wikidata.org
Status: VERIFIED FIXED
Product: Wikimedia
Classification: Unclassified
Site requests (Other open bugs)
unspecified
All All
: High enhancement (vote)
: ---
Assigned To: Sam Reed (reedy)
: shell
Depends on:
Blocks: 29782
  Show dependency treegraph
 
Reported: 2013-03-20 19:24 UTC by Lydia Pintscher
Modified: 2013-08-22 14:42 UTC (History)
10 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Lydia Pintscher 2013-03-20 19:24:30 UTC
Please run the rebuildTermSearchKey script on Wikidata.
Comment 1 Daniel Kinzler 2013-03-21 08:20:40 UTC
That's extensions/Wikibase/repo/maintenance/rebuildTermsSearchKey.php, to be run on wikidatawiki.

This will rebuild the term_search_key column in the wb_terms table. That table is quite large, so this ma take a while. Database updates are done in batches, and the script waitrs for slaves to catch up before starting the next batch.
Comment 2 Greg Grossmeier 2013-03-21 16:13:31 UTC
Asher: Could
Comment 3 Greg Grossmeier 2013-03-21 21:17:11 UTC
So, I didn't mean to save that.... Asher, I think Rob emailed CT about this. Sorry for the noise...
Comment 4 Andre Klapper 2013-03-27 10:58:15 UTC
According to IRC, CT Woo asked Peter to review and needs to follow up with him.
Comment 5 Greg Grossmeier 2013-03-27 20:02:23 UTC
Per https://bugzilla.wikimedia.org/show_bug.cgi?id=46613 this has been reviewed by Asher and is OK to run.
Comment 6 Daniel Kinzler 2013-03-27 20:46:32 UTC
Great! So, here's what needs to happen:

1) run rebuildTermSearchKey.php. It will log a line after each batch, e.g.:

  Updated 500 search keys, up to row 20999.

Should the script die or be stopped for some reason, it can be restarted from the row it left off, using --start-row 20999. The batch size can be tuned using --batch-size.

When the script is done, mark the last row it reported (we'll need it later!)

2) tell the wiki to start using the index, with $wgWBRepoSettings['withoutTermSearchKey'] = false; (or rather, by removing the line from the config that forces this to true).

3) re-run rebuildTermSearchKey.php from the row it finished at, to cover items that have been added/modified between the time the script finished and the time the settings were changed.
Comment 7 Daniel Kinzler 2013-03-27 20:48:43 UTC
oh, in case we lose the info which row to restart from, use this:

  select min(term_row_id) from wb_terms where term_search_key = '';

(off the top of my head)
Comment 8 Greg Grossmeier 2013-03-27 20:51:30 UTC
(btw, that link was supposed to be http://rt.wikimedia.org/Ticket/Display.html?id=4801, not bug 46613)

Thanks Daniel.

Peter: Would this go on your plate, or someone else in Ops?
Comment 9 Sam Reed (reedy) 2013-03-27 22:00:57 UTC
reedy@hume:/home/wikipedia/common/php-1.21wmf12$ mwscript extensions/Wikibase/repo/maintenance/rebuildTermsSearchKey.php wikidatawiki
...Update 'Wikibase\RebuildTermsSearchKey' already logged as completed.
reedy@hume:/home/wikipedia/common/php-1.21wmf12$ mwscript extensions/Wikibase/repo/maintenance/rebuildTermsSearchKey.php wikidatawiki --force

Fatal error: Class 'ObservableMessageReporter' not found in /home/wikipedia/common/php-1.21wmf12/extensions/Wikibase/repo/maintenance/rebuildTermsSearchKey.php on line 59
Comment 10 Daniel Kinzler 2013-03-27 22:18:50 UTC
(In reply to comment #9)
> Fatal error: Class 'ObservableMessageReporter' not found in
> /home/wikipedia/common/php-1.21wmf12/extensions/Wikibase/repo/maintenance/
> rebuildTermsSearchKey.php
> on line 59

Looks like we forgot to make that class available in production. Fixed in I6a1a7244975.
Comment 11 Andre Klapper 2013-04-03 22:45:25 UTC
Now that I6a1a7244975 is merged Reedy might try again?
Comment 12 Andre Klapper 2013-04-08 15:28:10 UTC
Reedy: Could you try again?
Comment 13 Sam Reed (reedy) 2013-04-09 22:17:57 UTC
(In reply to comment #12)
> Reedy: Could you try again?

I did try before. Got to 25-30% completion, but then was cancelled due to replication lag issues.

Seems to be working ok again..

reedy@hume:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildTermsSearchKey.php wikidatawiki --only-missing --force
Updated 100 search keys, up to row 22873110.
Updated 100 search keys, up to row 22873222.
Updated 100 search keys, up to row 22873330.
Comment 14 Sam Reed (reedy) 2013-04-18 16:46:10 UTC
Done!

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links