Last modified: 2014-10-12 18:16:21 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T66600, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 64600 - write script that touches every item and thereby updates database tables
write script that touches every item and thereby updates database tables
Status: NEW
Product: MediaWiki extensions
Classification: Unclassified
WikidataRepo (Other open bugs)
master
All All
: High normal (vote)
: ---
Assigned To: Wikidata bugs
u=dev c=backend p=13
:
Depends on:
Blocks: 52385
  Show dependency treegraph
 
Reported: 2014-04-29 12:41 UTC by Lydia Pintscher
Modified: 2014-10-12 18:16 UTC (History)
8 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Lydia Pintscher 2014-04-29 12:41:29 UTC

    
Comment 1 Marius Hoch 2014-05-01 20:12:56 UTC
Is this about updating all tables in one run? What purpose does this have (in contrast to the recently merged rebuildItemsPerSite.php)
Comment 2 Marius Hoch 2014-05-03 22:40:08 UTC
Should this really do null-edits (which at times can show up in the items history) or rather only run all of the secondary database updates? I guess secondary storage updates are enough, thus the bug title is slightly incorrect.
Comment 3 Aude 2014-05-07 12:00:46 UTC
if we can do just secondary updates without the null edits, that's what we need and want, imho.

also, I would very much like the ability to specify which secondary updates to do, in cases where we don't want to do all of them.
Comment 4 Aude 2014-05-07 12:01:14 UTC
e.g. things like hitting CirrusSearch with unnecessary updates wouldn't be nice
Comment 5 Daniel Kinzler 2014-05-13 14:32:11 UTC
This is a pretty generic script, and should support a couple of modes for different use cases. I'm not saying all of these need to be implemented right now, just that we should have them in mind when designing this.

So, for a list of page IDs (optionally, page titles, possibly from a DB query), either:

* just invalidate the parser cache by calling Title::invalidateCache()
* re-parse (null-edit) 
** re-parse later (using RefreshLinksJob2 or similar)
* re-apply data updates (using Content::getSecondaryDataUpdates(), using cached ParserOutput object if possible)
** would be nice to be able to filter the updates by class name

The different modes should be implemented by separate classes that get called by the main script which iterates over ranges of target pages (strategy pattern).

On the command line, it would perhaps be nice to use verbs to describe these modes, e.g. call this as "updatePages.php invalidate", "updatePages.php parse-later", "updatePages.php update-secondary", etc.
Comment 6 Gerrit Notification Bot 2014-06-16 04:55:30 UTC
Change 139768 had a related patch set uploaded by Legoktm:
Add simple script to "touch" pages

https://gerrit.wikimedia.org/r/139768
Comment 7 Kunal Mehta (Legoktm) 2014-06-16 05:00:01 UTC
(In reply to Daniel Kinzler from comment #5)

> ** re-parse later (using RefreshLinksJob2 or similar)

RefreshLinksJob2 has been deprecated since 1.23, and I'm not really sure what's the preferred way to do it now. I implemented the 3 other types though.
Comment 8 tobias.gritschacher 2014-06-16 12:41:08 UTC
Per hooman: "We had that discussion on IRC: As null edits can cause changes in the serialization they *can* show up in the edit history, thus we don't want them (AFAIR)."
Comment 9 tobias.gritschacher 2014-08-19 10:10:52 UTC
Needs more discussion.
Comment 10 Gerrit Notification Bot 2014-09-08 07:49:11 UTC
Change 139768 abandoned by Legoktm:
Add simple script to "touch" pages

Reason:
Don't have time to work on this, feel free to restore if the code is useful.

https://gerrit.wikimedia.org/r/139768

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links