Wikimedia Bugzilla is closed!

Belay that, {{tracked}} works.

As does the fr implementation. I guess the question here is "how long does it take for TemplateData implementations to show up?"

(In reply to comment #2)
> As does the fr implementation. I guess the question here is "how long does it
> take for TemplateData implementations to show up?"

Do you mean,

* "how long does it take newly-added TemplateData hinting to become available in the API for use in VisualEditor?",
* "how long does it take the VisualEditor transclusions dialog to show with the TemplateData hinting when adding a template in?", or
* "how long does it take the VisualEditor transclusions dialog to show with the TemplateData hinting on load?"

… or something else?

I'd go with A and C, Chris. If I add TemplateData to a template, go to an implementation of it that already exists and click the jigsaw piece, how long do I need between those actions to see the TemplateData?

(In reply to comment #4)
> I'd go with A and C, Chris. If I add TemplateData to a template, go to an
> implementation of it that already exists and click the jigsaw piece, how long
> do I need between those actions to see the TemplateData?

A: This should be instantaneous - if not, action=purge on the template should do it; if not, this is an issue we need to follow-up in general with how MW works.
B: This is not yet done (bug 49778 which is being worked on right now).
C: This should be near-instantaneous (but we're about to change how this works to make it faster).

For A & C in particular as you've asked, seconds or even less. Was this during a time of serious server strain, maybe? Otherwise it sounds worrying.

Nope; remember that time I added it to the bugzilla tracked template, and nothing happened? Well, it started working finally some time today. I'm going to try implementing it in a couple of other minor places and look at what happens.

(In reply to comment #6)
> Nope; remember that time I added it to the bugzilla tracked template, and
> nothing happened?

No.

> Well, it started working finally some time today.

That's not good. Did you purge the template, or did it get purged somehow (e.g. it was edited)?

> I'm going to try implementing it in a couple of other minor places and look
> at what happens.

OK.

https://en.wikipedia.org/wiki/User:Okeyes_%28WMF%29/sandbox?veaction=edit <- check out the talkpage template.

Argh, that's not good - https://en.wikipedia.org/w/api.php?action=templatedata&titles=Template:Talkback is not updated even though it appears in transclusion, even after purge. Timo, thoughts?

The behavior seems inconsistent. Today, TemplateData was available in transclusions immediately after adding it to the template's documentation subpage. (tested with https://fr.wikipedia.org/wiki/Mod%C3%A8le:Unit%C3%A9 )

Though it is possible that VisualEditor and/or TemplateData do something wrong, in general there's nothing relevant in either VisualEditor or TemplateData that can delay the updating of the data.

The template data is changes by users by editing the wikitext of a Template page (or a page transcluded into a Template page). When they click "Save page", the MediaWiki parser detects the <templatedata> tag, tells the TemplateData extension to make sense of it, which then adds (or replaces) the parsed data in the database.

There is no deferred update here, it is replaced directly.

The API request to fetch templatedata grabs it directly from the database. This isn't specifically cached in anyway.

The only caching layer is in your browser cache and the general cache in the Wikimedia cluster for HTTP requests to content domain names (e.g. "en.wikipedia.org") which is Squid.

https://en.wikipedia.org/wiki/Template:Mileposts was updated 48 hours ago and still hasn't shown. That seems like a time period even the squids should blink at.

Indeed, I cannot see templatedata hinting on en.wp for _any_ templates supposedly annotated per https://en.wikipedia.org/w/api.php?action=query&list=pageswithprop&pwppropname=templatedata&pwpprop=ids|title|value&format=jsonfm and https://en.wikipedia.org/wiki/Wikipedia:VisualEditor/TemplateData . Escalating priority.

The extension page says:

"The TemplateData extension works through having a <templatedata> tag defined in the wikitext of a template page (optionally it can be transcluded into the template page as well). This is designed to be compatible with the common layout many wikis have of transcluding template documentation from a separate page (but is not required)."

And this seems to work partly, e.g. for https://en.wikipedia.org/wiki/Template:Italic_title.  TemplateData is at https://en.wikipedia.org/wiki/Template:Italic_title/doc and it comes up at https://en.wikipedia.org/w/api.php?action=templatedata&titles=Template:Italic%20title&format=jsonfm , but it doesn't seem to actually work in VE (e.g. no description).

Then, https://en.wikipedia.org/wiki/Template:Dead_link appears set up similar, with TemplateData at https://en.wikipedia.org/wiki/Template:Dead_link/doc .  But it does not come up at https://en.wikipedia.org/w/api.php?action=templatedata&titles=Template:Dead%20link&format=jsonfm .  It does come up at https://en.wikipedia.org/w/api.php?action=templatedata&titles=Template:Dead%20link/doc&format=jsonfm (explicitly asking for the /doc page).

Wouldn't we avoid this delay if the template data were in a specific namespace, instead of being saved in one page, which is transcluded in another, which on its turn may be transcluded another, and so on, until the data finally gets to the template? See bug 50512.

I added[1] templatedata to en.wiki [[Template:Redirect]] yesterday (through the /doc transclusion) and also noticed that the main template page wasn't updating (I waited for about 10 minutes).

Then I did a null-edit on Template:Redirect and it was instantly there.

Roan mentioned something earlier (IRC or IRL, don't remember) about this perhaps being caused by the JobQueue (which we use to invalidate pages transcluding other pages, in this case the transclusion of /doc into Template:. *NOT* the transclusion of Template into articles, of course, that'd be insane).

CC-ing Roan, Tim and Aaron who may have more ideas about the cause of this.

As for TemplateData itself, it is triggered as a extension parser tag (<templatedata>) and during the parsing it adds page properties which MediaWiki ends up storing in the page_props db table. The API does a read query from that same table. Pretty basic.

So what could cause the read query to the page_props table to be delayed (or perhaps even not happen at all?) from the page properties added by a page that is transcluded (Template:Foo/doc) into a template page (Template:Foo).

JobQueue? Database replication? Something else?

Note that in my testing yesterday on Template:Redirect I ruled out HTTP caching (API cluster Squid) by making unique urls each time.


[1] https://en.wikipedia.org/w/index.php?title=Template%3ARedirect%2Fdoc&diff=562472124&oldid=562471783

Thought I'd drop a note and say that when I tried a null edit on https://en.wikipedia.org/wiki/Template:Mileposts, the overall template description loaded right away on https://en.wikipedia.org/wiki/British_Columbia_Highway_7. However, the parameter description (km=) did not load until I deleted the template from the article and reinserted the template anew.

Timo has narrowed down the issue to bug 5382 - we're working now to see whether we can fix that bug and so resolve this, or whether that will put too much weight on the cluster.

Observation:

- Edite <templatedata> block on Template:Pagetype/doc.
- View https://en.wikipedia.org/w/api.php?format=jsonfm&action=templatedata&titles=Template:Pagetype/doc
  (with random query params to bust frontend cache)
  It shows the updated data.
- View https://en.wikipedia.org/w/api.php?format=jsonfm&action=templatedata&titles=Template:Pagetype
  (with random query to bust frontend cache)
  It shows the old version still.
- View https://en.wikipedia.org/wiki/Template:Pagetype
  It shows the updated version in the HTML version of the data
  (not sure whether the html update was triggered by a deferred update
  as a result of me editing the sub page or by me viewing the page,
  either way the output is updated).

- View https://en.wikipedia.org/w/api.php?format=jsonfm&action=templatedata&titles=Template:Pagetype/doc
  (with random query to bust frontend cache)
  It shows the updated data.
- View https://en.wikipedia.org/w/api.php?format=jsonfm&action=templatedata&titles=Template:Pagetype
  (with random query to bust cache)
  It shows the old version still.

- Check https://en.wikipedia.org/w/api.php?format=jsonfm&action=query&meta=siteinfo&siprop=statistics and saw there are no pending jobs (not sure if this data is accurate though, it seemed to be accurate as edit count was going up on each refresh).


- Make a null edit to "Template:Pagetype".

- View https://en.wikipedia.org/w/api.php?format=jsonfm&action=templatedata&titles=Template:Pagetype
  (with random query to bust cache)

  It now shows the latest data, update succeeded.


So what is causing this to not be auto updated? I think bug 5382. But having trouble figuring out how come bug 5382 exists since we do have LinkUpdate and RefreshLinks2 which are triggered on edit from the job queue and update all pages linking to it, and the code in LinkUpdate includes all link tables and page properties.

You appear to just be seeing the effect of weeks of job queue lag, i.e. a system problem, not a MediaWiki problem.

Tim's change https://gerrit.wikimedia.org/r/#/c/72679/ (deployed) seems to have made it possible to force templatedata updates by just performing a null edit on the affected template, which is now safe to do on highly-used templates. Note that a null edit is an edit made by just pressing "Save page" without making _any_ changes to the template code itself.

What would be nice is if action=purge would force templatedata updates, like null edits do. This is necessary as many templates are protected so only admins can do null edits. Any editor can do an action=purge (I believe).

Moving this to be a site issue, and re-grading. Thanks to Tim's fix this isn't critical any more (users can 'just' null-edit touch the relevant template to cause TemplateData content to come through), but it's very irritating and we need to fix it.

(In reply to comment #23)
> Thanks to Tim's fix this isn't
> critical any more (users can 'just' null-edit touch the relevant template to
> cause TemplateData content to come through)

Note: This workaround only works for templates that aren't widely used.

Widely-used templates are protected, and users can't null-edit them unless they are an admin (which drastically reduces the population of people who can add and edit TemplateData on the exact templates which would benefit from TemplateData most).

*** Bug 69647 has been marked as a duplicate of this bug. ***