Last modified: 2006-02-25 00:08:02 UTC

Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 939 - changes to links in a template do not update 'pagelinks', 'categorylinks', etc for pages including it
changes to links in a template do not update 'pagelinks', 'categorylinks', et...
Status: RESOLVED FIXED
Product: MediaWiki
Classification: Unclassified
Templates (Other open bugs)
unspecified
All All
: Normal major with 5 votes (vote)
: ---
Assigned To: Nobody - You can work on this!
http://en.wikibooks.org/wiki/Category...
:
: 1824 1993 2235 3349 3854 4111 4929 (view as bug list)
Depends on: 1065
Blocks:
  Show dependency treegraph
 
Reported: 2004-11-24 15:37 UTC by Wikipedia:en:User:Paddu
Modified: 2006-02-25 00:08 UTC (History)
13 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Wikipedia:en:User:Paddu 2004-11-24 15:37:42 UTC
http://en.wikibooks.org/wiki/Programming:C_plus_plus/archive1 says "{{delete}}",
http://en.wikibooks.org/wiki/Template:Delete says "[ [ Category:Candidates for
speedy deletion ] ]", but
http://en.wikibooks.org/wiki/Category:Candidates_for_speedy_deletion says
nothing about "Programming:C_plus_plus/archive1".

Similarly for http://en.wikibooks.org/wiki/User:Paddu/C_plus_plus_talk also is
not listed.
Comment 1 Wikipedia:en:User:Paddu 2004-11-24 15:42:15 UTC
Oh! the category page started listing after I edited both the two pages. What's
going on?

Changing summary to:

"Category doesn't list all articles transcluding a template linking to the
category unless the articles are edited once more after the transclusion."
Comment 2 Rowan Collins [IMSoP] 2004-11-24 16:37:47 UTC
This is a known issue, as described at
http://meta.wikimedia.org/wiki/Help:Template#A_category_tag_in_a_template.3B_caching_problem


Basically, the category tag was added to the template *after* the template was
added to the page; currently, this doesn't trigger an update of the
'categorylinks' table in the database for pages that include that template.
Viewing the page gets the category tag parsed, so you see the "Categories: " box
at the bottom; but only editing each page gets the actual database updated.

I can't find another bug open for this, so I'll try and summarise the summary a bit.
Comment 3 Rowan Collins [IMSoP] 2004-11-28 15:25:27 UTC
Actually, this appears to be true for the 'links' table as well, as demonstrated
by a recent case at
http://en.wikipedia.org/wiki/Wikipedia:Help_desk#Whatlinkshere_not_updating
Basically, links in the template show up in Special:Whatlinkshere, but editing a
link in the template leaves a load of erroneous entries in the whatlinkshere for
the old target page. 

And the same goes for imagelinks: compare the pages listed on
http://en.wikipedia.org/wiki/Image:Science-symbol-2.png (as using the image)
with those on
http://en.wikipedia.org/w/wiki.phtml?title=Special:Whatlinkshere&target=Template%3ASci-stub
(those using the template that uses the image)

This is awkward, because recalculating the links, categorylinks, etc tables for
every inclusion every time a page is edited could take a significant amount of
time to process (since it requires re-parsing the whole page). AIUI, this
recalculation is normally only done when each individual article is saved. 

Perhaps a flag could be added to the 'cur' table to say "recalculate links on
next display", so that the page would only need *viewing*, not *editing* to
eliminate these effects (still not ideal, but it eliminates the need to create a
massive queue of pages to parse whenever a widely-used template is edited).
Since the cache should be invalidated anyway, the extra flag would just cause
the parser to update the links, categorylinks, etc once it was done.

The alternative is to have some logic when a template is edited that determines
what changes need to be made [e.g. remove 'links' entry for "foo", add
'categorylinks' entry for "Category:Bar"], and repeats them for each target
page. But this is dodgy, because *removing* a link from the template doesn't
guarantee that there isn't a link somewhere *else* in the article, so we could
be removing still-valid entries from the tables.
Comment 4 J. Noel Chiappa 2004-12-10 20:07:50 UTC
(In reply to comment #3)

> this is dodgy, because *removing* a link from the template 
doesn't guarantee that there isn't a link somewhere *else* in 
the article, so we could be removing still-valid entries from 
the tables.

Ooooh, good catch! I had mentally designed the same sort of 
optimization code which you had done (at the start of your 
post, above the clipped comment above - create a list of 
deleted links, etc) and I hadn't seen this bug. There's 
simply no way to fix this without mindblowing amounts of 
hair/work. (See the end of this post, if anyone really cares.)

The right fix is probably the simple one you mixed, which is 
to mark all articles which include that template (and we have 
to make sure template recursion works too) so that they will 
be re-parsed next time they are asked for. It's probably not 
much more work at run-time, and is more than an order of 
magnitude simpler implementation-wise.

There is yet *another* catch here though, which is that *any* 
article could be a template. (I have templates in User: space 
I transclude.) So really each article would need an "I am a 
template" flag bit, which is set any time another article 
transcludes it, and would never be cleared thereafter. 
(Reference counts, which could be cleared, get really hairy; 
skipping discussion for now, ask me if you care.) Still, fast 
at run-time, easy to implement. And you could put off this 
fix until later, so that non-Template: templates would still 
have the bug until the "used as template" bit is added.

(Here's how I would do it the other way if I *had* to. First 
I would parse the "template" [i.e. any page that's 
transcluded, see above] in "before" and "after" states, and 
from each one create a sorted list of categories and links 
[i.e. removing duplicates]. Then look for ones that were 
removed from the "before" list - those are the ones where you 
can go to *their* links lists and *potentially* remove all 
the articles which include that template. I say "potentially" 
because for each article, you'd either have to re-parse the 
source looking for duplicates [your bug above, this is the 
expensive, brute-force solution], or keep a list of 
links/categories which are in the article source [i.e. 
without expanding templates], and check that list for a match 
before deleting that article page. Far more complexity [extra 
storage, run-time cost, especially on pages which don't do 
any transclusions - although I suppose you could optimize 
them out] - than it's worth.) 



Comment 5 Rowan Collins [IMSoP] 2004-12-11 00:16:06 UTC
(In reply to comment #4)
> There is yet *another* catch here though, which is that *any* 
> article could be a template. (I have templates in User: space 
> I transclude.) So really each article would need an "I am a 
> template" flag bit, which is set any time another article 
> transcludes it, and would never be cleared thereafter. 
> (Reference counts, which could be cleared, get really hairy; 
> skipping discussion for now, ask me if you care.) 

Well, if there was (as is being proposed) a seperate 'inclusionlinks' table (or
an 'inclusion' linktype field in a merged 'links' table) it ought to be possible
to create a query which for any page would return all pages transcluding that
page. For most pages outside the Template: namespace, it would simply return
nothing, so nothing would be flagged for re-parsing; but for anything that *was*
used as a template, it would list all the pages that needed flagging. Having
this seperation is a must anyway, because without it we'd be recalculating links
on articles that just happen to link to the template, and it's ugly enough that
we purge their cache entries (cf bug 734).

I'm going to create a bug for making this distinction in the DB, for ease of
reference, and set it to blocking this.
Comment 6 Zigger 2005-04-17 03:53:48 UTC
*** Bug 1824 has been marked as a duplicate of this bug. ***
Comment 7 Jeff Bonham 2005-04-19 18:29:12 UTC
This also affects images included in templates - the "following pages link to this file" list  
that is automatically generated on image pages does not include pages that included the  
template before the image was changed to something else.  
  
Setting to "major" because this breaks the broken link table even further.  
Comment 8 Tim McCormack 2005-04-20 18:35:35 UTC
If anybody wants an example to play with: On commons there is a template
(inbound links:
[http://commons.wikimedia.org/w/index.php?title=Special:Whatlinkshere&target=Template%3AGray%27s_Anatomy_plate])
that had [http://commons.wikimedia.org/wiki/Category:Gray%27s_Anatomy_plates a
category] added later.
Comment 9 Brion Vibber 2005-04-27 01:57:04 UTC
*** Bug 1993 has been marked as a duplicate of this bug. ***
Comment 10 Jamie Bliss 2005-04-27 02:05:00 UTC
As an alternative solution, should the "purge" action cause the page to be
reparsed? Meaning that using purge could replace null edits.
Comment 11 Brion Vibber 2005-04-27 02:34:41 UTC
(In reply to comment #10)
> As an alternative solution, should the "purge" action cause the page to be
> reparsed? Meaning that using purge could replace null edits.

Purge _does_ cause the page to be reparsed. It does not, however, alter the links tables.
Comment 12 ABCD 2005-04-27 02:36:28 UTC
Here's a question - should &action=purge update the links table?
Comment 13 Brion Vibber 2005-05-23 22:16:35 UTC
*** Bug 2235 has been marked as a duplicate of this bug. ***
Comment 14 Dashiva 2005-06-01 18:28:31 UTC
I use a small parser modification for this. action=submit&nulledit=true or action=nulledit in the url will do the same as a regular null 
edit without visiting the edit page. Could even make a page link for it, but it's still just a band-aid. A fix so that updating the 
template cascades *-links table updates would be much better.
Comment 15 Puzzlet Chung 2005-06-17 02:26:02 UTC
*** Bug 1392 has been marked as a duplicate of this bug. ***
Comment 16 Puzzlet Chung 2005-06-17 04:04:13 UTC
I don't know this could be the compromise, but how about putting backlinks via
templates as subnodes, just like how we have backlinks via redirects? For
example, if article A and B contain Template:T which has a link to X, X's
Special:Whatlinkshere should look like:

< X
The following pages link to here (up to 500 are shown):
* Template:T
** A
** B
* Eks (redirect page)
** pages that link to Eks
* other pages that link to X directly

If A and B also links to X directly, list them on the first level as well. And
once the link in Template:T is deleted, the special page would look like:

< X
The following pages link to here (up to 500 are shown):
* Template:T (obsolete)
** A
** B
* Eks (redirect page)
** pages that link to Eks
* other pages that link to X directly

I'm not a technician guy, but I think implementing this would be easy as making
Whatlinkshere to include the inclusion lists of the templates that link to the
page which is the subject of the special page. But to make this to work, we need
to separate lists of "inclusions" and "actual links" (i.e. [[:Template:T]]) of
templates, which Whatlinkshere pages don't seem to distinguish the difference.
Comment 17 Christopher Beland 2005-07-15 05:12:17 UTC
Making the system update properly in the first place can be done with or without
user-visible changes.  Certainly that should happen regardless.  I also like the
interface change you suggest; it would help a a lot of category-tidying work go
more smoothly.  I see bug 1392 has been reopened; I wonder if these two
questions are now being considered separately or not.
Comment 18 Jamie Bliss 2005-07-16 16:43:06 UTC
(In reply to comment #17)
> ...
> I see bug 1392 has been reopened; I wonder if these two
> questions are now being considered separately or not.

I believe that both this and bug 1392 would require the same DB changes. But to
totally fix bug 1392 may also require editing SpecialWhatlinkshere.php.
Comment 19 Kellen 2005-08-14 22:28:26 UTC
This is a major issue for the wikibooks Cookbook -- we have most of our recipes
under category "Recipes" via a template, but would like to move them to category
"Recipe" -- if we just update the template, we'll have to apply several hundred
null edits to refresh each recipe page. A timely fix for this would save us a
lot of grunt work.
Comment 20 Brion Vibber 2005-09-04 05:20:49 UTC
*** Bug 3349 has been marked as a duplicate of this bug. ***
Comment 21 Quietust 2005-10-05 22:50:29 UTC
A command to flush and completely regenerate all link tables (by effectively
performing a null edit on every single page in the wiki) would be useful (though
slow) as a temporary solution to this problem; however, I don't recall if such a
command currently exists.
Comment 22 Rowan Collins [IMSoP] 2005-10-05 23:44:28 UTC
(In reply to comment #21)
> A command to flush and completely regenerate all link tables 

That'll be the 'maintenance/refreshLinks.php' command-line script. :)

Comment 23 lɛʁi לערי ריינהארט 2005-10-06 08:19:36 UTC
(In reply to comment #21)
> A command to flush and completely regenerate all link tables (by effectively
> performing a null edit on every single page in the wiki) would be useful (though
> slow) as a temporary solution to this problem; however, I don't recall if such a
> command currently exists.

related topic to this:
bug 2098: action=purge and categories - implement a more effectiv action=update
or action=rebuild
Comment 24 Ævar Arnfjörð Bjarmason 2005-11-01 11:50:29 UTC
*** Bug 3854 has been marked as a duplicate of this bug. ***
Comment 25 bdk 2005-11-30 09:32:40 UTC
*** Bug 4111 has been marked as a duplicate of this bug. ***
Comment 26 Rob Church 2006-01-02 14:51:15 UTC
Bump: Tim, how does the templatelinks stuff affect this bug now?
Comment 27 Brion Vibber 2006-02-09 18:32:00 UTC
*** Bug 4929 has been marked as a duplicate of this bug. ***
Comment 28 Tim Starling 2006-02-25 00:08:02 UTC
This is fixed in CVS and on Wikimedia sites. Will be released in MediaWiki 1.6. 

A "job queue" system has been implemented. There may be some delay between when the template is 
edited and when the relevant changes are reflected in the links tables. The length of the delay 
might need some fine tuning, delays of up to a day are to be expected until we sort out the system 
administration and performance aspects of this.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links