Last modified: 2007-04-25 22:43:00 UTC
When the same macro is transcluded into a page more than once - even where the uses of each transclusion have different arguments - sometimes the page fails, inexplicably. Look at the page for http://en.wikipedia.org/wiki/List_of_counties_in_Wisconsin (List of counties in Wisconsin), which uses the County1 template for every entry. This is done to make the internal page "cleaner" since the information in each entry in the table is the same, it makes sense to use a template call to generate each record in the table. However, after the same macro is called about 15 times, for some reason the page renderer partially fails and doesn't reneder all of the macro. After it partially fails a couple of times, then it completely "forgets" the macro exists, and presumes that the macro is missing, it can no longer see it again. Even changing the macro call to prefix it with :Template: does not fix the problem, it still thinks the template is in the main name space.
You seem to be hitting the inclusion size limit; check the HTML source of the rendered page for a series of comments indicating this.
It's hitting the pre-expand limit, which could be caused by either large comments in the template, or by the large #switch templates.
BTW, I think I fixed the problem where the links were going to the wrong namespace, so when it does hit the limit, it should link the right page now.
I know what the problem is. If a macro calls another macro, when it reads the included macro, it counts every line of the included macro even if it will only generate one line, or even nothing at all. So a macro using 50 or 60 lines that is included but only generates one line, will count as 50 lines. So if there are, say, 50 called macros in a {{#switch:}} statement, each of which of the called macros has 50 lines, and if only one of the macros is activated, it will generate one line but count as if it is generating 2500 (50*50) lines.
Precisely. This behavior may be changed at some point in the future, but currently that's how it works.
Yeah, I agree. But now that I think about it, I don't think the pre-expand limit should really be cumulative. Although it should be possible to work around this with a bunch of ugly meta templates, it's probably better to relax that rule a bit so parser functions will work here.
As long as things are processed, limits will apply to them. Closing.
The request is for them *not* to be processed, i.e., for {{#if:0|{{foo}}|{{bar}}}} to never actually look at or parse the contents of Template:foo, having figured out that it doesn't have to.
They're already not being processed; it was failing the pre-expand size limit. This is a cumulative limit on template sizes prior to expansion; which corresponds to how much bigger the article would get if all the templates were included but not expanded. Not a very realistic metric I think, but a reasonable approximations as long as expansions strictly "expand" the templates, i.e. when parser functions aren't used. This could be worked around simply by replacing parser functions with a heap of convoluted meta-templates, or fixed by making pre-expand size non-cumalative, since you wouldn't have more than one unexpanded template at a time.
The reason for the pre-expand limit is it takes time for the parser to crunch through all the text. Anything that's actually parsed is counted because anything that's parsed takes time to parse, whether it's displayed or not. We had pages that were taking ten seconds to parse because of all the templates. Currently unused inclusion paths must be parsed by the parser, so they're added to the size. The idea is to make the parser smart enough to not parse them to begin with. If you have an idea for a heap of convoluted meta-templates that will handle this as desired, please use them. They'll improve performance, and thereby evade the parser include size limit.
In this case, I think it was templates like {{#switch:{{{1}}}|NJ=New Jersey|NY=New York|...}} That were used repeatedly, creating a lot of pre-expand text. Changing that to something like {{statename/{{{1}}}}} and creating 50 sub-templates should decrease the pre-expand text. What you're talking about sounds more like bug 8314 that my understanding of this one.
Actually, I was doing it in the form of something similar to ((#switch:(((1)))|NJ={{StateName|34}} or whatever so it used the FIPS code, or something like that, so that in the event of a change only the lowest-level underlying template would be needed, and the changes would "cascade up" to all other templates that used them. Only problem with that is that it in effect counted all the templates that were never referenced, so that if it was CT, then CT would call the one template, it was still presuming that all 50 subtemplates would be called for this one entry, as well as presuming that all of the other 49 entries would each call their 50 sub-entries as well, leading to 2500 macro calls and the effective expansions. Having heard the arguments about inclusion time, I guess I can understand why it is putting this limit in this way - a macro expansion that takes 10 seconds is unacceptable, and preventing this is a worthy goal - I never thought about it. I do want to thank everyone who looked into this issue and provided such rapid feedback. While I was curious about why this was happening, it was more important that I find a way to fix it, and I have. Both on allowing the template to work, and a way to make sure someone else in the future, when new entries are added (I suspect there will be some new state(s) add to the United States eventually, maybe Canada will take the original invitation provided in the Articles of Confederation and accept the offer to have its provinces become states, maybe Texas decides to take up the automatic provision to allow itself to divide into more states, or maybe Guam and/or Puerto Rico will be added.) Well, what I did to fix the problem was simply to put a non-included note on each transclusion template listing all of the templates related to the particular function and cross referencing them, reminding any future editor to change all of them if new information is added. It's not as foolproof as making it fully automatic but at least it works, which is primary and fundamental. But, there is a question about why it counts entries that cannot be included and will never be included, e.g. when it selects CT or NJ, why is it still expanding the other entries that will never be used? I've discovered it did this when I did a {{subst:county1|parameters}} in order to make the original fix, and what I noticed is that it broke the one macro into its 50 component macro calls when the case statement was executed. If I compile the equivalent of a case statement in a programming language, or even if it's interpreted, the interpreter takes the case branch that matches and does not execute the rest of the code in that case statement. The language processor skips over the remaining unexecuted case branches until the case end delimiter is found. Now, granted there may be some execution time to skip over the unexecuted branches, it seems like it's excessive to presume that every case branch has to be expanded or that it's impossible to tell which branch will be executed. Unless the decoding of called macros (inclusion) is done before branch prediction takes place, e.g. the macro is read and decoded into its component parts then the parameters are examined to see which parts are used. Would there be an objection if I took a look at the code on how this is done and offerred suggestions?
Having read the other bug 8314 I can understand that there are arguments either way with respect to interpreting the value of a parameter before inclusion or doing so after inclusion. What might be asked is, which would be faster or which would be better as far as usage factors. For example, if the use of parameter interpretation in one way or another spoils caching then that's obviously unacceptable (is caching done at the static page level or at the wiki code level?) Or perhaps it has to do with whether a parameter is static or dynamic at rendering time. Maybe one of these execution forms will only work with "late binding," e.g. the parameters are translated after the wiki source has been extracted. I don't know how that would be possible, but, then I don't know the code so it's not my place to say.
It is all open source, so you're certainly welcome to look at it and see if there's a better way. It's really a matter of precedence; parser functions and templates expect their arguments to be pre-expanded, but in a case where they through away most of their arguments, and render the rest a second time, that's not ideal. What I partially did but never finished was to make {{defer:thing}} expand to {{thing}}, so you can explicitly defer substitutions when they can be done later. That's the best I could come up with.
*** This bug has been marked as a duplicate of 8314 ***
