Last modified: 2014-11-17 09:21:25 UTC

Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 5881 - Labeled Section Transclusion
Labeled Section Transclusion
Status: RESOLVED FIXED
Product: MediaWiki extensions
Classification: Unclassified
Extensions requests (Other open bugs)
unspecified
All All
: Normal enhancement with 9 votes (vote)
: ---
Assigned To: Steve Sanbeg
http://www.mediawiki.org/wiki/Labeled...
: patch, patch-need-review
: 6785 9514 (view as bug list)
Depends on:
Blocks: 9559 14724
  Show dependency treegraph
 
Reported: 2006-05-09 08:45 UTC by Dovi Jacobs
Modified: 2014-11-17 09:21 UTC (History)
7 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments
my implementation, from my local installation (3.65 KB, patch)
2006-09-20 19:03 UTC, Steve Sanbeg
Details
updated patch (2.19 KB, patch)
2006-10-06 19:29 UTC, Steve Sanbeg
Details
test cases (1.91 KB, text/plain)
2006-10-06 19:31 UTC, Steve Sanbeg
Details
enhancement to allow indivual markers to subst from templates. (2.26 KB, patch)
2006-10-06 19:54 UTC, Steve Sanbeg
Details
additional test cases for new enhancement (2.29 KB, text/plain)
2006-10-06 19:55 UTC, Steve Sanbeg
Details
reimplemented as an extension (1.89 KB, text/plain)
2006-11-03 20:34 UTC, Steve Sanbeg
Details
regression tests for extension (2.62 KB, text/plain)
2006-11-03 20:35 UTC, Steve Sanbeg
Details
reformat, work on edit links a bit. (6.61 KB, patch)
2006-11-07 23:15 UTC, Steve Sanbeg
Details
extension (7.18 KB, text/plain)
2006-11-10 21:00 UTC, Steve Sanbeg
Details
regression tests with closing / (2.99 KB, text/plain)
2006-11-10 22:38 UTC, Steve Sanbeg
Details
new extension with Nick's fixes (7.01 KB, text/plain)
2006-11-14 16:52 UTC, Steve Sanbeg
Details
extension, with better hook for recursive parsing. (6.02 KB, text/plain)
2006-11-15 23:42 UTC, Steve Sanbeg
Details
regerssion tests (+ a few more) (3.08 KB, text/plain)
2006-11-15 23:55 UTC, Steve Sanbeg
Details
regression tests (3.18 KB, text/plain)
2006-11-16 00:20 UTC, Steve Sanbeg
Details
extension, with fixed quoting (5.56 KB, text/plain)
2006-11-16 18:33 UTC, Steve Sanbeg
Details
regression tests (+2 more) (3.77 KB, text/plain)
2006-11-16 18:35 UTC, Steve Sanbeg
Details
Patch for bug in wfLstInclude() when article doesn't have any labeled section for the given label (652 bytes, patch)
2006-12-12 23:46 UTC, Cyril Dangerville
Details

Description Dovi Jacobs 2006-05-09 08:45:54 UTC
Request for a function that allows calling up a predefined *portion* of a page,
without transcluding any of the rest of the wiki text on that page.

Beginning and end points of sections to be made available for individual
transclusion could be determined by using marks such as in the following
example, in which "section 1" could be individually transcluded on any other
wiki page:

Beginning of section 1: <span id="bookmark">start1</span>

End of section 1: <span id="">end1</span>.

Those are just examples. Any section of text marked off with <span
id="x"></span> at its beginning and end could be transcluded individually.

Please note that this is intuitively the *opposite* of <noinclude></noinclude>.
These *exclude* parts of a template not meant for transclusion. The suggestion
here is for designating exactly what *will* be included, rather than what will
*not* be included.

Discussion of this began with the formatting of Bible verses at Wikisource.

See
[[http://en.wikisource.org/wiki/Wikisource:Scriptorium#Technical_Question:_.22Bookmarks.22]]

and
[[http://en.wikisource.org/wiki/Wikisource:Scriptorium#More_radical_technical_bookmark_question]]
Comment 1 Dovi Jacobs 2006-05-09 10:03:51 UTC
I apologize for a silly error above, which should read:

Beginning of section 1: <span id="start1"></span>

End of section 1: <span id="end1"></span>

The point is that sections could be defined using whatever labels make sense.
Comment 2 Phil Boswell 2006-05-09 10:23:18 UTC
Isn't this already accomplished using the <onlyinclude> tag?

Can you have more than one of those on a page?
Comment 3 Dovi Jacobs 2006-05-09 10:32:37 UTC
This would be much more flexible, allowing the definition of sections 1, 2, 5a,
78, a, b, c - any number of different sections, each with its appropriate tag name.

In Wikisource, these could be individual chapters of books (where the whole book
is on one page), single numbered paragraphs of Aristotle, or (as in the recent
discussion) individual verses of the bible.
Comment 4 Jesse (Pathoschild) 2006-05-09 10:51:32 UTC
Templates might be made to recognize sections, which would at least partially
accomplish the above. For example, {{foo#Section_78}} would transclude
everything within ==Section 78== on [[Foo]].
Comment 5 Zoran Obradovic 2006-05-09 10:52:32 UTC
This can already be done with parser functions, like #if and #switch.
Comment 6 Dovi Jacobs 2006-05-09 10:59:30 UTC
Thanks, Jesse, that is a good tool already available. I didn't know that was
possible. It is a bit clunky, however, in its being dependant on == === type
sections.

Thanks too Zoran, but please explain what you mean?
Comment 7 Phil Boswell 2006-05-09 11:06:29 UTC
The problem with transcluding sections is whether you include sub-sections.

For example, if ==Section 78== includes ===Section 78.1=== should it be possible to transclude 
just that portion between those two lines? or should transcluding "Section 78" always include 
all sub-sections?

In other words, if your source page [[foo]] looks like:
...
==Section 78 ==
aaaaa
===Section 78.1===
bbbbb
==Section 79 ==
...
should {{foo#Section 78}} cover the "aaaaa" AND "bbbbb" text or just the former? should you be 
able to specify just the former?

Also, does your partial transclusion include the associated section heading? Does 
{{foo#Section 78}} start with "==Section 78==" or not? If not, then what do you do with any 
sub-section headers which might be included? Should they remain at the given level or be 
adjusted to take account of their level relative to the "top-level" header being transcluded?
Comment 8 Dovi Jacobs 2006-05-09 11:12:07 UTC
Phil is right. Only exact markers placed at beginning and end would be able to
define units of text without ambiguity.

But still pardon me, what do #if and #switch accomplish here?
Comment 9 Zoran Obradovic 2006-05-09 12:03:05 UTC
something like this should do exactly what you want:

{{#ifeq:{{{section|name1}}}|name1|
text of section 1
}}

{{#ifeq:{{{section|name2}}}|name2|
text of section 2
}}

{{#ifeq:{{{section|name3}}}|name3|
text of section 3
}}
Comment 10 Zoran Obradovic 2006-05-09 12:05:22 UTC
Maybe that wasn't clear enough. You put that code on the page from which you
want to transclude text, and then transclude it with {{pagename}}, which will
display all text, or with {{pagename|section=name}}, which will display only the
section you want.
Comment 11 Dovi Jacobs 2006-05-09 13:56:55 UTC
Thanks. Is there a page that gives information on the #if and #switch functions?
Comment 12 Dovi Jacobs 2006-05-09 14:16:32 UTC
Tried it, and it doesn't seem to work, but rather transcludes the entire page:

User:Dovi/Bible (King James)/Genesis/45

Have I made an error someplace?

Also, I realized that by deliniating the text within the brackets (as in this
suggestion), there is no way to allow for overlapping sections. *Independent*
beginning and end markets would allow for theat.

Comment 13 Jesse (Pathoschild) 2006-05-09 16:40:08 UTC
[In reply to comment #10] Zoran Obradovic's suggestion does technically work,
but is far from sufficient to accomplish what Dovi is looking for. There are
several limitations to using ParserFunctions for this: the contents of the page
will be hidden on the page itself, since the parameter is not specified. This
can be circumvented, but it's tedious and messy. Another limitation is that the
pipe is used as a seperator in the ParserFunctions, so any pipe in the page will
have to be escape with {{!}}. Further, it would be extremely messy to implement
this on all pages, which is something that would need to be done to address the
scope of Dovi's proposal. If this is to be accomplished, it would be much better
to do so with new functionality.

[In reply to comment #7] The expected behaviour when calling {{:foo#Section}}
would be to include the section title. In those circumstances where this is not
desireable, it could be removed using ParserFunctions (which are more suited to
that particular task then to the greater proposal). Alternately, a new syntax as
suggested by Dovi might be better suited for more control. We could implement
this in the same general syntax as the Parserfunctions: "{{#section:name|start}}
[...] {{#section:name|end}}". This would have the benefit of being unambiguous,
avoiding any confusion with anchors and actual <span> containers.
Comment 14 Dovi Jacobs 2006-05-09 18:39:21 UTC
Jesse's final syntax suggestion:

{{#section:name|start}} [...] {{#section:name|end}}

sounds exactly like what I envisioned, but I had no idea how to express it as a
code suggestion.

The suggestion is clear and unambiguous. Plus it is completely flexible in terms
of how the text may be divided (even allowing for complementary overlapping
divisions such as exist in many classical texts).

If such a function were implemented, it would not only be an extremely useful
tool for Wikisource, but would probably find a great many positive
implementations in other contexts as well.
Comment 15 Brion Vibber 2006-05-09 20:46:22 UTC
Overlapping portions would interact very badly with formalized syntax;
this is not likely to be supported.
Comment 16 Dovi Jacobs 2006-05-10 03:42:16 UTC
The above syntax is not likely to be supported, or the use of it in overlapping
ways?
Comment 17 Dovi Jacobs 2006-05-10 03:43:23 UTC
(Just to clarify: in the vast majority of cases, text sections are sequential
anyways.)
Comment 18 Dovi Jacobs 2006-05-31 07:24:12 UTC
This proposal was unfortunately only clarified towards its end, and has been
further clarified since the last comments in the discussion here.

Current status is reflected here:

http://en.wikisource.org/wiki/Wikisource:Labeled_section_transclusion
Comment 19 Dovi Jacobs 2006-05-31 07:29:52 UTC
Two further notes:

1. This would also solve bug 3626 (transclusion of article introductions).

2. This bug should be closed here and reopened as "Labeled section transclusion"
since we are no longer talking about ParserFunctions.
Comment 20 Jennifer Zagurski 2006-05-31 22:38:47 UTC
This is a very important feature for Wikisource as it will allow all sorts of
comparitive texts to be created without the worry of errors in duplication.  As
you can imagine the difficulties involved in making sure all instances are
proofread equally in a text re-used in many places.
Comment 21 Dovi Jacobs 2006-06-20 11:04:33 UTC
Adding the current coding suggestion from the URL:

A. Mark off a section of text

A section of text (an individual chapter, subsection of a chapter, a numbered
paragraph in a classical text, an individual verse) might be marked off
something like this:

<begin section="Paragraph 1"/> ...text of the section...<end section="Paragraph 1"/>

The slashes inside the tags are XML-style, which is similar to <br/>,
<references/>, etc.

NB: Sections may overlap (different sectioning mechanisms may be used within the
same text), so a syntax where the end tag is ambiguous is not acceptable.
[edit]

B. Call up the section of text

When calling up that section, it would be done like this:

{{Text##Paragraph 1}}

The double-## syntax is suggested because a single "#" has the problem of being
normally used for regular wiki sections, so users might intuitively try
transcluding sections and be confused when it doesn't work. That is why "##" is
suggested as an alternative. Other non-title characters would work, too.

For further information, see:
http://en.wikisource.org/wiki/Wikisource:Labeled_section_transclusion
Comment 22 Aryeh Gregor (not reading bugmail, please e-mail directly) 2006-07-31 00:51:49 UTC
*** Bug 6785 has been marked as a duplicate of this bug. ***
Comment 23 Steve Sanbeg 2006-09-20 19:03:07 UTC
Created attachment 2375 [details]
my implementation, from my local installation

I've implemented this on my test installation, and copied a few test cases to
wikisource, which are linked from the talk page referenced in the description.
Comment 24 JeLuF 2006-09-24 08:12:12 UTC
You add a <section /> tag to the sanitizer. There is no such tag in HTML and
thus it should not be in that list.

Why is this needed at all? Instead of including a section, one could create a
third article containing teh section and including it into both articles. This
would avoid the introduction of even more markup.
Comment 25 Jennifer Zagurski 2006-09-24 16:37:07 UTC
Please read 

http://en.wikisource.org/wiki/Wikisource:Labeled_section_transclusion

And the talk page for many details as to why this function is needed.  The
suggestion you make is one already used at the Hebrew Wikipedia (although
instead of a third "article" they put it in the template namespace).   This
method is outlined on the page I linked above along with several other
alternatives.  The disadvantage listed there is "This method is extremely
labor-intensive, and also involves the creation of huge numbers of separate
pages."  I would also like to add it makes proofreading very diffucult when you
have to look up every sentance or two on a new page to proofread,  Basically it
does not scale well. Imagine every verse in the Bible on a seperate page then
multiply that times the seven or so seperate PD English translations available,
then imagine proofreading such a montrosity!  

Please anyone who thinks they have a better alternative read the above page on
Wikisource and be sure we have not already considered this.

Comment 26 Steve Sanbeg 2006-09-25 14:54:05 UTC
(In reply to comment #24)
> You add a <section /> tag to the sanitizer. There is no such tag in HTML and
> thus it should not be in that list.
> 

This was done primarily to allow a section marker to be placed into a template,
and allowed to pass through a normal transclusion; i.e. ot put <section
begin=discussion> in one template and <section end=discussion> in another, So
that a page that transcludes a lot of subpages could exclude all of the
discussions, thus making for a much smaller transclusion.  

This feature removes unused section markers when they're rendering, so it isn't
necessary for the sanitizer to remove them

It's not critical for the way it's used in WS, but it generally makes it more
flexible and more useful.

> Why is this needed at all? Instead of including a section, one could create a
> third article containing teh section and including it into both articles. This
> would avoid the introduction of even more markup.

That was my thought at first, too.  But we're not talking about one section;
there's already a markup for that, <onlyinclude>.  But that won't work with
multiple sections.  The editors have been understandably concerned about trying
to manage the editing when each page is split into potentially hundreds of tiny
articles, each of which is transcluded into two real articles.  

Even if we did split everything up like this, we could wind up seeing the same
kind of problems on article space in WS as we occasionally see in project space
in W when a page consists of an unbounded number of transclusions.


Comment 27 Steve Sanbeg 2006-10-06 19:29:04 UTC
Created attachment 2467 [details]
updated patch

I've decided to drop support for indirect LST (where begin/end can ceom from
sepeate templates).

It would be nice, but more work than I expected to get it working correctly,
and not important enough to hold up other projects that are waiting for it.

This obviates the Sanitizer changes, so this patch is simpler as well.
Comment 28 Steve Sanbeg 2006-10-06 19:31:45 UTC
Created attachment 2468 [details]
test cases

Now that I've been introduced to the regression testing system, I've ported the
test cases I previously posted on ws.
Comment 29 Steve Sanbeg 2006-10-06 19:54:15 UTC
Created attachment 2469 [details]
enhancement to allow indivual markers to subst from templates.

OK, I can't transclude markers from templates, but with a small block op on the
code, substing them will work.
Comment 30 Steve Sanbeg 2006-10-06 19:55:18 UTC
Created attachment 2470 [details]
additional test cases for new enhancement
Comment 31 Steve Sanbeg 2006-11-03 20:34:24 UTC
Created attachment 2631 [details]
reimplemented as an extension


This extension adds the <section> hook, and two parser functions, #lst to
include a section, and #lstx to exclude. It only needs to be included from
LocalSettings.php to be used.
Comment 32 Steve Sanbeg 2006-11-03 20:35:41 UTC
Created attachment 2632 [details]
regression tests for extension

These are the regression tests for this extension, and show most of its
functionality.

Note that this depends on bug 7801 to run the regression test, since parser
functions currently can't be regression tested.
Comment 33 Thomas Kindig 2006-11-07 22:26:34 UTC
I attempted to use this extension with MediaWiki version 1.5.8. I have not been
able to fathom the syntax change for setFunctionHook(), as I assume the
undefined function has been replaced with another function.

Fatal error: Call to undefined function: setfunctionhook() in
/var/www/html/home/extensions/LabeledSectionTransclusion.php on line 18

I don't mind updating the extension myself. I'm just not sure where to begin.
Any advise appreciated.
Comment 34 Steve Sanbeg 2006-11-07 23:15:37 UTC
Created attachment 2657 [details]
reformat, work on edit links a bit.

I hadn't thought too much about supporting older versions, and don't have much
experience with then; but I think this version should work and give you must of
the functionality.

There are two issues with these versions that I know about:
1) I'm trying to do this with parser functions, since those work with subst:. 
That is where the SetFunctionHook is used, which was introduced around 1.6.  I
added a check to disable that in pre 1.6, and added a transclude tag, i.e.
<transclude page=mypage include=mysection> (or exclude=mysection
[replace=something]

2) The recursive parsing function was introduced in 1.8; I've added a
workaround for that.

Of course, this is still in development, so any of that could change, but this
should work.
Comment 35 Alexandre Racine 2006-11-09 14:13:04 UTC
1- @Kindig : This extention will only work with 1.8 and above.
See here : http://www.mediawiki.org/wiki/Labeled_Section_Transclusion

2- So... I have tryed a couple of combination of tag and it does not work. What
is the correct way of using this thing? In other words, what is the correct syntax?
Comment 36 Steve Sanbeg 2006-11-09 16:25:52 UTC
(In reply to comment #35)

True; for now I'm more concerned with getting something we can use in wikisource
than supporting a lot of versions.  But it should be possible to make at least
some of it work in some older verions by checking the version number and
switching the behavior appropriately.

The main issues with older MWs are:

1) Recursive parsing, to parse the markup in the template, changed in 1.8. 
There is a workaround in the previous attachment that should allow this for
older versions.

2) Parser functions don't exist prior to 1.6 or 1.7, so the #lst syntax can't be
used in older verions.  So while 1.7+ can use {{#lst:mypage|mysection}}, 1.5
would use (in the previous attachment) would use <transclude page=mypage
section=mysection/>, which should work for any version from 1.5 on.

Regardless of which version you use, you mark a document with section tags, i.e.

<section begin=mysection/>this is a section<section end=mysection/>, then
transclude it with one of the above syntaxes.  Note that if you don't specify a
namespace to transclude, the extension uses the main namespace; i.e.
{{#lts:mypage... is different form {{#lts:template:mypage...

> 1- @Kindig : This extention will only work with 1.8 and above.
> See here : http://www.mediawiki.org/wiki/Labeled_Section_Transclusion
> 
> 2- So... I have tryed a couple of combination of tag and it does not work. What
> is the correct way of using this thing? In other words, what is the correct
syntax?

Comment 37 Rob Church 2006-11-09 17:23:26 UTC
When introducing a new extension, and a new kind of extension at that, it's
probably a good idea not to worry too much about supporting anything earlier
than the current stable branch; so concentrate on support for trunk and 1.8.
Comment 38 Steve Sanbeg 2006-11-10 21:00:33 UTC
Created attachment 2673 [details]
extension

OK, cool; luckily, stable & branch seem pretty similar w.r.t to this.

So I seem to have found the right voodoo to transclude from an extension.  This
version

* handles sections correctly, with the right TOC & edit links (whew!)
* handles cycles correctly, changing the loop into a self link
* drops (at least for now) non-parserfunction support, which couldn't do edit
sections.

Although I wish I understood noparse=true better, it does make things work,
allowing it to parse in the site (although not in the regression tests); all my
weird test cases look good.
Comment 39 Steve Sanbeg 2006-11-10 22:38:28 UTC
Created attachment 2674 [details]
regression tests with closing /

I've added the / to the <section /> tags; the current extension drops support
for the non-well-formed version
Comment 40 Steve Sanbeg 2006-11-14 16:52:16 UTC
Created attachment 2709 [details]
new extension with Nick's fixes

Implemented a few fixes recommended by Nick Jenkins:

* add closing ?>
* rm some dead code
* handle case where title is invalid.
Comment 41 Steve Sanbeg 2006-11-15 23:42:31 UTC
Created attachment 2711 [details]
extension, with better hook for recursive parsing.

Found a better hook for recursive parsing, so the parser can mark the headings
for us.  This simplifies the code somewhat, and fixes the issues that Nick
brought up.
Comment 42 Steve Sanbeg 2006-11-15 23:55:52 UTC
Created attachment 2712 [details]
regerssion tests (+ a few more)

Add Nick's test cases.	With improved parsing hooks in previous version, all
tests work now.
Comment 43 Steve Sanbeg 2006-11-16 00:20:47 UTC
Created attachment 2713 [details]
regression tests

fix error in last (js) test.
Comment 44 Steve Sanbeg 2006-11-16 18:33:04 UTC
Created attachment 2715 [details]
extension, with fixed quoting

Apparently, \Q..\E doesn't interpolate into variables, so call preg_quote()
instead.
Comment 45 Steve Sanbeg 2006-11-16 18:35:11 UTC
Created attachment 2716 [details]
regression tests (+2 more)

Add test for transcluded section heading, since implementing that was a lot of
the effort.

Test quoting by transcluding a section called "/", which produced warnings (and
didn't work) before.
Comment 46 Nick Jenkins 2006-11-17 01:41:21 UTC
Added to SVN in r17736.
Comment 47 Cyril Dangerville 2006-12-12 23:46:04 UTC
Created attachment 2853 [details]
Patch for bug in wfLstInclude() when article doesn't have any labeled section for the given label

Hi Mr Sanbeg, thanks for your extension! 
I am testing some use of it in DynamicPageList2.
I had just one bug when calling wfLstInclude($parser, $page, $sec) - directly
from DPL2 - on a page (title=$page) that doesn't have any section with
label=$sec. No match, so in particular, the $m[0] is empty, and therefore I get
the warning "Undefined offset: 0..." (where the $m[0][0][1] appears).
I know this is not the usual intended use of your extension, but when using
DPL, you don't expect every page in the output list to have labeled sections
for a given label.
Anyway, I made a quick fix (see attachment) to deal with it. You may fix that
better for SVN. Thanks.
Comment 48 Steve Sanbeg 2006-12-13 00:05:44 UTC
Cool, thanks.

It's supposed to just transclude an empty string when there's no section.  The
heading offset code is new, so that bug is recent.  I've added your fix and a
regression test so it won't break again to r18301.

(In reply to comment #47)
> Created an attachment (id=2853) [edit]
> Patch for bug in wfLstInclude() when article doesn't have any labeled section
> for the given label
> 
> Hi Mr Sanbeg, thanks for your extension! 
> I am testing some use of it in DynamicPageList2.
> I had just one bug when calling wfLstInclude($parser, $page, $sec) - directly
> from DPL2 - on a page (title=$page) that doesn't have any section with
> label=$sec. No match, so in particular, the $m[0] is empty, and therefore I get
> the warning "Undefined offset: 0..." (where the $m[0][0][1] appears).
> I know this is not the usual intended use of your extension, but when using
> DPL, you don't expect every page in the output list to have labeled sections
> for a given label.
> Anyway, I made a quick fix (see attachment) to deal with it. You may fix that
> better for SVN. Thanks.

Comment 49 Brion Vibber 2007-04-09 14:35:38 UTC
*** Bug 9514 has been marked as a duplicate of this bug. ***
Comment 50 Steve Sanbeg 2007-06-22 20:49:16 UTC
Brion enabled it on *.wikisource.org and test.wikipedia.org.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links