Last modified: 2013-06-18 16:11:59 UTC
Adding a music module which would have a text input and could produce scores (in PNG, PDF, SVG, etc.) and/or sounds (in MIDI, Ogg Vorbis, etc.). This would really help for music articles
Do you have any example of GPL software that can be added to the mediawiki software ?
http://en.wikipedia.org/wiki/GNU_LilyPond
see http://wikisophia.org/wiki/Wikitex http://meta.wikimedia.org/wiki/Music_markup The code is there since months - what is missing for including it as a mediawiki extension?
*** Bug 935 has been marked as a duplicate of this bug. ***
(In reply to comment #1) > Do you have any example of GPL software that can be added to the mediawiki > software ? What about the ABC music notation standard? http://abc.sourceforge.net/ There are tools to convert it to PostScript and MIDI, and they appear to be GPLed.
Created attachment 195 [details] A MediaWiki extension which supports on-the-fly creation of musical notation Hi, I created a MediaWiki extension which uses LilyPond (in safe mode) to produce musical notation. For short pieces, i.e. to describe a musical concept, you can use <lilypond>c4 e g</lilypond> If you want to hear it by clicking on the resulting picture, just use <lilymidi> instead of <lilypond>. And the best: you can also use it to write whole pieces in a team, by using <lilybook> instead of <lilypond>. The contents are then interpreted as a LilyPond file, and the pages are output as pictures. I did my own MediaWiki extension rather than use wikitex, because LilyPond is steering clear of TeX (text and music do have different dimensions). Ciao, Dscho
Dscho's mail to wikitech: http://mail.wikimedia.org/pipermail/wikitech-l/2004-December/026518.html
Created attachment 691 [details] Unpacked supplied .gz (lilypond extension)
Created attachment 694 [details] WikiPedia extension This is the LilyPond support written as an extension (requires mediawiki>=1.3)
i compiled the latest version of stable lilypond. From the look of the Wikipedia extention I can see the png in the tmp directory, but I am getting a "Failed to parse (PNG conversion failed; check for correct installation of latex, dvips, gs, and convert)". Please reply with which binary am I missing so I can proceed.
fixed most of the problem, had to change Lilypond.php. Current problem includes, the generated png file have excessive white space. Idea of fixing this include adding a variable to the \page syntax to alter the .ly file, or figure out how to use convert of Imagemagick using "autocrop" or -gravity center.
Created attachment 1141 [details] new version of LilyPond extension I updated it to the newest mediawiki (1.5.3) and a recent lilypond version (2.7.6). Works beautifully here.
The last comment is from over 9 months ago. What is the status of this after it was "working beautifully"?
Hi, well, after not hearing _anything_ about the chances to integrate it into wikipedia, I sorta lost interest in updating the extension. Having said that, if there is actually a _chance_ that _anybody_ is interested, I will be happy to address any raised issues. However, you have to convince me that I will not again invest much time and effort for _nothing at all_. As I said, last time I tried it, it was working beautifully. I have no reason to expect that this changed since my last test. Ciao, Dscho
There are a number of people strongly interested in this at Wikisource. Where an 1880 Encyclopedia of Music is curently being worked on. However I can not vouch for the whims of the developers. All I can promise is that I will continue to ask about it. Sorry if that isn't very convincing, but I don't know what else I can do to get this feature enabled.
So where is this demand I see on musipedia.org, that this is possible and that it works. I will realy love to be able to put music sheet in wikipedia (or wikisource), and I'm sure, I'm not the only one. I raise the priority, so that it could be a little bit accelerated if possible.
If I'm not mistaken, Musipedia doesn't run MediaWiki and isn't even a wiki, for that matter. The developers have already installed several extensions just for Wikisource (and some just for Wikinews). If active Wikisource contributors start pinging the wikitech-l mailing list, hopefully that could happen with this extension.
What's the status of this possible extension working with all the Wiki projects? I am particularly interested in the functionality of it with respect to Wikibooks. I've just recently joined up there and I want to be able to start writing musical examples. Thanks for the work you've done so far!
The status is basically that someone needs to get on IRC, bother Tim Starling about this, and hope he gets interested. He'd probably be the one to test and install any extensions for something like this.
(In reply to comment #16) > So where is this demand > I see on musipedia.org, that this is possible and that it works. I will realy love to be able to put > music sheet in wikipedia (or wikisource), and I'm sure, I'm not the only one. I raise the priority, so > that it could be a little bit accelerated if possible. I agree with your comment. I'm trying to write courses on music in wikiversidad (the spanish branch of wikiversity) but i couldn't because we're not able to write scores. I have seen too many comments requesting the same thing, but (as it happenned to me) never got answers to those comments. I'm just a musician and i want to write music in a wikiversidad article...but it discouraged me that it's too hard to find answers and to find someone that can include lilypond in mediawiki software. Excuse me, please, for my english. P.D.: the only place that led me here (thanks god, though i'm agnostic) was the IRC channel.
*** Bug 11485 has been marked as a duplicate of this bug. ***
*** Bug 11486 has been marked as a duplicate of this bug. ***
for the german language wikisource it would be very usefull to have lilypond aktivated. Wie have several historical songs and wnat to add the according notes to them. Thanks joergens.mi
It would very helpful to illustrate music articles on Wikipedia (I use the spanish version). I've tried the LilyPond extension on my machine and it works great. What would be the problem to install it on Wikipedia? (after 3 years of request.. :( )
(In reply to comment #24) > I've tried the LilyPond extension on my machine and it works great. What would > be the problem to install it on Wikipedia? (after 3 years of request.. :( ) Please read the previous comments before you post. As I stated in comment #19, someone needs to bring this to the attention of one of the core devs, namely Tim or Brion, so either 1) it can be reviewed for security and other issues and then either enabled, or 2) the problems can be specified and addressed by the author(s), then the extension enabled. If nobody takes it upon themselves to pursue the issue, nothing will get done.
(In reply to comment #25) > Please read the previous comments before you post. As I stated in comment #19, > someone needs to bring this to the attention of one of the core devs, namely > Tim or Brion, so either 1) it can be reviewed for security and other issues and > then either enabled, or 2) the problems can be specified and addressed by the > author(s), then the extension enabled. If nobody takes it upon themselves to > pursue the issue, nothing will get done. Thank you for your fast answer. I'd read the previous comments, but I guessed that in 7 months since your previous comment someone did asked on IRC (but I can't confirm that, because the discussions on IRC are not logged). I'll try to leave the request there to Tim or Brion. What is the server/channel? Thanks in advance, and excuse me for my limited english.
irc://irc.freenode.org/mediawiki or irc://irc.freenode.org/wikimedia-tech. You'll have to be persistent if you want to see this get dealt with: both of them are often busy and might disregard issues that aren't important enough for them to be bothered repeatedly about.
It would be very helpful for both Wikisource and the Commons to have LilyPond music files enabled. --[[User:Quadell]]
When this extension is ok, add it on fr.wikiversity project
*** Bug 14480 has been marked as a duplicate of this bug. ***
Would be a good feature for Wikisource. I hope we will be able to work with it soon. --~~~~
It would be great if we would have a tool like this at our de.wikisource Projekt. A very useful tool for books like this. http://de.wikisource.org/wiki/Allgemeines_Deutsches_Kommersbuch
> Would be a good feature for Wikisource. I hope we will be able to work with it > soon. After 4 years (and being just to install an existing plugin) it seems that the developers are not interested at all... There are other projects where you can submit musical sheets. Two of them are Mutopia: http://www.mutopiaproject.org/ and Musipedia: http://www.musipedia.org/ But is really a shame that we can't submit important pieces of world culture on Wikipedia :(
It is not *just* a matter of installing an existing plugin. The plugin must first be reviewed for security by a core developer. Since the plugin is basically a wrapper around LilyPond, LilyPond needs to be verified to be secure too (probably not manually, but by looking up statements from its developers, figuring out its basic execution model, etc.). This is not a trivial activity in terms of how much time it would take, and there is exactly one person who's both qualified to do it and who might possibly be willing at some point: Tim Starling. As I said more than a year ago in comment 19: ***************************************************************************** ***************************************************************************** ****** ANYONE WHO WANTS THIS SHOULD TALK TO TIM STARLING ABOUT REVIEW- ****** ****** ING IT, AND BE PERSISTENT IF HE DOESN'T RESPOND RIGHT AWAY. ****** ***************************************************************************** ***************************************************************************** No one has done this. Posting more "I want this too" comments to this bug is *not* likely to help, just as it hasn't helped for the last four years. What it *will* do is make it harder for interested readers to find the comments that are actually useful for making progress here, like comment 19, comment 25, comment 27, or this one. So please *stop posting comments if they don't add any useful info*. Anyone looking at this bug already *knows* that a significant number of people want this.
(In reply to comment #34) > It is not *just* a matter of installing an existing plugin. The plugin must > first be reviewed for security by a core developer. Since the plugin is > basically a wrapper around LilyPond, LilyPond needs to be verified to be secure > too (probably not manually, but by looking up statements from its developers, > figuring out its basic execution model, etc.). This is not a trivial activity > in terms of how much time it would take, and there is exactly one person who's > both qualified to do it and who might possibly be willing at some point: Tim > Starling. I can only say that Lilypond extension disable the macros for security reasons. But of course, that doesn't assure that the plugin is safe enough... > As I said more than a year ago in comment 19: > ***************************************************************************** > ***************************************************************************** > ****** ANYONE WHO WANTS THIS SHOULD TALK TO TIM STARLING ABOUT REVIEW- ****** > ****** ING IT, AND BE PERSISTENT IF HE DOESN'T RESPOND RIGHT AWAY. ****** > ***************************************************************************** > ***************************************************************************** > No one has done this. I tried. Maybe I was not so much persistent. But I don't like to be bothering and bothering anyone. The request is here, since 4 years ago. > Posting more "I want this too" comments to this bug is > *not* likely to help, just as it hasn't helped for the last four years. What > it *will* do is make it harder for interested readers to find the comments that > are actually useful for making progress here, like comment 19, comment 25, > comment 27, or this one. You say that developers will have to check the security. So, I don't see so much useful ways to do something from the users side. > So please *stop posting comments if they don't add any useful info*. Anyone > looking at this bug already *knows* that a significant number of people want > this. That is the point: so why we have to bother and insist and bother again the developers to make them realize that? Sorry, but I really don't understand that. And is the reason of my previous comment (plus, I think that putting a link to other project that allow the requested feature IS useful)
(In reply to comment #35) > I tried. Maybe I was not so much persistent. Apparently not. Try on IRC as well as e-mail. > You say that developers will have to check the security. So, I don't see so > much useful ways to do something from the users side. Users can only try to get the appropriate people to look at it. > That is the point: so why we have to bother and insist and bother again the > developers to make them realize that? Because Wikimedia has too few employees and is too disorganized right now to mandate that anyone is *required* to keep up with feature requests or submitted patches or much of anything else that's not critical to stop the site from falling to pieces. So you have to rely on the good old-fashioned "squeaky wheel gets the grease" method. Efficient it's not, but it's the state of things right now. When there are more senior developers, one might be assigned to review of patches and extension enable-ments, but there are only two now, and they have too much to do already without attempting to review every patch and extension that anyone submits.
My understanding is that a) safe mode is not secure, being trivially DoS-able by short infinite loop scripts b) safe mode will not work for many of the free scores available on the web The problems with LilyPond are sufficiently severe that I have, from time to time, researched alternative music renderers such as Philip's Music Writer that don't have an embedded scripting language. But if you really want to write a LilyPond extension, you could at least go to the trouble of reviewing its security, and dealing with the obvious problems in your code.
Please check out the new extention for music transcription at http://testwiki.smolenski.rs/index.php/ABC_Extension This is NOT Liliypond
(In reply to comment #38) > Please check out the new extention for music transcription at > http://testwiki.smolenski.rs/index.php/ABC_Extension > > This is NOT Liliypond Looks good to me, does it come with code? Does it use abc2ly like http://www.mediawiki.org/wiki/Extension:AbcMusic , or is it abc2ps, abcm2ps or jcabc2ps? Nothing wrong with a having a choice of solutions, right?
On second thoughts I don't like it so much. It doesn't let me bash the keyboard like Ben Folds in Jackson Cannery. That's kind of fun. LilyPond has a "special notehead" feature. http://lilypond.org/doc/v2.10/Documentation/user/lilypond/Special-noteheads#Special-noteheads I'm torn. Should I think like a sysadmin or a musician?
It's abcm2ps, and you don't need to be so sarcastic. Using LilyPond directly seems to be out of question for security reasons. The question is, is using abc2ly or abc2ps also out of question? That is, do you believe that somehow, through specially-crafted ABC notes, someone would be able to create a malicious LilyPond or PS file? If this is not a problem, then Extension:AbcMusic could be used, or a modified version of it that would use abcm2ps instead of abc2ly. If it is, a new tool, say, abc2svg could be made that would render ABC without the intermediate scripting language.
LilyPond is not out of the question, and I'm not being sarcastic. TeX has similar issues, and we managed to use that in a reasonably safe way.
So, if I'm understanding this, the options are either extend texvc to sanitize lilypond, which seems close to tex, using abc2ly to generate safe lilypond, or using some other converter to render abc without lilypond. Is there a reason to prefer one over the other, or is it just a matter of getting a working extension into SVN that doesn't pass lilypond code directly?
Yes, that's basically it. Full option tree is something like this: 1. Use LilyPond with a sanitizer. 2. Use ABC with... 2.1. abc2ly. 2.2. abc2ps. 2.3. custom renderer. The first question is that of usability: ABC is simpler to write than LilyPond, but LilyPond has more features. Should we use one or the other (or both)? Community input is needed here, for example I'd like to know did people have needs that could not be fulfilled by ABC? (Allgemeines Deutsches Kommersbuch mentioned here, I think could be done with ABC.) If the community feels that it needs LilyPond, then a sanitizer has to be written. I have no idea who would do it or how much would it take. If ABC is acceptable, it could theoretically be enabled tomorrow. Choice exists between using abc2ly (converts ABC to LilyPond, then converts LilyPond to PNG) and abc2ps (converts ABC to PostScript, then converts PostScript to PNG). The choice is mostly technical, and I intend to run some benchmarks in order to see what would be the faster solution. Using a custom renderer that would make SVG or PNG files directly from ABC would be needed only if the previous two are deemed unsafe, which doesn't seem to be the case. Another smaller issue is: if ABC is used, should it be extended so that it is more user-friendly? Community input is needed on this.
(In reply to comment #44) > Yes, that's basically it. Full option tree is something like this: > 1. Use LilyPond with a sanitizer. > 2. Use ABC with... > 2.1. abc2ly. > 2.2. abc2ps. > 2.3. custom renderer. > The first question is that of usability: ABC is simpler to write than LilyPond, > but LilyPond has more features. Should we use one or the other (or both)? > Community input is needed here, for example I'd like to know did people have > needs that could not be fulfilled by ABC? (Allgemeines Deutsches Kommersbuch > mentioned here, I think could be done with ABC.) > If the community feels that it needs LilyPond, then a sanitizer has to be > written. I have no idea who would do it or how much would it take. This is a non-choice. The community desperately wants to be able to transcribe music and LillyPond has no realistic chance of being enabled in the forseeable future. This bug was created in 2004 and no work is ongoing to make LilyPond safe to use. If there is any method that can be enabled within the month the community support for using that method will be there. If you need this proven, please explain what you will want for proof of support and I will work on it. > If ABC is acceptable, it could theoretically be enabled tomorrow. Choice exists > between using abc2ly (converts ABC to LilyPond, then converts LilyPond to PNG) > and abc2ps (converts ABC to PostScript, then converts PostScript to PNG). The > choice is mostly technical, and I intend to run some benchmarks in order to see > what would be the faster solution. Using a custom renderer that would make SVG > or PNG files directly from ABC would be needed only if the previous two are > deemed unsafe, which doesn't seem to be the case. > Another smaller issue is: if ABC is used, should it be extended so that it is > more user-friendly? Community input is needed on this. User-friendly is good :) But I think once an extention is enabled and people have a chance to actually work with it, then we can get better input on what sort of improvements are going to be most wanted. No one has had much of a chance to learn music notation yet, and I am not sure sure what we will find lacking when trancribing real texts. Birgitte SB
Sure there are choices. Although one extension was vetoed, the underlying application can still be used. The most visible choice to the community would be how much functionality would be lost by using ABC syntax instead of LillyPond. The others seem less drastic, since it shouldn't be too difficult to change the rendering pipeline if needed.
(In reply to comment #46) > Sure there are choices. Although one extension was vetoed, the underlying > application can still be used. The most visible choice to the community would > be how much functionality would be lost by using ABC syntax instead of > LillyPond. > > The others seem less drastic, since it shouldn't be too difficult to change the > rendering pipeline if needed. > I guess my point is that I don't see any real choice over lost functionality when we have had zero functionality for a very long time. The more technical stuff is a little lost on me, but I don't think the community would want to hold out for LilyPond if they can have music notes without it. Birgitte SB
(In reply to comment #46) > Sure there are choices. Although one extension was vetoed, the underlying > application can still be used. The most visible choice to the community would > be how much functionality would be lost by using ABC syntax instead of > LillyPond. More precisely, how much functionality would be lost by using ABC syntax instead of *crippled* LilyPond, with a carefully-selected whitelist of commands? LilyPond is much more powerful than ABC, but a lot of that is surely because it allows arbitrary scripts. I agree with Jennifer that it would be better to get whatever is fastest into place, then worry about details. If ABC is fastest to do and the eventual consensus is that it's not good enough, LilyPond can always be implemented at our leisure. They can both be enabled in parallel, ABC notation can be manually and/or automatically converted over time, and ABC can maybe be disabled eventually. In any event, ABC is likely to be good enough for the large majority of uses; in the few remaining cases a manually-created PNG or SVG can be used if people like, pending a more sophisticated solution. Note that I say this on general principle, knowing nothing about ABC, LilyPond, or indeed music or musical notation in general. :) But I think that on a technical basis, we should go with whatever's fastest to do in this case, even if it misses some functionality.
I notice there are a lot of features that are in ABC 2.0 that are not in ABC 1.6, including features very likely to be needed in Wikipedia, such as multi-voice staves (already used in the PNGs in [[Fugue]]). It might be nice to have a survey of the capabilities of the various ABC converters.
As a user of music writers who did know nothing about abc before this discussion I tried to get a quick grasp of the features of this product. If it is technically possible to integrate Lilypond to Wikipedia I would prefer that solution. abc is musically a clear second choice.
(In reply to comment #48) > (In reply to comment #46) > > Sure there are choices. Although one extension was vetoed, the underlying > > application can still be used. The most visible choice to the community would > > be how much functionality would be lost by using ABC syntax instead of > > LillyPond. > > More precisely, how much functionality would be lost by using ABC syntax > instead of *crippled* LilyPond, with a carefully-selected whitelist of > commands? LilyPond is much more powerful than ABC, but a lot of that is surely > because it allows arbitrary scripts. > Yes, and we should also consider that some pieces may already be typeset in standard format. So if the extension implements the full ABC format, we may be able to host some of this content. A crippled LilyPond, even if it has more functionality, would be more hit or miss. > I agree with Jennifer that it would be better to get whatever is fastest into > place, then worry about details. If ABC is fastest to do and the eventual > consensus is that it's not good enough, LilyPond can always be implemented at > our leisure. They can both be enabled in parallel, ABC notation can be > manually and/or automatically converted over time, and ABC can maybe be > disabled eventually. In any event, ABC is likely to be good enough for the > large majority of uses; in the few remaining cases a manually-created PNG or > SVG can be used if people like, pending a more sophisticated solution. > Or we could consider supporting multiple music formats the equivalent of hosting multiple image formats. Although it is more complicated since people would edit the code on wiki, if typeset music is available it would be easier to add if it didn't need to be converted first. > Note that I say this on general principle, knowing nothing about ABC, LilyPond, > or indeed music or musical notation in general. :) But I think that on a > technical basis, we should go with whatever's fastest to do in this case, even > if it misses some functionality. > Yeah, I'm with you there. I'm not even 100% sure that texvc could easily parse LilyPond, but I wasn't aware of ABC when I looked at that.
(In reply to comment #49) > I notice there are a lot of features that are in ABC 2.0 that are not in ABC > 1.6, including features very likely to be needed in Wikipedia, such as > multi-voice staves (already used in the PNGs in [[Fugue]]). It might be nice to > have a survey of the capabilities of the various ABC converters. abcm2ps does (partly) implement ABC 2.0 standard, in particular multi-voice staves, though it appears that it can't color different voices differently if that it what you had in mind. From the readme: > The main features of abcm2ps are quite the same as the abc2ps ones, > but they are closer to the current ABC standard (draft IV - 14/8/2003) Other tools, and I tried abc2ly, abc2ps, yaps and abc2midi can't convert abcm2ps' sample file (with a multi-voice staff) properly, so I guess the benchmarking isn't very needed...
(In reply to comment #48) > More precisely, how much functionality would be lost by using ABC syntax > instead of *crippled* LilyPond, with a carefully-selected whitelist of > commands? LilyPond is much more powerful than ABC, but a lot of that is surely > because it allows arbitrary scripts. What exactly is meant by "arbitrary scripts"? I have a fair amount of experience with LilyPond use, especially through the Mutopia repository (http://www.mutopiaproject.org), and I have not come across any Scheme code in real-world use. Macros are occasionally defined, but I don't think that the OP was referring to those. To the best of my knowledge, LilyPond is an extremely rich and powerful system by itself.
(In reply to comment #53) > (In reply to comment #48) > > > More precisely, how much functionality would be lost by using ABC syntax > > instead of *crippled* LilyPond, with a carefully-selected whitelist of > > commands? LilyPond is much more powerful than ABC, but a lot of that is surely > > because it allows arbitrary scripts. > > What exactly is meant by "arbitrary scripts"? I have a fair amount of > experience with LilyPond use, especially through the Mutopia repository > (http://www.mutopiaproject.org), and I have not come across any Scheme code in > real-world use. Macros are occasionally defined, but I don't think that the OP > was referring to those. To the best of my knowledge, LilyPond is an extremely > rich and powerful system by itself. > LilyPond is expressive enough that it's possible to write malicious code that could take a long time to, or possibly never, complete. The workaround for this would be similar to what is done for math; to tokenize the input, and make sure the commands are in a known whitelist of safe commands. That way, any music with a macro would be flagged as an error, since the might not be safe. So it would probably take longer to add LilyPond support than ABC (which shouldn't preclude adding LilyPond later) since someone would need to develop this preprocessor. Then, since we'd only support a subset of LilyPond, many valid files would not be supported, at least without modification; and we don't know what portion that would be.
> LilyPond is expressive enough that it's possible to write malicious code that > could take a long time to, or possibly never, complete. A really rudimentary research into LilyPond's documentation would show that the safe mode relies on GUILEs safe mode, which has been around so long that you can be certain that it is fleshed out. So, a preprocessor is not needed at all. The argument that many freely available LilyPond files do not work in safe mode is doubly bogus: 1) they can be fixed with relatively small effort, as most pieces should not require any special (and possibly unsafe) operations at all, 2) the extension is not about copying existing stuff into the wiki. It is about creating content collaboratively, so the unsafe features would simply not be used. I have to admit, though, that I am amazed how long this discussion would drag out. If I had known about the timely decision-making powers of the responsible persons, I do not know if I would have spent the time writing the extension in the first place. It is really a sorry story.
(In reply to comment #55) > A really rudimentary research into LilyPond's documentation would show that the > safe > mode relies on GUILEs safe mode, which has been around so long that you can be > certain that it is fleshed out. You're missing the fact that LilyPond's safe mode *does not claim* to prevent infinite loops. It *only* prevents operations like file access. If you would read the documentation that you condescendingly told us all to read, you would find: "--safe does *not* detect resource overuse. It is still possible to make the program hang indefinitely, for example by feeding cyclic data structures into the backend. Therefore, if using LilyPond on a publicly accessible webserver, the process should be limited in both CPU and memory usage." http://lilypond.org/doc/v2.9/Documentation/user/lilypond/Invoking-lilypond Therefore it *must* be run through some kind of sanitizer to avoid "cyclic data structures" or whatever. Killing the process if it eats too much CPU and RAM and having the render fail would also be a possible solution, but not a very nice one. > The argument that many freely available LilyPond files do not work in safe mode > is doubly bogus: The software's documentation appears to disagree: "Note that --safe will prevent many useful LilyPond snippets from being compiled." http://lilypond.org/doc/v2.9/Documentation/user/lilypond/Invoking-lilypond River has written [[mw:Extension:ABC]]. Given the author, if she's willing to say it's safe, it could probably be enabled without an extensive separate security review. It seems like the most useful way forward at this point, even if it's not as powerful as LilyPond.
Yes, infinite loops would allow attacks, that is correct. I could think of a number of strategies how to teach LilyPond to avoid that, and I am sure that at least one of those strategies would have been implemented already a long time ago, if the issue had been raised on the lilypond-devel list. (Just as the --jail option has been implemented already.) The documentation can say as much as it wants about useful snippets, the reality is that you can do almost everything you want without using Scheme, however. And certainly the rest of the things only matters when you want to produce a printed copy of the music which adheres to a certain standard. In other words, nothing I would expect a MediaWiki extension to support. In yet other words, it would be a safe bet that a --safe=no-scheme mode in LilyPond would do all ABC is capable of, and more. However, your last remark really boils it down to the point: ever since I submitted the initial version of the LilyPond extension, everybody seemed to tell me in all but writing that I should just go away, because ABC would be chosen eventually. That's perfectly okay. I have other interesting stuff to do.
I've got two points out of this long time discussion (5 years now). Long time only because there long times of inactivity in between. And it seems not only to be a discussion for technical reason but also a lot of politics. There seems to be two systems ABC and lilipond, both are more or less capable of doing the job. ABC less and lilipond more from my personal point of view. There is an '''old''' implementation which has been proven to work with lilipond and wikis and this solution is already in use on other mediawiki installations but not in wikimedia universum. And '''newer''' one based on ABC which translates to lilypond and using lilypond afterwards to to the job. It may be possible to work with ABC in an alternative mode via postscript but both modes must be tested first for stability, resources and performance. From this point of view it seems to be a clear joice to use lilipond directly without any stumbling intermediates. The big argument against lilipond - as far as I understand from the discussion - is the possibility for an attack by introducing malicous code (infinite loops). But as far as I can see there are possibility provided by lilipond itself to prevent exactly this. Using save mode, no-scheme and other. The parts of lilypond which will use these ''hazardous'' parts have no benefit for our purposes and lilypond can be used with this options disabled. Now my question is: Why should we use ABC when we can have the needed things directly in a save manner. I would beg to have exact look on this save mode of lilipond and if the risk is low enough activate it on the wikis. If there is a need for further testing within the community, I think the german language wikisource would be able to do some tests in the real live, We have several pieces of text where we need a music extension. Have a look here http://de.wikisource.org/wiki/Allgemeines_Deutsches_Kommersbuch:163 (roundabout 700 pages) or here http://de.wikisource.org/wiki/Seite:Die_Gartenlaube_(1895)_896.jpg
(In reply to comment #58) > The big argument against lilipond - as far as I understand from the discussion > - is the possibility for an attack by introducing malicous code (infinite > loops). But as far as I can see there are possibility provided by lilipond > itself to prevent exactly this. Using save mode, no-scheme and other. Nobody that I've seen has presented evidence that there are options in current versions of LilyPond that will prevent the possibility of unreasonable or even unbounded CPU/memory usage. The LilyPond documentation says safe mode does not do this. If there are other options like "no-scheme" (which I'm fairly sure has not been mentioned before and which I can't find in the LilyPond documentation), these need to be pointed out so that they can be considered. River (who is a long-time developer and root sysadmin) has said that the ABC extension should be no less safe than ImageMagick. If someone pursues that, it could therefore probably get enabled within a week on technical grounds. There is currently, to my knowledge, no LilyPond extension available that even claims to prevent trivial DoS attacks, and it will not be a credible contender until someone writes one. If you think you can write such an extension, but don't want to waste the effort when it might not get enabled, try asking Tim whether he'll agree in advance to review it. I agree that our review process is dysfunctional, but it's dysfunctional because of lack of trusted people willing to review things, and complaining about it is not going to fix that. You can either give up or make the best of it, your decision.
Sorry Aryeh Gregor, I'm not capable of doing the job you wants to urge me todo. :) I' working as a software engineer but I don't have any experiences in that type of software. But when the central Software team is shure that this - sorry from my point of view nearly unknown ABC - is the right choice and can be activated immediately why don't you do that. The community is asking for a usable method of typesetting music notes. And even if the the second best choice is the solution, than we should take that. Second best choice simply because even ABC is using the liliypond system, which seems to be a de facto standard and this ABC system is just the sanitizer your are talking about.
Sorry to be so frank, but it is _utterly stupid_ not to ask for a "really safe" mode on the LilyPond mailing list. It is _Open Source_! They will answer you, and if you are not pleased with what is available, chances are good that they will implement it! In the alternative, it is _utterly malicious_ if you want to corrupt the process of getting an extension into WikiPedia, and therefore refuse to take the appropriate steps. If you want to have an ABC extension (whose capabilities you would have to admit yourself are rather limited if you compared ABC to LilyPond) rather than a LilyPond extension, you should be honest enough to say it directly, and not dance around the issue.
(In reply to comment #61) > Sorry to be so frank, but it is _utterly stupid_ not to ask for a "really safe" > mode on the LilyPond mailing list. It is _Open Source_! They will answer you, > and if you are not pleased with what is available, chances are good that they > will implement it! *Any* interested party could have asked what Wikimedia's requirements were and asked themselves on the LilyPond list whether they could be implemented. You cannot blame Wikimedia sysadmins for failing to crusade on behalf of getting LilyPond enabled on Wikimedia. If you want that, you have to put in the work yourself. That includes ensuring that the objections to LilyPond are addressed. The objections were clearly stated from the first time an appropriate individual reviewed the software, more than a year ago, and if they weren't clear enough then you could have sought more clarity. If you think this should have been brought up on the LilyPond mailing list, why didn't you do so yourself? > In the alternative, it is _utterly malicious_ if you want to corrupt the > process > of getting an extension into WikiPedia, and therefore refuse to take the > appropriate steps. No one is under any obligation to get an extension enabled anywhere. If *you* want the extension enabled, *you* can take the appropriate steps. > If you want to have an ABC extension (whose capabilities you would have to > admit > yourself are rather limited if you compared ABC to LilyPond) rather than a > LilyPond > extension, you should be honest enough to say it directly, and not dance around > the issue. I don't personally care whether LilyPond or ABC is used. I would never use either one either as a reader or an author. I had never heard of either before I started looking into this bug report, and I have to this date never used either. I literally cannot even read musical notation. I am only putting any effort into this at all (as a volunteer MediaWiki developer who has no decision-making power on this but is familiar with how extensions get enabled) because it's clear a substantial number of users are interested. I am continuing to put effort into it, out of charity, despite your abusive language, because I still do care about the large number of users who are interested. A number of people have said that ABC would at least be a start, even if it would be somewhat inferior, and it looks like it could get enabled rapidly. I don't care who you want to blame for it, but an acceptable LilyPond extension does not exist right now, while an acceptable ABC extension does. For that reason, and that reason alone, I've suggested that it be enabled right now instead of waiting around indefinitely for a LilyPond extension. A LilyPond extension (if one is made that's acceptable to use) could be enabled in addition to an ABC extension if the community desired this at some later date.
What we're looking for here is much different from a typical "safe mode"; the proposed "no scheme mode" may be more appropriate, if that gets implemented. This doesn't need to be an ABC vs. LilyPond thing. Using one now doesn't mean we can't use the other later, or even that they can't coexist. At this point, it's more a matter of what can be ready the soonest, since a few projects are waiting for this functionality, and it looks like ABC is the furthest along. Yes, if there's a simple way to enhance LilyPond to make it useful for this, someone who is involved with LilyPond should ask for it. Then, someone should get the extension to use it, and advocate to have that enabled so that projects can use it. But if ABC is ready now, we should use it now. That will give the communities functionlity they've been waiting for, and should have little or no effect on whether LilyPond is used when ready.
> If you think this should have been brought up on the LilyPond mailing list Back when I provided my extensions, no such issues were raised. In the meantime, I got really frustrated. And the issue you raise was only mentioned 3 years after I showed some code. If you think that is any way to encourage people to do more work (as you suggested), then you are sorely mistaken. This issue will be a text-book example in some future book how not to handle issue reports. With the dates marked in red, and blinking. > At this point, it's more a matter of what can be ready the soonest The duration of this thread rather contradicts your statement.
(In reply to comment #64) > Back when I provided my extensions, no such issues were raised. In the > meantime, I got really frustrated. And the issue you raise was only mentioned > 3 years after I showed some code. It's perfectly fair and understandable if you didn't want to put more work into it because you felt dispirited by the lack of response. It's not remotely reasonable, on the other hand, to claim that anyone *else* is "utterly stupid" or "utterly malicious" for being uninterested as well. No one in the world is under any obligation to put any effort into getting problems with third-party extensions worked out so they can be enabled on Wikimedia, and no one can be blamed for the fact that it didn't happen. > > At this point, it's more a matter of what can be ready the soonest > > The duration of this thread rather contradicts your statement. No it doesn't, unless you think that the fact that everyone's waited four years already means they'd be happy waiting another four years.
I just want to add may points. It's very easy to say it's our job because we want to have it. And it's right to some extension. I would talk to the lillypond people if i could. I don't understand your objections against lilipond though I'm not able to get correctly communicated to the lillipond community. So please make it clear to a normal user with no knowledge about either lilipond nor wikimedia software what is going wrong then I'll try to be the parrot who talks to lillypond people. In german there is a game called "stille Post" A group of n people are sitting around a table. Person 1 tells person 2 something in a way that nobody else can hear it and writes it to a paper that no one can read at that moment. person 2 tells person 3 the secret and so on the last person in the row tells loud what message he receives this will be compared to the written message. There is allways a big laughing in the end. If you thing this is a usefull way to talk about security issues in such a big software system, lets give it a try. On the other hand if you, who knows these issues exactly, can easily give a clear an simple definition of the problem(s) communicate with the lillypond people and we will get a good fix in the end for lillipond. I don't think someone will burden the work to you but everybody would be happy if the problems can be fixed in a professional manner by a correct definition of the problem. If there is assistance needed, that has nothing to do with this technical aspects simply ask
We want LilyPond to be as safe to call as, say, ImageMagick (or ABC!). If it's configured correctly, we should be guaranteed that it will terminate in a reasonably short amount of time while using a reasonably small amount of memory/disk/network/etc. I think this is a clear enough requirement to relay. Alternatively, if LilyPond devs won't do this, someone could write a preprocessing sanitizer of some kind like we have for LaTeX, as long as it achieves the same effect. I'm not personally interested in spending time trying to do this myself, sorry. I'm already spending more time on this feature than I'd like given that I'll never use it. I'm not going to post any more about this unless I have something further to add.
Free sheet music is important for all wikisource projects.
removed shell keyword as there's nothing to do on the shell presently
As has been talking about from [[#61]], someone (Werner LEMBERG) started a discussion at lilypond-devel archive. If somebody else wants to see the discussion, go to http://lists.gnu.org/archive/html/lilypond-devel/2009-02/msg00019.html I'm also interested in some music markup for Wikimedia Projects (I'm from portuguese Wikibooks). Thanks to everybody for the effort spent until now!!! [[b:pt:Heldergeovane|Helder]]
There is a place when is allowed to use lilypond code to make sheet music, I don't remember its name...it is like "wiki..." and it says it's a sandboard. If we can use it in that wikipage, why cannot be used in wikipedia or wikiversity? I think that while we keep discussing if we should include lilypond or not, there are a lot of projects that can't be done by that reason. I could mention a manual to learn to read music...if we had a lilypond module everyone could improve that manual and make that project succeed. Someone once told me "Hey, but you can send a picture of the score and the people can see it"...but imagine for a moment that this thing happens but not with notes, but with TEXT: "Hey man, you can upload a picture of your essay about general relativity and the people could read it" and another guy (maybe a PHd), in the opposite side of the world, wants to edit the document because he thinks it should be more accurate...¿How could he do it?¿Maybe writing down all the essay and send a picture again with the text corrected, and so on? ¿Isn't it more difficult to improve the information in wikipedia that kind of "send and receive this pictures"? PLEASE, remember: "NOTES FOR A MUSICIAN ARE LIKE TEXT FOR A WRITER"
All I can do is to agree to Johann. Musicians do really need a tool for writing music like mathematicians do need their TeX for writing formulae. And math people have their tool since wikipedia started; musicians are waiting patiently since 2004 when this thread started. And there is no visible sign that this state will change. It would be better to get a working tool with deficiencies fast than to wait another five years of sleeping discussion until perhaps a perfect solution will be worked out. And a suboptimal solution can be improved during the time it can be used.
If the security issues discussed above are still a problem, then the "release first" approach might not be such a good idea.
Okay, to be _really_ frank, this whole talking in the wrong place (security issues should be raised with the maintainers! That should be _obvious_!) and the whole politicking and assuming-without-asking around this issue are really appalling. If there is somebody sane to work with, I am more than willing to address all issues. I will not work with people I lost respect for, though. So, if you're interested, these are the steps to be taken: 1) Pester the people who are responsible for not including LilyPond support for the _real_ reasons. 2) Sort out any handwaving arguments until you have clean and clear questions or issues. 3) Send technical issues off to the LilyPond devel list (I'll be waiting there). 4) Work with me to resolve the issues. 5) Go back to 1) (as I said, I lost respect for quite some of those people, so I am the wrong guy to do it).
to Dscho: I understand you're dissappointed and/or dissatisfied and/or annoyed, I don't really know exactly which one of these, since you invested some work to wikipond without result. But I'd suggest, to stick to the facts and to stay sober and to keep emotions out of the discussion. Arguments like your 1)-3) are rather counterproductive. I'd hope a rational discussion would more likely produce results.
(In reply to comment #75) > But I'd suggest, to stick to the facts and to stay sober and to keep emotions > out of the discussion. Arguments like your 1)-3) are rather counterproductive. I don't agree with you, brf. I think Dscho said what is most rational _to do_: Since #49 there was no comments from the Wikimedia developers about this bug and they had not discussed the issues with the LilyPond developers. (I was looking for this at http://lists.gnu.org/archive/html/lilypond-devel/2009-04/msg00273.html [Sorry for the delay in the answer Dscho, I was a little busy this two weeks =/]) But, it is needed to have a clear description of the problem in order to have a solution. In other words, even if someone could work on LilyPond to correct the problems, they would need to know what the problems are! In this sense, we need the participation of Wikimedia team... Helder
There has been no politicking here with regards to ABC vs. LilyPond. Tim Starling is the most likely to enable either, and he's said (comment #40) that he believes LilyPond to be musically much superior. But by the admission of its own documentation, LilyPond might use an arbitrarily large amount of resources if maliciously fed pathological input. That is the reason it is off the table, period. It's a DoS vector. If anyone (Wikipedia users, LilyPond developers, whatever) want to try getting that fixed, they're welcome to. Claiming that people have ulterior motives for going with ABC is not only offensive, but patently absurd. The concerns have been stated very clearly. Music support is not a top-priority feature and might not receive as much initiative from Wikimedia people as music lovers would like, but that's life. There are thousands of open bugs and most don't end up with much developer attention. It is not the duty of MediaWiki developers to try getting any *specific* bug or feature request fixed, even the paid ones. If *you* want to see a particular bug fixed, *you* have to try to persuade one or more suitable developers to give it more attention. Although I can't even read sheet music, I've been periodically pestering Tim about enabling ABC, since some users seem to want it a lot (even if it's markedly inferior to LilyPond, which everyone seems to agree on). He says maybe he'll enable it after the 1.15 release and the next scap. Even if ABC is enabled, LilyPond could still be considered at some later date if it's demonstrated to not be a security concern. In that case, ABC could be phased out from Wikimedia content in favor of LilyPond, and ABC eventually disabled. (In reply to comment #76) > I think Dscho said what is most rational _to do_: Since #49 there was no > comments from the Wikimedia developers about this bug and they had not > discussed the issues with the LilyPond developers. I am a MediaWiki developer and have commented repeatedly since comment 49. I don't have the ability to enable extensions on Wikipedia, I only have commit access, but I believe I understand the requirements adequately, and if I don't, I can ask people who do. If anyone requires clarifications on what is needed for LilyPond to be usable by Wikimedia, they can ask. So far, no one has. I have plenty of other things to do and don't care about this bug immensely, so I'm not going to be willing to take the initiative to dig up and pester LilyPond devs about getting this fixed. If LilyPond devs feel the same way, then I guess ABC will just be used forever. > But, it is needed to have a clear description of the problem in order to have a > solution. In other words, even if someone could work on LilyPond to correct the > problems, they would need to know what the problems are! In this sense, we need > the participation of Wikimedia team... I believe I've given a clear description of the problem in comment 67. If there's anything unclear about it, please ask.
Aryeh: thank you for proving all my points. And I mean all of them. That very much includes the unwillingness to clarify with the LilyPond team how serious said DoS vector might be. I can -- and will -- not work that way.
(In reply to comment #78) > Aryeh: thank you for proving all my points. > > And I mean all of them. > > That very much includes the unwillingness to clarify with the LilyPond team how > serious said DoS vector might be. > > I can -- and will -- not work that way. > How is Aryeh being unwilling here? He's pointed out what the DoS vector is (unlimited resource usage for evil input), and said to ask if anything needed clarification. The seriousness of the possibility of unlimited resource usage and the ease with which an attacker could craft input triggering that (by trying on their local LilyPond install first) seems to be pretty much proven to me. If you believe the DoS issue is no longer an issue (you seem to be suggesting this), please tell us (mentioning what was done to fix it, of course).
Roan: if you have a problem with MediaWiki, no matter if real or perceived, whom do you ask? Exactly. The developers. Not some random obscure bug tracker. Is that _not_ obvious???
(In reply to comment #77) > (In reply to comment #76) > > I think Dscho said what is most rational _to do_: Since #49 there was no > > comments from the Wikimedia developers about this bug and they had not > > discussed the issues with the LilyPond developers. > > I am a MediaWiki developer and have commented repeatedly since comment 49. I > don't have the ability to enable extensions on Wikipedia, I only have commit > access, but I believe I understand the requirements adequately, and if I don't, > I can ask people who do. If anyone requires clarifications on what is needed > for LilyPond to be usable by Wikimedia, they can ask. So far, no one has. I > have plenty of other things to do and don't care about this bug immensely, so > I'm not going to be willing to take the initiative to dig up and pester > LilyPond devs about getting this fixed. If LilyPond devs feel the same way, > then I guess ABC will just be used forever. I'm really sorry about my previous comment... I misunderstood the situation when I wrote "there was no comments from the Wikimedia developers about this bug": * I forgot your Comment #62 =( * I was thinking about Tim/Brion, because by comments like Comment #25 it seemed to me that it would be necessary a explanation of some of them about the "real reasons" that Dscho pointed in (1) Sorry again... Well, as I saw at Comment #55 and at http://lists.gnu.org/archive/html/lilypond-devel/2009-02/msg00022.html the item (b) pointed by Tim (in comment #37) is not really a issue, am I right? But in the same message Graham Percival said that "trying to keep lilypond within certain CPU-time limits is going to be hard". Would this be solved by doing what Dscho said at http://lists.gnu.org/archive/html/lilypond-devel/2009-02/msg00023.html ? After this, was said that "Scheme just should be disabled for the purpose of the MediaWiki extension." (http://lists.gnu.org/archive/html/lilypond-devel/2009-04/msg00265.html). How much of the issues would be solved with this? By the way, I've found this other discussion, now about a new extension ABC http://lists.wikimedia.org/pipermail/wikitech-l/2008-October/039988.html I would like to know if the 3 questions Brion asked about it are also to be considered in the context of LilyPond extension? A nice week to all! =D Bests regards, Helder
(In reply to comment #81) > Well, as I saw at Comment #55 and at > http://lists.gnu.org/archive/html/lilypond-devel/2009-02/msg00022.html > the item (b) pointed by Tim (in comment #37) is not really a issue, am I > right? I should think not. Even if most music won't work, it's a wiki, people can adapt it or rewrite it. > But in the same message Graham Percival said that "trying to keep lilypond > within certain CPU-time limits is going to be hard". Would this be solved by > doing what Dscho said at > http://lists.gnu.org/archive/html/lilypond-devel/2009-02/msg00023.html > ? You mean: > But we could add a simple timeout that says "if this fails to > terminate in 20 seconds, it errors _out_". I assume that would address all DoS concerns, if memory and disk use are also limited (either explicitly, or as a practical matter). I'd assume that any reasonable score could be created in well under 20 seconds. It wouldn't be ideal, though. It would lead to intermittent failure for input that's close to the limit, and it might cause occasional failures if the server is under high load briefly for some reason. For wikitext, which can also be very slow, we have limits like "no more than X of this instruction" instead, calibrated so as to make DoS unlikely. Likewise, for ImageMagick I believe we have pixel limits on what images it will try to resize. This way the software behaves consistently regardless of server load or other hard-to-control factors, but it's harder to do, of course. > After this, was said that "Scheme just should be disabled for the purpose of > the MediaWiki extension." > (http://lists.gnu.org/archive/html/lilypond-devel/2009-04/msg00265.html). How > much of the issues would be solved with this? I'd assume that disabling Scheme would be necessary, but not sufficient. > By the way, I've found this other discussion, now about a new extension ABC > http://lists.wikimedia.org/pipermail/wikitech-l/2008-October/039988.html > > I would like to know if the 3 questions Brion asked about it are also to be > considered in the context of LilyPond extension? I guess . . . those are infrastructural issues, not related to LilyPond itself but to how the images are stored and so on, issues that the MediaWiki extension would have to deal with. Currently we have a fairly ugly system for math images, where they're stored on NFS and not ever expired. I don't know if adapting a similar system would be acceptable or not. I'll try getting Tim or Brion to reply to your questions so you can get a more definite answer.
Hi all. I want to post a comment i did in 2007...i feel pity because nothing has changed since that year. Maybe i have to give up music and become a programmer or an engineer to fix this bug... Note: I prefer lilypond, it is not only superior, but it can be used in many languages, primary goal for wiki projects. Here is the 2007's comment: Good morning. The basic thing is that a music edition feature in wikiarticles (including wikipedia, wikiversity and "wikiversidad") is needed. Many people have asked for it but no answeres are receieved in help pages. I wish this feature be enabled to start a course on music in wikiversdad, but it just don't work in the way that i found in wikisophia.org. I encourage the use of lilypond as a language to create scores, with all its funcionality. It is necessary to listen to the music that is written via a "button" that allow the user to start the playing of what is written; another essential thing is that this "listening" capability can be opened in a new window in order to allow the pupil to listen to the music and to read the music simultaneously. In other hand, i wish it could be in the way that all wikis works; for example, we don't need to install anything additional to begin to write an article, we just click on the "edit" button and everything works fine...we want music to be edited that simply way. Remember we're not programmers, we're just musicians, and we want to worry about the music, not to begin a course on programming languages or algorithms to create a music course. Please, excuse me for my english; maybe this appeared a little "tough", but i'm not good speaking english. I just want to tell that i tried to be as kind as possible with my short knowledge of this language. Thank you very much.
I think wikiversity, wikibooks, wikisource, etc, are - and should be - less restrictive than wikipedia when it comes to trying new plugins. For example, wikiversity has a quiz module, and wikibooks support pdf generation, as opposed to wikipedia. Why not focus upon music edition for wikiversity, wikibooks and/or wikisource, and leave wikipedia for a while? Keep on the good work! /Magnus, Sweden (In reply to comment #84)
(In reply to comment #84) > Hi all. I want to post a comment i did in 2007...i feel pity because nothing > has changed since that year. Maybe i have to give up music and become a > programmer or an engineer to fix this bug... > Note: I prefer lilypond, it is not only superior, but it can be used in many > languages, primary goal for wiki projects. > > Here is the 2007's comment: > We're aware that LilyPond is superior and that people really want music module. We're also aware that this has taken a very long time, and that that sucks. I don't mean to be rude, but yet another post pointing out how useful a music module would be is not very useful or productive. (In reply to comment #85) > I think wikiversity, wikibooks, wikisource, etc, are - and should be - less > restrictive than wikipedia when it comes to trying new plugins. For example, > wikiversity has a quiz module, and wikibooks support pdf generation, as opposed > to wikipedia. > > Why not focus upon music edition for wikiversity, wikibooks and/or wikisource, > and leave wikipedia for a while? > Extensions that got enabled on these small wikis and not on big ones are usually extensions which are (expected to be) slow on large wikis due to their size. The DoS issue that exists (existed?) with the LilyPond extension is in a different ballpark, though: if attackers can make the servers grind to a halt by providing LilyPond with input specially crafted to take a huge amount of time/memory/both to process, it doesn't matter which wiki that input comes from; the size of the wiki is irrelevant here.
Aryeh has asked me to reply to comment 81 since he believes it asks "interesting questions". (In reply to comment #81) > Well, as I saw at Comment #55 and at > http://lists.gnu.org/archive/html/lilypond-devel/2009-02/msg00022.html > the item (b) pointed by Tim (in comment #37) is not really a issue, am I > right? Characterising the hearsay in comment #37 as "research" and then attacking it on that basis is rather bizarre. It's just something someone told me on IRC. All I wanted in comment #37 was a rigorous review of the issues. I still haven't seen such a review, just a contrary opinion on a mailing list. > But in the same message Graham Percival said that "trying to keep lilypond > within certain CPU-time limits is going to be hard". Would this be solved by > doing what Dscho said at > http://lists.gnu.org/archive/html/lilypond-devel/2009-02/msg00023.html > ? Yes, it's quite easy to do. If the lot of you got together and wrote code instead of arguing about it on a bug report, it'd be done by now. Then I'd have code to review. > After this, was said that "Scheme just should be disabled for the purpose of > the MediaWiki extension." > (http://lists.gnu.org/archive/html/lilypond-devel/2009-04/msg00265.html). How > much of the issues would be solved with this? Sounds like overkill. > By the way, I've found this other discussion, now about a new extension ABC > http://lists.wikimedia.org/pipermail/wikitech-l/2008-October/039988.html > > I would like to know if the 3 questions Brion asked about it are also to be > considered in the context of LilyPond extension? I'm not interested in writing requirements documents at this time. I will review code when it comes to me (and when I have time).
Just to put things straight: there was _no_ proper "this extension is not accepted because...". This is inacceptable, even from a MediaWiki big-whig. As I stated several times, I am willing to work on this, but only with people that have demonstrated the ability of working efficiently on this issue, which you, Tim, have not. As to your comment "If the lot of you got together and wrote code instead of arguing": this is just a preposterous mockery of the work I did. If you just lowered yourself to the ground of reality you would realize that it has taken all of _four_ _full_ years for you to comment _just_ _twice_. The worst part, both times you basically said "I'm not interested in writing requirements documents". In effect you are saying "I don't take your work, but I don't tell you why. Nya, nya, nyaaaaa!" That is just a waste of others (and my time). You are not a God, to waste my time like that. Shame on you. Yes, I am venting. But after 4 (in words, FOUR) years I very well deserve to do so. And if you do not acknowledge that, you deserve even less respect than I have left for you.
(In reply to comment #88) > Just to put things straight: there was _no_ proper "this extension is not > accepted because...". This is inacceptable, even from a MediaWiki big-whig. > My interpretation of Tim's comment is that this extension is not accepted yet because the issues mentioned in http://lists.gnu.org/archive/html/lilypond-devel/2009-02/msg00019.html haven't been addressed properly: the unbounded resource usage issue was acknowledged, but since that mailing list thread there hasn't been a clear "yes, this is still an issue" or "no, it's been fixed by now". As long as you (or another LilyPond developer) won't provide an clear and unambiguous yes or no as described above on each of the issues mentioned in that mailing list post, we really don't know what's going on and we can't really do anything. > As I stated several times, I am willing to work on this, but only with people > that have demonstrated the ability of working efficiently on this issue, which > you, Tim, have not. > > As to your comment "If the lot of you got together and wrote code instead of > arguing": this is just a preposterous mockery of the work I did. If there is still no (attempt at a) fix for the issues mentioned earlier (again, I don't know whether they have because no one's telling), then you guys have obviously not written any (relevant) code since your last patch (dating back to 2005) and have been arguing here. > If you just > lowered yourself to the ground of reality you would realize that it has taken > all of _four_ _full_ years for you to comment _just_ _twice_. > > The worst part, both times you basically said "I'm not interested in writing > requirements documents". In effect you are saying "I don't take your work, but > I don't tell you why. Nya, nya, nyaaaaa!" > > That is just a waste of others (and my time). You are not a God, to waste my > time like that. Shame on you. > > Yes, I am venting. But after 4 (in words, FOUR) years I very well deserve to > do so. And if you do not acknowledge that, you deserve even less respect than > I have left for you. > OK, so you've vented. Now can you address my (for a LilyPond developer relatively simple) questions from the first paragraph?
I think there is one big misunderstanding here. Most of the people who are requesting a music module arn't developers, neither wikimedia nor lillipond. They are simple users who have a need for incorporating notes to articles either wikipedia odr wikisource and begging for years for a solution of that. SO most of us have the problem that we have a wish an nobody is supporting us for years. Please could some of the skilled people who know how to incorporate or generate a musik module help the commiunity. As far as I can see you are using more time on arguing as an workin on that issue. Greetings
Hi, Tired of see that this doesn't seems to solve anything (afer 5 years), I wrote to Jimbo this mail with my thoughts. I think the matter is if music have enough relevance to be included in wikimedia projects and dedicate resources (-> developers time) to solve the needed issues to have a safe notation extension. Musicians think so, and wikimedia developers doesn't seem to do it. Let's see what is the Jimbo opinion... Best regards, Natanael. -------- Original Message -------- Subject: Do you think that music have cultural importance? Date: Sat, 09 May 2009 13:54:14 +0200 Hi Jimbo, Sorry to bother you... But, if you think the answer to the subject question is yes, could you please take a look to this bug report: https://bugzilla.wikimedia.org/show_bug.cgi?id=189 ? Is a 5 years bug report, where we are asking for a musical notation plugin for wikipedia/wikisource/wikiversity projects, which is an essential need for teach/learn/document music. All the answers we got is to bother more the developers, because they doesn't think that bug report have a priority. Thinking about wikimedia projects as cultural resources, I think it SHOULD have considered as a priority. Again, sorry to bother you, but I'm tired to see discussions there that doesn't seems to solve anything. Thanks in advance, and thank you also for your amazing projects and cultural resources for all the humanity :) Natanael.
It seems clear that there will be no fixed list of requirements. Either a LilyPond dev is willing to put in the work and submit it for eventual inspection when Tim finds the time, and is willing to then work on any totally new issues that might come up in the course of the review, without any advance guarantee that their work will ever get enabled on Wikipedia; or not. If so, that's great. If not, it looks like LilyPond support won't happen, and ABC will have to do. It's unfortunate that we don't have the resources to get every single bug fixed as nicely as everyone would like, but no software project does. There are thousands of open bugs here and a lot have unreviewed patches attached. If, in fact, nobody from LilyPond is willing to work on this without guaranteed returns (entirely understandable), and nobody on Wikimedia's side is willing to give guarantees in advance (this seems certain), then I suggest that everyone drop the issue until the status quo changes, rather than dwelling on it. Complaining is not going to give the relevant people more time that they can spend on things that are, to them, evidently low-priority. Now could I please ask everyone -- MediaWiki developers, LilyPond developers, Wikipedia users, and everyone else -- to stop using Bugzilla as a discussion forum, and only post further if you have anything to say *that will contribute to fixing the bug* (e.g., an expression of interest in working on a LilyPond solution, or a note on progress in enabling ABC). You're spamming a significant number of uninterested people with every uninformative post you make. The correct fora for recriminations and complaints (on both sides) are wikitech-l and/or lilypond-devel, if anywhere. If you really just have to respond to someone else's misplaced Bugzilla comment, you can do it by e-mailing them privately. Thank you.
General discussion should go on meta: http://meta.wikimedia.org/wiki/Music_markup http://meta.wikimedia.org/wiki/Talk:Music_markup Everyone interested should watchlist that page, and change your preferences to enable email modifications as shown in this image: http://www.mediawiki.org/wiki/File:Preferences_in_rel15-enec318.png
Even Open Office has now a module to incorporate lilypond. Have a look here http://ooolilypond.sourceforge.net/ Why is this not possible to do the same with our wikimedia projekts, especially the wikisource parts for nearly five years now this bug startet at 2004-08-22, it it may be the last bug open below 1000 There is also a need to have this for barrier-free access for disabled people.
OpenOffice runs on client machines and does not have to worry about DoS. If a file uses too much CPU to render, the user can just close the application. This doesn't work for a web app.
It schould be easy to run it on a lower priority and with a high nice level on a Unix maschine if there is any interest to do so. The open office was just another example that liliypond is something like a de facto standard and that some skilled people in the mediawiki world should put some effort on encorporating this needed tool in the mediawiki as an extension.
(In reply to comment #96) > It schould be easy to run it on a lower priority and with a high nice level No, this is not enough. At best, it limits the DoS potential to halting display of all new music, which is still unacceptable. The problem is perfectly solvable, but it's not trivial, and it will require effort. Nobody on Wikimedia's side looks interested at the moment, and nor does anyone from LilyPond given the lack of assurance that their code will end up being deployed. As I said in comment 92, please do not use Bugzilla as a discussion/advocacy forum. It just spams people and doesn't help anything. *Only* post a new comment if you're willing to help work on the problem, or have other *useful* information. You can go to wikitech-l, which MediaWiki developers/Wikimedia sysadmins pretty much all read, if you want to discuss this: https://lists.wikimedia.org/mailman/listinfo/wikitech-l You'll be more likely to get useful results there.
Trivial solution: Write a two line watchdog http://en.wikipedia.org/wiki/Watchdog_timer
Ok, I think I undesrtood why lilypond is not suitable to wiki projects. What about if any developers could help us trying to implement jmusic? As I see, it is very similar to Lilypond and has the ¿advantage? that it is not Lilypond.
(In reply to comment #90) > I think there is one big misunderstanding here. > > Most of the people who are requesting a music module arn't developers, neither > wikimedia nor lillipond. They are simple users who have a need for > incorporating notes to articles either wikipedia odr wikisource and begging for > years for a solution of that. SO most of us have the problem that we have a > wish an nobody is supporting us for years. > > Please could some of the skilled people who know how to incorporate or generate > a musik module help the commiunity. > > As far as I can see you are using more time on arguing as an workin on that > issue. > > > Greetings > Bingo!!!
At this moment, after 5 years, I'm not concerned about what language should be implemented. I think that ABC, Lilypond, Jmusic, etc., is good enough in order to stop this issue and begin using a music extension. I'm making a course on counterpoint and it seems it will be only with letters and no music :S. I stopped before my course on reading music because I prefered to wait this extension to be installed.
The ABC extension still seems like the most likely choice (because it's passed security review by a root), but it apparently needs some changes, and no one has been willing to spend the time on making them. According to River, "brion wanted some changes made to how it stores files, which got hung up on a rewrite of the file repo stuff so it never happened". There's no ETA; it looks to me like it will have to wait until a root or someone is willing to spend the time on it.
I've looked at the source code for abcm2ps now, and I think that from a security perspective, it would be better to use Lilypond or abc2ly. I've found 3 distinct buffer overflow vulnerabilities in abcm2ps, just with a quick review. No doubt fuzz testing would find more. It's the worst kind of C code, full of static buffers, with length checking implemented inconsistently. It also has two arbitrary local file reads and an arbitrary postscript injection feature. Lilypond is at least fairly well studied. If someone DoS's it we can always disable the extension.
Is there a recent version of the Lilypond extension available? It should be added to SVN for final review.
Further to that: note that PostScript is a Turing complete language. ImageMagick invokes Ghostscript, which is a complete implementation of it. You can do fun things like generating fractals: http://www.physics.uq.edu.au/people/foster/postscript.html So like Lilypond, abcm2ps+convert can have arbitrary running time, by way of the postscript injection feature. What I'd like to see from a Lilypond extension is: * Coding style and security as per the relevant pages on mediawiki.org * Generally good programming practice, like not pretending to be a math extension as in attachment 1141 [details] * Optionally, multiple input formats via the *2ly programs distributed with Lilypond * Optionally, Ogg Vorbis output and OggHandler integration * Optionally, remote rendering, say via Http::get() to a remote instance of MediaWiki. This would allow for a fixed resource pool, alleviating DoS concerns.
Could this be a solution/compromiss. MusixTex http://en.wikipedia.org/wiki/MusiXTeX. It seems to be completely Tex-based. Because we have tex already installed it may fit. link to the homepage http://icking-music-archive.org/software/indexmt6.html
My understanding of the status of this is that we don't have a specific, ready-to-deploy extension to review, so I'm removing the "needs-review" keyword.
The reason that there is no specific, read-to-deploy extension to review is that it took such a preposterously long time to get anywhere that the original author of the extension gave up on the Wikipedia project. Having said that, somebody dedicated enough to actually do something instead of just talking should be able to "fix" things in no more than half an hour.
Comment on attachment 1141 [details] new version of LilyPond extension marking attached patch obsolete since it is 5 years old.
Is it helpfull to mark it as obsolete? Is there any knowledege, wether this patch must be applied for the new version too? It would be much more helpfull, instead of downgrading, by removin inforamtion like this patch, to give assistance to get a music module up an running. greetings
See [[mw:Extension:LilyPond]]
This bug is to have a working music transcription extension installed on Wikimedia projects. That has not been resolved.
Just fixing the broken ling ov Mark. http://www.mediawiki.org/wiki/Extension:LilyPond What is ment by a working music transcription? Lilypond is working the Extension is working.
When I asked Tim about this a while back he said that before it would be installed on WMF servers that he wanted components addressed, and he added his requirements above (c103 & c105). They have yet to be attended to. We are looking forward to one day being able to have the extension running at English Wikisource for all those works with music and words. Not just for music based works, but also for things like Groves' Dictionary of Music and Musicians that is calling out for that gift.
(In reply to comment #114) > When I asked Tim about this a while back he said that before it would be > installed on WMF servers that he wanted components addressed, and he added his > requirements above (c103 & c105). They have yet to be attended to. > > We are looking forward to one day being able to have the extension running at > English Wikisource for all those works with music and words. Not just for > music based works, but also for things like Groves' Dictionary of Music and > Musicians that is calling out for that gift. Is it necessary to add a {{Security alert}}[1] to the extension's page (such as that on [[mw:Extension:AbcMusic]]), warning the user about the security problems/vulnerabilities pointed out by Tim on comment 103 and comment 105? [1] http://www.mediawiki.org/wiki/Template:Security_alert
After Nemo's recent mail [1] on wikisource-l, I created a rewrite of the LilyPond extension at http://www.mediawiki.org/wiki/Extension:Score Hopefully, this rewrite addresses comments 103 and 105 (sans the "optional" stuff), and also Tim Starling's recent review [2]. [1] http://lists.wikimedia.org/pipermail/wikisource-l/2011-December/001053.html [2] https://www.mediawiki.org/wiki/Special:Code/MediaWiki/98414#c27299
GrafZahl, thank you for your work on this! To get this installed ("deployed") on Wikimedia Foundation sites, you'll want to follow these steps: https://www.mediawiki.org/wiki/Writing_an_extension_for_deployment including requesting commit access to our Subversion source control repository: https://www.mediawiki.org/wiki/Commit_access_requests#Requesting_commit_access I'll start getting some reviews for you if possible. Thanks again.
This issue now depends on bug 33193 -- Review and deploy Score extension on Wikimedia Cluster.
Comitted Lilypond at r98414. Made some modifications to Score in response at r106459. Awaiting a new review.
Adding Owen Davis to cc, since he is working on a related project (as discussed during the Wikimania presentation https://wikimania2012.wikimedia.org/wiki/Submissions/We_want_scores! ).
... and at http://www.mediawiki.org/wiki/Mentorship_programs/Possible_projects#VisualEditor_plugins we are suggesting that this could be a VisualEditor plugin. http://www.mediawiki.org/wiki/Mentorship_programs/Possible_projects#VisualEditor_plugins
Score extension deployed.
Congratulations to all ! It makes my dream comes true today ! Thanks million times!
Exciting! Here's a small demo page on en.wp, with midi and Vorbis output enabled. https://en.wikipedia.org/wiki/User:Eloquence/Score Only issues I'm noticing so far: - instead of the TMH player, the native browser player is used on preview; - the audio player takes a long time to initialize (longer than the video player on pages with video).
(In reply to comment #124) > Exciting! Here's a small demo page on en.wp, with midi and Vorbis output > enabled. > > https://en.wikipedia.org/wiki/User:Eloquence/Score > > Only issues I'm noticing so far: > > - instead of the TMH player, the native browser player is used on preview; For me, the TMH player is eventually shown on preview, after a delay of about 10 seconds. > - the audio player takes a long time to initialize (longer than the video > player on pages with video). Yes, it seems to show up a long time after the load event fires, and long after the last HTTP request appears in firebug. I'm not sure what it could be doing with all that time. Whatever it is, it doesn't max out the CPU. Anyway, it's a subject for a separate bug report.
Filed as bug 47533.
Great news!! Thanks to everyone involved for making this happen!
Great addition to MediaWiki!
