Last modified: 2011-01-25 00:19:07 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T17878, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 15878 - MediaWiki:Robots.txt and $wgCapitalLinks = true on pl.wiktionary
MediaWiki:Robots.txt and $wgCapitalLinks = true on pl.wiktionary
Status: RESOLVED INVALID
Product: Wikimedia
Classification: Unclassified
Site requests (Other open bugs)
unspecified
All All
: Normal enhancement with 2 votes (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks: robots.txt 17004
  Show dependency treegraph
 
Reported: 2008-10-07 08:07 UTC by Piotr Kubowicz
Modified: 2011-01-25 00:19 UTC (History)
8 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Piotr Kubowicz 2008-10-07 08:07:38 UTC
Please add the following lines to robots.txt for pl.wiktionary:

Disallow: /wiki/Wikisłownik:Strony_do_skasowania
Disallow: /wiki/Wikis%C5%82ownik:Strony_do_skasowania
Disallow: /wiki/Wikis%C5%82ownik%3AStrony_do_skasowania
Disallow: /wiki/Wikisłownik:Bar
Disallow: /wiki/Wikis%C5%82ownik:Bar
Disallow: /wiki/Wikis%C5%82ownik%3ABar
Disallow: /wiki/Wikisłownik:Bar/
Disallow: /wiki/Wikis%C5%82ownik:Bar/
Disallow: /wiki/Wikis%C5%82ownik%3ABar/
Disallow: /wiki/Wikisłownik:Tablica ogłoszeń
Disallow: /wiki/Wikis%C5%82ownik:Tablica_og%C5%82osze%C5%84
Disallow: /wiki/Wikis%C5%82ownik%3ATablica_og%C5%82osze%C5%84
Disallow: /wiki/Wikisłownik:Tablica ogłoszeń/
Disallow: /wiki/Wikis%C5%82ownik:Tablica_og%C5%82osze%C5%84/
Disallow: /wiki/Wikis%C5%82ownik%3ATablica_og%C5%82osze%C5%84/
Comment 1 Siebrand Mazeland 2008-10-07 08:23:09 UTC
As far as I know you should configure this in your local MediaWiki:Robots.txt per bug 15601. If true, close as INVALID.
Comment 2 Piotr Kubowicz 2008-10-07 09:01:44 UTC
I have modified http://pl.wiktionary.org/wiki/MediaWiki:Robots.txt, but since there was no effect, I write here.
Comment 3 Siebrand Mazeland 2008-10-07 09:11:46 UTC
Changed summary to be in line with the problem description. The issue is not to add the pl.wp pages to the generic robots.txt, but it is a bug report on the robots.txt merge functionality.
Comment 4 Mike.lifeguard 2008-10-07 13:15:49 UTC
(In reply to comment #2)
> I have modified http://pl.wiktionary.org/wiki/MediaWiki:Robots.txt, but since
> there was no effect, I write here.
> 

I just checked for Meta, and it seems to have no effect there either. I doubt this should still be in Site requests as there seems to be a real bug with this feature (though adding things to the global robots.txt until it's fixed will be a workaround, I think).
Comment 5 JeLuF 2008-10-09 07:01:04 UTC
Case matters. You have to edit Mediawiki:robots.txt, not Mediawiki:Robots.txt
Comment 6 Piotr Kubowicz 2008-10-09 08:40:27 UTC
So why do for example MediaWiki:Common.js and MediaWiki:Deletereason-dropdown (not: MediaWiki:common.js or MediaWiki:deletereason-dropdown) work?

Mediawiki:robots.txt does not appear in Special:AllMessages, so how would you find out the correct spelling?
Comment 7 JeLuF 2008-10-09 10:26:40 UTC
robots.txt handling is no MediaWiki-Feature. So there's no default message here, and thus, the page is not listed in the Special:AllMessages view before it has been created.

http://pl.wiktionary.org/w/index.php?title=MediaWiki:Common.js
http://pl.wiktionary.org/w/index.php?title=MediaWiki:common.js

are different pages. One works, the other doesn't. Wiktionaries are case sensitive.
Comment 8 Raimond Spekking 2008-10-09 10:34:50 UTC
(In reply to comment #7)
> robots.txt handling is no MediaWiki-Feature. So there's no default message
> here, and thus, the page is not listed in the Special:AllMessages view before
> it has been created.
> 

robots.txt is defined in the WMF specific extension WikimediaMessages and therefore it is shown in Special:AllMessages. But that is not the point.

Regardless of this it is impossible to create a message [[MediaWiki:robots.txt]] on wikis with $wgCapitalLinks = true;
 which is the default for mostly all WMF wikis with exception of the Wiktionaries.

Please try to create [[de:MediaWiki:robots.txt]]. It's switches immediatly to [[de:MediaWiki:Robots.txt]].
Comment 9 Roan Kattouw 2008-10-09 12:39:49 UTC
(In reply to comment #8)
> robots.txt is defined in the WMF specific extension WikimediaMessages and
> therefore it is shown in Special:AllMessages. But that is not the point.
> 
> Regardless of this it is impossible to create a message
> [[MediaWiki:robots.txt]] on wikis with $wgCapitalLinks = true;
>  which is the default for mostly all WMF wikis with exception of the
> Wiktionaries.
> 
> Please try to create [[de:MediaWiki:robots.txt]]. It's switches immediatly to
> [[de:MediaWiki:Robots.txt]].
> 

The point here is that the issue was reported on plwiktionary, which has $wgCapitalLinks = false; . On wikis with $wgCapitalLinks = true; , editing [[MediaWiki:Robots.txt]] will work.
Comment 10 Mike.lifeguard 2008-10-09 15:01:13 UTC
(In reply to comment #9)
> (In reply to comment #8)
> > robots.txt is defined in the WMF specific extension WikimediaMessages and
> > therefore it is shown in Special:AllMessages. But that is not the point.
> > 
> > Regardless of this it is impossible to create a message
> > [[MediaWiki:robots.txt]] on wikis with $wgCapitalLinks = true;
> >  which is the default for mostly all WMF wikis with exception of the
> > Wiktionaries.
> > 
> > Please try to create [[de:MediaWiki:robots.txt]]. It's switches immediatly to
> > [[de:MediaWiki:Robots.txt]].
> > 
> 
> The point here is that the issue was reported on plwiktionary, which has
> $wgCapitalLinks = false; . On wikis with $wgCapitalLinks = true; , editing
> [[MediaWiki:Robots.txt]] will work.
> 

Meta, which has $wgCapitalLinks = true; uses MediaWiki:Robots.txt, yet it doesn't seem to work.

On pages which use __NOINDEX__ there is <meta name="robots" content="noindex,follow" />, however that is not so for pages which should have it because of MediaWiki:Robots.txt.
Comment 11 Happy-melon 2009-01-13 14:13:01 UTC
That's not how it works.  There are two ways of blocking spider access to pages: when a spider first visits a site, it looks for a file called "robots.txt" in the root of the site, and follows the rules there to exclude certain tranches of pages.  When it visits each individual page, it looks for the "robots" meta tag and, if one is present and tells it to go away, it does so, and 'forgets' that it was ever on the page.  Modifying [[MediaWiki:Robots.txt]] appends entries to the site /robots.txt file (or is supposed to, anyway); it doesn't affect meta tags on pages. 
Comment 12 Platonides 2009-10-08 21:40:26 UTC
(In reply to comment #4)
> I just checked for Meta, and it seems to have no effect there either. I doubt
> this should still be in Site requests as there seems to be a real bug with this
> feature (though adding things to the global robots.txt until it's fixed will be
> a workaround, I think).

http://es.wikipedia.org/robots.txt does work (look at the bottom)
Is it only working for wikipedias?
Comment 13 AlexSm 2009-11-23 17:55:06 UTC
At this moment I see the content of 
http://pl.wiktionary.org/wiki/MediaWiki:robots.txt
at the end of
http://pl.wiktionary.org/robots.txt
so marking this as INVALID, 
plus changing summary to reflect the solution.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links