Last modified: 2010-03-25 20:27:01 UTC
The content of [[commons:MediaWiki:Robots.txt]] does not seem to show up at http://commons.wikimedia.org/robots.txt . It should, shouldn't it? Is something wrong with the configuration?
It seems [[meta:MediaWiki:Robots.txt]] isn't either. Perhaps because they have a pedia suffix?
Yeah, looks like the relevant RewriteRules are only present in http://noc.wikimedia.org/conf/main.conf but not in http://noc.wikimedia.org/conf/remnant.conf where Commons and Meta are. Sprinkling a few suitably placed copies of # Make robots.txt editable via Mediawiki:robots.txt RewriteRule ^/robots.txt$ /w/robots.php [L] into that file would seem likely to fix it. (Also should probably do the same for the other relevant *.conf files.) CCing JeLuF on this -- he fixed bug 15663, so I assume he knows something about this robots.txt setup. ;)
Those files have interesting data, like the non-working wikipedia.geo.blitzed.org ServerAlias, or the robots file name. Why isn't robots.php in svn?
Fixed config for meta and commons. Added robots.php to SVN.