Last modified: 2008-08-14 02:31:09 UTC
Please exclude the following from being indexed by modifying robots.txt: *[[m:Talk:Spam blacklist]] and subpages *[[m:User:COIBot/LinkReports]] and subpages *[[m:User:COIBot/COIReports]] and subpages *[[m:User:COIBot/UserReports]] and subpages *[[m:User:SpamReportBot/cw]] and subpages *And the talk pages for all of the above
I agree with excluding crawlers from the bot report pages. I disagree with removing crawlers from [[m:Talk:Spam blacklist]] and its subpages. See my reasoning at http://meta.wikimedia.org/w/index.php?title=Talk:Spam_blacklist&oldid=992898#Excluding_our_work_from_search_engines
OK, per discussion, please do not exclude [[m:Talk:Spam blacklist]] or subpages. The rest listed above is fine to add.
Currently we are planning on having a bot edit some 47000 pages to add __NOINDEX__ - it would be much easier to have the bot reports not indexed with this addition to robots.txt We are still deciding whether to have the talk page and/or archives indexed or not, but we can manage that with the magic word; no action is required from the sysadmins.
Actually, it is ~25000 pages (enwiki was included in the 47000 figure), but the point remains.
Fixed by r37973 - __NOINDEX__ is applied to the pages in question through a template (& new ones use the magic word directly).
