Last modified: 2010-03-24 18:44:14 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T24932, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 22932 - Add charset to robots.txt
Add charset to robots.txt
Status: RESOLVED FIXED
Product: Wikimedia
Classification: Unclassified
Site requests (Other open bugs)
unspecified
All All
: Normal minor (vote)
: ---
Assigned To: Nobody - You can work on this!
http://thread.gmane.org/gmane.org.wik...
: shell
Depends on:
Blocks: robots.txt
  Show dependency treegraph
 
Reported: 2010-03-23 17:03 UTC by Platonides
Modified: 2010-03-24 18:44 UTC (History)
2 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Platonides 2010-03-23 17:03:00 UTC
From mediawiki-l discussion about not viewing robots utf characters http://thread.gmane.org/gmane.org.wikimedia.mediawiki/33601

robots.txt served from wikimedia wikis should have content-type text/plain;charset=utf-8 not just text/plain

robots.txt spec doesn't take content-types into account, no bot should be relying on a specific content-type.
http://www.robotstxt.org/orig.html
Comment 1 JeLuF 2010-03-23 19:36:22 UTC
Done.
Comment 2 Platonides 2010-03-23 22:05:49 UTC
Now it is Content-Type: text/html; charset=utf-8 
The mime should have remained text/plain, not changed to text/html.

The requested Content-Type is: text/plain; charset=utf-8
Comment 3 JeLuF 2010-03-24 18:44:14 UTC
Grmpf. Cut&Paste-Error, please excuse.

Should be fixed now.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links