Last modified: 2011-03-13 18:05:43 UTC
When I call the URL "http://de.wikipedia.org/wiki/Image:Beispiel.jpg", I recieve an HTTP 301 Moved Permanently with "Location: http://de.wikipedia.org/wiki/Bild:Beispiel.jpg". This is bad for bots as they have to know the local namespace names and so need a special engine parsing the HTTP redirs. A workaround would be to simply echo the page under the HTTP redirect so that a normal browser forwards, but a bot can see the image (or other namespace).
Uh, whatever HTTP client lib you are using should be able to cope with redirects, either implicitely or on demand. Whatever "special engine" are you talking about? Outputting the full page in adition to the redirect would cause server load that is useles for 99.9% of all requests. Depending on the client used, it may also load that even if following the redirect, wasting bandwidth on both ends. In short: when you use HTTP, make sure you support the HTTP standard.
Maybe self-written libraries?
Then fix them. As comment 1 points out, it's madness for us to do this.
Add to your bot the ability of, on getting a Location: header, resend the latest code to the new url. You can get all the namespace names on Special:Export.
