Last modified: 2013-11-07 08:27:09 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T43731, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 41731 - Correctly answer to OPTIONS method (Squids appear to reject CORS OPTIONS query before it ever gets to the API)
Correctly answer to OPTIONS method (Squids appear to reject CORS OPTIONS quer...
Status: RESOLVED FIXED
Product: Wikimedia
Classification: Unclassified
Site requests (Other open bugs)
wmf-deployment
All All
: Low minor (vote)
: ---
Assigned To: Mark Bergsma
: ops
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-11-03 08:55 UTC by Michael M.
Modified: 2013-11-07 08:27 UTC (History)
13 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Michael M. 2012-11-03 08:55:38 UTC
Steps to reproduce:

1. Go to https://de.wikipedia.org
2. Execute the following code:

var xhr = new XMLHttpRequest();
xhr.upload.addEventListener('progress', function () {}, false);
xhr.withCredentials = true;
xhr.open('GET', 'https://commons.wikimedia.org/w/api.php?action=tokens&type=edit&origin=https://de.wikipedia.org&format=json', true);
xhr.onreadystatechange = function() {
    console.log(xhr.responseText);
}
xhr.send();

This sends a request using the OPTIONS method with header
Access-Control-Request-Method: GET
This is correct according to the specs as I just learned:
http://dvcs.w3.org/hg/xhr/raw-file/tip/Overview.html -> set force preflight flag
http://dvcs.w3.org/hg/cors/raw-file/tip/Overview.html#resource-preflight-requests

The server answers with

HTTP/1.1 403 Forbidden

with a header
X-Squid-Error: ERR_ACCESS_DENIED 0
Comment 1 Piotr Sarnacki 2013-02-11 17:14:21 UTC
I try to make such request against wiktionary with similar results. Easy way to reproduce is to use curl:

curl -v -X OPTIONS "http://en.wiktionary.org/w/api.php?action=opensearch&search=a&format=json?origin=http:%2F%2Fen.wikipedia.org" -H "Origin: http://en.wikipedia.org"

I checked also other origins with no luck.
Comment 2 Antoine "hashar" Musso (WMF) 2013-03-18 10:31:54 UTC
The code in comment #0 does cross script request so unlikely to ever succeed.
Comment 3 Umherirrender 2013-03-22 19:06:17 UTC
Works when using the syntax suggest in Gerrit change #9624

$.ajax( {
	'url': 'https://commons.wikimedia.org/w/api.php',
	'data': {
		'action': 'tokens',
		'type': 'edit',
		'format': 'json',
		'origin': 'https://de.wikipedia.org'
	},
	'xhrFields': {
		'withCredentials': true
	},
	'success': function( data ) {
		console.log( data );
	}
} );

No idea, what the different to your code is. (I do not know how ajax() is implemented and if that makes a different).
Comment 4 Michael M. 2013-03-25 09:45:21 UTC
(In reply to comment #2)
> The code in comment #0 does cross script request so unlikely to ever succeed.

It is a cross script request, yes, but with the line
 xhr.withCredentials = true;
it will succeed as commons.wikimedia.org accepts CORS requests from de.wikipedia.org.

(In reply to comment #3)
> No idea, what the different to your code is. (I do not know how ajax() is
> implemented and if that makes a different).

There is no real difference, just a bit more jQuery-overhead. But with jQuery you can't add event listeners to xhr.upload (well, you can, if you really want to use uggly hacks).
Comment 5 Andre Klapper 2013-04-18 17:53:02 UTC
Low priority: Workaround in comment 3, plus Squid will get superseded by Varnish.
Comment 6 Bawolff (Brian Wolff) 2013-04-18 18:05:19 UTC
I'm not very familiar with CORS, but isn't OPTIONS requests only needed if you're using a method other than GET, HEAD, or POST or setting a caching header?


I don't think OPTIONS are needed for any valid use case for CORS on Wikimedia (However, I have never read the spec, and just skimmed right now for where the word OPTIONS is used, so I may misunderstand).


----

(In reply to comment #5)
> Low priority: Workaround in comment 3, plus Squid will get superseded by
> Varnish.

The upload varnishes give a 403 for options, so the transition to varnish probably doesn't matter.
Comment 7 Piotr Sarnacki 2013-04-18 21:07:29 UTC
> I don't think OPTIONS are needed for any valid use case for CORS on Wikimedia

CORS is needed if you want to do any XHR request from the different domain from the browser.

So it depends if you treat using API directly from the browser a valid use case, I would argue that it would be nice to have it working.
Comment 8 Bawolff (Brian Wolff) 2013-04-18 22:13:09 UTC
(In reply to comment #7)
> > I don't think OPTIONS are needed for any valid use case for CORS on Wikimedia
> 
> CORS is needed if you want to do any XHR request from the different domain
> from
> the browser.
> 
> So it depends if you treat using API directly from the browser a valid use
> case, I would argue that it would be nice to have it working.

I mean options should not be needed if you're only doing GET, HEAD or POST with CORS, even if from the browser. So you shouldn't need OPTIONS from the browser (I think. Not a cors expert)
Comment 9 Juliusz Gonera 2013-04-19 00:27:47 UTC
Bawolff, OPTIONS is needed if you want to bind to the upload progress event on POST AJAX requests. We want to use it in MobileFrontend to show a progress bar for image uploads.

See https://bugzilla.wikimedia.org/show_bug.cgi?id=44921 for details.
Comment 10 Yuvi Panda 2013-09-18 00:15:33 UTC
Is there any potential downside in just letting Squid accept OPTIONS and pass it through?
Comment 11 Faidon Liambotis 2013-10-29 09:17:29 UTC
Indeed, our Squid config blocks OPTIONS requests. This is definitely deliberate, with OPTIONS being explicitly singled out and seems to be there for many years.
I'll ask around to find out the rationale behind it -- please do as well, Brion might remember :) We need to find out soon anyway, since we're the Varnish switch is imminent.

acl options method OPTIONS
[...]
<? if ( $upload ): ?>
# Deny HTTP methods other than GET/HEAD
http_access deny !gethead
<? else: ?>
# Deny HTTP OPTIONS method requests
http_access deny options
<? endif ?>

What is the use case for this BTW? I know this is for CORS preflight requests, but what specifically? (from which domain to which, what it's going to be used for etc.)
Comment 12 Michael M. 2013-10-29 10:47:40 UTC
(In reply to comment #11)
> What is the use case for this BTW? I know this is for CORS preflight
> requests,
> but what specifically? (from which domain to which, what it's going to be
> used
> for etc.)

Uploading a file to Commons while the user is on some other domain (*.wikipedia.org or whatever), and showing upload progress messages to the user.

I want to use it in [[de:Benutzer:Schnark/js/screenshot.js]], a script to generate screenshots and uploading them directly, to show a progress bar while the screenshot is uploaded (this already works if you upload locally).
According to comment 9, Juliusz Gonera wants to do the same in MobileFrontend.

If you don't bind to the upload progress event, no OPTIONS request is needed, but then you can't show progress bars to users.
Comment 13 Mark Bergsma 2013-10-29 11:48:52 UTC
(In reply to comment #11)
> Indeed, our Squid config blocks OPTIONS requests. This is definitely
> deliberate, with OPTIONS being explicitly singled out and seems to be there
> for
> many years.
> I'll ask around to find out the rationale behind it -- please do as well,
> Brion
> might remember :) We need to find out soon anyway, since we're the Varnish
> switch is imminent.

If i recall correctly, we put that in to block all the WebDAV requests hitting the Squids from also busting through to the Apaches, for no need.

I suppose we can open up the OPTIONS method for the API (only).
Comment 14 Jon 2013-10-29 17:02:37 UTC
I can confirm that mobile web want to do exactly the same as Michael M. Thanks for looking into this guys.
Comment 15 Mark Bergsma 2013-10-31 12:38:24 UTC
(In reply to comment #13)
> (In reply to comment #11)
> > Indeed, our Squid config blocks OPTIONS requests. This is definitely
> > deliberate, with OPTIONS being explicitly singled out and seems to be there
> > for
> > many years.
> > I'll ask around to find out the rationale behind it -- please do as well,
> > Brion
> > might remember :) We need to find out soon anyway, since we're the Varnish
> > switch is imminent.
> 
> If i recall correctly, we put that in to block all the WebDAV requests
> hitting
> the Squids from also busting through to the Apaches, for no need.
> 
> I suppose we can open up the OPTIONS method for the API (only).

I've just deployed the following change in the Squid config:

 -http_access deny options
 +http_access deny options !api_php

So Squid should accept OPTIONS now, for ^/w/api.php requests only.

I'll work on a corresponding change for Varnish as well.
Comment 16 Mark Bergsma 2013-10-31 13:35:35 UTC
Varnish now accepts the OPTIONS method as well, as long as a corresponding Origin request header is present for CORS.
Comment 17 Jon 2013-10-31 19:01:27 UTC
Thanks so much for doing this Mark. I can't seem to get it working though...

I tried this on enwiki with the following steps:
* Visited https://en.m.wikipedia.org/wiki/Special:Uploads
* enabled upload progress bars in my JavaScript console:
mw.config.set( 'wgMFAjaxUploadProgressSupport', true )
* Attempted to upload an image

The ajax request failed:

OPTIONS https://commons.m.wikimedia.org/w/api.php?useformat=mobile&r=0.5664375186897814 Origin https://en.m.wikipedia.org is not allowed by Access-Control-Allow-Origin. load.php?debug=false&lang=en&modules=jquery%2Cmediawiki%2CSpinner%7Cjquery.triggerQueueCallback%2Cl…:128
XMLHttpRequest cannot load https://commons.m.wikimedia.org/w/api.php?useformat=mobile&r=0.5664375186897814. Origin https://en.m.wikipedia.org is not allowed by Access-Control-Allow-Origin. 

This is the request url info:

Request URL:https://commons.m.wikimedia.org/w/api.php?useformat=mobile&r=0.5664375186897814
Request Method:OPTIONS
Status Code:200 OK
Request Headersview source
Accept:*/*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-GB,en-US;q=0.8,en;q=0.6
Access-Control-Request-Headers:accept, content-type
Access-Control-Request-Method:POST
Cache-Control:no-cache
Connection:keep-alive
Host:commons.m.wikimedia.org
Origin:https://en.m.wikipedia.org
Pragma:no-cache
Referer:https://en.m.wikipedia.org/wiki/Special:Uploads
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36

Am I doing something wrong?
Comment 18 Michael M. 2013-11-04 09:15:33 UTC
(In reply to comment #17)
> Am I doing something wrong?

I can confirm that my code from comment 0 now does work (and it does work with POST instead of GET, too). But I can confirm that my script has problems, too, and I'm not sure whether I resolved them now or not (it might be just my browser cache).

Try to add the origin parameter in the URL even for POST requests, instead of the body. This seems to work.
Comment 19 Jon 2013-11-06 22:39:56 UTC
It does look like the origin has to be in the url not the body. Confirmed.
Comment 20 Bawolff (Brian Wolff) 2013-11-06 23:55:54 UTC
(In reply to comment #19)
> It does look like the origin has to be in the url not the body. Confirmed.

To quote the documentation:

 origin              - When accessing the API using a cross-domain AJAX request (CORS), set this to the originating domain.
                        This must be included in any pre-flight request, and therefore must be part of the request URI (not the POST body).
...

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links