I am building a web API. I found whenever I use Chrome to POST, GET to my API, there is always an OPTIONS request sent before the real request, which is quite annoying. Currently, I get the server to ignore any OPTIONS requests. Now my question is what's good to send an OPTIONS request to double the server's load? Is there any way to completely stop the browser from sending OPTIONS requests?
edit 2018-09-13: added some precisions about this pre-flight request and how to avoid it at the end of this reponse.
OPTIONS requests are what we call pre-flight requests in Cross-origin resource sharing (CORS).
They are necessary when you're making requests across different origins in specific situations.
This pre-flight request is made by some browsers as a safety measure to ensure that the request being done is trusted by the server.
Meaning the server understands that the method, origin and headers being sent on the request are safe to act upon.
Your server should not ignore but handle these requests whenever you're attempting to do cross origin requests.
A good resource can be found here http://enable-cors.org/
A way to handle these to get comfortable is to ensure that for any path with OPTIONS method the server sends a response with this header
Access-Control-Allow-Origin: *
This will tell the browser that the server is willing to answer requests from any origin.
For more information on how to add CORS support to your server see the following flowchart
http://www.html5rocks.com/static/images/cors_server_flowchart.png
edit 2018-09-13
CORS OPTIONS request is triggered only in somes cases, as explained in MDN docs:
Some requests don’t trigger a CORS preflight. Those are called “simple requests” in this article, though the Fetch spec (which defines CORS) doesn’t use that term. A request that doesn’t trigger a CORS preflight—a so-called “simple request”—is one that meets all the following conditions:
The only allowed methods are:
GET
HEAD
POST
Apart from the headers set automatically by the user agent (for example, Connection, User-Agent, or any of the other headers with names defined in the Fetch spec as a “forbidden header name”), the only headers which are allowed to be manually set are those which the Fetch spec defines as being a “CORS-safelisted request-header”, which are:
Accept
Accept-Language
Content-Language
Content-Type (but note the additional requirements below)
DPR
Downlink
Save-Data
Viewport-Width
Width
The only allowed values for the Content-Type header are:
application/x-www-form-urlencoded
multipart/form-data
text/plain
No event listeners are registered on any XMLHttpRequestUpload object used in the request; these are accessed using the XMLHttpRequest.upload property.
No ReadableStream object is used in the request.
Have gone through this issue, below is my conclusion to this issue and my solution.
According to the CORS strategy (highly recommend you read about it) You can't just force the browser to stop sending OPTIONS request if it thinks it needs to.
There are two ways you can work around it:
Make sure your request is a "simple request"
Set Access-Control-Max-Age for the OPTIONS request
Simple request
A simple cross-site request is one that meets all the following conditions:
The only allowed methods are:
GET
HEAD
POST
Apart from the headers set automatically by the user agent (e.g. Connection, User-Agent, etc.), the only headers which are allowed to be manually set are:
Accept
Accept-Language
Content-Language
Content-Type
The only allowed values for the Content-Type header are:
application/x-www-form-urlencoded
multipart/form-data
text/plain
A simple request will not cause a pre-flight OPTIONS request.
Set a cache for the OPTIONS check
You can set a Access-Control-Max-Age for the OPTIONS request, so that it will not check the permission again until it is expired.
Access-Control-Max-Age gives the value in seconds for how long the response to the preflight request can be cached for without sending another preflight request.
Limitation Noted
For Chrome, the maximum seconds for Access-Control-Max-Age is 600 which is 10 minutes, according to chrome source code
Access-Control-Max-Age only works for one resource every time, for example, GET requests with same URL path but different queries will be treated as different resources. So the request to the second resource will still trigger a preflight request.
Please refer this answer on the actual need for pre-flighted OPTIONS request: CORS - What is the motivation behind introducing preflight requests?
To disable the OPTIONS request, below conditions must be satisfied for ajax request:
Request does not set custom HTTP headers like 'application/xml' or 'application/json' etc
The request method has to be one of GET, HEAD or POST. If POST, content type should be one of application/x-www-form-urlencoded, multipart/form-data, or text/plain
Reference:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
When you have the debug console open and the Disable Cache option turned on, preflight requests will always be sent (i.e. before each and every request). if you don't disable the cache, a pre-flight request will be sent only once (per server)
Yes it's possible to avoid options request. Options request is a preflight request when you send (post) any data to another domain. It's a browser security issue. But we can use another technology: iframe transport layer. I strongly recommend you forget about any CORS configuration and use readymade solution and it will work anywhere.
Take a look here:
https://github.com/jpillora/xdomain
And working example:
http://jpillora.com/xdomain/
For a developer who understands the reason it exists but needs to access an API that doesn't handle OPTIONS calls without auth, I need a temporary answer so I can develop locally until the API owner adds proper SPA CORS support or I get a proxy API up and running.
I found you can disable CORS in Safari and Chrome on a Mac.
Disable same origin policy in Chrome
Chrome: Quit Chrome, open an terminal and paste this command: open /Applications/Google\ Chrome.app --args --disable-web-security --user-data-dir
Safari: Disabling same-origin policy in Safari
If you want to disable the same-origin policy on Safari (I have 9.1.1), then you only need to enable the developer menu, and select "Disable Cross-Origin Restrictions" from the develop menu.
As mentioned in previous posts already, OPTIONS requests are there for a reason. If you have an issue with large response times from your server (e.g. overseas connection) you can also have your browser cache the preflight requests.
Have your server reply with the Access-Control-Max-Age header and for requests that go to the same endpoint the preflight request will have been cached and not occur anymore.
I have solved this problem like.
if($_SERVER['REQUEST_METHOD'] == 'OPTIONS' && ENV == 'devel') {
header('Access-Control-Allow-Origin: *');
header('Access-Control-Allow-Headers: X-Requested-With');
header("HTTP/1.1 200 OK");
die();
}
It is only for development. With this I am waiting 9ms and 500ms and not 8s and 500ms. I can do that because production JS app will be on the same machine as production so there will be no OPTIONS but development is my local.
You can't but you could avoid CORS using JSONP.
After spending a whole day and a half trying to work through a similar problem I found it had to do with IIS.
My Web API project was set up as follows:
// WebApiConfig.cs
public static void Register(HttpConfiguration config)
{
var cors = new EnableCorsAttribute("*", "*", "*");
config.EnableCors(cors);
//...
}
I did not have CORS specific config options in the web.config > system.webServer node like I have seen in so many posts
No CORS specific code in the global.asax or in the controller as a decorator
The problem was the app pool settings.
The managed pipeline mode was set to classic (changed it to integrated) and the Identity was set to Network Service (changed it to ApplicationPoolIdentity)
Changing those settings (and refreshing the app pool) fixed it for me.
OPTIONS request is a feature of web browsers, so it's not easy to disable it. But I found a way to redirect it away with proxy. It's useful in case that the service endpoint just cannot handle CORS/OPTIONS yet, maybe still under development, or mal-configured.
Steps:
Setup a reverse proxy for such requests with tools of choice (nginx, YARP, ...)
Create an endpoint just to handle the OPTIONS request. It might be easier to create a normal empty endpoint, and make sure it handles CORS well.
Configure two sets of rules for the proxy. One is to route all OPTIONS requests to the dummy endpoint above. Another to route all other requests to actual endpoint in question.
Update the web site to use proxy instead.
Basically this approach is to cheat browser that OPTIONS request works. Considering CORS is not to enhance security, but to relax the same-origin policy, I hope this trick could work for a while. :)
you can also use a API Manager (like Open Sources Gravitee.io) to prevent CORS issues between frontend app and backend services by manipulating headers in preflight.
Header used in response to a preflight request to indicate which HTTP headers can be used when making the actual request :
content-type
access-control-allow-header
authorization
x-requested-with
and specify the "allow-origin" = localhost:4200 for example
One solution I have used in the past - lets say your site is on mydomain.com, and you need to make an ajax request to foreigndomain.com
Configure an IIS rewrite from your domain to the foreign domain - e.g.
<rewrite>
<rules>
<rule name="ForeignRewrite" stopProcessing="true">
<match url="^api/v1/(.*)$" />
<action type="Rewrite" url="https://foreigndomain.com/{R:1}" />
</rule>
</rules>
</rewrite>
on your mydomain.com site - you can then make a same origin request, and there's no need for any options request :)
It can be solved in case of use of a proxy that intercept the request and write the appropriate headers.
In the particular case of Varnish these would be the rules:
if (req.http.host == "CUSTOM_URL" ) {
set resp.http.Access-Control-Allow-Origin = "*";
if (req.method == "OPTIONS") {
set resp.http.Access-Control-Max-Age = "1728000";
set resp.http.Access-Control-Allow-Methods = "GET, POST, PUT, DELETE, PATCH, OPTIONS";
set resp.http.Access-Control-Allow-Headers = "Authorization,Content-Type,Accept,Origin,User-Agent,DNT,Cache-Control,X-Mx-ReqToken,Keep-Alive,X-Requested-With,If-Modified-Since";
set resp.http.Content-Length = "0";
set resp.http.Content-Type = "text/plain charset=UTF-8";
set resp.status = 204;
}
}
What worked for me was to import "github.com/gorilla/handlers" and then use it this way:
router := mux.NewRouter()
router.HandleFunc("/config", getConfig).Methods("GET")
router.HandleFunc("/config/emcServer", createEmcServers).Methods("POST")
headersOk := handlers.AllowedHeaders([]string{"X-Requested-With", "Content-Type"})
originsOk := handlers.AllowedOrigins([]string{"*"})
methodsOk := handlers.AllowedMethods([]string{"GET", "HEAD", "POST", "PUT", "OPTIONS"})
log.Fatal(http.ListenAndServe(":" + webServicePort, handlers.CORS(originsOk, headersOk, methodsOk)(router)))
As soon as I executed an Ajax POST request and attaching JSON data to it, Chrome would always add the Content-Type header which was not in my previous AllowedHeaders config.
I am currently working on a multi threaded proxy server that supports keep-alive connections. I see some weird issues while handling requests from firefox browser. I connect to my local proxy using localhost:10001/http://url, and I can access all the links on this host. The process is as below.
1. Create a socket bind it to port 10001
2. Accept connections and if a client is connected fork()
3. Keep on processing the client request as persistent connection.
Now the problem is that when I open a new tab in firefox to access a second url with different host with using localhost:10001/http://url2, the strange thing is that that request goes to my client socket connection created during first connection. I initially thought that it might be due to my code, but then i tried to do the same using telnet and all the new connections would create a separate process. Are there any specific settings that is making firefox browser do this??
HTTP keep-alive is a way to reuse an underlying TCP connection for multiple requests so that one can skip the overhead of creating a new TCP connection all the time. Since the target of the connection is the same all the time in your case it makes sense for the browser to reuse the same TCP connection. The comparison with telnet is flawed since with telnet you do a new TCP connection all the time.
If HTTP keep-alive gets used is specified by the HTTP version the Connection header and on the behavior of both server and client. Both server and client can decide to close the idle connection any time after a request was done, i.e. they are not required to keep it open after the request is done. Additionally they can signal that they like to have the connection open by using the Connection: keep-alive HTTP header or that they like to close after the request with Connection: close. These headers have default values depending on the HTTP version, i.e. keep-alive is on with HTTP/1.1 while off with HTTP/1.0 unless explicitly specified.
Apart from that the "proxy" you are implementing with the use of URL's like http://proxy/real-url is not a real HTTP proxy. A real HTTP proxy would be configured as a proxy inside the browser and the URL's you use would stay the same which also means that no URL rewriting would need to be done by the proxy. Worse is that your idea of a proxy effectively merges all hosts inside the same origin (i.e. origin is the proxy) and thus effectively disables a major security concept of the browser: the same-origin policy. This means for example that some rogue advertisement server would share with your implementation the origin with ebay and thus could get access to the ebay cookies and hijack the session and misuse it for identity theft
HTTP persistent connection is also used with the proxy, not only with the destination.
For firefox you could try to alter the behavior with the proxy by setting network.http.proxy.version to 1.0. But you'll have to enhance your proxy (and perhaps rethink completely its inner workings) to be able to deal with these reused connections. I'm sure it's not limited to Firefox.
Also make sure your proxy doesn't answer with HTTP/1.1 because it's not.
HTTP Response from my ESB service contains below HTTP header details;
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Server: Jetty(7.6.7.v20120910)
How to remove/hide server version in response header.
I have tried with camel- route configuration as below;
removeHeaders pattern="*"
But its not worked out. Meanwhile i tried to intercept the response by using cxf-out-interceptor, but Message contains only content-type & date in PROTOCAL_HEADERS.
Is there any configuration in Fuse container level to remove this header key from HTTP response?
With the jetty component I think you cannot. The endpoint implementation adds back the three headers Content-Type, Transfer-Encoding, and Server even after the header filter strategy gets applied.
If you have the flexibility, try the netty4-http component instead of jetty. Possibly it does not add headers after the strategy is applied/you remove the headers explicitly with removeHeaders.
You can try to add your custom bindings instead of DefaultBindings provided by CXF if you using CXF as web service provider. In that case, you can remove/add/overrider certain attributes when headers are populated back to CXF Exchange from Camel Exchange.
I'm writing a webservice. Could any one explain these above methods and give me some example about them? Thank for your help.
GET should be used to retrieve data with no other effect however you can use query params in url to post data using get but it is not a safe method.
The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line.Generally used to create new entity.
The PUT method requests that the enclosed entity be stored under the supplied Request-URI. Generally used to update existing entity.
The PATCH method applies partial modifications to a resource
The DELETE method requests that the origin server delete the resource identified by the Request-URI.
The TRACE method echoes the received request so that a client can see what (if any) changes or additions have been made by intermediate servers.
The HTTP CONNECT method method starts two-way communications with the requested resource. It can be used to open a tunnel usually to facilitate SSL-encrypted communication (HTTPS) through an unencrypted HTTP proxy.
The OPTIONS method allows the client to determine the options and/or requirements associated with a resource, or the capabilities of a server, without implying a resource action or initiating a resource retrieval.
HEAD Retrieve all resources in a collection (header only) i.e. The HEAD method asks for a response identical to that of a GET request, but without the response body. This is useful for retrieving meta-information written in response headers, without having to transport the entire content.
From this_link you can get a detail about these methods.I have used these resources to write these methods in short.
You can also get simplified details on this wikipidea page.
This stackoverflow link is also very descriptive for http methods.
And for the implementation part this open source Django_rest_code at github can be a very good example to look at how to implement these Http methods in Django(Python).
They are actions from the perspective of the client:
GET refers to the client requesting information in the form of a URL request to the server ie loading a web page full of data.
POST is the client sending information back to the server ie clicking submit on a text field.
PUT is very similar to POST except that the information sent back to the server must be identified under the supplied Request-URI
DELETE requests that the server delete the entity that the client has designated ie removing a blog post from your blog tells the server to forget that information.
Those are the 4 main methods through which clients and servers communicate, thus how information on the server is displayed to and controlled by the client.
I've been developing a small Silverlight client, which will talk to a REST service build using the WCF WEBAPI....
When the service is called using GET, it'll kick of a long running process, that'll generate a resource, so the service will return 'Accepted' and a URI in a Location header, to where the resource will be found.
Server: ASP.NET Development Server/10.0.0.0
Date: Fri, 18 Nov 2011 09:00:17 GMT
X-AspNet-Version: 4.0.30319
Content-Length: 3
Location: http://localhost:52878/myservice?fileid=f68201f6-9d77-4818-820e-e5e796e9710a
Cache-Control public, max-age=21600
Expires: 21600
Content-Type: text/plain
Connection: Close
Now, in my Silverlight client, I need to access this header information, however using the BrowserHTTP stack, this is not possible... so I've switched to the ClientHTTP, which makes it possible for me to access the header information returned.
However the ClientHTTP stack doesn't support Content Caching:
http://www.wintellect.com/CS/blogs/jprosise/archive/2009/10/14/silverlight-3-s-new-client-networking-stack.aspx
which is causing me troubles..... I wan't the same resource to be returned for 6 hours, before a new one is generated.
Is there a way to get the best of both... being able to access the Header info AND have content caching??
TIA
Søren
Stop using a header to return the information needed by the client code.
If you include the required information in the entity body using either raw or encoded in some message format (e.g. XML or JSON) then you can continue to use the BrowserHTTP and benefit from its caching.
Using the headers is the correct way to convey this information. That's why it's in the standard.
I don't do silverlight, but what I get from that post is that you will now need to implement the caching. Using the BrowserHttp leverages the browsers caching mechanism. Now using ClientHttp you are dropping closer to the metal and you will have to implement caching.