I am working on a REST client which uses libcurl to send HTTP POST Messages.
After receiving HTTP POST packet, My REST server needs to take some action, but it is unable to do that because some headers differ from the expected format.
Required HTTP Packet Format:
Hypertext Transfer Protocol
> Accept: text/html
> Content-Type: text/plain
Line-based text data: text/plain
>test string
Current HTTP Packet Format:
Hypertext Transfer Protocol
> Accept: */*
> Content-Type: application/x-www-form-urlencoded
Line-based text data: application/x-www-form-urlencoded
>test string
I have tried the following: (Reference: http://curl.haxx.se/libcurl/c/httpcustomheader.html)
curl = curl_easy_init();
if(curl)
{
struct curl_slist *chunk = NULL;
/* Add a custom header */
chunk = curl_slist_append(chunk, "Accept: text/html");
chunk = curl_slist_append(chunk, "Content-Type: text/plain");
/* set our custom set of headers */
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, chunk);
res = curl_easy_perform(curl);
But this code change has NO EFFECT on the outgoing HTTP POST packet.
Can anyone please suggest me what libcurl options can help me achieve the desired format?
Thanks
Related
I'm using Gatling to do some load tests on an application.
Therefore, I have to request a token from a keycloak instance, with a application/x-www-form-urlencoded header.
private val httpRequest = http("Get access token")
.post("https://********/auth/realms/mds/protocol/openid-connect/token")
.asFormUrlEncoded
.formParam("client_id", "********")
.formParam("client_secret", "********")
.formParam("grant_type", "client_credentials")
val getAccessToken = exec(
httpRequest
.check(status.is(200))
.asJson
.check(jsonPath("$.access_token").saveAs("access_token"))
)
The keycloak returns a 415 because of this:
=========================
HTTP request:
POST https://********/auth/realms/mds/protocol/openid-connect/token
headers:
accept: application/json
host: ********
content-type: application/json
content-length: 103
body:FormUrlEncodedRequestBody{contentType='application/json', charset=UTF-8, content=client_id=********&client_secret=********&grant_type=client_credentials}
=========================
=========================
HTTP response:
status:
415 Unsupported Media Type
headers:
....
body:
{"error":"RESTEASY003065: Cannot consume content type"}
Why is Gatling ignoring my content-type header?
The debugger shows, that the variable httpRequest has the correct header. What is changing the header to application/json
You're posting formParams that will generate a body encoded with application/x-www-form-urlencoded, and then you're trying to override the generated associated content-type header to application/json.
This is non-sensical, resulting in the server not being able to decode the body because the content-type doesn't match.
If you want to send JSON, send a JSON payload with a body, not a form.
I found the mistake after days of search.
the asJson does not convert the response to Json, instead it sets a Content-Type and Accecpt application/json header
I'm using angular to POST to an authentication endpoint; on the server side, I can see the request succeed, and proper CORS headers are set. Angular's origin is http://localhost:9000
On the server side, preflight OPTIONS requests always get a 200 back, so that seems OK.
On the client side, the $http.post always fails with an error code of 0, which from other research suggests something is still wrong with CORS. I've read the spec and tried a number of other answers, yet something is still missing.
Angular POSTs like this:
$http({
method: 'POST',
url: 'http://localhost:3000/auth/login',
data: {
username: $scope.username,
password: $scope.password
}
})
.then(function (response) {
/* etc. etc. */
}, function (response) {
/* This always triggers, with response.status = 0 */
console.log("ERROR: " + response.data);
console.log("Status: " + response.status);
console.log("Status text: " + response.statusText);
console.log("Headers: " + response.headers);
$scope.error = 'Something went wrong...';
});
Using curl to debug what the server is sending back, this is it:
< HTTP/1.1 302 Found
< X-Powered-By: Express
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET,PUT,POST,DELETE,OPTIONS
< Access-Control-Allow-Headers: Origin, Content-Type, Accept, Authorization, Content-Length, X-Requested-With
< Access-Control-Allow-Credentials: true
< Set-Cookie: ua_session_token=(blahblah); Path=/
< Location: /
< Vary: Accept
< Content-Type: text/plain; charset=utf-8
< Content-Length: 23
< Date: Mon, 28 Dec 2015 15:08:17 GMT
< Connection: keep-alive
This is why I'm at a loss, as per the specification, the server seems to be doing the right thing?
Here's what the server gets from the client in terms of request headers:
HEADER host localhost:3000
HEADER content-type application/json;charset=UTF-8
HEADER origin http://localhost:9000
HEADER content-length 38
HEADER connection keep-alive
HEADER accept application/json, text/plain, */*
HEADER user-agent Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/601.3.9 (KHTML, like Gecko) Version/9.0.2 Safari/601.3.9
HEADER referer http://localhost:9000/
HEADER accept-language en-us
HEADER accept-encoding gzip, deflate
UPDATE tried something else with no luck, based on this post. It would seem Access-Control-Allow-Headers is case-sensitive, and angular is sending on the request accept, origin, content-type. I tweaked the server to parrot back the same, with no luck.
Alright, after applying my head to my keyboard for several hours, I've fixed it.
The answer seems to be that angular really doesn't like getting redirects in response to POST. When I changed the server endpoint to return just a plain auth token as text (the same token it was setting as a cookie anyway) rather than returning a redirect, the angular POST started working like a charm and falling through to the success handler.
Not sure I got deep enough into this to know why angular was behaving in that way; by playing around with it I found that if the redirect the server sent was to a nonexistent (404) URL that this could be replicated, EVEN IF the original POST returned that valid redirect.
I'm trying to set a HTTP Header for all my REST calls with following code:
app.factory('authInterceptor', function ($rootScope, $q, $window) {
return {
request: function (config) {
config.headers = config.headers || {};
config.headers.Authorization = '12345678';
return config;
},
response: function (response) {
if (response.status === 401) {
// handle the case where the user is not authenticated
}
return response || $q.when(response);
}
};
});
app.config(function ($httpProvider) {
$httpProvider.interceptors.push('authInterceptor');
});
I currently don't have any authorization enabled on the server.
when I leave out the line "config.headers.Authorization = '12345678';" , then the REST call works well and I get my results. In the JS console I see
GET http://localhost:8080/rest/club/1 [HTTP/1.1 200 OK 7ms]
But when I put this line in to set the Header field, then I see following request in the javascript console
OPTIONS http://localhost:8080/rest/club/1 [HTTP/1.1 200 OK 2ms]
Why does setting Authorization Header change my method from "GET" to "OPTIONS"? And how can I set a custom Header and my request still work?
changing it to
config.headers["X-Testing"] = '12345678';
had the same result.
EDIT:
I tried the answer, I'm setting following HTTP Headers in the server:
response.getHeaders().putSingle("Access-Control-Allow-Origin", "http://localhost");
response.getHeaders().putSingle("Access-Control-Allow-Header", "X-Testing");
response.getHeaders().putSingle("Access-Control-Allow-Methods", "POST, GET, OPTIONS");
response.getHeaders().putSingle("Access-Control-Max-Age", 1728000);
my REST server is running on port 8080, the webserver for the html/JS on port 8000 (initially worked with file://... but moved to a separate webserver because Origin was null)
response.getHeaders().putSingle("Access-Control-Allow-Origin", "*");
or
response.getHeaders().putSingle("Access-Control-Allow-Origin", "http://localhost:8000");
didn't work either.
Must I return any content in the OPTIONS response? I tried 200 OK with the same content as the GET, but I also tried 204 No Content.
2nd EDIT:
here is what firefox sends and receives for the OPTIONS method:
You need to enable CORS in your REST service. As explained in MDN, once you add a custom header, the http protocol specifies performing a preflight,
Preflighted requests
Unlike simple requests (discussed above), "preflighted" requests first
send an HTTP request by the OPTIONS method to the resource on the
other domain, in order to determine whether the actual request is safe
to send. Cross-site requests are preflighted like this since they may
have implications to user data. In particular, a request is
preflighted if:
It uses methods other than GET, HEAD or POST. Also, if POST is used
to send request data with a Content-Type other than
application/x-www-form-urlencoded, multipart/form-data, or text/plain,
e.g. if the POST request sends an XML payload to the server using
application/xml or text/xml, then the request is preflighted. It sets
custom headers in the request (e.g. the request uses a header such as
X-PINGOTHER)
Addition to enabling CORS you also need to add a Access-Control-Allow-Headers header tag to accept your custom header (for the OPTIONS response). This is visible in the MDN Example,
HTTP/1.1 200 OK
Date: Mon, 01 Dec 2008 01:15:39 GMT
Server: Apache/2.0.61 (Unix)
Access-Control-Allow-Origin: http://foo.example
Access-Control-Allow-Methods: POST, GET, OPTIONS
Access-Control-Allow-Headers: X-PINGOTHER
Access-Control-Max-Age: 1728000
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 0
Keep-Alive: timeout=2, max=100
Connection: Keep-Alive
Content-Type: text/plain
UPDATE
As mentioned in the comments, the OPTION response's Access-Control-Allow-Headers is missing the last "s".
I'm trying to upload a file using POST
here's my request :
POST /upload.php HTTP/1.1
Host: localhost
Content-Type: multipart/form-data; boundary=---------------------------552335022525
Content-Length: 192
-----------------------------552335022525
Content-Disposition: form-data; name="userfile"; filename="12.txt"
Content-Type: text/plain
blabla
-----------------------------552335022525--
Using HTTP live headers firefox plugin everything works
but when putting it a char *buffer and send it with winsocksapi I get 400 Bad Request error
You need a blank line between the headers and the payload.
Content-Length: 192
-----------------------------552335022525
This is part of the HTTP protocol. HTTP request headers end with the first empty line (CR-LF by itself.) What you are sending is resulting in the string
-----------------------------552335022525
being taken (along with the following two lines) as a request header which, of course, it isn't. The server can't make head or tail of that, so it responds with 400 Bad Request.
Also, sending the Content-length is not necessary with multipart/form-data, nor even a good idea, as the wrong value could create problems. The MIME multipart format is self describing.
I have written a test code that first connects to a web server. It then requests a page, which has a login form, using GET. The GET works and a 200 status code is returned from the web server. After retrieving the page, I attempt to send a POST request to send some log in information but instead receive a 405 METHOD_NOT_ALLOWED status code in return.
Here is the code for sending the GET:
char data[273] = "GET /login HTTP/1.1\r\nHost: www.minecraft.net\r\nUser-Agent: Web-sniffer/1.0.37 (+http://web-sniffer.net/)\r\nAccept-Charset: ISO-8859-1,UTF-8;q=0.7,*;q=0.7\r\nCache-Control: no-cache\r\nAccept-Language: de,en;q=0.7,en-us;q=0.3\r\nReferer: http://web-sniffer.net/\r\n\r\n";
if ((val = send(sockfd, &data,272,0)) == -1){
perror("send");
}
And here is the code containing the POST command, username and password are changed so no personal information is given out:
char data2[405] = "POST /login HTTP/1.1\r\nHost: www.minecraft.net\r\nConnection: close\r\nUser-Agent: Web-sniffer/1.0.37 (+http://web-sniffer.net/)\r\nAccept-Charset: ISO-8859-1,UTF-8;q=0.7,*;q=0.7\r\nCache-Control: no-cache\r\nAccept-Language: de,en;q=0.7,en-us;q=0.3\r\nReferer: http://web-sniffer.net/\r\nContent-type: application/x-www-form-urlencoded\r\nContent-length: 38\r\n\r\nusername=ausername&password=apassword";
if ((val = send(sockfd, &data2,404,0)) == -1){
perror("send");
}
To make the packets readable:
GET /login HTTP/1.1[CRLF]
Host: www.minecraft.net[CRLF]
User-Agent: Web-sniffer/1.0.37 (+http://web-sniffer.net/)[CRLF]
Accept-Charset: ISO-8859-1,UTF-8;q=0.7,*;q=0.7[CRLF]
Cache-Control: no-cache[CRLF]
Accept-Language: de,en;q=0.7,en-us;q=0.3[CRLF]
Referer: http://web-sniffer.net/[CRLF]
[CRLF]
And the POST packet:
POST /login HTTP/1.1[CRLF]
Host: www.minecraft.net[CRLF]
Connection: close[CRLF]
User-Agent: Web-sniffer/1.0.37 (+http://web-sniffer.net/)[CRLF]
Accept-Charset: ISO-8859-1,UTF-8;q=0.7,*;q=0.7[CRLF]
Cache-Control: no-cache[CRLF]
Accept-Language: de,en;q=0.7,en-us;q=0.3[CRLF]
Referer: http://web-sniffer.net/[CRLF]
Content-type: application/x-www-form-urlencoded[CRLF]
Content-length: 38[CRLF]
[CRLF]
username=ausername&password=apassword
As you may notice, the User-Agent is Web-sniffer as I used web-sniffer.net to make these packets, which worked on their website as the POST command there returned a 302 FOUND status code.
I am wondering why this behavior is observed here when the exact same packets worked on web-sniffer.net, yet don't work here.