I implemented this but the response headers don't include Content-Length, even though I make sure the server replies that bit specifically. I also verified the response outside CN1 and it includes Content-Length. The full list of headers captured in ReadHeaders is (as seen from Android): null,Alt-Svc,Cache-Control,Connection,Content-Type,Date,ETag,Server,Transfer-Encoding,Vary,X-Android-Received-Millis,X-Android-Response-Source,X-Android-Selected-Protocol,X-Android-Sent-Millis,X-Cloud-Trace-Context,X-Powered-By. Right now to estimate download sizes I (1) call endpoint to get total size (2) call endpoint to get actual download and use NetworkManager progress listener, but it would be nice to be able to track progress with only one request (by using Content-Length). The vanilla RequestBuilder doesn't expose response headers so a direct usage of ConnectionRequest with readHeaders is needed. But the Content-Length is missing from getHeaderFieldNames
Note:
The reason why this wasn't working is because by default Android/CN1 sends a request with the header Accept-Enconding:gzip. This returns a chunked response that doesn't include the length header. I can't guarantee that this behavior matches every server response, but it does in my case (Node.js + Express)
To force a server to return a non-chunked response, set the header to "compress", "identity", or "deflate"
Example:
Rest.post(url).header("Accept-Encoding", "compress").fetchAsJsonMap(resp -> {...
Related
Tell me please, what is the main difference between options CURLOPT_MIMEPOST and CURLOPT_POSTFIELDS ?
What can be done with a CURLOPT_MIMEPOST - that cannot be done with CURLOPT_POSTFIELDS ?
CURLOPT_POSTFIELDS
Sends exactly the bytes you specify in the body of the HTTP request. With a default Content-type of application/x-www-form-urlencoded. libcurl will not add or encode the data in any way for you.
With the curl command line tool, you do this with -d.
CURLOPT_MIMEPOST
Makes libcurl send a "multipart formpost". That is a data stream using a format that allows the sender to send multiple "parts" of data to the server, each part being properly separated and identified. Each part has a name, content and its own set of headers. When an HTTP client "uploads a file", this is almost always done using multipart formposts.
Multipart formpost is structured data in the request body and this option helps you produce and send that format. An application can also produce that format by themselves should they prefer that and provide it with CURLOPT_POSTFIELDS or even using the callback CURLOPT_READFUNCTION.
With the curl command line tool, you do this with -F.
See Also
https://everything.curl.dev/libcurl-http/upload
I am attempting to make a GET request for the /fills and /orders endpoints and it works if I do not add query parameters, however, if I add the query parameter "product_id=ETH-USD," I then receive a 400 response with an empty message. Is there anything else I need to do to the Authorization Header when making a GET request with query parameters?
Works: https://api.pro.coinbase.com/fills
400 Response: https://api.pro.coinbase.com/fills?product_id=ETH-USD
So the answer to my question is yes, the Authorization header needs the Body section even for the GET request. The Body section, in this instance, should have ?product_id=ETH-USD in the header signature.
I am writing an apache module output filter that needs to consume a couple of internal-only response headers. These response headers are set by a perl based application running in the backend. The APR function I am using in my output filter is:
apr_table_get(r->headers_out, "x-my-response-header");
However, what seems to happen is that in my output filter I do not see the above response header set, up until the third or fourth bucket brigade - which is unfortunately already too late - I actually need to use the value of x-my-response-header to compute a new response header and set that in the response to the browser.
I insert my output filter this way:
ap_hook_insert_filter(insertOutputFilterHook, NULL, NULL, APR_HOOK_FIRST);
ap_register_output_filter(myFiltersName, myOutputFilter, NULL, AP_FTYPE_CONTENT_SET);
What I have verified:
The internal-only headers do appear on the HTTP response on my browser (haven't unset them yet)
The first two bucket brigade's buckets contain html page text
Questions:
What could be the reasons for why the internal-only response header not be set/visible in the first call to my output filter / first bucket brigade?
Is it possible to instead accumulate the first few bucket brigades and then start flushing them out once the internal only response header's value known?
When I'm using google.appengine.api.urlfetch.fetch (or the asynchronous variant with make_rpc) to fetch a URL that steadily streams data, after a while I will get a google.appengine.api.urlfetch_errors.DeadlineExceededError as expected. Since it is a stream that I want to sample, setting the deadline to a higher value can't ever help, unless the stream finishes (which I do not expect to happen).
It seems there is no possibility of getting the partially downloaded result. At least the API doesn't offer anything. Is it possible to
either request the downloaded part
or only ask for a certain amount of data (since I can estimate the stream's rate) to be downloaded?
[Clarification: Since it is a stream, requests with a Range header will be answered with 200 OK and not 206 Partial Content.]
In your call to urlfetch.fetch, you can set HTTP headers. The Range header is how you specify a partial-download request in HTTP:
resp = urlfetch.fetch(
url=whatever,
headers={'Range': 'bytes=100-199'})
if those are the 100 bytes you want. The HTTP status code you get should be 206 for such a partial download, etc (none of that's GAE-specific). See e.g http://en.wikipedia.org/wiki/Byte_serving for details.
I am getting a "System.Net.ProtocolViolationException: Operation is not valid due to the current state of the object." error when trying to call
var request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "GET";
request.ContentType = "text/xml";
request.BeginGetRequestStream(RequestCompleted, request);
I suspect this may be because you are performing a BeginGetRequestStream on a request object for which you have specified the "GET" method.
When performing a "GET" the server will not be expecting an entity body in the request hence you should proceed straight to BeginGetResponse. Also specifying a ContentType on the request is not necessary, it specifies the type of content being sent in the entity body of the request but as stated a "GET" doesn't send any content it only gets content.
I disagree with AnthonyWJones answer. I find nothing in the HTTP spec that prohibits a "GET" request from containing a message body. I think this has unfortunately become the de facto understanding of how HTTP works since there is generally no need (or way) to include a message body. Having said that, he is correct as to the cause of this specific exception. However, I think the BCL should be changed to allow it.
If anyone can point it out I would be very interested to know where the spec precludes this:
HTTP RFC 2616