I want to make a database where I store data from HAR files. Specifically, I want, by European standards, to store the following headers from HTTP requests and responses.
content-type
cache-control
pragma
expires
age
last-modified
host
In order to make an efficient MySQL database though, I need to categorise them in a way where I get the least amount of empty columns. Problem is, that I cannot find any information about which types of request/responses get specific headers.
The idea is to make different ISA tables for request and headers where I include the headers that will definitely be included in the corresponding request/response.
I think it might have to do with the method of the request, but I have not found ANY info about that online
In general, header fields are not specific to methods; more likely to the distintion between request and response, status code, and whether a payload is present in the request.
(The details are in RFCs 7230..5).
Related
If a request fails, HTTP POST is normally not idempotent (executing a failed request again might cause multiple inserts). What do you think about using the users session id as UUID v5 "namespace" and the JSON payload as the "name"? It would result in the same ID for multiple requests and the database would reject an additional insert.
There are APIs that specifically mark HTTP methods that are otherwise non-idempotent as idempotent.
POST being non-idempotent by default does not mean it's not allowed to be that, it just means that generic clients can't assume they are.
The best implementation I've seen is the Stripe API, that uses an Idempotency-Key as a HTTP header. The client defines this, and if 2 requests are received with an identical id, stripe knows how to handle the second. I think this is the best approach, and better than the idea of trying to construct a hash based on the request. A request looking identical does not mean the effect is the same, consider for example this POST request:
POST /increment
Content-Type: application/json
{ "increment-by": 2 }
If I send this request twice, I expect some id to be increased to 4, even if the request body was the same each time.
The Idempotency-Key lets a client control and inform the server if 2 requests were actually the same.
https://stripe.com/blog/idempotency
Followups:
Do I store the Idempotency-key as a separate column on the record?
I would be inclined to implement this feature globally as some kind of middleware.
Storing the Idempotency-key in something like Redis yields the risk of two realities (e.g. server creates db record and crashes before writing to Redis).
Use a transaction.
All you have to store about the key is that you've seen it before, and you only have to store it if the request was successful.
I want to make a database query from frontend (Angular) to backend. But I need to send lots of parameters for that.
As far as I understand, if we are not making any database changes, it is better to use GET as it uses cached entries. POST should be used used if we need to make changes on server/DB.
But if I want to send many parameters (some are serialized objects) and make no server side changes, will it be alright to use POST request in that case and embed all parameters in the POST body instead of sending a huge URL encoded GET request?
To first clear this up: responses to POST requests can be cached, as long as the origin server returns the proper caching response headers. However, browsers and proxy servers generally don't apply caching to POST requests.
That being said, with the proper encoding you can store a lot of information in the ~ 2 KB of a query string, so GET should be the way to go.
If you're certain you'll go beyond the limits of a GET request and you'll need to go the POST way while remaining cacheable, you could introduce a kind of "nonce", for example using a hash of the search parameters:
Client does a POST to /search, with the search parameters.
Server stores the parameters somewhere, for example in a database.
Server generates a nonce from the parameters, for example by hashing the search parameters, or the row ID.
Server redirects the client to the result page for that nonce: /search/123abc.
Client requests the /search/123abc search results page.
Server performs the search based on the nonce, and returns a response which is cacheable.
This will introduce one additional HTTP roundtrip, but enable caching cross-browser and through proxy servers.
I think you should use post in this situation which is more manageable and looks clean. For more benefit of post follow these links:
Link 1
Link 2
I'm pretty new to go and I want to build a CRUD rest API on GAE without views just simple JSON Rest API.
There is allot of frameworks out there,
go-http-routing-benchmark.
But I'm not sure which one will be most suitable for GAE.
My main concern is how to handle a safe and secure session .
As mentioned in a comment, you can start with the Go standard library, and only utilize 3rd party libs if you reach a point when the standard library is not sufficient for you (which point you may never reach).
If your clients are not browsers (you said you don't want any views) but any other arbitrary HTTP clients, an HTTP session may not be what you want. An HTTP session is usually managed by storing a session ID in an HTTP cookie which is automatically sent by the browser along with each HTTP request, and at the server side this session ID is read and an associated, server side data structue is looked up by it.
A common solution is to use some kind of secret information referred to as a key or API key. The idea is that if you want to grant access to someone, you generate a secret key (e.g. a random text) at server side which you store in the database. You send this key to the client who has to attach this to every API request he makes. At server side in the beginning of each API request you can check if the provided API key is valid (this also identifies the caller) and act accordingly.
The API key can be sent in various ways by the clients, e.g. as a URL parameter (strongly not recommended for unsecure HTTP requests but is perfectly fine for HTTPS requests), as an HTTP header field or as part of the request data structure. It is really up to you how you expect it, usually depends on how the requests look like (e.g. if they don't include any data, it's better to put the KEY in a header or URL parameter; if the clients are expected to send other, complex data which can be in the form of JSON text, it can be convenient to also include the API key in the JSON data too).
As part of a bookmarklet, I want to load in a script www.example.com/a.js
This file may change in the future so I want to do some versioning on it.
However, since I must hardcode the url in the bookmarklet, I can't use proper url-versioning .
What would be a good practice to do versioning instead?
I imagine using etag would probably work. I.e.: update the eTag on each version update and have the client request the file with the If-None-Match header.
Would this work?
have the client request the file with the If-None-Match header.
That I know of, there is no way to control or set the headers that a browser will use when loading a page resource such as images and script.
ETag is something that should work "out of the box". If the client sends an ETag header with the file then the browser should automatically send the If-None-Match header for the next request.
One frequently seen method to ensure a fresh file, although less efficient, is to add a time stamp or random number to the script like http://example.com/script.js?1234567. Of course that will force the client to download the script every time. You could instead use different date strings to force a download only once per day or week or month, etc.
And then there is just the good old standard caching headers expires and cache-control that you can use as well to force a fresh download every X period of time.
I need to send a multipart document to a server. IT seems however that the server expects the boundary to be a specific sequence "********". If I use multipart request, it seems to default to something else even when I add a request header.
That is indeed missing in the current version of Codename One (didn't expect people would need functionality like this).
We will add a setBoundary(String) method for the next update in 3 weeks or so.