If a request fails, HTTP POST is normally not idempotent (executing a failed request again might cause multiple inserts). What do you think about using the users session id as UUID v5 "namespace" and the JSON payload as the "name"? It would result in the same ID for multiple requests and the database would reject an additional insert.
There are APIs that specifically mark HTTP methods that are otherwise non-idempotent as idempotent.
POST being non-idempotent by default does not mean it's not allowed to be that, it just means that generic clients can't assume they are.
The best implementation I've seen is the Stripe API, that uses an Idempotency-Key as a HTTP header. The client defines this, and if 2 requests are received with an identical id, stripe knows how to handle the second. I think this is the best approach, and better than the idea of trying to construct a hash based on the request. A request looking identical does not mean the effect is the same, consider for example this POST request:
POST /increment
Content-Type: application/json
{ "increment-by": 2 }
If I send this request twice, I expect some id to be increased to 4, even if the request body was the same each time.
The Idempotency-Key lets a client control and inform the server if 2 requests were actually the same.
https://stripe.com/blog/idempotency
Followups:
Do I store the Idempotency-key as a separate column on the record?
I would be inclined to implement this feature globally as some kind of middleware.
Storing the Idempotency-key in something like Redis yields the risk of two realities (e.g. server creates db record and crashes before writing to Redis).
Use a transaction.
All you have to store about the key is that you've seen it before, and you only have to store it if the request was successful.
Related
I need a serverless communication between a server(less backend) and a client. The client asks for a token. When the client makes a request with this token, the backend generates a new token and sends it back to the client. When the client tries to make a request with a previous token, the backend rejects it. I don't want the backend to keep track, either in ram or in a database, a whitelist or a blacklist of valid/invalid tokens. The backend is allowed to have a static lookup table or/and a static rule/algorithm to perform this logic if needed (to use the information inside token's payload).
So, is it possible to achieve something like this ? Is there a way to apply certain kind of information inside each token to know wether you have accepted it once or not ?
In your scenario the server is stateless (at least regarding authentication) so you cannot use the state of the server to discriminate if a received token is already used.
Also the moment you generate the token it is before it's first usage, so you cannot inject into it anything that tells if it was used or not: simply you don't have that information at that time of course.
So basically if your only information containers are these two (stateless server and self generated token) the answer is no no matter how it is done; simply there is no place where this bit of information (is this token last one or not) can be placed in the moment the information is generated (at the first usage time).
Theoretically speaking you can send this information to a third party entity and ask it back when you need it.. ..but this is just cheating: if you are not accepting a DB or RAM or filesystem storage, I suppose that sending this information somewhere through an API or else is just an excluded option as the other ones.
May be you can try TOTP, https://en.wikipedia.org/wiki/Time-based_One-time_Password_algorithm. This is the same algorithm used for MFA.
Here is the python implementation, you can find implementation in other languages as well.
https://pypi.org/project/pyotp/2.0.1/
Backend:
Create a random key and save it static database.
When client request for token create a token(+base64 conversion) using random key and send back to client.
When your server gets the token(-base64 conversion). Using the same random key verify the received token.
The totp algorithm will ensure that the old tokens are not valid.
You token is usually valid for 30 secs. so you may want decide on managing the validity of tokens.
I have a page with multiple widgets, each receiving data from a different query in the backend. Doing a request for each will consume the limit the browser puts on the number of parallel connections and will serialize some of them. On the other hand, doing one request that will return one response means it will be as slow as the slowest query (I have no apriori knowledge about which query will be slowest).
So I want to create one request such that the backend runs the queries in parallel and writes each result as it is ready and for the frontend to handle each result as it arrives. At the HTTP level I believe it can be just one body with serveral json, or maybe multipart response.
Is there an angularjs extension that handles the frontend side of things? Optimally something that works well with whatever can be done in the Java backend (didn't start investigating my options there)
I have another suggestion to solve your problem, but I am not sure you would be able to implement such a thing as from you question it is not very clear what you can or cannot do.
You could implement WebSockets and the server would be able to notify the front-end about the data being fetched or it could send the data via WebSockets right away.
In the first example, you would send a request to the server to fetch all the data for your dashboard. Once a piece of data is available, you could make a request for that particular piece and given that the data was fetched couple of seconds ago, it could be cached on the server and the response would be fast.
The second approach seems a more reasonable one. You would make an HTTP/WebSocket request to the server and wait for the data to arrive over WebSocket.
I believe this would be the most robust an efficient way to implement what you are asking for.
https://github.com/dfltr/jQuery-MXHR
This plugin allows to parse a response that contains several parts (multipart) by having a callback to parse each part. This can be used in all our frontends to support responses for multiple data (widgets) in one requests. The server side will receive one request and use servlet 3 async support (or whatever exists in other languages) to ‘park’ it, sending multiple queries, writing each response to the request as each query returns (and with the right multipart boundary).
Another example can be found here: https://github.com/anentropic/stream.
While both of these may not be compatible with angularjs, the code does not seem complex to port there.
I want to make a database query from frontend (Angular) to backend. But I need to send lots of parameters for that.
As far as I understand, if we are not making any database changes, it is better to use GET as it uses cached entries. POST should be used used if we need to make changes on server/DB.
But if I want to send many parameters (some are serialized objects) and make no server side changes, will it be alright to use POST request in that case and embed all parameters in the POST body instead of sending a huge URL encoded GET request?
To first clear this up: responses to POST requests can be cached, as long as the origin server returns the proper caching response headers. However, browsers and proxy servers generally don't apply caching to POST requests.
That being said, with the proper encoding you can store a lot of information in the ~ 2 KB of a query string, so GET should be the way to go.
If you're certain you'll go beyond the limits of a GET request and you'll need to go the POST way while remaining cacheable, you could introduce a kind of "nonce", for example using a hash of the search parameters:
Client does a POST to /search, with the search parameters.
Server stores the parameters somewhere, for example in a database.
Server generates a nonce from the parameters, for example by hashing the search parameters, or the row ID.
Server redirects the client to the result page for that nonce: /search/123abc.
Client requests the /search/123abc search results page.
Server performs the search based on the nonce, and returns a response which is cacheable.
This will introduce one additional HTTP roundtrip, but enable caching cross-browser and through proxy servers.
I think you should use post in this situation which is more manageable and looks clean. For more benefit of post follow these links:
Link 1
Link 2
I'm pretty new to go and I want to build a CRUD rest API on GAE without views just simple JSON Rest API.
There is allot of frameworks out there,
go-http-routing-benchmark.
But I'm not sure which one will be most suitable for GAE.
My main concern is how to handle a safe and secure session .
As mentioned in a comment, you can start with the Go standard library, and only utilize 3rd party libs if you reach a point when the standard library is not sufficient for you (which point you may never reach).
If your clients are not browsers (you said you don't want any views) but any other arbitrary HTTP clients, an HTTP session may not be what you want. An HTTP session is usually managed by storing a session ID in an HTTP cookie which is automatically sent by the browser along with each HTTP request, and at the server side this session ID is read and an associated, server side data structue is looked up by it.
A common solution is to use some kind of secret information referred to as a key or API key. The idea is that if you want to grant access to someone, you generate a secret key (e.g. a random text) at server side which you store in the database. You send this key to the client who has to attach this to every API request he makes. At server side in the beginning of each API request you can check if the provided API key is valid (this also identifies the caller) and act accordingly.
The API key can be sent in various ways by the clients, e.g. as a URL parameter (strongly not recommended for unsecure HTTP requests but is perfectly fine for HTTPS requests), as an HTTP header field or as part of the request data structure. It is really up to you how you expect it, usually depends on how the requests look like (e.g. if they don't include any data, it's better to put the KEY in a header or URL parameter; if the clients are expected to send other, complex data which can be in the form of JSON text, it can be convenient to also include the API key in the JSON data too).
Iam writing a C program to interact with HTTPs server. Server is expecting the data without
any assignments(Ex: normally a request can be "https://xz.aspx?name=google" where as is it
possible to send the name "https://xz.aspx?google"). Currently server is getting an entry
log for my request but not able to fetch request data.
1.Is it possible to send a value with out assignment?
2.Will .net look for default assignments?
3.Is there anything else to probe?
The data you're sending is just whatever you put in the query part of the request-uri in the HTTP request.
That data can be almost anything you like (as long as it is using letters that are valid according to RFC2616). The "assignments" concept is not something HTTP knows or uses, it is just a common way for clients and servers to deal with the data.
so... Yes, you can send a value "without assignment" with curl. Weather the receiver will like it or understand it is a completely different matter.