I'm implementing a correlation ID within my applications and would like some feedback on the design for it. The primary concern is that the correlation ID should be available for all logs.
Lets say I have a web (front-end) application which is serving pages to my users. It talks with two API's which provide data. The API's are not exposed to the user, and so all requests 'begin' in the front-end app.
The API's job is simple, they consume the correlation ID as provided in all headers from the front-end app (X-Correlation-ID) and print it in any logs.
The front-end app has to generate the ID, add it to the headers for outgoing requests, but it must also consume the ID.
My question is this: How does the front-end app store the correlation ID?
My first thought was that it would modify the incoming request and add the header if it did not exist, however this would make the incoming request somewhat 'unreliable' as it is now modified.
Another thought is perhaps it is stored as some kind of application global that is cleared per request.
Correlation ids i.e. ids typically attached to headers like Request-ID, X-Request-ID, X-Trace-ID, X-Correlation-ID are typically issued per request.
You seem to want to store it locally on the client though. What you describe sounds more like a “session id” that gets reset when the client “restarts”. If that is the case, then you simply use local/session storage or cookies to store and clear it when needed.
Do keep in mind that first sentence above though. Correlation ids are typically used per request. What I usually do:
Generate an id on the client per request
Pass it to the API via one of the aforementioned headers
Whoever gets the request first (some API gateway, HA Proxy etc) checks for the existence of the header and proxies it further downstream. So do any services calling other services. This is usually provided as a service:tool to most services/teams so that they don’t forget to do it.
Profit?
That’s what heroku does for example. Same for many other services / companies.
Goes without saying, you can combine the two ids, the “session” one you refer to plus the ones generated per request to get a better view of what is going on in logs etc
As per my design, I'd suggest to always check (intercept using middlewares/filters on backend) each request from the client at the first point of contact at the backend (load balancer/ gateway / controller) and check for the Trace-ID/Correlation-ID at the request header, if present then forward the request as it is, if not present (because this is the first call from that new client), then generate the random ID (as Trace-ID) and attach that newly generated random ID to the Request Header and reroute/pass on the request further down the application.
One more thing, while sending back the response, make sure to again add this ID generated earlier to the Response Header, so that the client can receive this unqiue ID and save it on localstorage/cookie for further calls so that the client could easily be traced using that trace ID.
As you requested for logs, now since you have that correlation ID/trace ID, you can log them anywhere you want and you can easily determine the complete flow of the client's request in case of any issues using this unique ID.
Steps:
Check for the Correlation ID in the Request Header (at Backend)
If already present, allow the request to pass through to the required service.
If not present, generate a unique random ID and add a custom Header in the Request Header.
Again, before sending the response, again add this custom Header to the Response header for client.
I hope that answers your query.
Related
I have a rest api which has organisations as a top level object and all other data belongs to an organisation e.g. orders, invoices, projects etc..
Users have the ability to belong to multiple organisations and on my frond end SPA I want them to be logged into one application at a time with the ability to switch.
Should I be storing the current organisation id on the client side and passing it with every request or should i have a rest endpoint that sets the current org id on the user table etc, which will also mean when they come back and login it is already set to the last organisation they accessed.
Should I be storing the current organization id on the client side and passing it with every request?
Yes,
a general practice is sending the information in each API because API are stateless.
Should I have a rest endpoint that sets the current org id on the user table?
NO you should not, one of the reason is let say user logs in browser1 you will store that organizationId in table, same user login from browser2 this will override your previous Id
Now if you go back to browser1 it will have inconsistent data unless you had designed a mechanism not to override the previous value.
Hope that make sense
In general, stateless APIs are fairly common. For instance, you might pass an Auth Token (JWT) with every single request, which identifies the User and could potentially provide other info about the User as well.
If you're using Laravel, you might want to look into this package to handle JSON Web Tokens: https://github.com/tymondesigns/jwt-auth. You could then serialize your User object on Login, which could also include their current Organization. The frontend can then pass this token with every request (as an Authorization Header)
I have the following code in my angular app declaration - an API key for Facebook (to implement Share button):
.run(function($FB){
$FB.init('9xxxxxxxxxxxx94');
})
So i know the general answer to this - 'API keys should be kept on the server side', however I don't see how i actually implement this.
The share call-method is made on the front end, so even if my server kept the API key and sent it, surely it's still visible on the front end, else how would the share button work?
So my question, how do I hide that Facebook API Key?
Thanks.
Requesting the key
The first thing that happens is that the client will request a key. This will only happen on certain pages like the sign up and log in pages. The idea here is that we want to make sure that only users browsing with a known client (in this case the official website or core client as it’s called) are allowed to take actions like creating or authenticating a user.
So when the client app requests the login page the server generates a unique token based on information sent in the request. The information used is always something the server knows, something the client knows, and something both know. So for example the server can generate a unique key based on User agent + current time + secret key. The server generates a hash based on this information and then stores a cookie containing only the hash on the client machine.
Setting permissions
At this point our key really isn’t a key anymore. It has been transformed into an access token. The server should then take this access token and store it for later retrieval. You can put the key in a database but since data of this type needs to be retrieved often I would suggest using a key-value store like Redis to cut down on database reads/writes and boost performance.
When you store the token you should also store a separate piece of data to indicate what permissions are associated with the token. In this case our token is acting only as a way to register and authenticate users so we store it next to a value that indicates who the token belongs to (the app’s web UI) and what permissions it has (limited to create and authenticate users). We treat it just like we would any other API client that way we can capture stats and control how it is used.
Authorizing a request
When the client then makes the POST request to create a new user or log in the server will check to see if the client sent an identifying cookie along with the request. If not, we reject the request. If it does send the cookie, the server should once again generate the hash using the values used previously (these values are either already known or sent with the request anyway so we’re not really taxing the server much) compare it to the cookie being sent to us, and if the values match allow the request to proceed.
Sources - Securing API Keys
OR
Simply send a request to your Server and let him handle your request with the hidden API-key and just return the result of your request to your front-end.
I am creating an Angular application, and I am having trouble wrapping my head around the proper way to ensure my application and its users is secure.
I've been reading around many stack discussions, but I believe I am missing some core understanding of what is happening, please correct any errors you see written below.
So far I have a Sinatra server with many (currently mostly hypothetical) resource routes. A user can create an account using an email address and password that is stored in a database after being hashed with BCrypt. When a user logs in, the record is retrieved from the database by email and the password checked for authentication. It is from this point I am not sure how to proceed.
Prior to this I have simply set a session variable and had the server check that the variable exists in order to correctly route logged in users. Now my application is (currently) a single HTML page that uses Angular and ui-router to display different content, so most of the requests are simply returning JSON content.
It is my understanding that Restful applications should generally not use sessions, or rather that the server should respond identically to identical requests and not have its own data that shapes a response. But if I do not store something in a session variable, how could the server know that the client making the request has the correct permissions? And are sessions not stored in the browser anyway, thus not part of the server?
I believe from what I have read, it is possible to create a token which is essentially a large random string, return that string to the client and also store it in a database with a timestamp. The client then provides this token when making requests and the server hits the database to verify it exists and valid. But would the client not also have to store that string in a cookie? I suppose the angular application could store the token in a variable, which would persist while using the ui-router but not if the users navigates using the address bar.
I also do not understand how Basic Auth may or may not fit into this picture. Any help would be greatly appreciated, as well as a pointer to some good resources where I may find a better understanding of these concepts in general.
You want to read up on JWT. There are JWT libraries for Ruby and Angular.
I know you aren't using Node for your backend but a very easy way to see all the pieces working together is to run the angular-fullstack Yeoman generator. It uses JWT and the code is easy to follow.
As far as I can see, whatever you are doing with your sessions can work just fine.
This can be a sample JSON response from the server in case the user is not loged in :
{
"errorCode": 1,
"error": "User not logged in",
"data": {}
}
You can set your own error codes and handle what you want to do. You will send any data only if the user is logged in. For all the pages which don't require authentication, you can set data to whatever you want.
On the angularJS side, you can handle based on error codes, you can redirect the user to the login page and so forth.
The alternate way to support the same on multiple platforms is to use token based approach. The token based approach in simple words work this way.
The user logs in for the first time with his / her credentials.
The server verifies these information and creates a token from which the server is able to decode the user id.
Whenever the client makes the requests, it passes its token with every request.
As the server can decode the user information from the token, it sends or doesn't send the data based on whether that's a right token or not.
The token depends on a secret value. It can be same for all the users or differnet for each based on how you want to implement.
This is all done and you can look at
http://jwt.io/
As #andy-gaskell mentioned, you can look at
http://angular-tips.com/blog/2014/05/json-web-tokens-introduction/
I'm very bad at explaining. Please let me know if any of this doesn't make sense.
you are missing the point of the REST concept. One of the main concepts in the REST apis is that the server should be stateless - this means that you should not store sessions or other "state" in your web server. Every HTTP request happens in complete isolation. Every request should include all data needed by the server to fulfill the request.
But if I do not store something in a session variable, how could the
server know that the client making the request has the correct
permissions?
You can store request scoped variables. This means that they should be only active during the same request. You can store the current logged in user in the request scoped variable. In that way you can get the current user in your invocation of the business method. I'm not familiar with Sinatra but here is the doc: http://www.sinatrarb.com/intro.html#Request/Instance%20Scope
But would the client not also have to store that string in a cookie?
of course you should store your access token in the client side
https://stormpath.com/blog/where-to-store-your-jwts-cookies-vs-html5-web-storage/
as #Andy Gaskell suggest take a look at JWT and fullstack application code generators and forget about the basic auth because it's really "basic".
more useful links:
If REST applications are supposed to be stateless, how do you manage sessions?
http://www.sitepoint.com/php-authorization-jwt-json-web-tokens/
In the single page webapp I've recently built I'm getting data for my models using Restangular module. I'd like to add real-time updates to the app so whenever any model has been changed or added on the server I can update my model list.
I've seen this working very well in webapps like Trello where you can see the updates without refreshing the web page. I'm sure Trello webclient uses REST API.
What is a proper way to architect both server and client to archive this?
First of all, your question is too general and can have a lot of solutions that depend
on your needs and conditions.
I'll give you a brief overview for a single case when you want to leave REST APIs
and add some realtime with web sockets.
Get all data from the REST -- Sokets for notifications only.
Pros: Easy to implement both server side and client side. You only need to emit events on the server with
info about modified resource (like resource name and ID), and catch these events on the client side and fetch
data with REST APIs.
Cons: One more request to the server on every notification. That can increase traffic dramaticaly when you have a lot of active clients for a single resource (they will generate a lot of reverse requests to the server).
Get initial load from the REST -- Sockets for notifications with data payload.
Pros: All info comes with the notification and will not cause new requests to the server, so we have less traffic.
Cons: Harder to implement both server side and client side. You will need to add data to all the events on the server. You will need to fetch data from all the events on the client side.
Updated according to the comment
As for handling different types of models (just a way to go).
Client side.
Keep a factory for each model.
Keep in mind that you need realtime updates only for displayed data (in most cases), so you can easily
use memory caching (so you can find any entity by its ID).
Add listener for every type of changes (Created, Updated, Deleted).
In any listener you should call some initObject function, that will find entity in the cache by ID and extend it, if there is no entity with such ID, just create a new one and add it to cache.
Any Delete just removes an entity from the cache.
Any time you need this resource, you should return the link to cache object in order to keep two way databinding (that is why I use extend and not =). Of course, you need to handle the cases like: "User is editing the resource while notification about deleting comes".
Server side.
It is easier to send all the model then just modified fields (in both cases you must send the ID of resource).
For any Create, Update, Delete event push event to all engaged users.
Event name should contain action name (like New, Update, Delete) and the name of resource (like User, Task etc.). So, you will have NewTask, UpdateTask events.
Event payload should contain the model or just modified fields with the ID.
Collection changes can be handled in two ways: with add/update/remove items in collection or changing all the collection as a whole.
All modifications like PUT, POST, DELETE are made with REST of course.
I've made a super simple pseudo gist for the case 1). https://gist.github.com/gpstmp/9868760 but it can be updated for case 2) like so https://gist.github.com/gpstmp/9900454
Hope this helps.
I have a Servlet when a request comes it checks for the user id and then if id is not there it creates a new user id in the database. But if I get multiple requests with a very short delay then all those request tend to see that there is not user at the moment and create multiple users with the same name. I just don't want to make the user id field unique to solve this problem. Other than the user id I store some related data as well.
I need to know how to keep a DB locked until one Servlet request is finished processing.
You need to make your servlet code synchronized.
Easy way is to make your servlet implement SingleThreadModel.
http://www.javatpoint.com/SingleThreadModel-interface
But this is not a good approach as your servlet will handle only one thread/request at a time. Good solution is to synchronize the part where you check and generate the uid.