I'm pretty new to go and I want to build a CRUD rest API on GAE without views just simple JSON Rest API.
There is allot of frameworks out there,
go-http-routing-benchmark.
But I'm not sure which one will be most suitable for GAE.
My main concern is how to handle a safe and secure session .
As mentioned in a comment, you can start with the Go standard library, and only utilize 3rd party libs if you reach a point when the standard library is not sufficient for you (which point you may never reach).
If your clients are not browsers (you said you don't want any views) but any other arbitrary HTTP clients, an HTTP session may not be what you want. An HTTP session is usually managed by storing a session ID in an HTTP cookie which is automatically sent by the browser along with each HTTP request, and at the server side this session ID is read and an associated, server side data structue is looked up by it.
A common solution is to use some kind of secret information referred to as a key or API key. The idea is that if you want to grant access to someone, you generate a secret key (e.g. a random text) at server side which you store in the database. You send this key to the client who has to attach this to every API request he makes. At server side in the beginning of each API request you can check if the provided API key is valid (this also identifies the caller) and act accordingly.
The API key can be sent in various ways by the clients, e.g. as a URL parameter (strongly not recommended for unsecure HTTP requests but is perfectly fine for HTTPS requests), as an HTTP header field or as part of the request data structure. It is really up to you how you expect it, usually depends on how the requests look like (e.g. if they don't include any data, it's better to put the KEY in a header or URL parameter; if the clients are expected to send other, complex data which can be in the form of JSON text, it can be convenient to also include the API key in the JSON data too).
Related
I am building a frontend application using react and I am wondering whether it is risky to expose API endpoint URLs (ex: https://myapi.com/v1/getitems) on GitHub, assuming that the endpoint has several security measures that are not exposed such as CORS and JWT Token Bearer Authentication. I would assume not, since, if someone were to take the endpoint and send requests, they would need a token and be allowed to do so by CORS.
Is there any security risk in doing so?
Yes. Don't add the base url of your api on github in plain view. Even though you might have CORS and Authorization, that doesn't stop a malicious actor to keep spamming your back-end.
What you do is create a .env file in your root folder. You can use a library like #beam-australia/react-env and there are others as well.
You add in the .env file the values that are important for your environment and that are usually secrets, and you want them to not be visible in your code. You do it like so:
API_URL="https://myapi.com/v1"
And then you access this variable in your code with env("API-URL") ( in the #beam-australia/react-env case, but others libraries work the same). Keep in mind that you need to add .env in .gitignore so that the .env file is not pushed to github.
Regarding requests, you can make one like so:
fetch(`${env("API_URL}/getitems`)
This way your code will be stripped of the API's base url and if someone sees your code it will see only the endpoint, not the full url.
Publishing the code of the API is risky on its own. Somebody can find a vulnerability in it and instantly hack it. If you add the address of the API to the code you help this kind of attacks. They can get the address with some investigation; OSINT and social engineering too, but better to reduce the attack surface.
As of the secrets, they must never be near to the code, because you or another developer can accidentally publish it. It happened too many times with many developers, so better to take this seriously. If you want to keep the address in secret, then you must extract it from the code and put it in the configuration of the production environment which is imported from a totally different location than your code. Using environment variables like Alex suggested is a good idea for this. Nowadays it is common to use docker, which has a way to manage secrets, so you don't need to reinvent the wheel: https://docs.docker.com/engine/swarm/secrets/ Another aspect that the configuration belongs to the actual instance. In OOP terms you want to describe the injected properties of an object in its class or in a global variable, which is an antipattern.
As of client side REST clients like javascript applications running in the browser or Android/iOS apps, you must not publish your secrets along with the REST client, otherwise it is not a secret anymore. In that case you need a server side part for the REST client and for example sign and encrypt JWT there with a secret key. It is your decision whether this server side part of the REST client sends the HTTP request to the REST API and in that case you can hide the URI of the REST API or it just manages the JWT and the client side part of the REST client sends it. If the server side part of the REST client sends the HTTP request to the REST API, then you can even use traditional sessions with session cookies between the client side and the server side parts of the REST client, but you cannot use them between the server side part of the REST client and the REST API where the communication must be stateless. Though it does not make much sense to have a separate REST API if you don't have multiple different REST clients in this scenario e.g. browser clients for JS and JSless browsers, Android and iOS clients, fully automated clients running on servers, etc. So don't confuse the REST client - REST API relationship with the browser - HTTP server relationship, because they are not necessarily the same. Most of the REST clients run on servers, not in the browser.
If a request fails, HTTP POST is normally not idempotent (executing a failed request again might cause multiple inserts). What do you think about using the users session id as UUID v5 "namespace" and the JSON payload as the "name"? It would result in the same ID for multiple requests and the database would reject an additional insert.
There are APIs that specifically mark HTTP methods that are otherwise non-idempotent as idempotent.
POST being non-idempotent by default does not mean it's not allowed to be that, it just means that generic clients can't assume they are.
The best implementation I've seen is the Stripe API, that uses an Idempotency-Key as a HTTP header. The client defines this, and if 2 requests are received with an identical id, stripe knows how to handle the second. I think this is the best approach, and better than the idea of trying to construct a hash based on the request. A request looking identical does not mean the effect is the same, consider for example this POST request:
POST /increment
Content-Type: application/json
{ "increment-by": 2 }
If I send this request twice, I expect some id to be increased to 4, even if the request body was the same each time.
The Idempotency-Key lets a client control and inform the server if 2 requests were actually the same.
https://stripe.com/blog/idempotency
Followups:
Do I store the Idempotency-key as a separate column on the record?
I would be inclined to implement this feature globally as some kind of middleware.
Storing the Idempotency-key in something like Redis yields the risk of two realities (e.g. server creates db record and crashes before writing to Redis).
Use a transaction.
All you have to store about the key is that you've seen it before, and you only have to store it if the request was successful.
I have an application written in angularjs and a dropwizard backend. All API calls are ajax, with the exception of file downloads, which is done by performing a redirect to a standard GET request.
All API calls are secured through a token which is passed as a Token header. We use SSL for all APIs.
The download GET request works but I'm having a hard time figuring out how to secure it. I have no way of setting a custom header, which is required to pass the token. So theoretically, I'm left with two options, clearly none of them acceptable: 1. Pass the token as one the GET parameters 2. Leave the download unsecured.
Any ideas how to secure file download?
Putting a secret token in a URL query parameter isn't great because URL tend to be leakable, for example through history/logging/referrers. There are ways to mitigate this: for example you could have the server side issue a download token that is only good for one use or for a limited amount of time. Or the client could pass a time-limited token created using a signature over the secret token that the server side could verify.
Alternatively you could, just for this one interface (eg path-limited, quitckly-expiring) put the token in a cookie.
Another approach is to download the whole file through AJAX, thus allowing you to set the header as normal. Then you have to present the content as a downloadable local resource, which requires a cocktail of browser-specific hacks (eg using data: or filesystem: URLs, and potentially links with the download attribute). Given the complication this isn't usually worth bothering with, especially if the file is very large which may present further storage constraints.
I've read quite a few SO threads about authentication and authorization with REST and Angular, but I'm still not feeling like I have a great solution for what I'm hoping to do. For some background, I'm planning to building an app in AngularJS where I want to support:
Limited guest access
Role-based access to the application once authenticated
Authentication via APIs
All of the calls to the REST API will be required to occur over SSL. I'd like to do build the app without breaking RESTful principles, namely not keeping session state stored on the server. Of course, whatever is done vis-a-vis authorization on the client-side has to be reinforced on the server side. Since we need to pass the entire state with each request, I know I need to pass some sort of token so that the backend server receiving the REST request can both authenticate and authorize the call.
With that said, my main question is around authentication - what are the best practices here? It seems there are lots of different approaches discussed, here's just a few that I've found:
http://broadcast.oreilly.com/2009/12/principles-for-standardized-rest-authentication.html
http://frederiknakstad.com/2013/01/21/authentication-in-single-page-applications-with-angular-js/
http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html
There was a similar question asked (AngularJS best practice application authentication), but unless I'm misunderstanding the answer, it seems to imply that a server session should be used, which is breaking RESTful principles.
My main concern with the Amazon AWS and the George Reese article is it seems to assume that the consumer is a program, rather than an end user. A shared secret can be issued to a programmer in advance, who can then use it to encode calls here. This isn't the case here - I need to call the REST API from the app on behalf of the user.
Would this approach be enough? Let's say I have a session resource:
POST /api/session
Create a new session for a user
To create a session, you need to POST a JSON object containing the "username" and "password".
{
"email" : "austen#example.com",
"password" : "password"
}
Curl Example
curl -v -X POST --data '{"username":"austen#example.com","password":"password"}' "https://app.example.com/api/session" --header "Content-Type:application/json"
Response
HTTP/1.1 201 Created {
"session": {
"id":"520138ccfa4634be08000000",
"expires":"2014-03-20T17:56:28+0000"
}
}
Status Codes
201 - Created, new session established
400 - Bad Request, the JSON object is not valid or required information is missing
401 - Unauthorized, Check email/password combo
403 - Access Denied, disabled account or license invalid
I'm leaving out the HATEOAS details for clarity. On the backend, there would be a new, limited duration session key created and associated with the user. On subsequent requests, I could pass this as part of the HTTP headers:
Authorization: MyScheme 520138ccfa4634be08000000
Then the backend servers would be responsible for digesting this out of the request, finding the associated user and enforcing authorization rules for the request. It should probably update the expiration for the session as well.
If all this is happening over SSL, am I leaving the door open to any kind of attacks that I should be protecting against? You could try to guess session keys and place them in the header, so I suppose I could additionally append a user GUID to the session key to further prevent brute force attacks.
It's been a few years since I've actively programmed and I'm just getting back into the swing here. Apologies if I'm being obtuse or unnecessarily reinventing the wheel, just hoping to run my ideas by the community here based on my reading thus far and see if they pass the litmus test.
When someone asks about REST authentication, I defer to the Amazon Web Services and basically suggest "do that". Why? Because, from a "wisdom of the crowds" point of view, AWS solves the problem, is heavily used, heavily analyzed, and vetted by people that know and care far more than most about what makes a secure request than most. And security is a good place to "not reinvent the wheel". In terms of "shoulders to stand on", you can do worse than AWS.
Now, AWS does not use a token technique, rather it uses a secure hash based on shared secrets and the payload. It is arguably a more complicated implementation (with all of its normalization processes, etc.).
But it works.
The downside is that it requires your application to retain the persons shared secret (i.e. the password), and it also requires the server to have access to that a plain text version of the password. That typically means that the password is stored encrypted, and it then decrypted as appropriate. And that invite yet more complexity of key management and other things on the server side vs secure hashing technique.
The biggest issue, of course, with any token passing technique is Man in the Middle attacks, and replay attacks. SSL mitigates these mostly, naturally.
Of course, you should also consider the OAuth family, which have their own issues, notably with interoperability, but if that's not a primary goal, then the techniques are certainly valid.
For you application, the token lease is not a big deal. Your application will still need to operate within the time frame of the lease, or be able to renew it. In order to do that it will need to either retain the user credential or re-prompt them for it. Just treat the token as a first class resource, like anything else. If practical, try and associate some other information with the request and bundle it in to the token (browser signature, IP address), just to enforce some locality.
You are still open to (potential) replay problems, where the same request can be sent twice. With a typical hash implementation, a timestamp is part of the signature which can bracket the life span of the request. That's solved differently in this case. For example, each request can be sent with a serial ID or a GUID and you can record that the request has already been played to prevent it from happening again. Different techniques for that.
Here is an incredible article about authentication and login services built with angular.
https://medium.com/opinionated-angularjs/7bbf0346acec
This SO question do a good job of summing up my understanding of REST
Do sessions really violate RESTfulness?
If you store a token in a session you are still creating state on the server side (this is an issue since that session is typically only stored on the one server, this can be mitigated with sticky sessions or other solutions).
I'd like to know what your reasoning is for creating a RESTful service though because perhaps this isn't really a large concern.
If you send a token in the body along with every request (since everything is encrypted with SSL this is okay) then you can have any number of servers (load balanced) servicing the request without any previously knowledge of state.
Long story short I think aiming for RESTful implementations is a good goal but being purely stateless certainly creates an extra layer of complexity when it comes to authentication and verifying authorization.
Thus far I've started building my back-ends with REST in mind, making URIs that make sense and using the correct HTTP verbs, but still use a token in a session for the simplicity of authentication (when not using multiple servers).
I read through the links you posted, the AngularJS one seems to focus just on the client and doesn't seem to explicitly address the server in that article, he does link to another one (I'm not a Node user so forgive me if my interpretation is wrong here) but it appears the server is relying on the client to tell it what level of authorization it has which is clearly not a good idea.
I am trying to use Play 2.x with backbone.js for a project. My intention is to create RESTful APIs at the server end (all response bodies are in JSON and all request bodies are in JSON also).
I would like to use Facebook OAuth (server side) to authenticate my requests. For this purpose I'm using play-authorize for OAuth. The issue I am having is that the user session information is stored in the Session Object in Play. I don't really want to use play-templates in my HTML code, how can I use the Session Object on the client side without the play-templates.
Also what measures can I use to prevent CSRF/XSS attacks while using Play.
Maybe I'm misunderstanding your question, but be aware that Play is stateless. That means that the "Session" is stored in a cookie that gets sent to the server. You can store string values to that cookie and access them from the browser.
Now, you don't want to store critical values in there, but something that the server side code recognizes and lets you work with to solve your problem.