I'm developing the restful web app that using some popular web framework on the backend, say (rails, sinatra, flask, express.js). Ideally, I want to develop client side with Backbone.js. How do I let only my javascript client side interact with those API calls? I don't want those API calls to be public and be called by curl or simply by entering the link on browser.
As a first principle, if your API is consumed by your JS client, you have to assume, that it is public: A simple JS debugger puts an attacker into a position, where he can send a byte-for-byte identical request from a tool of his choice.
That said, if I read your question correctly, this is not, what you want to avoid: What you really don't want to happen is, that your API is consumed (on a regular basis) without your JS client being involved. Here are some ideas on how to if not enforce, then at least encourage using your client:
I am sure, your API has some sort of authentication field (e.g. Hash computed on the client). If not, take a look at This SO question. Make sure you use a salt (or even API key) that is given to your JS client on a session basis (a.o.t. hardcoded). This way, an unauthorized consumer of your API is forced into much more work.
On loading the JS client, remember some HTTP headers (user agent comes to mind) and the IP address and ask for reauthentication if they change, employing blacklists for the usual suspects. This forces an attacker to do his homework more thoroughly again.
On the server side, remember the last few API calls, and before allowing another one, check if business logic allows for the new one right now: This denies an attacker the ability to concentrate many of his sessions into one session with your server: In combination with the other measures, this will make an abuser easy detectable.
I might not have said that with the necessary clarity: I consider it impossible to make it completely impossible for an abuser to consume your service, but you can make it so hard, it might not be worth the hassle.
You should implement some sort of authentication system. One good way to handle this is to define some expected header variables. For example, you can have an auth/login API call that returns a session token. Subsequent calls to your API will expect a session token to be set in an HTTP header variable with a specific name like 'your-api-token'.
Alternatively many systems create access tokens or keys that are expected (like youtube, facebook or twitter) using some sort of api account system. In those cases, your client would have to store these in some manner in the client.
Then it's simply a matter of adding a check for the session into your REST framework and throwing an exception. If at all possible the status code (to be restful) would be a 401 error.
There's an open standard now called "JSON Web Token",
see https://jwt.io/ & https://en.wikipedia.org/wiki/JSON_Web_Token
JSON Web Token (JWT) is a JSON-based open standard (RFC 7519) for
creating tokens that assert some number of claims. For example, a
server could generate a token that has the claim "logged in as admin"
and provide that to a client. The client could then use that token to
prove that they are logged in as admin. The tokens are signed by the
server's key, so the server is able to verify that the token is
legitimate. The tokens are designed to be compact, URL-safe and usable
especially in web browser single sign-on (SSO) context. JWT claims can
be typically used to pass identity of authenticated users between an
identity provider and a service provider, or any other type of claims
as required by business processes.[1][2] The tokens can also be
authenticated and encrypted.[3][4]
Set a SESSION var on the server when the client first loads your index.html (or backbone.js etc.)
Check this var on the server-side on every API call.
P.S. this is not a "security" solution!!! This is just to ease the load on your server so people don't abuse it or "hotlink" your API from other websites and apps.
Excuse me #MarkAmery and Eugene, but that is incorrect.
Your js+html (client) app running in the browser CAN be set up to exclude unauthorized direct calls to the API as follows:
First step: Set up the API to require authentication. The client must first authenticate itself via the server (or some other security server) for example asking the human user to provide the correct password.
Before authentication the calls to the API are not accepted.
During authentication a "token" is returned.
After authentication only API calls with the authentication "token" will be accepted.
Of course at this stage only authorized users who have the password can access the API, although if they are programmers debugging the app, they can access it directly for testing purposes.
Second step: Now set up an extra security API, that is to be called within a short limit of time after the client js+html app was initially requested from the server. This "callback" will tell the server that the client was downloaded successfully. Restrict your REST API calls to work only if the client was requested recently and successfully.
Now in order to use your API they must first download the client and actually run it in a browser. Only after successfully receiving the callback, and then user entry within a short frame of time, will the API accept calls.
So you do not have to worry that this may be an unauthorized user without credentials.
(The title of the question, 'How do I secure REST API calls', and from most of what you say, that is your major concern, and not the literal question of HOW your API is called, but rather BY WHOM, correct?)
Here's what I do:
Secure the API with an HTTP Header with calls such as X-APITOKEN:
Use session variables in PHP. Have a login system in place and save the user token in session variables.
Call JS code with Ajax to PHP and use the session variable with curl to call the API. That way, if the session variable is not set, it won't call and the PHP code contains the Access Token to the API.
I set up a Connected App, a Python application to programmatically access Salesforce objects on behalf of a user (offline access).
The app works and I can generate an access_token:
$ curl https://login.salesforce.com/services/oauth2/token -d "grant_type=password" -d "client_id=MY_APP_CLIENT_ID" -d "client_secret=MY_APP_SECRET" -d "username=my#user.com" -d "password=my_password"
{"access_token":"00D09000000KDIX!AQoAQNi1234","instance_url":"https://my_instance.salesforce.com","id":"https://login.salesforce.com/id/12345/12345","token_type":"Bearer","issued_at":"1606401330889","signature":"abc/def"}
So far so good.
Now I wanted to switch to a web-server-based flow that uses refresh tokens, but I'm stumped. Where do I get the initial refresh_token to send alongside grant_type=refresh_token? The docs seem to assume I already have a refresh_token and just want to generate another access_token based off that, which is not the case.
What are the actual steps and necessary calls, end-to-end?
List of docs that I found and read, but made me no wiser:
https://help.salesforce.com/articleView?id=remoteaccess_oauth_tokens_scopes.htm&type=5
https://help.salesforce.com/articleView?id=remoteaccess_oauth_refresh_token_flow.htm&type=5
https://developer.salesforce.com/forums/?id=906F0000000AgInIAK
https://help.salesforce.com/articleView?id=connected_app_create_api_integration.htm&type=5
Here's the Salesforce documentation on the Web Server OAuth flow. It runs like this; note that user interaction is involved, so curl by itself won't be enough to demonstrate the flow clearly.
You direct the user to the Salesforce login UI, in their web browser, to get yourself an an authorization code:
https://login.salesforce.com/services/oauth2/authorize?client_id=<YOUR CONNECTED APP CLIENT ID>&redirect_uri=<CALLBACK URL ON YOUR SERVER>&response_type=code
The user interacts with the authorization page to approve your application.
The user is then redirected to the callback URL in your application that you provided in the call (note: this also has to be set up as a callback in your Connected App definition), e.g.,
https://YOUR_SERVER.com/oauth2/callback?code=<AUTHORIZATION CODE>
Your app can present UI here if you want but the point is to ingest the authorization code.
The callback URL can be on localhost. That's how, for example, the Salesforce CLI implements authorization of orgs; it spins up a local web server to receive the callback.
At this point, the user interaction is done. Your application makes a POST request to Salesforce's /services/oauth2/token endpoint to exchange the authorization code you received for an access token.
If your Connected App is set up with the refresh_token scope, you'll also get back at that time a refresh token that you can store and use to obtain new access tokens in the future, using the refresh token flow you already identified.
For a headless application, it can be easier to go straight to JWT (if that's your ultimate goal). I have an example of how to pair JWT authentication with the simple_salesforce Python library. It takes a little bit of initial setup to populate the certificate on the Connected App and assign Preapproved Profiles (or better, Permission Sets), but once the setup is done it's very smooth and never requires any user interaction.
I want to implement a backend server that can read (to perform some action) users gmail every time a new mail is received. I am able to figure out that using gmail API users.watch, my server can be notified every time a new email is received. Now, for fetching new mails from gmail my server needs User credentials (Auth token) that are provided by the user at the time of opting in to be watched. Is there anyway these credentials can be sent to my server along with the push notification (maybe using users.watch API).
One method I came across to achieve the same is to store auth and refresh token in a DB, that will be accessible only by my server. But it will be better if the purpose can be achieved without storing credentials in the DB.
When the user authenticates your application you are given a refresh token assuming that you requested offline access. You should store this in a secure place associated with the user on your server.
When you get a push notification you should then retrieve the refresh token that you have stored on your server and use that to request a new access token that you can use to access the users data.
The push notification system has no way of sending you the authorization nor would it be a very wise idea if it was storing your authorization.
At work we are making an SPFx Web Part React client app that deploys to SharePoint as a Web Part. Our back-end is a ASP.NET Core 2.2 Web API that is secured using Azure Portal's built in Authentication feature. The front-end is using AadHttpClient that magically handles the authentication by taking the context of the current page (SharePoint) that has the user already logged in. Doing so, silent authentication occurs and the API call is successfully made with authentication successfully passed. The AadHttpClient is supposed to magically bundle up the token in the request header that gets sent to the back-end Web API. I still need to debug the live development app and see how to retrieve the Bearer Token in the back-end Web API. These are my next probable steps?
Would I just probably use 'string bearerToken = Request.Headers.....;' or 'string bearerToken = Request.Headers["KeyValue"]' to get the token itself?
Assuming I can get this Bearer Token, how can I check the caller's user information? Is it just var userName = User.Identity.Name;? Or would I or could I use the token and some how make a call to Microsoft Graph API to view the user's info?
If you are using ASP.NET Core and using default authentication then things are bit easier. From documentation you can see that several tokens are injected in the request header based on Identity provider so in your case you have to look for following headers which Azure AD injects. These headers would contain ID Token which you would need to verify the claims and get user information.
X-MS-TOKEN-AAD-ID-TOKEN
X-MS-TOKEN-AAD-ACCESS-TOKEN
X-MS-TOKEN-AAD-EXPIRES-ON
X-MS-TOKEN-AAD-REFRESH-TOKEN
Ideally all the claims are injected automatically in ClaimsPrincipal
you can find more here
Official Docs
How To extract Token
Using the twilio mobile SDK on Android/iOS, to make a voice call to support operators at the backend (purely voip, no phone at either end. Support operators will be using a backend web app that uses the twilio api/twiml to receive and queue calls)
We have one very important requirement -- to be able to pass the user-id from the mobile app to the backend when placing the call. This is required for the backend web-app to be able to automatically pull up the user's information for the support operator who picks up the call in-browser.
Is this possible with the mobile SDK by any method? Any insight would be appreciated
Edit: Can I perhaps use this user-id as the callerid parameter when dialling, and have it read at the web-app side?
After a day of working with Twilio functions, I came up with the following solution.
Mobile app gets access token, hits the Twilio voice endpoint, is routed to the queue, wait music plays etc.
Operator (web app) gets access token, hits the Twilio voice endpoint, and automatically dequeues the first caller from the queue (i.e is connected to them)
In the TwiML for the operator dequeuing, there is a parameter url which is a TwiML URL that will be fetched and executed on the queued caller's end before the caller is connected to the operator. Twilio passes a bunch of parameters to this URL, including the caller identity and the operator's call sid
We will host this URL at our backend, and store a map of {caller identity, operator call sid}.
The operator web app will poll the backend once the operator dials, until the caller identity corresponding to the current operator call sid is returned. Or the backend can send some kind of push notification to the operator web app. This is up to the implementation (we will use websockets)
For more info please see https://www.twilio.com/docs/voice/twiml/queue
Edit - Twilio javascript API 1.5 supports access tokens instead of capability tokens, so it is simpler to use them for both mobile app and web app.
Edit 2 - Tested this flow today and works fine :-)