Handlig large number request on the http server - database

I am planning to create a website for questioning and answering purpose. The website will contain the more functionality in the future. So the request will be increasing from the future point of view towards the website.
for example, the request may be like 2000 or more per second. Also the processing of request will requires some validation like user-name/password. After the request is validated, it is required save the data to the database.
I also want to know how to check the data write speed from multiple user's request from the server at a time.
Please let me know TIA...

Related

I want API JSON response data but don't want it to be shown in network tab as its not specific to logged in user yet very sensitive?

So basically I have some information which I have gathered with lot of effort, this information is not specific to user so I cannot totally depend on my user logged in token, this is somewhat some generic info that needs to be present on the website as info and currently this information is coming directly as JSON from API calls in network tab which makes me insecure that this info can be copied and used by 3 person without my concern.
What can I do to stop this ?
The only foolproof way to keep the client from possibly being able to see and possibly manipulate sensitive information is to not send it to the client in the first place. There's no other way. While you can make it harder for the client to do so by, for example, encrypting on the server and decrypting on the client side inside long, convoluted, minified code - it's still possible.
If each user has information specific to them that you'd like to display, the right approach would be to validate the client's login token and then send back the data for that user (but not the data for any other user). If the client sees that the information is incorrect (like if someone else was using the computer), you can prompt them to log out and log back in with their actual credentials, resulting in another request that'll be verified by the server and which will return the data needed to display what the user is looking for.
If you're worried about third-party snoopers, you should also, of course, use SSL on your site. (Unfortunately there's not much you can do about other vulnerabilities the client may have, such as malicious browser extensions, drivers, and hardware - either don't send the info in the first place, or bear the small risk.)

Web3JS - Front or backend

I have a question about web3js.
If my site will load data directly from blockchain (MetaMask) without loading them into the database, and immediately through js bring them to the site then:
1. If a large number of events (1-10 kk) then analyze them on the side of the browser is a bad idea? Need to record everything in the database and the user to publish the analyzed information from the database?
2. When a user enters the site, then in fact his browser parses the information out of the blockchain (if using MetaMask)? Not my server. And I do not care how many connections go through web3js at the same time
What the best?
1. Get all data with web3js and show them on site.
2. Or get all data and writing to database, then use ajax for dinamic show data?

Angular - Can't see how to hide this API Key

I have the following code in my angular app declaration - an API key for Facebook (to implement Share button):
.run(function($FB){
$FB.init('9xxxxxxxxxxxx94');
})
So i know the general answer to this - 'API keys should be kept on the server side', however I don't see how i actually implement this.
The share call-method is made on the front end, so even if my server kept the API key and sent it, surely it's still visible on the front end, else how would the share button work?
So my question, how do I hide that Facebook API Key?
Thanks.
Requesting the key
The first thing that happens is that the client will request a key. This will only happen on certain pages like the sign up and log in pages. The idea here is that we want to make sure that only users browsing with a known client (in this case the official website or core client as it’s called) are allowed to take actions like creating or authenticating a user.
So when the client app requests the login page the server generates a unique token based on information sent in the request. The information used is always something the server knows, something the client knows, and something both know. So for example the server can generate a unique key based on User agent + current time + secret key. The server generates a hash based on this information and then stores a cookie containing only the hash on the client machine.
Setting permissions
At this point our key really isn’t a key anymore. It has been transformed into an access token. The server should then take this access token and store it for later retrieval. You can put the key in a database but since data of this type needs to be retrieved often I would suggest using a key-value store like Redis to cut down on database reads/writes and boost performance.
When you store the token you should also store a separate piece of data to indicate what permissions are associated with the token. In this case our token is acting only as a way to register and authenticate users so we store it next to a value that indicates who the token belongs to (the app’s web UI) and what permissions it has (limited to create and authenticate users). We treat it just like we would any other API client that way we can capture stats and control how it is used.
Authorizing a request
When the client then makes the POST request to create a new user or log in the server will check to see if the client sent an identifying cookie along with the request. If not, we reject the request. If it does send the cookie, the server should once again generate the hash using the values used previously (these values are either already known or sent with the request anyway so we’re not really taxing the server much) compare it to the cookie being sent to us, and if the values match allow the request to proceed.
Sources - Securing API Keys
OR
Simply send a request to your Server and let him handle your request with the hidden API-key and just return the result of your request to your front-end.

What is the best approach to work with data while using token based authentication

I am building an sample application that lets user store comments.
I've created the registration and login process. When the user registers, his details are stored in a MySQL database and a token is returned to the browser. Now he can access the Profile page.
When an existing user logs in he is redirected to the profile page. The profile page is accessible only when a user registers or logs in.
After logging in, I want to show all his comments if he has already added them.
My frontend is in Angular and backend use Laravel. For authentication I use Satellizer.
I want to know, what is the best approach while playing with data, considering the fact that the user will add, edit his comments. Should I use localstorage and store data in a key value pair or should I create a json file which gets updated everytime the user adds a comment or makes a change.
I wanted to know what is the most efficient way to deal with data from server so that the application is fast even when it scales to a 10000 users and lot of data for each user.
Thanks
You should be updating it on the server when changes are made rather than only relying on localstorage. You can use localstorage to cache, but it should only be for immutable data, it shouldn't really be used for data that is going to change.
So in this case you'll be adding and updating new comments via your API (ideally a RESTful one!). Once you've made a change, you could store the comments locally and only update them when the user makes a new comment, however you'll quickly run into issues where the data is invalid on different clients. (i.e. if you update the comments on a different computer, the other computer won't be aware).
Alternatively, you could cache the comments and then simply ping the server to find out if new comments have been added. This could be using a HEAD request for example to check the last modified date on your comments resource.
You can store comments data locally on user browser, but you should properly manage it.
I don't how much load your server will have and if the time invested now worths it.
You can fetch comments and store them locally
User adds a comment, then you update locally and send a request to the server
You need to track the request response, if requests fail so notify user and remove comments from local.
if request was successful so you can continue on your way.
** facebook uses this "success first" approach
user does an action, and he see it happens instantly, in the background it could take few seconds, only if it fails they will notify you.
** look at their commenting process, when you comment, it appears instantly, no loading... but in the BG the load happens.

Securing Angular Application

I am creating an Angular application, and I am having trouble wrapping my head around the proper way to ensure my application and its users is secure.
I've been reading around many stack discussions, but I believe I am missing some core understanding of what is happening, please correct any errors you see written below.
So far I have a Sinatra server with many (currently mostly hypothetical) resource routes. A user can create an account using an email address and password that is stored in a database after being hashed with BCrypt. When a user logs in, the record is retrieved from the database by email and the password checked for authentication. It is from this point I am not sure how to proceed.
Prior to this I have simply set a session variable and had the server check that the variable exists in order to correctly route logged in users. Now my application is (currently) a single HTML page that uses Angular and ui-router to display different content, so most of the requests are simply returning JSON content.
It is my understanding that Restful applications should generally not use sessions, or rather that the server should respond identically to identical requests and not have its own data that shapes a response. But if I do not store something in a session variable, how could the server know that the client making the request has the correct permissions? And are sessions not stored in the browser anyway, thus not part of the server?
I believe from what I have read, it is possible to create a token which is essentially a large random string, return that string to the client and also store it in a database with a timestamp. The client then provides this token when making requests and the server hits the database to verify it exists and valid. But would the client not also have to store that string in a cookie? I suppose the angular application could store the token in a variable, which would persist while using the ui-router but not if the users navigates using the address bar.
I also do not understand how Basic Auth may or may not fit into this picture. Any help would be greatly appreciated, as well as a pointer to some good resources where I may find a better understanding of these concepts in general.
You want to read up on JWT. There are JWT libraries for Ruby and Angular.
I know you aren't using Node for your backend but a very easy way to see all the pieces working together is to run the angular-fullstack Yeoman generator. It uses JWT and the code is easy to follow.
As far as I can see, whatever you are doing with your sessions can work just fine.
This can be a sample JSON response from the server in case the user is not loged in :
{
"errorCode": 1,
"error": "User not logged in",
"data": {}
}
You can set your own error codes and handle what you want to do. You will send any data only if the user is logged in. For all the pages which don't require authentication, you can set data to whatever you want.
On the angularJS side, you can handle based on error codes, you can redirect the user to the login page and so forth.
The alternate way to support the same on multiple platforms is to use token based approach. The token based approach in simple words work this way.
The user logs in for the first time with his / her credentials.
The server verifies these information and creates a token from which the server is able to decode the user id.
Whenever the client makes the requests, it passes its token with every request.
As the server can decode the user information from the token, it sends or doesn't send the data based on whether that's a right token or not.
The token depends on a secret value. It can be same for all the users or differnet for each based on how you want to implement.
This is all done and you can look at
http://jwt.io/
As #andy-gaskell mentioned, you can look at
http://angular-tips.com/blog/2014/05/json-web-tokens-introduction/
I'm very bad at explaining. Please let me know if any of this doesn't make sense.
you are missing the point of the REST concept. One of the main concepts in the REST apis is that the server should be stateless - this means that you should not store sessions or other "state" in your web server. Every HTTP request happens in complete isolation. Every request should include all data needed by the server to fulfill the request.
But if I do not store something in a session variable, how could the
server know that the client making the request has the correct
permissions?
You can store request scoped variables. This means that they should be only active during the same request. You can store the current logged in user in the request scoped variable. In that way you can get the current user in your invocation of the business method. I'm not familiar with Sinatra but here is the doc: http://www.sinatrarb.com/intro.html#Request/Instance%20Scope
But would the client not also have to store that string in a cookie?
of course you should store your access token in the client side
https://stormpath.com/blog/where-to-store-your-jwts-cookies-vs-html5-web-storage/
as #Andy Gaskell suggest take a look at JWT and fullstack application code generators and forget about the basic auth because it's really "basic".
more useful links:
If REST applications are supposed to be stateless, how do you manage sessions?
http://www.sitepoint.com/php-authorization-jwt-json-web-tokens/

Resources