I am new to web development and database and am trying to implement password authentication with reasonable security and speed. I have read about hashing the password and append a salt unique to each user in order to deter people from generating rainbow tables.
My question deals with the time I have to search in order to verify a user. Since I don't know who is trying to connect at any given time, it seems to me that I would need to retrieve every field from the salt column then hash the submitted password + each unique salt and then finally compare each output to the hashed strings in the table?
So I have to submit a separate query for each combination of hash(password+salt)? That seems like it would be awfully slow. Am I missing a trick that would speed up the process? Or is it simply a matter of sucking it up and sacrificing speed for better security? Or am I mistaken and with the speed of today's computers it isn't an issue at all?
You cannot authenticate the user based only on a password. Password is a verification that the user is who they say they are, so you need some sort of user identifier — name or whatever. Table then looks like users(..., name, password, ...), you do SELECT password WHERE name = "foo" and proceed with verification from there. Convenient form is to keep all parameters needed to generate derived key inside the password field, e.g. like this:
algo$salt$password hash
For hashing itself, you don't want it to be fast — see key derivation functions like PBKDF2, bcrypt or scrypt. In general, tuning the parameters so that it takes about a second to derive one key is a nice way to make brute forcing infeasible.
Related
So I am encrypting fields in the database, but I don't think i can encrypt the user's username or email because I use those fields to find the user. I could hash them instead, but since I don't think I can use a unique salt per username/email someone could just use a rainbow table to find the hidden username/email.
I guess this it is ok to not encrypt them? I would like to make the website as secure as possible. Would hashing them make sense? I could find a user by their _id instead of username/email, but I wouldn't have their _id until I find the user.
What I am doing currently:
const user = await new db.userModel({
email: email,
username: username,
stuff: cryptr.encrypt(stuff),
});
////
const user = await db.userModel.findOne({
email: email
}).exec();
EDIT: I guess hashing would not make sense, since I cannot un-hash the username/email. Not sure what I was thinking.
This question should really be asked on infosec Stackexchange.
Encryption is usually pointless as a defense against hackers, because you need to access the key to use the encrypted database. That means that you need to save it next to the database. If your server becomes compromised, then the hacker will simply decrypt the database with the key. Of course it's better than doing nothing, because it is possible (maybe improbable) that the hacker will compromise only the database. Horatiu Jeflea hovewer mentions other important reasons to encrypt the database, which you should especially consider if you are not working on the project alone.
Hashing usernames is only possible if you don't need to display them, but usernames are usually public anyway, so it improves security very little.
Hashing emails is an interesting problem. You asked for them for a reason. You presumably need to contact your users. If you hash them, you won't be able to do it and if you don't need to do it, then you don't need to (and shouldn't) save them in the first place. If however they are only part of the authentication process, then it would be possible solution.
Rainbow tables can't really break modern hashing algorithms, although the biggest mitigation is aforementioned salt, which will make it MUCH harder for most attack on the hash. You also have to make sure that you are not using vulnerable ones like md5, but safe one like sha256 or sufficient bcrypt. You should hash the passwords with one of those. Also note that you could use the same salt for all the hashed fields and the salt could even be (this would reduce the security little bit) one of the public fields (username?). There are very little excuses for not using salt.
In summary: you can't hash them and encryption is probably not worth it unless you can sufficiently isolate the key, or need it because of something else than external hackers
I guess this it is ok to not encrypt them?
In most cases, yes. Passwords should be hashed (+ salt..) and sensitive data should be encrypted. But username or email in most cases should not be sensitive.
But let's assume they will be encrypted in DB.
You are storing encrypt(username) in your DB, so in order to search for that, instead of using username, use encrypt(username). Of course sorting may cause some headaches, but finding the user should be efficient.
Think of encrypting (in your case) not about hackers, but about people who are reading those records. For example developers who are investigating a production issue or a DBA, you want some (not all) fields hard for them to read. Storing the key on a different machine, best in a Key Management Tool, will add an extra security layer.
It is recommended to never trust the user and filter all input before storing them in the database, to avoid sql injections and such.
What about user password ? Should I perform any kind of sanitization before storing them ? If I do so, user typing they password to log in might not match a sanitized version.
( Password are encrypted before being stored).
What's the best practice?
It is recommended to never trust the user and filter all input before storing them in the database, to avoid sql injections and such.
No, not at all. It's recommended to avoid SQL injection by using prepared statements, and to avoid HTML/script injection by properly escaping special characters when displaying data.
Should I perform any kind of sanitization before storing them
No, you shouldn't. And you shouldn't encrypt passwords either. You should instead salt and hash them, using a strong cryptographic algorithm like Bcrypt.
Should I perform any kind of sanitization before storing them ...
In addition to JB Nizet's answer with respect to storage, you should filter them prior to accepting one. Consider, Password1 meets NIST security requirements, but its also on [nearly?] every attacker's Top N password list. Obviously, you don't want an attacker to be able to use their favorite password list against you.
The lists themselves can be rather large. I've seen sizes of 10,000, 100,000, and 10 million. The lists can be compressed down to 20KB or 40KB using a Bloom filter. Then, when the user selects a password, check the candidate password against the filter. The Bloom filter will produce some false positives, but that's OK. Simply ask the user to select a different password.
Here's a GitHub with a 10,000-password list: noloader/Password-Lists. Another list is available at danielmiessler/SecLists. Also see Peter Gutmann's Engineering Security, Chapter 6: Passwords.
So from what I have seen it is impossible to decrypt any hashing algorithm such as MD5 or SHA-1 without brute forcing it or using rainbow tables. This seemed to confuse me on a few aspects of using hashes. These confusing points are:
What would be the point of hashing in the first place if they cant be decrypted?
How would hashed passwords be able to be used in a database?
Also since people say it is like the modulo operation, what, if anything, is preventing multiple inputs to equate to the same hash?
If somebody simply does SHA1 or MD5 on a password, then they get almost no protection.
That's why it's important to understand the right way to handle "password hashing". Please read Our password hashing has no clothes
To answer your questions:
You can verify the user without "decrypting the hash": you simply "hash" the user entered password (along with salt and other parameters) upon login and verify that it matches the expected result that is stored in the database.
See 1 and the Troy Hunt link
People who say it is like a modulo operation are making a bad analogy: they are non-experts on this subject. Anyway, the properties of the "hash" function make it hard to find collisions, and the salt prevents two users with the same password from having the same password "hashes" in the database.
Other resources:
Salted password hashing - Doing it right
Method to Protect Passwords in Databases for Web Applications -- advanced reading: solves other problems with current solutions to protecting passwords. If you wonder why I put "hash" in quotes above, this will explain it.
It's not like a modulo. A hash is reasonably guaranteed to be unique based on the input. If you enter your passwords in a database as hashes, then all you need to do is to hash the password entry and check it against what you have stored in the database. This way, you are not storing readable passwords in the database which are openly visible to others. Normally, you would have a private key, some salt and something unique like a timestamp included in your hashing algorithm to ensure that it cannot be easily spoofed.
This may help you further:
http://searchsqlserver.techtarget.com/definition/hashing
Even if the process of hashing is basically non decryptable, the problem as pointed before is that each hash is nearly unique, so that means using websites like md5decrypt which contains a lot of different words and their encrypted hashes, one may find the password he is looking for.
That is if the password isn't strong enough in the first place. Obviously one shouldn't use the password "password" for instance because it will probably be found in most of the websites like md5decrypt.
What you should do to protect passwords on your website is actually simple. First, don't use old hashes like md5 or sha1. Use at least sha256, and if you've enough sql storage, sha384 or sha512. You should know that most of the online hashes database are only about the most commonly used hashes (let's say md5,sha1,sha256 in most cases). So you should find a hash type that isn't very represented on online database.
Then you should (you have actually to) use salt when encrypt users passwords, that is add some word, letters, whatever, to the password before you encrypt it, then store that salt somewhere so you can still allow people to log in. You could also add a pepper to the salt to make the all thing stronger.
While using the salt, try to find a way that hackers won't think about, for instance double the salt, or triple it, or try different ways to concat the salt and the actual password, etc. You could also make a double encryption with double salt, like sha512(sha384()), which would be almost impossible to find.
But, please, do not store unencrypted passwords !
I've been doing a lot of research to determine best practices for storing passwords for a system I'm currently developing. So far, I've decided that I'm going to be using a SHA512 hash with an RNG to generate a salt for each password (obviously best practice against Rainbow Tables etc).
Would storing the passwords in two separate fields in the table be too simplistic to determine the password representation in the database (there's a PasswordHash field and a PasswordSalt field)? It may seem like security through obscurity, but I was thinking of storing the salt and the password hash together in one field concatenated together.
Would this "help" at all?
It doesn't really matter much. The salt doesn't give away any secrets, in fact, it helps keeping secrets. If you want to impress your database designer friends, keep it in a field of its own. If you want to impress your optimizing friends, keep it with the hash. It's a matter of style, really.
Regarding the salt; you might want to make it long enough to prevent against rainbow table attacks: http://en.wikipedia.org/wiki/Rainbow_table#Defense_against_rainbow_tables
Store it seperately.
Remember what you are protecting against: The scenario is someone gets a copy of your database and therefore can execute a rainbow-table lookup against the fields, if not salted.
It really doesn't matter if the attacker knows the salt; it's just to stop him from using pre-generated rainbow tables.
I'm writing Baby's First Web Application. My first task has been to set up an authentication system, which I think I've done okay on. I'm new to the whole thing, though, so:
When the user reports that he's forgotten his password, I e-mail him a temporary replacement password in plain text. It's perhaps not the most secure way to handle the situation, but it's how I do it for now. I do force him to change it at the next login, and the technique I use is to carry a "must-change" field in the database, set to true for users who've been sent the e-mail.
My question: Is a separate database column the best tactic under the circumstances, or is there something better I can do?
A separate column is quite reasonable.
Operating systems typically have a "password expiration timestamp" field which doubles as a "must change at next logon" flag simply by setting the timestamp to 0 (AKA January 1, 1970). Web sites do not usually have password expiration dates, in which case a plain boolean flag suffices.
I presume you are storing passwords hashed and salted. If not, do so. If so, you could store meta-data in the salt. E.g. the salt is [0-9a-z]{8}, but for temp passwords it is ____[0-9a-z]{4}. (before downvoting, people, read on!) The point in this, is that a separate field might get edited separately from the hash field. Of course that should never happen, but it can happen. (failing queries, moronic sysadmins, people who have discovered phpmyadmin and think they understand the system, etc) Keeping the "state" of the password in the salt, prevents such mayhem: on validation of the password, you will always be able to see that you validated against a temp password, and you will always be able to identify the user who needs to get a "enter new password" prompt.
My practice has always been to overload the email validation (where you send an email to the registrant to make sure that the registrant owns the address) to also function as a password reset mechanism. I use certain information about the user (username, id, email, and importantly, the current password hash in DB) to make a hash, which is included in a URL that's emailed to the user, at which point they can set a new password of their choosing.
That being said, the "best practice" vis-a-vis user authentication is 95% of the time to user a library that someone else has written and tested extensively. Just search Google for one that's appropriate for your framework.