I'm currently doing some research on encryption and have read the whitepaper of 1Password (Password Manager) and other tools. I would like to write a small application that can do something similar, just for testing purposes to understand the whole thing better. My approach would be the following:
Master Password - only known by the user, not stored on the system or anywhere else, has to be typed in manually.
Security Key - a 26-digit randomly-generated key stored in the Windows Credential Manager / macOS Keychain, only on the user's system
User enters Master Password, Security Key is retrieved. The security key (even if it is indirectly known because it is on the user's system) is combined with the master password and hashed (SHA-512) and then used as a key when encrypting / decrypting the database (AES-256-CBC). The hashed combination is not stored on device or saved.
Now four questions arise here:
is it bad if the initialization vector is known? It should be generated randomly, but it must be stored somewhere so that it is always available.
do I see it right, that if a component (Master Password or Security Key) is missing, no access is possible? Master Password forgotten => data gone, Key deleted and no longer known => data gone?
accordingly, attacker must know master password AND key to gain access? If he has the key, he has to guess the master password, if he has the master password, he has to get the key. Is this understood correctly?
Thanks a lot!
The initialization vector (or nonce) is public and is stored unencrypted with the data. Different algorithms and cipher modes require different things of it, but it should always be generated either from a key derivation function or from a secure CSPRNG.
If you implement this approach securely, yes, it would be impossible to recover the data without both components. I would use scrypt or Argon2i to turn the passphrase into a key, and then use that plus the security key as the salt in HKDF to derive a key-encrypting key. There are other secure approaches as well, including just using the security key as the salt in scrypt or Argon2i. All of these approaches would require both piece of data to have any hope of decryption.
Related
I'm working on adding the option to encrypt uploaded files on my website by giving each file a "password", but I'm not sure of the best way to keep track of the encryption keys for each individual file. For example, when retrieving the file if the user enters the wrong password I'm pretty sure it'll download the file, but it won't be unencrypted properly. Is it best to store the hashed & salted passwords for each file in a database and match up there first? Or is there a better way to do it?
If you store the encryption keys as hashed (and salted) values, there is no way to retrieve the original encryption key. If you hash something (with a strong hashing algorithm), you cannot get the original back.
The best mechanism depends on the operating environment (OS, plus potentially other software installed), and on the requirements.
Depending on the specific requirements, it may be best not to store the encryption key at all. The point is that the user wants to protect data and the user knows the key. If it is possible to perform the encryption on the client side, the key would never have to traverse the network at all (e.g. encrypt in JavaScript). If the encryption must happen on the server, ensure the key is sent via an https connection, use it in memory to encrypt the file, and remove from memory (if your language supports immediately removing something from memory... e.g. in C# use a SecureString).
If you must keep the encryption key (which is a security issue on several levels), you will have to state more information about the operating environment.
I'm working on a CMDB like application, where I have to store our security credentials (servers usernames & passwords, ...).
I'm looking for the best way to store them securely with those constraints :
Most users will NOT have access to all credentials (depending on user role)
We don't want all passwords being encrypted with the same key (already tried : when a user leave the company, it's a pain to change the key...)
Indeed, we don't want any private key to be hard written in app source, or even stored anywhere (in our previous version, private key was stored between our ears...)
We need to realize passwords strength audits (ie. parse decrypted passwords from a script)
There must not be any case where we can not access our credentials anymore (lost key, ...) => we don't want unauthorized persons to look at them but we don't want to loose them either => solution for this constraint could be regular export into a physical locker...
I'm not asking about application (https, ...) or database (no public access, ...) security concerns themselves but only about the storage side (could even NOT be in a database...? encrypted files or something...) : Is it possible to prevent someone, even having access to app code or database content (worst case scenario), to be able to read decrypted credentials ?
I'm aware that I'm asking for some magic solution, but I want to know it if it exists ;o)
The general case of what you're asking to do is not possible. All types of modern cryptography are mechanical advantage. That is, they use a small secret to guard a larger secret. If you can't keep the small secret safe, there is no safety. If you want the ability to give passwords on a password by password basis to someone, you are effectively giving them the secret -- the passwords -- that they would need to gain access to the items in question.
This very problem is why federated identity systems (Kerberos/Active Directory/etc.) systems exist -- to allow a central machine to authenticate users without exposing secrets to said users. But using a federated identity system requires cooperation between the system-to-be-logged-in-to and the identity service.
Certain highly-sensitive information (payment info, usernames, passwords, etc.) should be encrypted before it can be persisted to my database.
Later, that information has to be decrypted in order to be fetched from persistence and used at a later time.
If I use, say, AES256 to encrypt a billing address, I'll still need to store that AES256 key/passphrase in persistence as well.
If the whole point behind encrypting information that is going into a database is to protect that information in case someone hacks into my database, and I'm storing the key to decrypt that same information in the database, then what's the point of encrypting the data in the first place?
If someone hacks into my database, they'll be able to find the persisted key and decrypt any encrypted data they want to anyways.
Am I missing something here?
There is an old saying "Encryption is easy, key management is hard". And that very much applies here.
If you need to store data in an encrypted format (you frequently don't because you only need to hash the data not encrypt it), you do not want the encryption key to be stored in the database. You want the key to be accessible when your applications need to decrypt the data but you don't want people like the DBA that has access to all the encrypted data to be able to get the key. You want to make sure that the key is backed up so that you can recover the data but you don't want those backups to comingle with your database backups. Key management, therefore, becomes a very thorny problem to solve.
In the vast majority of cases, you want to purchase some third-party key management solution that can deal with these contradictions. Just like you don't want to implement encryption algorithms on your own, you don't want to do key management on your own. The folks that attempt to solve key management on their own generally do not succeed.
A better option would be to use certificates and this can easily be done in most RDBMS.
The best option regarding passwords is to hash them. This is a one way hash, and is not decrypted. Basically, when a user logs in, you hash their input password, and compare the hash against the one stored in your db for a match - and a successful login.
Regarding payment information, you will need a random generated private key. Depending on the system and implementation this can be stored a number of different ways.
You can store this in a config file, encrypted using an RSA container for example so it is not readable.
There are other solutions as well.
You can also encrypted db connection strings and the like with the RSA container method above to help prevent anybody actually seeing you db username password your application will use to access the db.
I have a web service using HMAC to verify signed requests. In this scenario there is a key (password) assigned to each caller of my service. The user of course gets that key, but I also need to store the key in my database so that my service can verify the signature of each request coming in.
So this is a case where I really do need to store passwords in the database in a form that I can retrieve and use them. I can't use the best practice of only storing a salted hash of the password in the database.
I could encrypt the keys but then I need to store the encryption key somewhere. This is a fairly common scenario for secured RESTful web services, so the likes of Amazon (AWS) and Microsoft (Azure) have to deal with this problem.
What are best practices in this situation?
The only time password should be stored in a database is if the password is needed to connect to some other system. If one merely needs to check the credentials supplied by some other entity, one should store a hash of the password.
Even if it's necessary to use a password to connect with another party, if some other credentials are needed for you to perform such access (e.g. someone logs into your system, and then you log into another system on their behalf) it's desirable when practical to store the external password encrypted with a hash of the supplied password (but NOT the same hash as the one stored in the database!). If there are multiple credentials which could be used to log into your server, store a separate copy of the encryption key for the remote password, encrypted using each acceptable credential.
If you really need the password (in order to connect to another system, for example), I would recommend putting in somewhere far away from the other information. Perhaps another database, an encrypted file on the file system, etc. This is so if someone gets your main database, they are not guaranteed to also get the passwords in question.
It may be obvious but you want the location of the passwords to be encrypted with another key (to make it less likely that someone who somehow gained access to the primary data source will also gain access to the password data store).
It sounds like the best practice for your scenario would be to use public key cryptography.
In some project we have very that even our staff is not suppose to have access to. In theory, we have policies to ensure they don't. In practice, we are in Africa and policies don't mean a lot, no matter how strongly you enforce it.
I would like to know is there is a way to encrypt data in your database so:
each user password encrypt and decrypt its own data, and its own data only;
data is decrypted as late as possible in the process to ensure maximum security to the user. Ideally it would be on the client side I guess, but I'd love to hear that it's possible to do some crazy thing I don't know about on the server side.
data is still searchable. Is that even possible?
My first idea was: "if a customer want THAT level of protection, then give him its own hosting on a virtual machine and encrypt the hardrive, then all maintenance must be done with it's allowance".
I can't come up with a fancy strategy just how I've implemented this:
Keep in mind that you have to re-encrypt everything when the user changes his password. I'm using always the same encryption key but the key is encrypted using the user's plaintext password. So I just have to re-encrypt the key. The user's password is stored as a salted hash so nobody can decrypt the key and the data even if he sees the hash.
It works like this:
User enters his plaintext password
Create salted hash
Check if the generated hash matches the one in the database (authentication)
If yes, then decrypt the key for the data using his plaintext password
Decrypt stored data using the key
This won't give you 100% security but improves it.
Here are a few things I can think of:
You should encrypt data stored when it is stored in the and when you read it back. Use a solution that integrates at an RDBMS level rather than the data layer.
For the transport of data to and from the application, use HTTPS web services.
If you have a Desktop application, do not store any data and log files etc locally.
If it is a web app, make the app HTTPS as well.
Security is bound to make the app a little slower than using plain data, but that's the price you will pay.
It really depends on what and where (on the client or server) you are doing with the data.
For example, your application don't need to know the password itself to verify it during authentification. Best practice for this use case is to store only a cryptographic hash (e.g. sha1) of the password and a random salt. That is sufficient to verify it, but giving only the hash and salt, it would take a nearly infinte amount of time to figure out the plain password.
Encryption can be a soultion if you have to exchange data over unsecure channels. But keep in mind that in order to process the data you have to decrypt them. So if de- and encryption is done on the same machine, it's rather pointless. And if decryption is required it doesn't matter how late you are going to do it, because of the key must be given anyway.
You can use encryption to secure the communication between the server and the client, for example. You could even generate messages on the server that only the client will be able to read and vice versa using asynchronous encryption. So once the message was generated on the server and encrypted using the client's public key even the server isn't able to read the message anymore, because of the private key only the client knows is required for the decryption.
What you denfinetly can not solve by cryptography is, when you have data on the server, that the server should be able to read in order to process them but human users unrestricted with priveleages to this server shouldn't.