In some project we have very that even our staff is not suppose to have access to. In theory, we have policies to ensure they don't. In practice, we are in Africa and policies don't mean a lot, no matter how strongly you enforce it.
I would like to know is there is a way to encrypt data in your database so:
each user password encrypt and decrypt its own data, and its own data only;
data is decrypted as late as possible in the process to ensure maximum security to the user. Ideally it would be on the client side I guess, but I'd love to hear that it's possible to do some crazy thing I don't know about on the server side.
data is still searchable. Is that even possible?
My first idea was: "if a customer want THAT level of protection, then give him its own hosting on a virtual machine and encrypt the hardrive, then all maintenance must be done with it's allowance".
I can't come up with a fancy strategy just how I've implemented this:
Keep in mind that you have to re-encrypt everything when the user changes his password. I'm using always the same encryption key but the key is encrypted using the user's plaintext password. So I just have to re-encrypt the key. The user's password is stored as a salted hash so nobody can decrypt the key and the data even if he sees the hash.
It works like this:
User enters his plaintext password
Create salted hash
Check if the generated hash matches the one in the database (authentication)
If yes, then decrypt the key for the data using his plaintext password
Decrypt stored data using the key
This won't give you 100% security but improves it.
Here are a few things I can think of:
You should encrypt data stored when it is stored in the and when you read it back. Use a solution that integrates at an RDBMS level rather than the data layer.
For the transport of data to and from the application, use HTTPS web services.
If you have a Desktop application, do not store any data and log files etc locally.
If it is a web app, make the app HTTPS as well.
Security is bound to make the app a little slower than using plain data, but that's the price you will pay.
It really depends on what and where (on the client or server) you are doing with the data.
For example, your application don't need to know the password itself to verify it during authentification. Best practice for this use case is to store only a cryptographic hash (e.g. sha1) of the password and a random salt. That is sufficient to verify it, but giving only the hash and salt, it would take a nearly infinte amount of time to figure out the plain password.
Encryption can be a soultion if you have to exchange data over unsecure channels. But keep in mind that in order to process the data you have to decrypt them. So if de- and encryption is done on the same machine, it's rather pointless. And if decryption is required it doesn't matter how late you are going to do it, because of the key must be given anyway.
You can use encryption to secure the communication between the server and the client, for example. You could even generate messages on the server that only the client will be able to read and vice versa using asynchronous encryption. So once the message was generated on the server and encrypted using the client's public key even the server isn't able to read the message anymore, because of the private key only the client knows is required for the decryption.
What you denfinetly can not solve by cryptography is, when you have data on the server, that the server should be able to read in order to process them but human users unrestricted with priveleages to this server shouldn't.
Related
I'm working on adding the option to encrypt uploaded files on my website by giving each file a "password", but I'm not sure of the best way to keep track of the encryption keys for each individual file. For example, when retrieving the file if the user enters the wrong password I'm pretty sure it'll download the file, but it won't be unencrypted properly. Is it best to store the hashed & salted passwords for each file in a database and match up there first? Or is there a better way to do it?
If you store the encryption keys as hashed (and salted) values, there is no way to retrieve the original encryption key. If you hash something (with a strong hashing algorithm), you cannot get the original back.
The best mechanism depends on the operating environment (OS, plus potentially other software installed), and on the requirements.
Depending on the specific requirements, it may be best not to store the encryption key at all. The point is that the user wants to protect data and the user knows the key. If it is possible to perform the encryption on the client side, the key would never have to traverse the network at all (e.g. encrypt in JavaScript). If the encryption must happen on the server, ensure the key is sent via an https connection, use it in memory to encrypt the file, and remove from memory (if your language supports immediately removing something from memory... e.g. in C# use a SecureString).
If you must keep the encryption key (which is a security issue on several levels), you will have to state more information about the operating environment.
Certain highly-sensitive information (payment info, usernames, passwords, etc.) should be encrypted before it can be persisted to my database.
Later, that information has to be decrypted in order to be fetched from persistence and used at a later time.
If I use, say, AES256 to encrypt a billing address, I'll still need to store that AES256 key/passphrase in persistence as well.
If the whole point behind encrypting information that is going into a database is to protect that information in case someone hacks into my database, and I'm storing the key to decrypt that same information in the database, then what's the point of encrypting the data in the first place?
If someone hacks into my database, they'll be able to find the persisted key and decrypt any encrypted data they want to anyways.
Am I missing something here?
There is an old saying "Encryption is easy, key management is hard". And that very much applies here.
If you need to store data in an encrypted format (you frequently don't because you only need to hash the data not encrypt it), you do not want the encryption key to be stored in the database. You want the key to be accessible when your applications need to decrypt the data but you don't want people like the DBA that has access to all the encrypted data to be able to get the key. You want to make sure that the key is backed up so that you can recover the data but you don't want those backups to comingle with your database backups. Key management, therefore, becomes a very thorny problem to solve.
In the vast majority of cases, you want to purchase some third-party key management solution that can deal with these contradictions. Just like you don't want to implement encryption algorithms on your own, you don't want to do key management on your own. The folks that attempt to solve key management on their own generally do not succeed.
A better option would be to use certificates and this can easily be done in most RDBMS.
The best option regarding passwords is to hash them. This is a one way hash, and is not decrypted. Basically, when a user logs in, you hash their input password, and compare the hash against the one stored in your db for a match - and a successful login.
Regarding payment information, you will need a random generated private key. Depending on the system and implementation this can be stored a number of different ways.
You can store this in a config file, encrypted using an RSA container for example so it is not readable.
There are other solutions as well.
You can also encrypted db connection strings and the like with the RSA container method above to help prevent anybody actually seeing you db username password your application will use to access the db.
I am currently working on a service which requires users to pick a 4-digit password/pin, because it is a mobile service. I am encrypting those passwords with either 256 or 2048bit encryption and it will be hashed. The account is blocked after 4 wrong entries, and can only be entered by mobile phone. Would it be hard to crack those PINs? I am asking this because sensitive information is being stored. The database is connected to a web application, the application is loaded to the phone using twilio. The thing I am most scared for is that the database is being hacked via the web. What would be a good way to keep sensitive data secure?
If someone gets hold of the database, you would be pretty much screwed:
If you just encrypt the 4-digit passwords, an attacker can just build a table of the 10000 possible encrypted strings and can trivially decrypt the PINs.
If you use salt strings (and encrypt not PIN, but PIN+salt and store crypted(PIN+salt) alongside with salt), people have to make a per-password effort, but there are still only 10000 possibilities for each password (which is not very much).
Which means, yes, by all means you should keep the database off the web. (If the web application is only ever accessed through twilio, you can reject connections from any other IP range).
Since you're using twilio, just make sure that twilo only talks to your web service using a secure protocol and reject any requests that you aren't sure are coming from a trusted source (that is, twilo). No real need for a pin at all.
This is a huge webpage on how to setup ssl between your web server and twilo. It even has a php example.
http://www.twilio.com/docs/security
If you use PKCS#1 1.5 or 2.0 RSA encryption (view the standards) you will also encrypt a random padding. This means that in transit, the PIN's cannot be compared, as long as the padding is kept secret and is truly random (this is not a salt which should be made public).
As for the database, it would be a good idea to move it out of the normal operations as much as possible. Create a simple service that just checks the PIN after decryption, make sure you don't have buffer overruns etc. on that, and if possible, use a different machine and access rights than the production server. Really test this part well, as the interface is small, it should not be difficult.
If you and the phones are up to it, you might want to try ECC, but that's not for the meek. RSA encrypt normally uses a small public exponent (0x010001 is highly recommended) so it is faster than ECC for the phone. On the server (and during key creation) ECC is much faster. I would not recommend symmetric cryptography (AES/3DES) for this.
Oh, and include the public encryption key in the application (for implicit trust), don't send it over from the server. Keep the private key secret and inaccessible for anything other than the already mentioned service.
The interface you describe sounds secure to me. It's secure enough for ATMs!
Are the encrypted PINs easy to crack? Yes, there's only 10000 possible combinations and a rainbow table can be generated of all the possible encrypted values unless you salt. However that would require access to the encrypted PINs which means the attacker already has a copy of your database.
So really you need to ensure your database server is secure. There's a lot of variables that could make it insecure so it's a big question. Instead you could rely on third-party solutions like Amazon S3 or others and concentrate on coding instead of security. Let them do the hard work!
You should use good quality password salting technique to prevent against hacking passwords. Check the wikipedia article to know more about salting. Salt Article
I have a web service using HMAC to verify signed requests. In this scenario there is a key (password) assigned to each caller of my service. The user of course gets that key, but I also need to store the key in my database so that my service can verify the signature of each request coming in.
So this is a case where I really do need to store passwords in the database in a form that I can retrieve and use them. I can't use the best practice of only storing a salted hash of the password in the database.
I could encrypt the keys but then I need to store the encryption key somewhere. This is a fairly common scenario for secured RESTful web services, so the likes of Amazon (AWS) and Microsoft (Azure) have to deal with this problem.
What are best practices in this situation?
The only time password should be stored in a database is if the password is needed to connect to some other system. If one merely needs to check the credentials supplied by some other entity, one should store a hash of the password.
Even if it's necessary to use a password to connect with another party, if some other credentials are needed for you to perform such access (e.g. someone logs into your system, and then you log into another system on their behalf) it's desirable when practical to store the external password encrypted with a hash of the supplied password (but NOT the same hash as the one stored in the database!). If there are multiple credentials which could be used to log into your server, store a separate copy of the encryption key for the remote password, encrypted using each acceptable credential.
If you really need the password (in order to connect to another system, for example), I would recommend putting in somewhere far away from the other information. Perhaps another database, an encrypted file on the file system, etc. This is so if someone gets your main database, they are not guaranteed to also get the passwords in question.
It may be obvious but you want the location of the passwords to be encrypted with another key (to make it less likely that someone who somehow gained access to the primary data source will also gain access to the password data store).
It sounds like the best practice for your scenario would be to use public key cryptography.
My web application stores external website login/passwords for interaction with them. To interact with these websites I need to use the original password text, so storing just the hash in my database is not going to work.
How should I store these passwords?
Edit:
I am concerned if someone gets access to my server. If I use some kind of 2-way encryption and they have server access then they can just check how the passwords are decrypted in my backend code.
It seems to me that you want to store passwords in a similar fashion as Firefox and Chrome. So why not look at how they do it?
This is how Chrome does it:
http://www.switchonthecode.com/tutorials/how-google-chrome-stores-passwords
If you MUST do this, you should use a two-way encryption. There are a lot algorithms (ciphers) for this, but basically you encrypt your data with an encryption key, and use the same key for decrypting them again.
Choosing the right cipher depends on which are supported by the programming language of your choice, but examples are:
Blowfish
3DES
Skipjack
They come in different complexity and some are harder to crack than others. You should realize though, that no two-way encryption is safe from cracking, given enough time. So it all depends on, how sensitive these passwords are.
/Carsten
Decide what you are protecting them against. Options include (but are not limited to): Accidental disclosure, disclosure by you, disclosure in transmission, disclosure due to code error, disclosure due to physical theft of hardware, etc.
If this is a web application, and each user is storing his/her own set of passwords, then you might encrypt these passwords with their login password to your application. If this is an application that each user installs separately, and which keeps its own local database, you could have an optional master password (like Firefox does).
If you are just ensuring that the data is safe if the hardware is stolen, you might use a full disk encryption solution like TrueCrypt or PGP WDE, or Ubuntu, Debian, or Fedora's built-in approach, and require a PIN or password on every boot.
If you just care about secure transmission, have code to ensure that you use transport security, and don't worry about encrypting the data in your database.
I would go about this in the following way.
Protect data against hardware being stolen:
Use disc encryption as discussed in previous posts.
Protecting data if server is compromised (hacked):
I would use two different servers for this project, one worker server and one front server.
A) Worker server
This has the DB with passwords etc,
it also connects to other services.
To connect to worker server, users
can do it through an API. API should
have on function, insertUserData,
which allows userdata to be inserted,
API escaped all the input.
API uses
a DB user which only has input
privilegies on the userData table.
This would be the only way to contact
this server.
Only allow SSL
connections.
This server in turn runs chron jobs that connect to external services, pulls data from them and populate it's DB. Use a different DB with different user privileges.
This server runs another chron JOB which connects to the front server and pushes new data to front server.
Minimal amount of services running
Only SSH/SCP from your IP, tight firewalling. Block if connections exced X / min etc as they only would do an occasional insert.
NO FTP etc.
B) Front server
Receives data from Worker server, never uses the passwords itself. Only way to contact worker server is through API mentioned above, only for new user information. This is where all users login to see their information etc.
The problem with doing it all on the same server, if you get hacked the hacker can sit and sniff all incoming data / passwords etc.. so even if they are stored / encrypted / decrypted securely, with some patience he would sniff them all.
When the application is first run, it will generate a random key. This key will be used to encrypt and decrypt sensitive data. Store the key in a local file, and set the file permissions so that nobody else can read it. Ensure that the user running the web server has no login access (this is a good idea anyway).
Possible ways to break this system:
Get root access.
Get sudo access.
Deploy a malicious application on the web server - this application will then have access to the key, and may be able to send it elsewhere.
As long as you take reasonable precautions against all of these, you should be OK.
EDIT: Come to think of it, you could just store the sensitive data right in the key file. Encryption would provide an extra layer of security, but it wouldn't be a very strong layer; if an attacker gets access to the file, chances are good that he also has access to the DB.