I am creating a mobile app that uses Google App Engine (python) for the backend. Users sign in with Twitter on the app, and the auth token and secret are passed to the backend (over https) so that the server can authenticate with Twitter and also periodically sync friends and followers in a background task. Because they are used by the background thread, I want to store the information in the datastore so they can be retrieved and used later.
Right now, during development and testing, I just put these in the datastore in plain text. But I'd like to add a little more security by storing it encrypted and decrypting it when its needed. Thank you for any help!
For general account passwords, I use
security.generate_password_hash(raw_password, length=12)
based on how webapp2_extras stores the passwords. But this approach wouldn't allow me to retrieve the data. Is there anything similar that allows for encryption and decryption?
Normally for password storage you would use a unidirectional (One way) encryption technique so that no one can work out what the password is and then take the user supplied values and compare them to the stored values. This way you're never really storing the actual password and it's less likely to be stolen.
What you're looking for is a bidirectional encryption technique where by you provide the value and a key to create an encrypted value and can apply the key to the encrypted value to get the original.
You haven't stated which language you're using so I cannot provide a good example, however I suggest looking at techniques such as AES. Please keep in mind that if you choose an encryption technique with a short key it will be much easier to brute force. Any encryption that is bidirectional is at risk of easier brute force and once the key has been determined ALL passwords are at risk of being decrypted. Most languages have some form of support for AES and similar encryption techniques.
There are many techniques available, some much newer and more secure so do some research and see what you deem 'secure enough'.
Read more here: http://en.wikipedia.org/wiki/Advanced_Encryption_Standard
Related
I've been learning about ways to keep databases for web applications more secure in School and one of the things mentioned was the encryption of data of the user's personal information. Now when I'm looking through examples of web systems I've noticed that it's normally just the password that is hashed, for obvious reasons, but should there be some sort of encryption when it comes storing a user's details such as their address? Is it safe to have these stored in plain text as long as you have measures in place to keep your database secure or should this be encrypted?
Sorry if this is the wrong place to ask but thought I would be able to get a good answer here
Yes, any Personally Identifiable Information (PII) should be encrypted in transit and at rest because it may be used for identity theft. There are recent instances of hackers getting into systems and stealing data that is then potentially sold on eg the Travelex data breach https://www.bbc.co.uk/news/business-51017852. Encrypting the data makes it valueless even if hackers do get into your network.
This is different to password hashing which is a one way mechanism to allow comparison of provided credentials against stored credentials without ever needing to convert the stored value to plaintext. PII data does normally need to be decrypted by the applications that use it.
How are you supposed to store users passwords in a Cloudant DB ? By users, I mean users of an application that uses Cloudant as a backend.
I've searched the docs, but I found nothing on that topic.
There's a _users database, in which you can create users and add a "password" field, but the password is a regular field that the DB admin (and possibly others) can read.
Is there a built-in way to hide it from view or encrypt it ?
EDIT
I've found a piece of the puzzle, the CouchDB security feature that encrypts user's passwords.
Since CouchDB 1.2.0, the password_sha and salt fields are
automatically created when a password field is present in the user
document. When the user document is written, CouchDB checks for the
existence of the password field and if it exists, it will generate a
salt, hash the value of the password field and hash the concatenation
of the password hash and the salt. It then writes the resulting
password into the password_sha field and the salt into the salt field.
The password field is removed.
This has the following implications: Clients no longer have to
calculate the password salt and hash manually. Yay.
Now what's missing is the link between that underlying DB feature and Cloudant (just setting the password field in the user document is not working).
EDIT 2
Found that other question which is similar to this one - it's a broader problem, but specifically for web apps. There's an accepted answer from #JasonSmith that addresses my question:
Can I use CouchDB security features
Answer's "yes you can"
Cloudant does not yet have the newer CouchDB feature where the server
will automatically hash the password for you
But the CouchDB doc states that this features is included in the 1.20 version from 2013! How is that a "newer" feature?
From the doc, I gather that Cloudant uses CouchDB 1.61.
To recap:
the feature exists,
it's a CouchDB security feature existing in the CouchDB version that Cloudant uses,
Cloudant can be configured to use CouchDB security features
So... the missing link is really really small...
As you've discovered, Cloudant does not automatically hash passwords server side, as introduced in Couch 1.2. Also, it only supports the simple password scheme: salted SHA1 (which you may find insufficient). That's how passwords are supposed to be saved (not plain text).
It also misses a bunch of other security features, such as special access rules to the _users database (described here).
Hashing passwords "automatically" can be accomplished by an update function (special access rules could be implemented through show/list functions). I have done this myself:
function (doc, req) {
var body = JSON.parse(req.body || '{}') || {};
if (doc == null) doc = {
_id: req.id,
type: 'user'
};
doc.name = body.name;
doc.roles = body.roles;
doc.salt = req.uuid;
doc.password_scheme = 'simple';
doc.password_sha = hex_sha1(body.password + doc.salt);
return [doc, { json: doc }];
}
Get hex_sha1 from here. Set the above as an update function in a design doc on the _users database. You can also use this as a validation function.
Then instead of PUTing a user into the database, you PUT the same JSON to the update function, and it generates the salted hash before committing it to the database.
If salted SHA1 is not enough for your purposes you can't rely on _users on Cloudant, as is.
Not knowing more about your design, I can't really give much advice.
But I should warn you that, thanks to poor _users support, it's e.g. nearly impossible to effectively implement a 2-tier architecture on Cloudant. I'd be glad to be contradicted by someone who knows better, but after banging my head against this for months (and nagging support), this is the conclusion I've come to.
Eventually, you'll need an application layer to do user management, either through _users or API keys. Once you have such a layer, that's where you can hash passwords, and/or skip the _users database and do user management some other way. Every sample posted by Cloudant eventually does this, as soon as things get complicated enough (and none of the samples scale to tens of thousands of users, btw).
Finally, to #Anti-weakpasswords, who says you must go with PBKDF2 and huge iteration counts.
This is sound advice regarding saving passwords in general, but:
this doesn't work with Cloudant, at all;
it doesn't really work very well with CouchDB either.
First, as stated, if salted SHA1 is all that Cloudant supports, period.
But even for CouchDB, it's bad advice. With basic HTTP auth, you're sending the password on every single request. Key stretching with huge iteration counts on every single request would put tremendous pressure on the server, so large iteration counts are not recommended (that's why the docs have a 10 in there). If you're going down that road, you need to make sure you always use _session and cookies, and avoid basic auth like the plague.
More likely, if you take security seriously, you need to get a layer between the client and the database that handles user management some other way, and have decoupled database users/roles with strong enough passwords not to need strong hashing at all.
Clusers just came out! It may be useful to you. Clusers is a user account creator meant for Cloudant and CouchDB. It uses the older "password_scheme: simple" which Cloudant and older CouchDB.
First, you really, really do need to read Thomas Pornin's canonical answer to How to Securely Hash Passwords.
Read it right now.
Now read that CouchDB link, and see one of the recommended ways to produce password_sha for 1.3 (and if you're not on at least 1.3, get there).
{
"_id": "org.couchdb.user:username",
"_rev": "1-227bbe6ddc1db6826fb6f8a250ef6264",
"password_scheme": "pbkdf2",
"iterations": 10,
"name": "username",
"roles": [
],
"type": "user",
"derived_key": "aa7dc3719f9c48f1ac72754b28b3f2b6974c2062",
"salt": "77bac623e30d91809eecbc974aecf807"
}
Make certain that password_scheme is pbkdf2!
See that "iterations": 10 in the same? You need to bump that up by a huge amount - I'd say try a number in the low hundreds of thousands and see how it runs; SHA-1 is very cheap these days.
As far as Cloudant goes, here's a Github repository with some code to have Cloudant use the CouchDB _users.
I am currently working on a service which requires users to pick a 4-digit password/pin, because it is a mobile service. I am encrypting those passwords with either 256 or 2048bit encryption and it will be hashed. The account is blocked after 4 wrong entries, and can only be entered by mobile phone. Would it be hard to crack those PINs? I am asking this because sensitive information is being stored. The database is connected to a web application, the application is loaded to the phone using twilio. The thing I am most scared for is that the database is being hacked via the web. What would be a good way to keep sensitive data secure?
If someone gets hold of the database, you would be pretty much screwed:
If you just encrypt the 4-digit passwords, an attacker can just build a table of the 10000 possible encrypted strings and can trivially decrypt the PINs.
If you use salt strings (and encrypt not PIN, but PIN+salt and store crypted(PIN+salt) alongside with salt), people have to make a per-password effort, but there are still only 10000 possibilities for each password (which is not very much).
Which means, yes, by all means you should keep the database off the web. (If the web application is only ever accessed through twilio, you can reject connections from any other IP range).
Since you're using twilio, just make sure that twilo only talks to your web service using a secure protocol and reject any requests that you aren't sure are coming from a trusted source (that is, twilo). No real need for a pin at all.
This is a huge webpage on how to setup ssl between your web server and twilo. It even has a php example.
http://www.twilio.com/docs/security
If you use PKCS#1 1.5 or 2.0 RSA encryption (view the standards) you will also encrypt a random padding. This means that in transit, the PIN's cannot be compared, as long as the padding is kept secret and is truly random (this is not a salt which should be made public).
As for the database, it would be a good idea to move it out of the normal operations as much as possible. Create a simple service that just checks the PIN after decryption, make sure you don't have buffer overruns etc. on that, and if possible, use a different machine and access rights than the production server. Really test this part well, as the interface is small, it should not be difficult.
If you and the phones are up to it, you might want to try ECC, but that's not for the meek. RSA encrypt normally uses a small public exponent (0x010001 is highly recommended) so it is faster than ECC for the phone. On the server (and during key creation) ECC is much faster. I would not recommend symmetric cryptography (AES/3DES) for this.
Oh, and include the public encryption key in the application (for implicit trust), don't send it over from the server. Keep the private key secret and inaccessible for anything other than the already mentioned service.
The interface you describe sounds secure to me. It's secure enough for ATMs!
Are the encrypted PINs easy to crack? Yes, there's only 10000 possible combinations and a rainbow table can be generated of all the possible encrypted values unless you salt. However that would require access to the encrypted PINs which means the attacker already has a copy of your database.
So really you need to ensure your database server is secure. There's a lot of variables that could make it insecure so it's a big question. Instead you could rely on third-party solutions like Amazon S3 or others and concentrate on coding instead of security. Let them do the hard work!
You should use good quality password salting technique to prevent against hacking passwords. Check the wikipedia article to know more about salting. Salt Article
In some project we have very that even our staff is not suppose to have access to. In theory, we have policies to ensure they don't. In practice, we are in Africa and policies don't mean a lot, no matter how strongly you enforce it.
I would like to know is there is a way to encrypt data in your database so:
each user password encrypt and decrypt its own data, and its own data only;
data is decrypted as late as possible in the process to ensure maximum security to the user. Ideally it would be on the client side I guess, but I'd love to hear that it's possible to do some crazy thing I don't know about on the server side.
data is still searchable. Is that even possible?
My first idea was: "if a customer want THAT level of protection, then give him its own hosting on a virtual machine and encrypt the hardrive, then all maintenance must be done with it's allowance".
I can't come up with a fancy strategy just how I've implemented this:
Keep in mind that you have to re-encrypt everything when the user changes his password. I'm using always the same encryption key but the key is encrypted using the user's plaintext password. So I just have to re-encrypt the key. The user's password is stored as a salted hash so nobody can decrypt the key and the data even if he sees the hash.
It works like this:
User enters his plaintext password
Create salted hash
Check if the generated hash matches the one in the database (authentication)
If yes, then decrypt the key for the data using his plaintext password
Decrypt stored data using the key
This won't give you 100% security but improves it.
Here are a few things I can think of:
You should encrypt data stored when it is stored in the and when you read it back. Use a solution that integrates at an RDBMS level rather than the data layer.
For the transport of data to and from the application, use HTTPS web services.
If you have a Desktop application, do not store any data and log files etc locally.
If it is a web app, make the app HTTPS as well.
Security is bound to make the app a little slower than using plain data, but that's the price you will pay.
It really depends on what and where (on the client or server) you are doing with the data.
For example, your application don't need to know the password itself to verify it during authentification. Best practice for this use case is to store only a cryptographic hash (e.g. sha1) of the password and a random salt. That is sufficient to verify it, but giving only the hash and salt, it would take a nearly infinte amount of time to figure out the plain password.
Encryption can be a soultion if you have to exchange data over unsecure channels. But keep in mind that in order to process the data you have to decrypt them. So if de- and encryption is done on the same machine, it's rather pointless. And if decryption is required it doesn't matter how late you are going to do it, because of the key must be given anyway.
You can use encryption to secure the communication between the server and the client, for example. You could even generate messages on the server that only the client will be able to read and vice versa using asynchronous encryption. So once the message was generated on the server and encrypted using the client's public key even the server isn't able to read the message anymore, because of the private key only the client knows is required for the decryption.
What you denfinetly can not solve by cryptography is, when you have data on the server, that the server should be able to read in order to process them but human users unrestricted with priveleages to this server shouldn't.
I have a database that many different client applications (a smattering of web services, some java apps and a few dot net applications) connect to. Not all of these are running on windows (Sadly, otherwise it would make this an easy answer question with just enabling windows authentication for database connections). At the moment, the passwords are stored in various configuration / properties files lying around the systems. Ideally, only the support staff have access to the servers where the files are running, but if someone else gains access to one of the servers, they would have enough database permissions to get a fair whack of data as it stands now.
My question then, What is the best way to keep the passwords configurable, without having it too easily available to the casual human reader?
Edit Just to clarify, DB server is Windows Server 2003, running MSSQL 2005.
PS: I don't see any questions that this duplicates, but if there are, please feel free to close this one.
I'm assuming you want to hide the passwords from casual observers. If they were evil, steely eyed observers with access to all the source code on one of the machines that connects, then they can get the password with a bit of reverse engineering.
Remember that you do not need to use the same protection for each different client. A few steps:-
Create different database accounts for different systems that access your database
Limit access on the database to only what they need using your inbuilt database GRANTs
Store a triple DES (or whatever) key inside a password manager class on your database. Use this to decrypt an encrypted value in your properties file.
We have also considered having the application prompt for a pass-phrase on startup but have not implemented this as it seems like a pain and your operations staff then need to know the password. It's probably less secure.
Let's assume the following common scenario:
You use the same code base for all environments and your code base has the database passwords for each environment.
The personnel (sysadmins, configuration managers) that have access to your production application server are allowed to know the production database passwords and no one else.
You don't want anyone with access to the source code to know what the production passwords are.
In a scenario like this, you can encrypt and store the production passwords in property files that your application. Within the application you can include a class that reads the passwords from the property file and decrypts it before passing it to the database driver. However, the key and the algorithm used to decrypt the password are not part of the source code but rather passed to the application as a system property at runtime. This decouples the knowledge of the key from the application source code and anyone with access to just the application source code will no longer be able to decrypt the password because they do not have access to the application's runtime environment (app server).
If you are using Java take a look at this for a more concrete example. The example uses Spring and Jasypt. I am confident that some thing like this can be extrapolated to other environments like .Net
At my old workplace we used to have a system whereby all passwords were encrypted (using Triple DES or whatever we were using at the time). The passwords were often stored in properties files (this was in a Java system).
When the password needed to be changed, we could simply use "!plaintext" as the value, and then our code would load it up, encrypt it, and store the encrypted value back in the properties file.
This meant that it was possible to change the password without knowing what the original value was - not sure if that's the kind of thing you were asking for!
It sounds like there is no easy answer (because of the different types of applications that connect)... really, the only issue I see is the Java Apps which seem to connect directly to your database. Is that correct?
If so, here's what you can do:
1) Change any client-side applications that connect directly to the DB to go through a service. (If they have to connect directly, then at least give them a first step to "get password" from a service, then they can connect directly).
2) Store the passwords in the web.config file (if you chose to do .Net web services), and then encrypt the "connection strings" section of the file.
Don't use passwords, server to server authentication can usually be performed by using a key file or a client cert or some other way other than a password.
You could use a reversible encryption algorithm e.g. Blowfish to store the passwords as a stopgap measure. There should be a number of free libraries you can use to build this into all your programs that need this access.
Bruce Schneier's page on Blowfish
Wikipedia article on Blowfish
For the java stuff, if you're using an app server see if you can define a data source, and your apps can get at the data source using JNDI. That way, managing the datasource (including connection details) is handled by the app server, and your application code has to do is ask for a datasource.
NTLM Authentication or LDAP-based (Active Directory) authentication should be available to you with a bit of effort. This would allow you to use your "windows authentication" across applications.
It may mean a bit of a migration for your operations staff, but SSO for a set of applications is nice.
Yes I have to agree with the option of storing the (salted) hashes. I would recommend a (salted) SHA256 hash of the password stored in the database. Also don't forget to enforce secure password rules.
My interpretation of your question is that you are asking specifically how to store configuration passwords that your code will use to connect to services it depends on such as a database or third party API. In that case, you may want to consider using a service which provides a secrets container such as Hashicorp's Vault.
You can think of vault as a web service your application can connect to in order to lookup the secrets your application needs at application runtime.
As an example, lets assume your application needs to connect to a database but you don't want to store your database credentials with your application source code in your version control system. Furthermore, lets assume that you want the database credentials used by your application to be different each time your application starts. In this case, you could enable and configure the database secret back end in vault. This means that vault will dynamically create your database credentials as a service, and then provide your application with a revocable leased token for some duration of time. Vault, of course, will allow you to store any secret in it.
Vault provides secure ways for your application to connect to it. One such authentication method uses what is known in vault as the Cubbyhole Secrets Engine.
Using encryption is not a good idea. If someone compromize the key he can decrypt it. Use a hash algorith with salt to store paswords. Hash algorithms are one way so its not reversible. But they are vulnerable to dictionary attacks so use salt (concatane plain text with something long and verbose than hash it). It also protect database from internal attacks.