We use SQL Server, soon to be upgraded to the latest version of SQL Server, which I believe is SQL Server 2017. I'd like to be able to move some of our databases to the cloud, at least for development. Our company has instituted a lot of security protocols, and some (important) people in the company are uncomfortable with PII information or corporate information being on the cloud. Not that I blame them. I've read that SQL Server databases can be encrypted, and have read some of the documentation. This may be the solution to me being able to put some of our data in the cloud.
I can't get my head around how one could write a query against an encrypted database, without that database being at least temporarily decrypted. Would the fields referenced in queries or views be converted to encrypted values for the query to run? Or, is there some other way this is handled. I saw some stats on the overhead of TDS, and while it's not pretty, it may be doable. Could anyone explain in not quite layman's terms how this encryption is handled?
When you are doing a search against encrypted data, you would have to encrypt your search query in with the same algorithm the data was encrypted against. This eliminates the possibility for LIKE searches and for you to be able to show the data back in plain text however.
I can't get my head around how one could write a query against an encrypted database
There is difference if you encrypt the database or data.
Often the database itself can encrypt the database (such as MSSQL), or the hosting infrastructure can encrypt the data storage. In that case - the data themselves are stored encrypted, but the encryption is transparent to the authenticated clients (authenticated client will query the data as before, nothing changes). The key is managed by the infrastructure or database.
Another option is storing the data already encrypted by the client, e.g. encrypting only PII by the client app.
Storing all data already encrypted would make either data not searchable or you will need to use deterministic (semantically non-secure) encryption. This is what previous answer suggested. For this there's even a nice tool CryptDB which act as an encryption proxxy over database queries.
Related
I am looking into ways to build a data warehouse that would house confidential data for 1+ clients. The requirement is that our organization can never obtain access to the decrypted data. There would be a web application clients utilize to write data and query data (basic select statements including ranges).
I've been learning about SQL Server Always Encrypted w/ Secure Enclaves (and Azure SQL Always Encrypted w/ Secure Enclaves) and am wondering about key management strategies.
Based on what I've read here: https://learn.microsoft.com/en-us/sql/relational-databases/security/encryption/always-encrypted-enclaves?view=sql-server-ver16 and even here https://aws.amazon.com/blogs/modernizing-with-aws/sql-server-always-encrypted-with-secure-enclaves/ , it seems like at the end of the day, someone in an organization utilizing this, is going to maintain access of the actual keys that could be used to decrypt the data.
Microsoft suggests role separation such that a DBA may have access to the database, but not access to the keys used for encryption and a Security Admin has access to the keys but not the actual database.
But, hypothetically, if the DBA and Security Admin wanted to be malicious together, they could technically obtain access to the data together. Are there strategies that can be employed to help mitigate this potential risk?
I'm wondering if the key management could be placed onto the client(s) instead of one of our own Security Admins - that way, technically, it would be completely impossible for our organization to ever have access to the data in a decrypted form. This would mean that for each client, there would be a column master key that they would store/manage and pass to us (encrypted). We'd use this to programmatically retrieve the column encryption key(s) relevant to their querying.
I've never worked around architectures involving confidential computing and am curious if the direction I am headed seems reasonable and/or if folks have any thoughts, advice, or could point me to any documentation involving key management strategies where clients house their own keys to access their data hosted by some other service.
I am going to store sensitive data like SSN and bank account numbers in my SQL Server database. The database is located on a server separate from where the application code is run, and I have Security Groups set up to only allow inbound access to the db server from my app server IP. I have already implemented the IDataProtector API and am using it to store encrypted passwords in my db, so I was going to use this to store the SSN and bank info as well.
But I read about Always Encrypted today, which is a SQL Server-based method you can use to encrypt specific columns. From the Microsoft page: "Always Encrypted allows clients to encrypt sensitive data inside client applications and never reveal the encryption keys to the Database Engine (SQL Database or SQL Server). As a result, Always Encrypted provides a separation between those who own the data and can view it, and those who manage the data but should have no access."
If I'm mostly concerned with hackers getting into my db and stealing data, which one of these methods is more secure? The master key for Always Encrypted would be stored in the Windows Certificate Store on the app server. I honestly don't know where the keys for IDataProtection are stored since it's been awhile since I set it up. I don't have anything explicit in Startup in services.AddDataProtection().
After much research, I think I'll go with Data Protector for now. The main decision point was that I don't want to use Azure Key Vault for centralized key storage, and Always Encrypted does not offer many alternatives.
I found that I can't use Windows Certificate Store for the column master keys because my EC2 instances are on a load balancer/auto-scaler, and so the Windows environment could be rebuilt at any time and lose the keys.
I got a question about encryption in SQL: encrypting data in the application's code (C#) and storing the encrypted data in the database has much less overhead compare to using the build-in SQL encryption options.
I like that:
You don't have to worry about making sure you'll be able to decrypt the data if you restore the database on a different server or when using SQL replication. I know that you can generate a special key that will allow you to do that but again, too much overhead.
It creates a total separation between the encryption method and the encrypted data.
Validating user password is done via the application's code by comparing user input with the hashed password. I don't think anyone need to run a query to check how many users have a specific password so we are good there.
On the other hand, if the encrypted data is a credit-card #, running a query to know when a specific card was used is not an unusual task.
To solve this problem, I'm thinking to use a SQL-CLR function to validate encrypted data. It will be used only for validation and not for encryption.
I wonder how safe is this going to be, especially if I would want to create some stored procedures that will relay on this user function to be able to query for data.
Thanks!
You are just asking for PCI problems I think. Why do you need to know when a specific card was used? For purposes of analytics/reporting or investigations/disputes/etc.
Your merchant processor (I am assuming you are a merchant based on the post) likely has ways to tokenize the actual card number at the point of sale. That token will still allow you to do analytics/reporting.
Storing encrypted data in the database, which was encrypted in the application layer, and then providing a way to decrypt in the database would likely ruin all the benefits of encryption.
Could you encrypt the card number in your C# app and then query that database using the already encrypted value?
Can you help with “validate encrypted data”?
We are using Sybase SQL Anywhere 11. We need to encrypt some of our tables in our database. I followed the instruction and did it. We selected the "strong" option with encryptionKey and AES256_FIPS algorithm. But there are something I am not clear about them.
It will require encryptonKey when we create the database, remove the database and start the database server but it will NOT require encryptionKey when we stop the database server and connect to the server to create tables and add data. Why there is NO encryptionKey asked when we connect to it or try to stop the server? I am doing something wrong?
don't know how to test the encryption? I still can see all plain text in the encrypted tables when I use Sybase Central tool. If somebody knows the database user name and password, he/she can connect to the database and read the content without the encryptionKey. is this right?
I can't speak specifically to Sybase's system, but database encryption in general is designed to encrypt at the file system level to prevent backdoor access to the database's data (i.e. reading the binary file and parsing it yourself rather than going through the RDBMS), not to prevent access or obfuscate data through the standard database interface (it's expected that you'll use your RDBMS's security mechanisms to do this).
Most likely the SQL Anywhere tool captures the encryption key when you first enter it (i.e. at server startup) and then stores it in memory to decrypt for other operations. As #Adam Robinson said, the data needs to be encrypted in the filesystem or there's a back door by taking the "analog solution" and just parsing the raw file, which is a nice segue to your second question...
If you want to test the encryption, try locating and opening the file outside of the Sybase tools and see if you can spot any identifiable data. I would start with a simple text search for a string (create a cell with a unique string you can search for) and make sure its not there in the encrypted file. To verify, you could also parse a non-encrypted version of the database and make sure that you can find the string there.
I don't want customers to be able to make backups of my sql server database and access the tables data etc.
I know there are some products that will encrypt the data in the tables, and their product will decrypt it when displaying in my application.
What products do you guys know of? What options do I have?
(This is a business requirement, however silly it might seem to some hehe).
Update
This is for sql server 2008 express
The problem with encrypting data inside the database is that as long as the database lives on the client's machine (as you indicated, they're running SQL 2008 Express, so I'm betting it lives on the client's desktops or laptops) then they can get into the data. They can set up security on the instance so that they have SA privileges, and from there, they can get the data, period. There's no way around that.
What you have to do is encrypt the data before it hits the database: encrypt it in your application. Inside the app, encrypt the data that you want to store in each sensitive field. As another poster indicated, you don't want to encrypt ID fields because those are used for indexing.
There is the 3rd party xp_crypt. It's been around for years.
It's an extended stored proc (that is, DLL)
SQL Server 2008 supports database encryption natively. Check the documentation for Transparent Data Encryption (TDE).
You can encrypt stored procedures, which can protect your logic.
TDE is available only Enterprise edition.
I can't find if it supports native sql encryption - but you could find this out with a little searching. But if it did you could probably set the database master key with your application and keep all of the decryption/encryption code in your application.
If it doesn't support native encryption, you might want to creat/find your own encryption functions in your application language and lock away the keys in your code.
Transparent Data Encryption will encrypt the database on disk, but is unencrypted in memory, so appropriate security would also be necessary to ensure unauthorised users cannot access the table. As it's an Enterprise-only feature, you can safely move away from it.
SQL Server 2005 and above have built-in encryption features - have a look at Books Online, and especially Chapter 5 - Encryption of Adam Machanic's Expert SQL Server 2005 Development book (technically, Lara Rubbelke wrote chapter 5 though).
Note that you'll only want to encrypt some columns - those that you'll never try to look up, as encrypted columns are pretty much useless for indexing. Adam Machanic's book suggests ways to solve this problem.
Another solution for transparent SQL Server encryption is DbDefence
Free for databases less than 77 MBs.