What is the difference between using SQL Server SSL (Encrypted=true in the connection string) + TDE, vs using SQL Server Always Encrypted?
With regards to RGPD, is one more adapted than the other?
Always Encrypted exists to solve more than just the issue of making sure data is encrypted in-transit. In fact, that's not even the primary concern that Always Encrypted solves.
The big issue that Always Encrypted solves is that with Transparent Data Encryption (TDE), the keys and certificates which secure the encrypted data are themselves stored in the database. This could be a concern for someone considering putting their SQL Server database in the cloud, because the cloud provider then ultimately has the secrets for decrypting the data.
With Always Encrypted, the Column Encryption Key (CEK), which is used to encrypt/decrypt column data, is stored in the database in its encrypted form. But here's the kicker - the key used to encrypt/decrypt the CEK is stored outside the database, leaving the database unable to decrypt the data on its own.
All the database can do is
Provide the encrypted CEK,
provide the location of the CMK, and
serve/store pre-encrypted data.
It's up to the client to get the Column Master Key (CMK) from the key/certificate store wherever that's located, then use the CMK to decrypt the CEK, and use the decrypted CEK to encrypt/decrypt data.
So that's the conceptual difference. Here are a couple pages that go into the details of it:
Overview of Key Management for Always Encrypted (Microsoft docs)
SQL Server Encryption: Always Encrypted (Redgate article)
Be aware that Always Encrypted comes with some hefty drawbacks with regard to querying data and other things. This article gives a pretty good list of limitations. Some of these drawbacks can be mitigated using Always Encrypted with secure enclaves.
Related
My project has a requirement to encrypt a sensitive field column in SQL server table but the encryption/decryption key shall be kept outside SQL environment to ensure maximum data security.
Thycotic server is one secret server to store and secure passwords. I am trying a POC to check if encryption key can be generated via Secret server and can be used to encrypt table column in SQL server.
I haven't found much related to this on the Thycotic website or on google.
How can this be achieved if feasible?
I have a lack of understanding regarding a particular area of TDE within Microsoft SQL Server (in my case, SQL Server 2016). Specifically what certificates/keys are required to restore an encrypted database on another server.
Microsoft Docs highlight the encryption hierarchy, running from service master key, database master key, certificate and database encryption key. It highlights the necessity of creating backups of the certificate to prevent data loss. It doesn't really mention anything of backing up anything above the certificate in the hierarchy.
However a Redgate article describes backing up the service master key and database master key in addition to the certificate.
In practice I am able to restore an encrypted database on another server using only the saved certificate/private key.. so what am I missing? If the certificate is protected by a master key that is not available, how does it work? Is that part of the encryption 'internal-only' - if so is there any instance where I would need to restore the service master key or database master key from a backup? I suppose keeping a backup of those wouldn't hurt anyway?
Many thanks
I've successfully restored a TDE-encrypted database onto a server that has only the certificate that was used for TDE. Said another way, the destination server had neither matching a matching database master key (DMK) on the master database nor a matching service master key (SMK). So long as the DMK for the master database is encrypted with the SMK on the target and the TDE certificate's private key is loaded into master and encrypted with that DMK, you should be good to go.
But! For something like this, you shouldn't take my (a random guy on the internet) word for it. This fundamentally affects your ability to restore your database. Restore-ability is Job One™ for a DBA, you should try it and convince yourself that not only what I'm saying is true but also that you can do it.
Also, as part of that restore plan, take that certificate (along with its private key), back it up to disconnected media, and put it somewhere for safe keeping (I like to give it to the legal department).
I am undecided on two methods to hide data in a single column in SQL Server DB.
1. SQL Always Encrypted
2. SQL Dynamic Data Masking
After reading articles for both, my understanding is that Always Encrypted may be more secure than Dynamic Data Masking as the data stored is encrypted instead of the putting a mask on top of the password.
However, what are the impacts when it comes to daily usage in this scenario if the user simply wants to hide the password field from database administrators?
Data to be hidden: Password
Functions using Password column: Reset password, change password, insert new password row when creating new user
Any advice would be appreciated.
P.S. My initial plan was to use SHA1+Salt hashing to password fields and storing hash values instead of password. However, if the requirements are simply to hide the field in DB, I find that the other 2 methods above may be sufficient.
If you want to hide the PW from the DB administrator, then dynamic data masking is off the table. This simply masks it at the presentation layer.
AlwaysEncrypted moves the responsibility of encryption to the client application, meaning the encryption and decryption is handled outside of the application, leaving the data unreadable to the database administrator generally speaking.
So, it depends on your definition of hiding. If you want it hidden from administrators... Always Encrypted is where you need to focus.
EDIT
Always Encrypted is only available in 2016 for starters, but column level encryption has been available since 2005. But to answer your question, no, the DBA doesn't have to have the keys to the kingdom. AE was designed to handle the encryption and decryption at the driver level. The information below is from the official 70-764 exam reference by Victor Isakov:
At a high level the AE architecture works as shown in Figure 1-5:
The client application issues a parameterized query. It uses the new Column Encryption Setting=Enabled; option in the connection string
The enhanced ADO.NET driver interrogates the database engine using the [sp_describe_parameter_encryption] system stored procedure to determine which parameters target encrypted columns. For each parameter that will require encrypting the driver retrieves the encryption algorithm and other information that will be used during the encryption phase
The driver uses the Column Master Key (CMK) to encrypt the parameter values before sending the ciphertext to the database engine.
The database engine retrieves the result set, attaching the appropriate encryption metadata to any encrypted columns, and sends it back to the client application. The data is encrypted both at rest within the database and in flight from the database engine to the client application.
The client application’s driver decrypts any encrypted
Under no circumstances should the password ever be stored on a database table, or anywhere for that matter, period.
You should hash the password and store the hashed value. Consider using SHA 2, or higher, with salting.
See NIST for more details: https://csrc.nist.gov/Projects/Hash-Functions
SQL Server 2008+ supports up to SHA2-512:
https://learn.microsoft.com/en-us/sql/t-sql/functions/hashbytes-transact-sql?view=sql-server-2017
DECLARE #HashThis NVARCHAR(4000)
SET #HashThis = CONVERT(NVARCHAR(4000),'dslfdkjLK85kldhnv$n000#knf');
SELECT HASHBYTES('SHA2_512', #HashThis);
To see how the password can be salted, see the article below. It is worth pointing out that hashing and salting should be handled at the application layer because the DBA can see the password that is being salted in SQL Profiler. Be very careful.
https://www.mssqltips.com/sqlservertip/3293/add-a-salt-with-the-sql-server-hashbytes-function/
Additional Info:
https://www.mssqltips.com/sqlservertip/4037/storing-passwords-in-a-secure-way-in-a-sql-server-database/
Final point: If, by any chance, you are performing this exercise as a means to authenticate users in a web application, then you should look for authentication modules for the specified application framework. For example, ASP.NET comes with Microsoft's Identity 2.0 system, which handles the authentication process end-to-end.
https://learn.microsoft.com/en-us/aspnet/core/security/authentication/identity?view=aspnetcore-2.1&tabs=visual-studio
I would be using shared hosting for my SQL Server. I wanted to encrypt the primary key column(auto generated ID) and a varchar column. I searched around and came across TDE. But, being a shared hosting, i cannot use it. So, was looking for any alternatives. The data is already there in the tables, so encrypting from application is not possible now. And there are many SQL statements that search using the above said varchar column. So, performance is also a concern.
Thanks,
Dev
encrypt the primary key column(auto generated ID)
And how will you be able to find a record if its ID is encrypted?? Answering 'I serach by encrypted ID' disqualifies you for not salting the key...
And now for the real issue. You said that you'll be deploying on shared hosting, but you do not mention what kind of protection are you expcecting that ecnryption will provide. The problem is key management. Data will be encrypted with a key and the server will need to decrypt that dtaa somehow. No matter how you turn the problem arround, the root key used to decrypt all the other keys will be also on shared hosting, and hence you will not achieve anything but a slight bump on the road to your data. To put up a fence, the key must be retrieved somewhere from ouside the scope of the shared hosting, eg. your applicaiton will ask for the password to decrypt the root key when interacting with the users, which is by all practical means impossible. Note that
TDE would had solved nothing since the root of trust has the very same problem in TDE or in columnar encryption. If you need privacy, use private hosting.
And to answer the question:
to encrypt data you use ENCRYPTBYKEY.
to decrypt data use DECRYPTBYKEY
the encryption key should be encrypted with a certificate and the certificate should be encrypted with a password, or with the database master key. See Encryption Hierarchy.
you open the encryption key in the session using OPEN SYMMETRIC KEY
you open the master key in the session using OPEN MASTER KEY
you search encrypted dtaa by creating a hash and searching the hash, see Indexing encrypted data
And if you use columnar encryption you never encrypt the primary key. Doing so is completely non-sensical. And claiming any sort of privacy in shared hosting environment is a pipe dream. The only thing you could protect agfainst, at best, is accidental media loss (the hosting HDD turning up at a flea market).
Dev,
I think in your case the only choice is to encrypt from application.
Alternatively you can rename your table, substitute the VIEW instead of renamed table. Encrypt all existing data:
update real_table set field1 = call_encrypt_function(field1), field2 = call_encrypt_function(field2)
In that view you can do select call_decrypt_function(field1),call_decrypt_function(field2) from real_table. For insert and updates you need to master INSTEAD OF INSERT and INSTEAD OF UPDATE triggers. Of course you need to use WITH ENCRYPTION for that objects. I saw that approach in XP_CRYPT, however I prefer free solutions. SQL Server provides basic encryption functions for free.
I've been trying to figure out a good way to encrypt sensitive columns in my DB. I thought the built-in encryption mechanisms of SQL Server would do the trick but either I'm missing something or doing it wrong.
The original plan was to create a table with columns that were encrypted with a symmetric key, and have a view select the data from the table unencrypted. However, I was unable to figure out how to use the DecryptByKey method in the view select statement. Plus it occurred to me that the data would be unencrypted going TO and FROM the view, so unless the connection was secure then it would sorta be pointless.
Then I thought to bring all the encryption/decryption to my app. I figured that
If the DB was completely unable to decrypt its own data, then someone infiltrating the DB wouldn't be able to do much at all.
It would save the server the effort of trying to decrypt/encrypt the info, as encryption/decryption in the DB could affect performance globally instead of just on a single workstation.
So as it sits, my app has "hard-coded" IVs and Keys for each column that needs to be encrypted. It sends the encrypted info to the DB, and receives encrypted info from the DB. This is just for messing around mind you, I know I have to put the IVs and keys somewhere else...they simply aren't safe in the app code.
I was thinking of this crazy idea:
The client app would contain a single Key and IV. The server would contain the Keys/IVs of all of the encrypted columns in a single table. However, the values of the Keys/IVs would be encrypted with the Key/IV that the client app held.
On startup, the client app would load all the Keys/IVs from the DB into memory, and decrypt them as needed to view the data selected from the server.
There could also be a relation which would join users with keys they were allowed to use. So the app would only decrypt columns that the user was authorized to see.
Do you think the idea is a win or loose? And how have some of you implemented encryption given a client-app/SQL Server scenario?
YOu loose. Point. No chance to use indices etc.
If you want safety, put it on a safe server and load Enterprise Edition and use database file encryption.
Consider putting in a middle tier to handle the encryption/decryption for you. Assuming you can put it on the server you can keep control of the bits and not worry about the client app (which may be somewhat out of your control) from being decompiled (and exposing keys).