I am looking for an alternative to symmetric key encryption to savely store sensitive data in a Microsoft SQL database. The reason for this is a few days ago I had an error during the night (at 3 am) where the database responded my status call, which is used for health checks of the backend, with an error
A severe error occurred on the current command. The results, if any, should be discarded.
(The call I am using for health check is only calling my rest api - going through the web service to the database, does a select count(*) from Member and returns the count.)
After that error every api call which used sensitive data from the database returned
Please create a master key in the database or open the master key in the session before performing this operation.
My monitor service said that the backend was up again after 2 minutes automatically but the master key was not working anymore. I fixed it with the following command
open master key decryption by password = 'password'
alter master key add encryption by service master key
the morning after but in the meantime the backend was not working correctly. So the failover didn't really did its job (because I had to do something manually to get everything working again).
So I am trying to achieve to store sensitive data easily in the database (must be able to decrypt it again) and to have a working failover without doing anything manually too.
Thanks for your input!
What i think I'm reading is that you have some sort of HA technology in play (e.g. availability groups). If that's the case, care needs to be taken to ensure that both sides of the topology can open the database master key.
Fortunately, it's fairly easy to do that. You can backup and restore service master keys (SMK). So, you'll backup the SMK from the secondary node and restore it to the primary node. SQL Server will decrypt anything currently encrypted with the old key and re-encrypt with the new.
Related
My name is Hari varma, I am working as a SQL DBA.
Currently I am facing issues with Always Encryption in SQL Server 2017 Development Edition in our testing server.
I don't have any experience in Always Encryption and TDE. One of my client asked me to do encryption on database.
I have done some testing on Always Encryption and I was able to encrypt and decrypt the column data by using doing the following:
On the SQL Server instance
-->Options-->Additional connection Parameter-->Column Encryption Setting = Enabled
After I enabled the column encryption I am able to view the encrypted data in the table.
However I am not able to insert, update, and delete data in this encrypted column.
Also I need to be able to set permissions on users who are allowed / not allowed to view the data on this encrypted column.
Which permissions I need to give on a particular user and provide any prerequisites for Always Encryption and TDE.
First of all it's important to understand that your SQL Server instance does not know the keys used for encrypting and decrypting data when using Always Encrypted. The keys are kept externally, usually in a key store such as Windows Certificate Store or Azure Key Vault. This means that SQL Server itself cannot encrypt or decrypt the data - this instead has to be done by a client application that has access to the keys.
However I am not able to insert, update, and delete data in this encrypted column.
I assume you are attempting to insert, update, and delete data directly through SSMS or something similar. This is only possible to a limited extent. This is because SSMS (which is your client application in this case) needs to be able to encrypt the data before sending it to your SQL Server.
Read more about inserting data into columns that are encrypted via Always Encrypted in SQL Server here (using SSMS).
A brief summary of how to insert encrypted data via SSMS:
You need to enable the column encryption setting in your connection string. This is done under Options>>Additional Connection Parameters when you connect to your SQL Server instance in SSMS. Add this text in there: Column Encryption Setting=Enabled
Once you've connected to your database and opened a query window, you need to enable parameterization for always encrypted. This is done in SSMS under Query>>Query Options>>Execution>>Advanced>>Enable Parameterization for Always Encrypted.
When you've completed the two steps above you'll be able to insert data into an encrypted column like this:
DECLARE #ParameterToBeEncrypted NVARCHAR(100) = 'Decrypt me';
INSERT INTO dbo.MyTable(MyEncryptedColumn) VALUES (#ParameterToBeEncrypted);
This works because your client application (SSMS) is able to encrypt the value that you're initializing #ParameterToBeEncrypted with before sending it to SQL Server. This only works if your current user has access to the column encryption key. SQL Server will never see the plain/non-encrypted value ('Decrypt me') - it will only see the encrypted value that should be inserted into the encrypted column.
Which permissions I need to give on a particular user and provide any prerequisites for Always Encryption
It's a combination of permissions in SQL Server and being able to access the keys used for encrypting and decrypting the data. The necessary database permissions are VIEW ANY COLUMN MASTER KEY DEFINITION and VIEW ANY COLUMN ENCRYPTION KEY DEFINITION.
You can read more about the necessary permissions here.
If you want to encrypt a set of existing data in your database, then your best bet is to write your own client application (e.g. in C# or similar) or create a SSIS package (which would serve as a client application). The client application or SSIS package should read the data from the database, encrypt the data outside of the database, and then send it back to the database as encrypted data.
Environment: Win 2012 and SQL 2014 Standard edition.
Issue: I am doing a daily restore of the production database into our test server. The production database has a encryption key. The restore to the test server is a SQL Job that runs nightly.
Items Tried: I have tried to include a step in the sql job to decrypt the key in the test environment: open master key decryption by password = ''. I have tried using EXEC, sp_executesql and embedding the commands in a stored procedure. The only thing that really works of when I open up management studio and run the command manually.
Results From the Job: The job runs successfully but does nothing. I adding logging and there is nothing indicating any errors. All the log say is Begin Executing.
Question: Does anybody know how I can embed the open master key decryption by password = '' step into the sql job where the command with work.
I think the issue that you have is that you're successfully opening the master key within that session, but other sessions don't see that. You subsequently need to re-encrypt the database master key with the test server's service master key. Luckily, once you've opened the key with the password (as you already have), it's as easy as:
alter master key add encryption by service master key;
Also, you shouldn't need to do anything fancy in your open master key… statement. That is, no need to wrap it in sp_executesql or any of that.
I have two instances of SQL Server 2012 Enterprise but struggling to restore a TDE database from a set of backups (full, differential and log) from instance one to instance two. I am getting the error 'Cannot find server certificate with thumbprint'.
Instance one and two both have master keys with the same password, I have backed up the certificate and private file from instance one and restored to instance two. If I look in the Security > Certificates folder of the master DB on instance two I can see it listed.
If I try a restore despite have imported the same certificate and it being present I still get the error 'Cannot find server certificate with thumbprint', can somebody please advise as to what to do next?
If I try reimporting the certificate again I get the message 'A certificate with name 'ServerCert' already exists or this certificate already has been added to the database.'
I have also tried restoring/replacing the database on instance one from the backup sets and they work, just not when restoring to instance two.
Any help would be much appreciated :-)
Despite retrying the same process several times, I removed the TDE, certificate and master keys across all instances and started from scratch with the exception of re-backing up the database. Despite all passwords and certificate names being the same still no joy.
I then removed the TDE, certificate and master keys across all instances, restarted both instances, then created a master key, then the cert, reapplied the TDE, then carried out all new full, differential and transaction log backups all on instance one. Backup up the certificate from instance one, then on instance two created the same master key, imported the backup up certificate from instance one and then carried out the database restore and all is well.
None of my T-SQL changed in any way, the only think I can gather is some sort of security failed to correctly apply, maybe a tempdb issue.
I have read others having the same issues, I hope this helps others.
We implemented database mirroring for WCF and set up failover partner in connectionstring as following
....Data Source=10.0.13.21,1433;Failover Partner=10.0.13.31,1433; ...
10.0.13.21 - primary server
10.0.13.31 - secondary server
but somehow we get error:
the underlying provider failed
when the primary server is in failover state and the application is supposed to connect to secondary server. Everything works again after resetting IIS.
Another strange thing is that the switch from secondary back to primary server is working but not from primary to secondary.
Anybody knows possible reason for this?
thanks in advance
I think you have to completely close your connection in the application and maybe do some retry code. Clearing connection pools may also be necessary.
And also bear in mind that depending upon the size of the redo queues for the mirroring there may be a delay before the failover partner database is available again.
Here's a couple of links which I would read http://www.sqlskills.com/BLOGS/PAUL/post/A-SQL-Server-DBA-myth-a-day-(1130)-database-mirroring-failover-is-instantaneous.aspx
and http://technet.microsoft.com/en-us/library/cc917713.aspx
I have backed up an encrypted DB (symmetric key/certificate) and
restored it on a different server.
Unfortuantely we're having problems with the decryption... hoping
someone can help.
In the restored db, I can see the Symmetric Key and the Certificate in
SSMS, but when I try to Open the key using the cert ( open symmetric
key KeyA decryption by certificate CertB )I get the
following very descriptive error:
Msg 15466, Level 16, State 1, Line 1
An error occurred during decryption.
Any ideas?
Thanks in advance.
http://blogs.msdn.com/lcris/archive/2007/11/16/sql-server-2005-restoring-the-backup-of-a-database-that-uses-encryption.aspx answers this:
"When you restore a database that uses encryption features, there is only one thing you need to take care off - if the database master key (DbMK) needs a service master key (SMK) encryption, you need to regenerate this encryption. Note that this encryption is made by default when you create the DbMK, but it may be intentionally dropped, if you want tighter control of access to the encrypted data. Anyway, if you did have such SMK encryption for the DbMK, the steps to regenerate it are the following:
OPEN MASTER KEY DECRYPTION BY PASSWORD = 'password'
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
CLOSE MASTER KEY
That's it - the database encryption features should now work as when the backup was taken. Also note that it doesn't matter if you restore the database on the server where the backup was taken or elsewhere. The only thing that matters for this procedure is that you know one of the passwords protecting the DbMK "
The master key was decrypted by the service master key on the source server and we were decrypting the master key with password on the destination. I altered the master key to be decrypted by the service master key and it's working now.
The problem you are probably experiencing is that the Database Master Key for the servers is different. To my understanding the other keys are based off of this and it could cause problems when trying to decrypt the data. Check out the encryption hierarchy for a description of the steps that go into data encryption.
I hope this answer helps and isn't too off-track. :)
http://social.msdn.microsoft.com/forums/en-US/sqlsecurity/thread/34c9c35c-2d08-4873-abfd-aae40240dfe7/?prof=required
That link worked for me, follow the 2 links to backup/restore
You can do the restore from the destination server using a UNC, you do not have to copy the file.