I'm working on an enterprise application and for some security reason I have to enable Always Encrypted. DBA and Production IT team did what needed from their side:
I can generate this script from using SQL server management studio under security -> always encrypted key
USE [DE_NAME]
/****** Object: ColumnMasterKey [*****] Script Date: 23/12/2022 17:30:17 ******/
CREATE COLUMN MASTER KEY [*****]
WITH
(
KEY_STORE_PROVIDER_NAME = N'AZURE_KEY_VAULT',
KEY_PATH = N'https://xxxxxxxxx.vault.azure.net/keys/******/####'
)
GO
What mean that Alaways Encrypted is configured in the DB?
Production IT team :
Create Keyvault
Create a Service Principal/application Registration in Azure AD
register Name(clientId) and secret in KeyPass
At keyvault level added role Assignement
From my side, I added the conf as descripted in Microsoft documentation:
https://learn.microsoft.com/en-us/sql/connect/jdbc/using-always-encrypted-with-the-jdbc-driver?view=sql-server-ver16
Application inf :
Spring boot
Java 8
mssql diver : 9.2.1.jre8
I added mssql-jdbc_auth-9.2.1.x64.dll and sqljdbc_auth.dll to -Djava.library.path
Test code :
try (Connection sourceConnection = DriverManager.getConnection("jdbc:sqlserver://server;DatabaseName=db_name;encrypt=true;trustServerCertificate=true;integratedSecurity=true;columnEncryptionSetting=Enabled;keyVaultProviderClientId=xxxxxxxx-xxxx-xxxx-xxxx-xx71xx1axxd3;keyVaultProviderClientKey=XXX7X~XXXMv21mXqXxX4FBi7XoXmxxxXDSZXw")) {
PreparedStatement stmt = sourceConnection.prepareStatement("INSERT INTO Table_name(id, aaa, bbb, ccc, ddd, eee) VALUES (?, ?, ?, ?, ?, ?)");
stmt.setInt(1, 999999);
stmt.setString(2, "local_test_0");
stmt.setInt(3, 989898);
stmt.setString(4, "local_test_0");
stmt.setString(5, "local_test_0");
stmt.setString(6, "local_test_0");//col encrypted
stmt.executeUpdate();
} catch (SQLException e) {
e.printStackTrace();
}
When I run this code I get the following error message :
com.microsoft.sqlserver.jdbc.SQLServerException: Operand type clash: varchar(12) encrypted with (encryption_type = 'DETERMINISTIC', encryption_algorithm_name = 'AEAD_AES_256_CBC_HMAC_SHA_256', column_encryption_key_name = 'CEK-XXXX-Shared-XX-XX', column_encryption_key_database_name = 'XXXX') collation_name = 'SQL_Latin1_General_CP1_CI_AS' is incompatible with nvarchar(132) encrypted with (encryption_type = 'DETERMINISTIC', encryption_algorithm_name = 'AEAD_AES_256_CBC_HMAC_SHA_256', column_encryption_key_name = 'CEK-XXXX-Shared-XX-XX', column_encryption_key_database_name = 'XXXX')
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:262)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1632)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:602)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:524)
It's like the driver didn't encrypt data before inserting it.
Could you help please ?
Related
I want to connect Azure MS SQL Database with Azure Databricks via python spark. I could do this with pushdown_query if I run Select * from.... But I need to run ALTER DATABASE to scale up/down.
I must change this part
spark.read.jdbc(url=jdbcUrl, table=pushdown_query, properties=connectionProperties)
otherwise I get this error Incorrect syntax near the keyword 'ALTER'.
Anyone can help me. Much appreciated.
jdbcHostname = "xxx.database.windows.net"
jdbcDatabase = "abc"
jdbcPort = 1433
jdbcUrl = "jdbc:sqlserver://{0}:{1};database={2}".format(jdbcHostname, jdbcPort, jdbcDatabase)
connectionProperties = {
"user" : "..............",
"password" : "............",
"driver" : "com.microsoft.sqlserver.jdbc.SQLServerDriver"
}
pushdown_query = "(ALTER DATABASE [DBNAME] MODIFY (SERVICE_OBJECTIVE = 'S0')) dual_down"
df = spark.read.jdbc(url=jdbcUrl, table=pushdown_query, properties=connectionProperties)
display(df)
I have been able to read/write from Databricks into SQL Server table using JDBC driver. However this time I have to execute a command before I write to a SQL Server.
I need to execute this command on SQL server: SET IDENTITY_INSERT <sqlserver_table_name> ON
How to do this from Databricks ? Any help/pointers are appreciated. Thanks.
You can't do this with the JDBC Spark Connector (or the SQL Server Spark Connector), but it's trivial when using JDBC directly in Scala or Java. When using JDBC directly you have explicit control of the session, and you can issue multiple batches in the same session, or multiple statements in the same batch. EG
%scala
import java.util.Properties
import java.sql.DriverManager
val jdbcUsername = dbutils.secrets.get(scope = "kv", key = "sqluser")
val jdbcPassword = dbutils.secrets.get(scope = "kv", key = "sqlpassword")
val driverClass = "com.microsoft.sqlserver.jdbc.SQLServerDriver"
// Create the JDBC URL without passing in the user and password parameters.
val jdbcUrl = s"jdbc:sqlserver://xxxxxx.database.windows.net:1433; . . ."
val connection = DriverManager.getConnection(jdbcUrl, jdbcUsername, jdbcPassword)
val stmt = connection.createStatement()
val sql = """
SET IDENTITY_INSERT <sqlserver_table_name> ON
"""
stmt.execute(sql)
//run additional batches here with IDENTITY_INSERT ON
connection.close()
And you can always use the Spark Connector to load a staging table, then use JDBC to run a stored procedure or ad-hoc SQL batch to load the staging data into the target table.
I have used Always Encrypted to encrypt values in a column. My connection string looks like this (everything is working fine and the client is able to decrypt the values):
Driver={ODBC Driver 13 for SQL Server};Server=10.10.10.1\INST2;Database=test;Trusted_Connection=yes;ColumnEncryption=Enabled;
Then I try to update one of the encrypted values using the following code:
ALTER PROCEDURE dbo.usp_SecurityUsers_GetDetails
(
#Firstname NVARCHAR(128)
)
AS
BEGIN;
UPDATE SecurityUsers
SET FirstName = #Firstname
WHERE login = 'system.export.service'
SELECT Login, FirstName, LastName
FROM SecurityUsers
END;
the SP is called like this:
Set cmd = Server.CreateObject("ADODB.Command")
Set cmd.ActiveConnection = conn
cmd.NamedParameters = True
cmd.CommandText = "[dbo].[usp_SecurityUsers_GetDetails]"
cmd.Parameters.Append cmd.CreateParameter("#Firstname", adLongVarWChar, adParamInput, 128, "system_text")
cmd.CommandType = adCmdStoredProc
Set recordSet = cmd.Execute
But I get the following error:
[Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Encryption
scheme mismatch for columns/variables '#P1'. The encryption scheme for
the columns/variables is (encryption_type = 'PLAINTEXT') and the
expression near line '1' expects it to be (encryption_type =
'RANDOMIZED', encryption_algorithm_name =
'AEAD_AES_256_CBC_HMAC_SHA_256', column_encryption_key_name =
'CEK_SecurityLevelNormal', column_encryption_key_database_name =
'test') (or weaker).
It's very strange, because I am able to update the field if the code is not encapsulated in stored procedure and directly executed in the SQL Server Management Studio.
Also, calling the SP directly in the SSMS works.
EDIT:
Updating to .net 4.6.3 does not change anything. What I am seeing is, if I change the connection string ColumnEncryption=Enabled; to Column Encryption Setting=Enabled;, I am getting different error:
[Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Operand type
clash: ntext is incompatible with nvarchar(128) encrypted with
(encryption_type = 'RANDOMIZED', encryption_algorithm_name =
'AEAD_AES_256_CBC_HMAC_SHA_256', column_encryption_key_name =
'CEK_SecurityLevelNormal', column_encryption_key_database_name =
'test')
I was feeling that the issue must be in the way I am trying to modify the data as I am able to read it. And it was so lame.
In my stored procedure I have nvarchar(256). In the ASP I am using adLongVarChar instead adVarWChar. The error seems more reasonable now:
Encryption scheme mismatch for columns/variables '#P1'.
The conclusion is one is able to read encrypted data and modify it using ADO and ODBC driver.
I have used SQLdependency with SignalR to show alerts to users.. The code is as follows:
public IEnumerable<AlertInfo> GetData(long UserId)
{
using (var connection = new SqlConnection(ConfigurationManager.ConnectionStrings["yafnet"].ConnectionString))
{
connection.Open();
using (SqlCommand command = new SqlCommand(#"SELECT [AlertID],[AlertNote],[AlertDetails],[AlertDate],[Location]
FROM [dbo].[Alerts] where [UserID]=" + UserId + " AND [IsViewed]=0", connection))
{
// Make sure the command object does not already have
// a notification object associated with it.
command.Notification = null;
SqlDependency.Stop(ConfigurationManager.ConnectionStrings["yafnet"].ConnectionString);
SqlDependency.Start(ConfigurationManager.ConnectionStrings["yafnet"].ConnectionString);
SqlDependency dependency = new SqlDependency(command);
dependency.OnChange += new OnChangeEventHandler(dependency_OnChange);
if (connection.State == ConnectionState.Closed)
connection.Open();
using (var reader = command.ExecuteReader())
return reader.Cast<IDataRecord>()
.Select(x => new AlertInfo()
{
AlertID = x.GetInt64(0),
AlertNote = x.GetString(1),
AlertDetails = x.GetString(2),
AlertDate = x.GetDateTime(3),
Location = x.GetString(4)
}).ToList();
}
}
}
It is working fine on localhost. But after uploading to Azure server, this method throws the following error:
Message":"An error has occurred.","ExceptionMessage":"Statement 'RECEIVE MSG' is not supported
in this version of SQL Server.","ExceptionType":"System.Data.SqlClient.SqlException","StackTrace":"
\r\nServer stack trace: \r\n at System.Data.SqlClient.SqlConnection.OnError(SqlException exception
, Boolean breakConnection, Action`1 wrapCloseInAction)
What could be the issue?
Actually your SQL Server database must have is_broker_enabled = 1.
You need to check whether it's enabled or not.
To verify this, use the command SELECT name, is_broker_enabled FROM sys.databases.
If your database shows result "1" it's okay and if "0" then you must enable it using this command ALTER DATABASE yourdb SET ENABLE_BROKER.
But the bad news is Azure SQL database shows it enabled but it no longer supports is_broker_enabled.
To do this, you need to install the full instance of SQL Server to Azure VM.
I am developing an application based on google sql cloud . The Mysql database is local.
The code below to insert a row works :
Statement stmt = connection.createStatement();
String sql = "INSERT INTO USER(name) VALUES ('smith')";
stmt.executeUpdate(sql);
However, the code below, that uses PreparedStatement rather that Statement does not work :
String sql = "insert into USER(email) values(?)";
PreparedStatement pstmt = connection.prepareStatement(sql);
pstmt.setString(1, email);
int success = pstmt.executeUpdate();
No rows are inserted in the database and success = 0 (return code for error)
Thanks for your help