Using the AdventureWorksDW2017 database, I encrypted the dbo.DimCustomer.CustomerAlternateKey using deterministic encryption. I also created a target table called dbo.TargetTable where I also encrypted the dbo.TargetTable.CustomerAlternateKey column with deterministic encryption using the same key.
create table TargetTable
(
[CustomerKey] int,
[CustomerAlternateKey] nvarchar(15)
)
I have a connection setup in SSMS with column encryption setting = "enabled" so I can see the plaintext value of the encrypted column in dbo.DimCustomer and dbo.TargetTable.
When I write the following bits of code, they work just fine:
declare #keyvalue nvarchar(15) = 'AW00011000'
select *
from [dbo].[DimCustomer]
where CustomerAlternateKey = #keyvalue -- works
declare #keyvalue nvarchar(15) = 'AW00011000'
insert into TargetTable (CustomerKey, CustomerAlternateKey)
select
CustomerKey,
CustomerAlternateKey
from
[dbo].[DimCustomer]
where
CustomerAlternateKey = #keyvalue -- works
If I wrap the code in a stored procedure, it does not work and throws out the standard encryption error.
create or alter procedure uspMoveToTarget
#keyvalue nvarchar(15)
as
begin
insert into TargetTable (CustomerKey, CustomerAlternateKey)
select
CustomerKey,
CustomerAlternateKey
from [dbo].[DimCustomer]
where CustomerAlternateKey = #keyvalue
end
exec uspMoveToTarget 'AW00011000' -- does not work
declare #variable varchar(15) = 'AW00011000'
exec uspMoveToTarget #variable -- does not work
This is the error I get:
Msg 206, Level 16, State 2, Procedure uspMoveToTarget, Line 0 [Batch Start Line 38]
Operand type clash: varchar is incompatible with nvarchar(15) encrypted with (encryption_type = 'DETERMINISTIC', encryption_algorithm_name = 'AEAD_AES_256_CBC_HMAC_SHA_256', column_encryption_key_name = 'CEK_Auto2', column_encryption_key_database_name = 'AdventureWorksDW2017')*
Using deterministic encryption looked like a better bet than randomised because is seems like you can actually work with the data but if we can't wrap the code in stored procedures, the refactoring of our ETL process is going to be a larger effort than I initially thought.
Thanks in advance
Operand type clash: varchar is incompatible with nvarchar(15)
There is no implicit conversion of encrypted values.
Try
declare #variable nvarchar(15) = N'AW00011000'
exec uspMoveToTarget #variable -- works
The parameter is always required, but in some cases SSMS will parameterize the batch for you. But this doesn't work for stored procedure calls. You can see that the stored procedure parameter requires encryption like this:
EXEC sp_describe_parameter_encryption N'exec uspMoveToTarget #keyvalue', N'#keyvalue nvarchar(15)';
Related
In the below procedure, I am trying to return a value based on an input that is in JSON format. To protect against an invalid JSON input, I am using the ISJSON() function. However, SQL Server appears to be evaluating JSON_VALUE() in the 2nd statement prior to the ISJSON() validation check in the 1st statement.
CREATE TABLE dbo.Table1(
ColKey integer NOT NULL,
ColText nvarchar(255) NOT NULL,
CONSTRAINT [PK_Table1] PRIMARY KEY CLUSTERED ([ColKey] ASC));
GO
INSERT INTO dbo.Table1(ColKey, ColText)
VALUES (1, 'Test String');
GO
CREATE PROCEDURE [dbo].[usp_GetTextFromJSON]
#JSON nvarchar(255),
#TextOut nvarchar(255) OUTPUT
AS
BEGIN
IF ISJSON(#JSON) <> 1
THROW 50000, 'Invalid JSON', 1; -- Alternatively: RETURN -1;
SELECT TOP 1 #TextOut = ColText
FROM dbo.Table1
WHERE ColKey = JSON_VALUE(#JSON, N'$.Col1Key')
RETURN 0;
END;
GO
Executing the procedure with an invalid JSON string does not trigger the expected exception.
DECLARE #return_value int,
#TextOut nvarchar(255);
EXEC #return_value = [dbo].[usp_GetTextFromJSON]
#JSON = N'{Col1Key:1}', -- Invalid JSON: ISJSON(N'{Col1Key:1}') = 0
#TextOut = #TextOut OUTPUT;
SELECT #TextOut;
returns:
Msg 13609, Level 16, State 1, Procedure dbo.usp_GetTextFromJSON, Line 10 [Batch Start Line 29]
JSON text is not properly formatted. Unexpected character 'C' is found at position 1.
Changing the execution code to the following returns successfully.
DECLARE #return_value int,
#TextOut nvarchar(255);
EXEC #return_value = [dbo].[usp_GetTextFromJSON]
#JSON = N'{"Col1Key":1}', -- Valid JSON
#TextOut = #TextOut OUTPUT;
SELECT #TextOut;
Why is SQL Server evaluating JSON_VALUE() in the 2nd statement before evaluating ISJSON in the 1st statement?
It''s a compilation issue not an execution issue. When you run the good one first the plan is generated and cached and the bad one then works as you want.
Probably it tries to evaluate JSON_VALUE(#JSON, N'$.Col1Key') against the initial parameter value so it can then use the histogram on ColKey to get row estimates.
You can try assigning the parameter to a local variable to avoid this.
CREATE PROCEDURE [dbo].[usp_GetTextFromJSON] #JSON NVARCHAR(255),
#TextOut NVARCHAR(255) OUTPUT
AS
BEGIN
DECLARE #vJSON NVARCHAR(255) = #JSON;
IF ISJSON(#vJSON) <> 1
THROW 50000, 'Invalid JSON', 1; -- Alternatively: RETURN -1;
SELECT TOP 1 #TextOut = ColText
FROM dbo.Table1
WHERE ColKey = JSON_VALUE(#vJSON, N'$.Col1Key')
RETURN 0;
END;
I have written a stored procedure that we will use to dynamically generate merge code and synchronise datasets.
The merge code generates reliably but fails to execute with a
'NOT VALID IDENTIFIER'
error. I think it has something to do with how I am escaping varchar values but I can't seem to resolve it
I would appreciate any insight into where I am tripping up.
--CREATE TABLE dbo.temp_share_test
--(grp_id int not null,
--co_code varchar(10) not null,
--sp_no INT not null,
--sp_code varchar(10) not null,
--sp_type varchar(10),
--sp_colour varchar(10));
--alter table dbo.temp_share_test add constraint pk_temp_share_test
--primary key (grp_id, co_code, sp_no, sp_code);
--insert into dbo.temp_share_test (grp_id, co_code, sp_no, sp_code, sp_type, sp_colour)
--values
--(2,'A',1,'X',1,'GREEN'),
--(2,'A',2,'Y',2,'BLUE'),
--(2,'A',3,'X',3,'YELLOW');
DECLARE #sql VARCHAR(MAX)
SET #sql =
'MERGE dbo.temp_share_test AS t
USING (SELECT * FROM dbo.temp_share_test WHERE grp_id = 2 AND co_code = ''A'') AS s
ON (t.grp_id = 2 AND t.co_code = ''H'' AND t.sp_no = s.sp_no AND t.sp_code = s.sp_code)
WHEN NOT MATCHED THEN INSERT (grp_id, co_code,sp_code,sp_no,sp_type,sp_colour)
VALUES (2, ''H'',s.sp_code,s.sp_no,s.sp_type,s.sp_colour);'
exec #sql;
I suggest to use sp_executesql overEXEC:
DECLARE #sql NVARCHAR(MAX) =
'MERGE dbo.temp_share_test AS t
USING (SELECT * FROM dbo.temp_share_test WHERE grp_id = 2 AND co_code = ''A'')
AS s
ON (t.grp_id = 2 AND t.co_code = ''H'' AND t.sp_no = s.sp_no
AND t.sp_code = s.sp_code)
WHEN NOT MATCHED THEN INSERT (grp_id, co_code,sp_code,sp_no,sp_type,sp_colour)
VALUES (2, ''H'',s.sp_code,s.sp_no,s.sp_type,s.sp_colour);'
EXEC sp_executesql #sql;
DBFiddle Demo
When dealing with dynamic SQL I highly recommend to read: The Curse and Blessings of Dynamic SQL
Thank you for the responses.
Simply adding brackets to the EXEC statement ('EXEC (#sql)') did allow command line execution but I was still not able to run from within the stored procedure. But all good now.
lad2025's link was a brilliant reference for both method and best practice. Thank you.
SUM: key to executing a dynamic sql merge statement from within a stored procedure
- use 'EXEC sp_executesql' with parameters. Far safer and avoids quote issues
- use nvarchars
I have a database with different tables (all the same structure) where I'd like to run a stored procedure having a parameter that defines which table to query.
I can't seem to figure it out:
CREATE SCHEMA test;
GO
First I created a schema
CREATE TYPE DataType as TABLE (
[datetime] [datetime] NULL,
[testVar] [bigint] NULL)
GO
Then I created the table type
USE [TestDataFiles]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [test].[testing]
(
-- Add the parameters for the stored procedure here
#datetime datetime,
#t DataType READONLY
)
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON
select top(10) *
from #t
where [datetime] > #datetime
END
GO
Then I created the stored procedure.
Exec test.testing #t = 'table.1', #datetime = '2017-01-01'
However when I call it I get the following error:
Msg 206, Level 16, State 2, Procedure test, Line 0 [Batch Start Line 0]
Operand type clash: varchar is incompatible with DataType
Same happens with:
Exec test.testing #t = [table.1], #datetime = '2017-01-01'
I have seen an example where in the procedure between the begin and select you put something like:
INSERT INTO table.1
( datetime, testVar)
But table.1 (or table.2 etc as I have a list of tables) has data and I don't want to change it.
Unless I'm meant to create a dummy table like I did the TYPE?
The examples I've found online havent been useful.
To do that you will need to use dynamic SQL
The basic procedure is to build up a string that will hold the statement you will execute, then execute it
declare #SQL nvarchar(1000)
declare #t as nvarchar (1000)
set #t = 'MyTable'
set #Sql = 'Select * from ' + #t
exec sp_executesql #sql
You have to pass parameter of type DataType. So, create variable of that type and pass it into stored procedure like
declare #table1 DataType
INSERT INTO #table1(datetime, testVar) values (..., ...)
Exec test.testing #datetime = '2017-01-01', #t = #table1
I have Encrypted one column mob_no in a table.
after encrypted the column, stored procedure is getting error.
I have added the sp below
create procedure get_cut
#mobNo varchar(50),
#custId int
As
Begin
if(#mobNo = null or #mobNo = '')
Begin
select #mobNo = mob_no
from table1 where cust_id = #custId
End
select cust_name from tbl_cust where mob_no = #mobNo and cust_id = #custId
End
When run this sp I got the bellow error
Msg 33299,
Encryption scheme mismatch for columns/variables '#mobNo'.
The encryption scheme for the columns/variables is (encryption_type = 'PLAINTEXT')
and the expression near line '9' expects it to be (encryption_type = 'DETERMINISTIC',
encryption_algorithm_name = 'AEAD_AES_256_CBC_HMAC_SHA_256',
column_encryption_key_name = 'xxxx', column_encryption_key_database_name = 'mydb')
(or weaker).
The following equality condition
#mobNo = ''
in your stored procedure will be evaluated in the server. Since you are using Always Encrypted, the column corresponding to mobNo is stored in the server as binary data. the server will not be able to compare the values in the database against '' because the server does not know the encrypted binary value corresponding to ''
You can modify the stored procedure to take 3 arguments instead of 2 as follows:
create procedure get_cut
#mobNo varchar(50),
#custId int,
#emptyString varchar(50)
...
if(#mobNo is null or #mobNo = #emptyString)
...
Note: is null instead of = null
you can pass the value '' as
a parameter to your SqlCommand, if you are using an application to execute your stored procedure or
if you are using Sql Server Management Studio to execute your stored procedure, lookup Parameterization for Always Encrypted, to understand how to send plaintext values targeting encrypted columns
As an example, From SSMS (Requires at least SSMS version 17.0.) you can do the following:
Declare #emptyStringVar varchar(50) = ''
Declare #mobNoVar varchar(50) = '000-000-0000'
EXEC get_cut
#mobNo varchar = #mobNoVar
#custId int = 1234
#emptyString #emptyStringVar
I need to update a bigint value in a database with a concatenation of two strings. SQL does not like this. DailyArchive.LogicalAccount is a bigint, #padding and a.AccountNumber are varchars.
Update dbo.DailyArchive
SET DailyArchive.LogicalAccount = CONCAT(#padding, a.AccountNumber)
FROM #_AccountList a
JOIN dbo.DailyArchive ON a.AccountNumber = DailyArchive.LogicalAccount
I receive the following error message:
Msg 8114, Level 16, State 5, Procedure updateNumbers_ArchiveDB, Line
15 Error converting data type varchar to bigint.
How I am executing the procedure:
declare #p1 dbo.AccountListType
insert into #p1 values(N'Account Number',N'Account Type')
insert into #p1 values(N'7463689',N'Basic')
insert into #p1 values(N'1317893',N'Premium')
insert into #p1 values(N'2806127',N'Basic')
exec updateNumbers_ArchiveDB
#_AccountList=#p1,
#padding=N'111',
#proc_dateStart='2008-01-04 11:24:46',
#proc_dateEnd='2008-01-04 11:24:46'
Try removing the line...
insert into #p1 values(N'Account Number',N'Account Type')
These will not convert into a BIGINT.
Even if a value is NULL, it would still be OK, since the CONCAT will just ignore it, ie...
CONCAT(NULL,'111') will give 111.