What can I do so that EF becomes aware of the output clause in SPs and generates the result object accordingly?
INSERT INTO goodtable
(token,
ip, long_ip,
) OUTPUT INSERTED.*
VALUES
(#token,
#ip, #long_ip,
);
What I currently do to circumvent this is writing a dummy select, generate the objects and the comment out the dummy select leaving the output. This is not a good solution for a long term run.
Please do not suggest changing the SQL.
Have you tried. when importing the stored procedure into EF, using function import, to add the returns a collection of option.
I've never tried it, but can't see a reason for it not to work.
Related
I'm working with a masked database on my QA server using SQL Server Standard (64-bit) 14.0.1000.169. This is my structure:
CREATE TABLE [dbo].[Test](
[Column1] [VARCHAR(64)] NULL,
[Column2] [VARCHAR(64)] NULL
)
GO
INSERT INTO [dbo].[Test]
VALUES ('ABCDEFG', 'HIJKLMN')
I've masked the column with the following code:
ALTER TABLE [dbo].[Test]
ALTER COLUMN [Column1] VARCHAR(64) MASKED WITH (FUNCTION = 'default()');
It works as expected when I perform the following query using a non-allowed user:
SELECT [Column1], [Column2]
FROM [dbo].[Test]
FOR JSON PATH
-- RESULT: '[{"Column1":"xxxx", "Column2":"HIJKLMN"}]'
But it doesn't work when the same non-allowed user saves the result in variable (the main goal):
DECLARE #var VARCHAR(64)
SET #var = (SELECT [Column1], [Column2] FROM [dbo].[Test] FOR JSON PATH)
SELECT #var --it should show a valid JSON...
-- RESULT: 'xxxx' <-- JSON LOSES ITS STRUCTURE
-- DESIRED RESULT: '[{"Column1":"xxxx", "Column2":"HIJKLMN"}]' <-- VALID JSON
Main problem: JSON looses its structure when a masked column appear in the SELECT and "FOR JSON PATH" clause is present.
We want to get a valid JSON even if the data column is masked or not, or even if sa user or not.
I've tested using NVARCHAR or doing a CAST in the masked column, but the only way we get the desired result is using a #tempTable before use the "FOR JSON PATH" clause.
How can I do for SELECT a masked column and save it to
VARCHAR variables without loose JSON structure?
Any help will be appreciated.
NOTE: The SA user is default allowed to see unmasked data (so the JSON doesn't loose its structure), but we want to execute it on a non-allowed user and return a valid JSON, not only 'xxxx'.
It does indeed appear to be a bug. Repro is here. Although see below, not so sure.
When using FOR JSON, or for that matter FOR XML, as a top level SELECT construct, a different code path is used as compared to placing it in a subquery or assigning it to a variable. This is one of the reasons for the 2033-byte limit per row in a bare FOR JSON.
What appears to be happening is that in the case of a bare FOR JSON, the data masking happens at the top of the plan, in a Compute Scalar operator just before the JSON SELECT operator. So the masking happens on just the one column.
PasteThePlan
Whereas when putting inside a subquery, a UDX function operator is used. The problem is that the Compute Scalar is happening after the UDX has created the JSON or XML, whereas it should have been pushed down below the UDX in the plan.
PasteThePlan
I suggest you file the bug with Microsoft, on the Azure Feedback site.
Having gone over this a little, I actually think now that it's not a bug. What does seem to be a bug is the case without nesting.
From the documentation:
Whenever you project an expression referencing a column for which a data masking function is defined, the expression will also be masked. Regardless of the function (default, email, random, custom string) used to mask the referenced column, the resulting expression will always be masked with the default function.
Therefore, when you select any masked column, even in a normal SELECT, if you use a function on the column then the masking always happens after any other functions. In other words, the masking is not applied when the data is read, it is applied when it is finally output to the client.
When using a subquery, the data is fed into a UDX function operator. The compiler now senses that the final resultset is a normal SELECT, just that it needs to mask any final result that came from the masked column. So the whole JSON is masked as one blob, similar to if you did UPPER(yourMaskedColumn). See the XML plan in this fiddle for an example of that.
But when using a bare FOR JSON, it appears to the compiler as a normal SELECT, just that the final output is changed to JSON (the top-level SELECT operator is different). So the masking happens before that point. This seems to me to be a bug.
The bug is even more egregious when you use FOR XML, which uses the same mechanisms. If you use a nested FOR XML ..., TYPE then you get just <masked /> irrespective of whether you nest it or not. Again this is because the query plan shows the masking happening after the UDX. Whereas if you don't use , TYPE then it depends if you nest it. See fiddle.
Do any of you know of a command or function that would produce the line number of the query window inside of select statement? For example, if you are on line 34 it you would have select statement and the results would show 34. I am not looking to use row_number() function because we are not referencing a table.
You can't do what you want without altering the stored procedure and adding PRINT 'something' statements.
While this tactic can be useful for quick-and-dirty stored procedure debugging, it does not beautify the code and, is not considered sustainable. There are better methods. Maybe look at a tutorial like this: https://www.sqlshack.com/debugging-stored-procedures-sql-server-management-studio-ssms/
(Opening the following on behalf of a Snowflake client...)
When I try to insert into the table it threw below error:
Numeric value 'abc_0011O00001y31VpQAI' is not recognized
Have check the table DDL and found only 3 columns defined as NUMBER and rest as VARCHAR.
I checked the SELECT query and didnot find any string value in those NUMBER Datatype columns. Also tried searching in all the Varchar columns for the value 'abc_0011O00001y31VpQAI' , I didn't find any
I know one thing Snowflake doesn't always shows correct error. Am I missing anything here? Is there any way to fix it?
Both COL4_MRR and COL5_QUANTITY are NUMBER
INSERT INTO TABLE
(COL1_DATE, COL2_CD, COL3_CUST_NAME, COL3_LOC_NAME,COL4_MRR,COL5_QUANTITY)
SELECT
'2019-10-03' AS COL1_DATE ,
'AE' AS COL2_CD
,CUSTOMER_NAME AS COL3_CUST_NAME
,LOCATION_NAME AS COL3_LOC_NAME
,MRR_BILLED as COL4_MRR
,QTY_BILLED as COL5_QUANTITY
FROM SCHEMA.V_TABLEA
union all
SELECT
'2019-10-03' AS COL1_DATE ,
'BE' AS COL2_CD
,CUSTOMER_NAME AS COL3_CUST_NAME
,LOCATION_NAME AS COL3_LOC_NAME
,NULL as COL4_MRR
,QTY_BILLED as COL5_QUANTITY
FROM SCHEMA.V_TABLEB
I created a table_D same as original TABLE and tried inserting into it , it worked fine . Then Inserted into Original TABLE from table_D , it worked again .
Deleted those rows from original TABLE and reran the job , it worked fine.
There was no issue with data as all was Number only, I even tried with TRY_TO_NUMBER too. It inserted the data without any changes to the code.
...............
Client is currently waiting on a next day run to re-test to determine if this is either a bug or an issue with their data. In the meantime, we are interested to see if anyone else has run into similar challenges and have a viable recommendation. THANK YOU.
The error typically means you are trying to insert non-numeric data (like 'abc_0011O00001y31VpQAI') into a numeric column. It seems like the customer did everything right in testing and TRY_TO_NUMBER() is a great way to verify numeric data.
Do the SELECT queries run fine separately? If so, then I would check whether there might be a potential mismatch in the datatype of the columns and make sure they are in the right order.
I would also check whether or not the header is being skipped in the file (that may be where the 'abc_0011O00001y31VpQAI' is coming from since the customer did not see it in the data).
SELECT queries work fine, I tried creating a new table with same DDL as original and tried loading into that new table, it worked fine. Not sure why it is not loading into the original table
I have a table with two columns hora_entrada and hora_saida and when running this select statement:
select hora_entrada, hora_saida
from Controlo_de_Entrada_saidas
it shows this result:
What I want to do is to combine these two columns into one that if Hora_saida = "Não saiu", it shows the data in hora_entrada but if hora_saída has an actual hour in there, it shows the data in hora_saida instead.
So basically this should be my result:
I don't know if I'm making myself clear, I actually don't even know where to start or if its even possible but any help would be appreciated.
Try using a CASE expression:
SELECT
hora_entrada,
hora_saida,
CASE WHEN hora_saida = 'Não saiu'
THEN hora_entrada ELSE hora_saida END AS new_column
FROM yourTable;
This logic assumes that any value in the hora_saida column which is not Não saiu is a valid timestamp. If not, then we could add additional logic to check for this as well.
If you are open/able to changing your data model, you might want to consider just storing NULL values for the missing data. SQL Server (and most other databases as well) has a built-in function COALESCE, which can be used to replace NULL with a backup value. This would avoid needing to use a lengthy CASE expression as you see above.
I use CakePhp for a little application. In this application I store some store procedures (I use mssql). Now I want to execute theses store procedures and want to use the result set.
If I call a store procedure without any parameters i get want I want without any problems. But when i call another store procedure with parameters i get the error "Invalid cursor state"). I did some research and i understood that there it seemed to be that i execute multiple select because of the result-state.
In CakePhp i call:
This is running without any problems
$res = $db->query("Exec storeProcedureWithoutParams");
Here I get the error with the cursor
$res = $db->query("Exec storeProcedureWithParams 'param1'");
The store procedures are very simple:
CREATE PROCEDURE [dbo].[storeProcedureWithoutParams]
AS
SELECT [u1]
,[u2]
,[u3]
,[u4]
,[u5]
,[u6]
FROM [testdb].[dbo].[test]
CREATE PROCEDURE [dbo].[storeProcedureWithParams]
#test nvarchar(50)
AS
SELECT [u1]
,[u2]
,[u3]
,[u4]
,[u5]
,[u6]
FROM [testdb].[dbo].[test]
where [u6] <> #test
If I execute the both queries directly at the database I get two return. The first one is the result of the select-statement and the second one is the result-set. If I understood it right, is this the problem because i get two results (thats why the cursor-error). BUT I don't know how to solve it and I don't know how to get only the result of the select-statement without the return-state?
Can anybody help me?
Thank you very much!
Greetz
V
Sorry for the late answer, but I'll add it here for testing purposes, as there aren't any full answers for this question.
Not sure 100%, but seems that when there is some kind of message output along the query results, it gives the "invalid cursor state". Adding SET NOCOUNT ON it solved to me.