I'm writing a sql script, and I'd like to use Management Studio to develop the query, and a C# program to run it in production.
My query contains parameters, like so;
SELECT * FROM TABLE
WHERE id = #id
I can feed in a value for #id in the C# program, and that works nicely. However, I also want to declare default values for testing in Management Studio. So I really want to write something like this pseudocode;
if not declared #id
declare #id int
set #id=43
end if
SELECT * FROM TABLE
WHERE id = #id
Is there any way to check to see if a variable name has already been taken?
You can't do exactly what you're after. I'd suggest either:
1) wrap the script up as a sproc and give defaults for the params
2) include a comment block at the top of the script that you can then uncomment when running in SSMS:
/*
-- Default variables for testing
DECLARE #Id INTEGER
SET #Id = 43
*/
SELECT * FROM Table WHERE id = #Id
I've managed to make some progress by marking out the default variables in the script, like so;
/** DEFAULTS **/
declare #id int
set #id = 43
/** END DEFAULTS **/
Then preprocessing the script in my C# program, like so;
script = RemoveBlock(script, "DEFAULTS");
And implementing the function like so;
public static string RemoveBlock(string script, string blockName)
{
if (script == null) { return null; }
var startTag = string.Format("/** {0} **/", blockName);
var endTag = string.Format("/** END {0} **/", blockName);
var startTagIdx = script.IndexOf(startTag);
if (startTagIdx == -1) { return script; }
var endTagIdx = script.IndexOf(endTag, startTagIdx + startTag.Length);
if (endTagIdx == -1) { return script; }
var endOfEndTag = endTagIdx + endTag.Length;
var beforeBlock = script.Substring(0, startTagIdx);
var afterBlock = script.Substring(endOfEndTag);
return beforeBlock + afterBlock;
}
So the C# program runs a version without the variables but with parameters.
Related
I am copying data from a Snowflake table into an S3 external stage:
COPY INTO '#my_stage/my_folder/my_file.csv.gz' FROM (
SELECT *
FROM my_table
)
However this code runs daily and I don't want to overwrite my_file.csv.gz but rather keep all the historical versions. However I haven't found a way to create dynamic paths:
SET stage_name=CONCAT('#my_stage/my_folder/my_file', '_date.csv.gz');
COPY INTO $stage_name FROM (
SELECT *
FROM my_table
);
COPY INTO IDENTIFIER($stage_name) FROM (
SELECT *
FROM my_table
);
None of the later 2 queries work!
My question: How can I create dynamic Stage paths in Snowflake? Thanks
Here's a stored procedure you can use and modify. Note that the line with the comment to modify your copy into statement uses backticks instead of single or double quotes. In JavaScript, that allows use of single or double quotes in the string, multi-line constants, and replacement tokens in the form ${variable_name}
create or replace procedure COPY_TO_STAGE(PATH string)
returns variant
language javascript
as
$$
class Query{
constructor(statement){
this.statement = statement;
}
}
// Start of main function
var out = {};
// Change your copy into statement here.
var q = getQuery(`copy into '${PATH}' from (select * from my_table);`);
if (q.resultSet.next()) {
out["rows_unloaded"] = q.resultSet.getColumnValue("rows_unloaded");
out["input_bytes"] = q.resultSet.getColumnValue("input_bytes");
out["output_bytes"] = q.resultSet.getColumnValue("output_bytes");
} else {
out["Error"] = "Unknown error";
}
return out;
// End of main function
function getQuery(sql){
cmd1 = {sqlText: sql};
var query = new Query(snowflake.createStatement(cmd1));
query.resultSet = query.statement.execute();
return query;
}
$$;
Once you define it, you can use SQL variables as the input if you want:
SET stage_name=CONCAT('#my_stage/my_folder/my_file', '_date.csv.gz');
call copy_to_stage($stage_name);
This won't work. Unfortunately using variables for identifiers does not work for stages. You might need to create a Stored procedure with Dynamic SQL:
https://docs.snowflake.com/en/sql-reference/stored-procedures-usage.html#label-example-of-dynamic-sql-in-stored-procedure
So you can just call this procedure every day or generating a SP with several parameters for the path (Stage), the query which will be executed and the target filename.
(Submitting on behalf of a Snowflake User...)
QUESTION:
Is it possible to nest multiple functions inside of a function and pass all the parameters required?
for example...
CREATE OR REPLACE FUNCTION "udf_InteractionIndicator"("ROH_RENEWAL_SYSTEM_STATUS1" VARCHAR(100), "GOLS_OPPORTUNITY_LINE_STATUS" VARCHAR(100)
, "ROH_CLIENT_CURRENT_TEMPERATURE1" VARCHAR(100)
, "ROH_PO_ATTACHED" VARCHAR(100)
, "ROH_PO_NUMBER" VARCHAR(100)
, "RT_PAID_OVERRIDE" VARCHAR(100), "ROH_RENEWAL_OPPORTUNITY_STATUS1" VARCHAR(100)
, "ROH_RENEWAL_CONVERSATION_DATE" DATE, "ROH_APPROVAL_RECEIVED_DATE" DATETIME)
RETURNS NUMBER(1,0)
AS
$$
CASE WHEN ("udf_RenewalNoticeSentIndicator"("ROH_RENEWAL_SYSTEM_STATUS1", "ROH_CLIENT_CURRENT_TEMPERATURE1"
, "GOLS_OPPORTUNITY_LINE_STATUS"
, "ROH_PO_ATTACHED", "RT_PAID_OVERRIDE"
, "ROH_RENEWAL_OPPORTUNITY_STATUS1")) = 1
AND (ROH_RENEWAL_CONVERSATION_DATE IS NOT NULL
OR ("udf_AuthorizedIndicator"(ROH_APPROVAL_RECEIVED_DATE, "ROH_PO_ATTACHED", "ROH_PO_NUMBER")) = 1
OR ("udf_PaidIndicator"("GOLS_OPPORTUNITY_LINE_STATUS")) = 1
OR ("udf_ChurnIndicator"("GOLS_OPPORTUNITY_LINE_STATUS")) = 1
)
THEN 1 ELSE 0 END
$$
;
I've received the recommendation to:
...create a SQL UDF or JavaScript UDF. A JavaScript UDF can only
contain JavaScript code, and an SQL UDF can contain only one SQL
statement (no DML and DDL). In case of nesting, SQL UDF can call
another SQL UDF or a JavaScript UDF but the same is not true with the
JavaScript UDF(it only contains JavaScript code).
CREATE OR REPLACE FUNCTION udf_InteractionIndicator_nested(ID DOUBLE)
RETURNS DOUBLE
AS
$$
SELECT ID
$$;
create or replace function js_factorial(d double)
returns double
language javascript
strict
as '
if (D <= 0) {
return 1;
} else {
var result = 1;
for (var i = 2; i <= D; i++) {
result = result * i;
}
return result;
}
';
CREATE OR REPLACE FUNCTION udf_InteractionIndicator(ID DOUBLE)
RETURNS double
AS
$$
select udf_InteractionIndicator_nested(ID) + js_factorial(ID)
$$;
select udf_InteractionIndicator(4);
+-----------------------------+
| UDF_INTERACTIONINDICATOR(4) |
|-----------------------------|
| 28 |
+-----------------------------+
HOWEVER, I'm trying to accomplish this with a SQL UDF. It makes sense that a nested function can be created as long as they use the same parameter. I'd like to create a function that accepts say 8 parameters and the underlying functions may reference all, some or none of the parent function parameters. That is where I run into an issue... THOUGHTS?
(A consultant in our community offered the following answer...)
With a JavaScript UDF the design will be much more compact and maintainable, if your use case is that there is a "main" function that breaks down work into subfunctions which will only be invoked from main.
Then you simply define all underlying functions within the main function, which is possible with JavaScript but not with an SQL UDF, and then you are free to use the main parameters everywhere.
CREATE OR REPLACE FUNCTION MAIN_JS(P1 STRING, P2 FLOAT)
RETURNS FLOAT
LANGUAGE JAVASCRIPT
AS '
function helper_1(p) { return p * 2; }
function helper_2(p) { return p == "triple" ? P2 * 3 : P2; }
return helper_1(P2) + helper_2(P1);
';
SELECT MAIN_JS('triple', 4); -- => 20
I am using Dapper's QueryMultipleAsync method to read multiple result set. My stored procedure checks some conditions (for example a user might be trying to get someone else's data by sending an Id to my API) to determine whether the data should return or not.
I, on C# side, first need to read the return value (not result set) to determine if data is returned or SP simply returned 3 (which means insufficient rights). To illustrate the case:
IF #UserRole < 10
BEGIN
RETURN 3; -- Insufficient rights.
END
IF #IsCurrentUserOwner = 0
BEGIN
RETURN 5; -- Not owner.
END
-- Get users.
SELECT
Id
, [Name]
, LastName
FROM
Users
-- WHERE ...
-- Get chat messages.
SELECT
*
FROM
ChatMessages
-- WHERE ...
I know that output and return values are written at the end of reader so I can only read return parameter when all the data (result set) is read. So, I always have to read the result set first then return/output parameters.
What if my SP looked like this:
-- ...some code above...
IF #IsCurrentUserOwner = 0
BEGIN
RETURN 5; -- Not owner.
END
DECLARE #RType TINYINT = NULL
-- some other code here to get #RType value here...
SELECT #RType -- To make this a result so QueryMultiple's reader can read this.
-- ...some other code to get users and chat messages...
To describe the problem:
#RType variable could be 5 as well as SP's return value. When I read the result first (because output/return parameters are at the end of reader), how do I know the value I just read (which is 5 in this case) is the #RType or return value? (they are convertible)
This is how my C# code roughly looks like:
var parameters = new DynamicParameters();
parameters.Add("#RetVal", dbType: DbType.Int32, direction: ParameterDirection.ReturnValue);
var dbResult = await Context.SqlConnection.QueryMultipleAsync(sql: "my_sp_name", param: parameters, commandType: CommandType.StoredProcedure)
// Some code here...
using (var reader = dbResult)
{
var rType = reader.Read<byte>(); // <----- How to know if I read #RType or just return value of SP since they are convertible to each other?
var users = reader.Read<User>();
var chatMessages = reader.Read<ChatMessage>();
// ...
}
var returnValue = parameters.Get<int>("#RetVal");
if (returnValue == 5)
{
return "You are not allowed to see this data";
}
How do you recommend me to handle the case?
Your C# code uses async - await (QueryMultipleAsync) and currently Dapper doesn't support for Async methods that no perform a SELECT statement.
There is an open issue about that in Github:
https://github.com/StackExchange/Dapper/issues/591
Option 1
You can split your stored procedure into 2 parts:
1. Check the Return value.
2. Getting the select statements.
Option 2
ALTER PROCEDURE dbo.sp
... your params..
#Output INT OUTPUT
AS
IF #UserRole < 10
BEGIN
SET #Output = 3; -- Insufficient rights.
END
IF #IsCurrentUserOwner = 0
BEGIN
SET #Output = 5; -- Not owner.
END
-- Get users.
SELECT
Id
, [Name]
, LastName
FROM
Users
-- WHERE ...
-- Get chat messages.
SELECT
*
FROM
ChatMessages
-- WHERE ...
Option 3
Use ADO.NET
I wasn't really clear on your question, but the snippet below should work. Since you are defining the #RetVal as ParameterDirection.ReturnValue you should be able to access it before accessing the actual readers. I was able to verify this works in SQL SERVER.
var returnValue = parameters.Get<int>("#RetVal");
if (returnValue == 5 || returnValue == 3)
{
return "You are not allowed to see this data";
}
using (var reader = dbResult)
{
var rType = reader.Read<byte>(); // <----- How to know if I read #RType or just return value of SP since they are convertible to each other?
var users = reader.Read<User>();
var chatMessages = reader.Read<ChatMessage>();
// ...
}
here is my stored procedure
ALTER PROCEDURE mysp_TestOne
#p1 int
AS
BEGIN
SET NOCOUNT ON;
if #p1 > 0
return 5;
SELECT 1 F1
union
Select 2 F1;
SELECT 1 F1
union
Select 2 F1;
END
GO
and more complete snippet of c#;
using (SqlConnection cn = new SqlConnection(connectionString))
{
var sql = #"mysp_TestOne";
var parameters = new DynamicParameters();
// #p1 triggers whether to return query results or just the returnvalue
parameters.Add("p1", 0, direction: ParameterDirection.Input);
parameters.Add("#RetVal", 0, direction: ParameterDirection.ReturnValue);
var dbResult = await cn.QueryMultipleAsync(sql, parameters, commandType: CommandType.StoredProcedure);
var returnValue = parameters.Get<int>("#RetVal");
if (returnValue == 5 || returnValue == 3)
{
return "You are not allowed to see this data";
}
using (var reader = dbResult)
{
var rType = reader.Read<byte>(); // <----- How to know if I read #RType or just return value of SP since they are convertible to each other?
var users = reader.Read<User>();
var chatMessages = reader.Read<ChatMessage>();
// ...
}
}
HTH
I have a stored procedure that works fine but it has inside it three "select"s.
The selects are not from an inner temporary table.
This is mainly the format of the procedure:
ALTER PROCEDURE [dbo].[STProce]
#param1 int,
#param2 int,
#param3 int,
#param4 int,
#param5 int
AS
select #param1 as p1, #param2 as p2, #param3 as p3
.
.
.
select #param4 as p4
.
.
.
select #param5 as p5
I'm executing the procedure from another procedure and need to catch it there.
I created a table and inserts into it the "exec" from the procedure, like that:
CREATE TABLE #stalledp
(
RowNumber INT,
fldid INT,
fldLastUpdated datetime,
fldCreationDate datetime,
fldName nvarchar(255),
fldPending nvarchar(255)
)
INSERT INTO #stalledp (RowNumber,fldid,fldLastUpdated,fldCreationDate,fldName,fldPending)
EXEC spDebuggerViews_GetStuckWorkflowInstances #workflowSpaceId='00000000-0000-0000-0000-000000000000',#pageNum=1,#pageSize=100000,#orderByColumn=N'fldid',#sortOrder=1,#workflowInstanceId=0,#stuckInstanceType=1,#createdDateFrom='1900-01-01 00:00:00',#createdDateTo='9999-01-01 23:59:59',#updatedDateFrom='1900-01-01 00:00:00',#updatedDateTo='9999-01-01 23:59:59'
Afterwards I receive this error:
Column name or number of supplied values does not match table definition.
The order and name of columns of the table is exactly like the procedure returns.
Is there a possibility to catch only one of the tables that the procedure returns and avoid the other? I cannot change the procedure at all.
I tried declaring a table the same fields as the first select of the procedure and I get an error says that
Thank you in advance!
If all of the result sets returned are of the same structure, then you can dump them to a temp table as you are trying to do. However, that only gets you so far because if the data in the fields cannot be used to determine which result set a particular row came from, then you just have all of the result sets with no way to filter out the ones you don't want.
The only way to interact with multiple result sets individually, regardless of them having the same or differing structures, is through app code (i.e. a client connection). And if you want to do this within the context of another query, then you need to use SQLCLR.
The C# code below shows a SQLCLR stored procedure that will execute a T-SQL stored procedure that returns 4 result sets. It skips the first 2 result sets and only returns the 3rd result set. This allows the SQLCLR stored procedure to be used in an INSERT...EXEC as desired.
The code for the T-SQL stored proc that is called by the following code is shown below the C# code block. The T-SQL test proc executes sp_who2 and only return a subset of the fields being returned by that proc, showing that you don't need to return the exact same result set that you are reading; it can be manipulated in transit.
C# SQLCLR proc:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
public class TheProc
{
[Microsoft.SqlServer.Server.SqlProcedure]
public static void Get3rdResultSetFromGetStuckWorkflowInstances()
{
int _ResultSetsToSkip = 2; // we want the 3rd result set
SqlConnection _Connection = null;
SqlCommand _Command = null;
SqlDataReader _Reader = null;
try
{
_Connection = new SqlConnection("Context Connection = true;");
_Command = _Connection.CreateCommand();
_Command.CommandType = CommandType.StoredProcedure;
_Command.CommandText = "tempdb.dbo.MultiResultSetTest";
// (optional) add parameters (but don't use AddWithValue!)
// The SqlDataRecord will be used to define the result set structure
// and act as a container for each row to be returned
SqlDataRecord _ResultSet = new SqlDataRecord(
new SqlMetaData[]
{
new SqlMetaData("SPID", SqlDbType.Char, 5),
new SqlMetaData("Status", SqlDbType.NVarChar, 30),
new SqlMetaData("Login", SqlDbType.NVarChar, 128),
new SqlMetaData("HostName", SqlDbType.NVarChar, 128),
new SqlMetaData("BlkBy", SqlDbType.VarChar, 5),
new SqlMetaData("DBName", SqlDbType.NVarChar, 128)
});
SqlContext.Pipe.SendResultsStart(_ResultSet); // initialize result set
_Connection.Open();
_Reader = _Command.ExecuteReader();
// Skip a predefined number of result sets
for (int _Index = 0;
_Index < _ResultSetsToSkip && _Reader.NextResult();
_Index++) ;
// Container used to move 1 full row from the result set being read
// to the one being sent, sized to the number of fields being read
Object[] _TempRow = new Object[_Reader.FieldCount];
while (_Reader.Read())
{
_Reader.GetValues(_TempRow); // read all columns
_ResultSet.SetValues(_TempRow); // set all columns
SqlContext.Pipe.SendResultsRow(_ResultSet); // send row
}
}
catch
{
throw;
}
finally
{
if(SqlContext.Pipe.IsSendingResults)
{
SqlContext.Pipe.SendResultsEnd(); // close out result set being sent
}
if(_Reader != null && !_Reader.IsClosed)
{
_Reader.Dispose();
}
_Command.Dispose();
if (_Connection != null && _Connection.State != ConnectionState.Closed)
{
_Connection.Dispose();
}
}
return;
}
}
T-SQL test proc:
USE [tempdb]
SET ANSI_NULLS ON;
IF (OBJECT_ID('dbo.MultiResultSetTest') IS NOT NULL)
BEGIN
DROP PROCEDURE dbo.MultiResultSetTest;
END;
GO
CREATE PROCEDURE dbo.MultiResultSetTest
AS
SET NOCOUNT ON;
SELECT 1 AS [ResultSet], 'asa' AS [test];
SELECT 2 AS [ResultSet], NEWID() AS [SomeGUID], GETDATE() AS [RightNow];
EXEC sp_who2;
SELECT 4 AS [ResultSet], CONVERT(MONEY, 131.12) AS [CashYo];
GO
EXEC tempdb.dbo.MultiResultSetTest;
To do:
Adjust _ResultSetsToSkip as appropriate. If you only want the first result set, simply remove both _ResultSetsToSkip and the for loop.
Define _ResultSet as appropriate
Set _Command.CommandText to be "spDebuggerViews_GetStuckWorkflowInstances"
Create the necessary parameters via SqlParameter (i.e. #workflowSpaceId='00000000-0000-0000-0000-000000000000',#pageNum=1,#pageSize=100000,#orderByColumn=N'fldid',#sortOrder=1,#workflowInstanceId=0,#stuckInstanceType=1,#createdDateFrom='1900-01-01 00:00:00',#createdDateTo='9999-01-01 23:59:59',#updatedDateFrom='1900-01-01 00:00:00',#updatedDateTo='9999-01-01 23:59:59')
If needed, add input parameters to the SQLCLR proc so that they can be used to set the values of certain SqlParameters
Then use as follows:
INSERT INTO #stalledp
(RowNumber,fldid,fldLastUpdated,fldCreationDate,fldName,fldPending)
EXEC Get3rdResultSetFromGetStuckWorkflowInstances;
There is a way to get the first record set but the others, I'm afraid, you're out of luck.
EXEC sp_addlinkedserver #server = 'LOCALSERVER', #srvproduct = '',
#provider = 'SQLOLEDB', #datasrc = ##servername
SELECT * FROM OPENQUERY(LOCALSERVER, 'EXEC testproc2')
EDIT: If you only need to check the other result set for columns to be not null you could predefine the expected results sets like so:
EXEC testproc2 WITH RESULT SETS (
(a VARCHAR(MAX) NOT NULL, b VARCHAR(MAX) NOT NULL),
(a VARCHAR(MAX) NOT NULL)
);
If the query within the stored procedure returns null values a exception is raised at that point in procedure. This will only work on sql server 2012 and upwards though.
I'm updating a long list of records. In my code, everything run as predicted until it execute the query. I get an
Incorrect syntax near 'TempUpdatePhysicalCityStateZip'
(my stored procedure name). I've tested it with SQL Server Management Studio and it runs fine. So, I'm not quite sure where I got it wrong. Below is my stored procedure and code:
ALTER PROCEDURE [dbo].[TempUpdateCityStateZip]
#StoreNo nvarchar (11),
#City nvarchar(50),
#State nvarchar(2),
#Zip nvarchar(5)
AS
BEGIN
SET NOCOUNT ON;
UPDATE StoreContact
SET City = #City, State = #State, Zip = #Zip
WHERE StoreNo = #StoreNo
END
Here is my code:
Dictionary<string, string> CityStateZipList = getCityStateZipList(dbPath);
using (SqlConnection conn = new SqlConnection(dbPath))
{
conn.Open();
SqlCommand cmdUpdate = new SqlCommand("TempUpdateCityStateZip", conn);
foreach (KeyValuePair<string, string> frKeyValue in CityStateZipList)
{
cmdUpdate.Parameters.Clear();
string[] strCityStateZip = frKeyValue.Value.Split(' ');
cmdUpdate.Parameters.AddWithValue("StoreNo", frKeyValue.Key.ToString());
foreach (String i in strCityStateZip)
{
double zipCode;
if (i.Length == 2)
{
cmdUpdate.Parameters.AddWithValue("State", i);
}
else if (i.Length == 5 && double.TryParse(i, out zipCode))
{
cmdUpdate.Parameters.AddWithValue("Zip", i);
}
else
{
cmdUpdate.Parameters.AddWithValue("City", i);
}
}
cmdUpdate.ExecuteNonQuery();
}
}
I believe you can get that puzzling error message if you don't specify the command type:
cmdUpdate.CommandType = CommandType.StoredProcedure;
Don't you need the # sign before the parameter?
cmdUpdate.Parameters.AddWithValue("#State", i);
FWIW, Thats kind of a dirty piece of code there, you will probably have many issues trying to maintain that. For performance reasons you may want to parse out the CityStateZipList before you open the connection, that way you aren't keeping it open longer than you need.