I have procedure in Snowflake and would like to call it from my Timer Triggered Azure Function App.
That procedure expects a parameter which is of type string. Following is my code snippet to connect to Snowflake and calling that procedure with parameter.
using (IDbConnection conn = new SnowflakeDbConnection())
{
//Connect to Snowflake
conn.ConnectionString = Environment.GetEnvironmentVariable("SnowflakeConnection");
conn.Open();
using (IDbCommand cmd = conn.CreateCommand())
{
if (conn.State == ConnectionState.Open)
{
cmd.CommandText = "SP_Snowflake_Procedure";
//cmd.CommandType = CommandType.StoredProcedure;
var date = cmd.CreateParameter();
date.ParameterName = "RUNDATE";
date.DbType = DbType.String;
date.Value = "2018-01-01";
cmd.Parameters.Add(date);
using (IDataReader dr = cmd.ExecuteReader())
{
/****************
Logic to work on data
received from SP
*****************/
}
}
}
}
When control comes to cmd.ExecuteReader(), it's failing with error:
Snowflake.Data: SQL compilation error: syntax error line 1 at position 0 unexpected 'SP_Snowflake_Procedure'.
I don't understand this Snowflake, how to call a procedure. I had a thought of, it is way similar to MS SQL. But I am wrong. I couldn't even find proper related documents.
I could use same code without procedure call but simple SELECT statement and worked fine.
Suggest me any changes here.
I can't tell from the code if you're using the ODBC driver for Snowflake or the .NET driver for Snowflake. The ODBC driver supports more features than the .NET driver, but I think executing SPs should be supported in both.
You'll need to make the call using a SQL statement that executes a query (as opposed to methods that execute non-query). It will return a table with a single row with the return from the SP. It will contain a single column with the name of the SP and the scalar value of the SP (basically what would be returned to the SQL worksheet if run in the web UI).
Here's a sample SP to test in case you need a simple one:
create or replace procedure EchoString(stringValue string)
returns VARCHAR
language JavaScript
as
$$
// Note that variables passed to Snowflake stored procedures
// muat be all CAPITAL letters when used in the body of the
// procedure code.
return STRINGVALUE
$$;
--Run the stored procedure to echo the value.
call EchoString('Echo this string.');
Here's how to call the SP from a C# project using an ODBC connection:
OdbcConnection DbConnection = new OdbcConnection("DSN=Snowflake;pwd=******");
OdbcCommand DbCommandSetup = DbConnection.CreateCommand();
DbConnection.Open();
// These two lines are only required if you get a message about no running warehouse.
// It will depend on how your calling user is set up in Snowflake.
DbCommandSetup.CommandText = "use warehouse TEST;";
DbCommandSetup.ExecuteNonQuery();
OdbcCommand DbCommand = DbConnection.CreateCommand();
DbCommand.CommandText = "call TEST.PUBLIC.ECHOSTRING('Echo this string.')";
OdbcDataReader DbReader = DbCommand.ExecuteReader();
// Note: If you define a Snowflake SP, DB, or schema in mixed case without double quoting
// the name, Snowflake will uppercase it in the catalog. You can call it from here without
// converting to upper case as long as it's not double quoted (escaped \") in the string.
Related
I am using dependency injection with Entity Framework Core and the context is created through a scope factory:
using (var scope = this._scopeFactory.CreateScope())
{
var context = scope.ServiceProvider.GetRequiredService<thecontext>();
//run code here
}
A transaction is started after all the objects are created:
using (var trans = context.Database.BeginTransaction())
{
try
{
This works well but during the process I need to insert records by running a stored procedure that is outside the transaction. In other words even if the transaction is aborted I still need the results of this stored procedure insert to persist. In addition any inserts from the 'isolated' stored procedure must be available to the process running under the transaction as well as outside the current scope.
The normal method of executing the stored procedure is by getting the connection and attaching the transaction to a new command. But I need to either use a new connection that is outside the current scope so it is not bound by the transaction or perhaps there is another way?
string sql = $"theProc";
var cnn = _context.Database.GetDbConnection();
if (cnn.State == ConnectionState.Closed)
cnn.Open();
DbCommand cmd = cnn.CreateCommand();
cmd.CommandText = sql;
cmd.CommandType = CommandType.StoredProcedure;
if (_context.Database.CurrentTransaction != null)
cmd.Transaction = _context.Database.CurrentTransaction.GetDbTransaction();
I ended up doing something like:
var cnnstr = _context.Database.GetConnectionString();
var isolatedCnn = new SqlConnection(cnnstr);
isolatedCnn.Open();
using (isolatedCnn)
{
This works but is ugly for a number of reasons so I'd like to find a better solution.
My ultimate goal is to run sql queries against sql-server and capture the returned data in a spreadsheet. The following code roughly reflects my current set-up and it works. The design allows me to read sql codes from text files and submit it to a sql-server. "Sub ExecuteCRUD" submits a first sql script to prepare data and dumps the result into a temp table. "Function loadRecordset" submits a relatively simple select query and captures the returned data in a recordset, which I then use to populate a spreadsheet.
There are a couple "variables" in my setup that could potentially be relevant for discussion.
My 4 set of Sql Codes
The ConnectionString (Part of my vba code)
The rest of my vba codes
a. Dbo.ConnectionString = "Provider=MSDASQL;DRIVER=SQL
Server;SERVER=myserver;UID=id;PWD=password;DATABASE=database;"
b. Dbo.ConnectionString = "Provider=SQLOLEDB;Data
Source=myserver;Initial Catalog=database;User
ID=id;Password=password;"
Initially, all I changed was the connection string. The immediate result was connection string version a works perfectly. Using b version, my setup would fail without any errors from sql-server.
Using Connection string version b, "Sub ExecuteCRUD" (data preparation step) would still work smoothly. I can verify that the temp table is created in tempdb as a result of my first sql script.
"Function loadRecordset" would run through the lines without any errors up to and including "rs.open". (I checked for errors, none whatsoever from the ado connection).
Only subsequent codes, when using the recordset to copy out the data would get an error: "Operation is not allowed when the object is closed."
Through some testing, I narrowed down the issue to the sql codes, sort of.
I have to reiterate here. My initial set of sql codes worked completely fine when using the ODBC provider. Things only went sideways using the OLEDB provider.
Using the OLEDB provider, the "offending" sql code was Use databaseABC. Furthermore, using ado, my setup submits 4 sets of sql codes to the sql server. The first set of sql codes prepares data (creating tables, inserting data, creating index, using while loops to populate data, using recursive ctes, etc). In this first set of sql codes, Use databaseABC was also included, and it would execute successfully. The other 3 set of sql codes submitted were only select queries aimed at obtaining data. When Use databaseABC was included in the select query sql codes, the operation failed without any errors. After I took out the Use databaseABC, everything would run correctly in the OLEDB provider world.
Use databaseABC is not a necessary part of the select queries. Using it saves me the trouble of specifying database names all the time in the join clauses.
At this point, my curiosity is two fold
why Use databaseABC causes failures, specifically only when using OLEDB provider, more specifically only when running select queries.
When the failure occurred, should sql-server or the driver generate any errors? What would be the proper way of checking and verifying that?
Private Dbo As New ADODB.Connection
Private rs As ADODB.Recordset
Public Sub ConnectServer()
If Dbo.State = adStateClosed Then
Dbo.ConnectionString = "Provider=SQLOLEDB;Data Source=*server*;" _
& "Initial Catalog=*database*;User ID=*id*;Password=*pwd*;"
Dbo.Open
End If
End Sub
Public Sub ExecuteCRUD(ByVal Sql As String)
On Error GoTo PANIC
Dbo.Execute Sql
Exit Sub
PANIC:
Debug.Print Err.Descript
Stop
End Sub
Public Function loadRecordset(ByVal Sql As String) As Long
On Error GoTo PANIC
Set rs = New ADODB.Recordset
With rs
.CursorLocation = adUseClient
.CursorType = adOpenStatic
.LockType = adLockReadOnly
.Source = Sql
Set .ActiveConnection = Dbo
End With
rs.Open
loadRecordset = rs.RecordCount
Exit Function
PANIC:
Debug.Print Err.Description
loadRecordset = 0
Stop
End Function
What is the best way to update a table from a List inside an SSIS Script Task?
We have a shared class library. I have used the dll in the script task to do most of the necessary work. The dll method then returns a List which contains data related to the processes that it ran. It is my job to write this list to a table.
I'm thinking I will loop through each item in the List and and run the update SQL statement.
For brevity, I did not paste the SQL statement, but it is actually an Upsert using MERGE.
Actually, I wish there were a way to output the List to the input of an Execute SQL Task, but I'm unsure if that is possible.
Here is what I have so far. As you can see it is unfinished.
private void UpdateEtlData(List<ProcessStatitics> statistics)
{
var connection = GetOhioConnectionString();
// will loop thru each item in statistics and run the
// following sequence. This code is unfinished, but
// I will use properties inside each statistic to form the
// query
foreach(statistic in statistics)
{
SqlCommand command = new SqlCommand();
command.Connection = connection;
command.CommandText = ""
}
}
The easiest thing for you to do here is to create your SQLCommand outside of the loop, and set it up with parameters to write your data. This blog post covers it well: http://csharp-station.com/Tutorial/AdoDotNet/Lesson06
Steps are:
// 1. declare command object with parameter
SqlCommand cmd = new SqlCommand(
"Insert into CityList (CityName) values (#City)", conn);
// 2. define parameters used in command object
SqlParameter param = new SqlParameter();
param.ParameterName = "#City";
// 3. add new parameter to command object
cmd.Parameters.Add(param);
// When you want to assign the value
param.Value = inputCity;
Then in your loop you can assign your value from the list to param.Value, and call command.ExecuteNonQuery
I am using ServiceStack.Ormlite for SQL Server and just updated from 3.9.71 to 4.0.33.0 and now transactions for direct commands are failing. I can get ORMlite transactions working or direct commands, but not both.
The complication is I am doing some very complicated DB commands and since Sql.In() for a large list of GUIDs is massively slow I have a workaround which uses db.CreateCommand() and then passes the GUID list in as a custom table type.
Thus I need a single transaction to span across ORMLite commands and direct db commands. The following code used to work and now it fails.
For instance the following code used to work. I now get errors saying that the CreateCommand() should use the transaction. When I directly try then I get the indicated cast exception:
using (var db = DB.Connection.OpenDbConnection())
{
using (var transaction = db.OpenTransaction())
{
// Some ORMLite code
db.Delete<SomeType>();
using (var command = db.CreateCommand())
{
// Direct DB command
command.CommandText = "Delete from SomeTable where ...";
command.Parameters.Add(GUIDList)
command.ExecuteNonQuery();
}
}
}
Clarification:
In the code OpenTransaction() will work for the OrmLite code, but fail on the CreateCommand code. BeginTransaction() will fail for the OrmLite code.
The actual error is at command.ExecuteNonQuery(): ExecuteNonQuery requires the command to have a transaction when the connection assigned to the command is in a pending local transaction. The Transaction property of the command has not been initialized.
To use Transactions in OrmLite you should use the OpenTransaction() API, e.g:
using (var trans = db.OpenTransaction())
{
//...
}
I've added a couple new API's to be able to use an OrmLite transaction with a raw ADO.NET IDbCommand in this commit.
Use a managed OrmLite DB Command
Use a managed OrmLite command with OpenCommand() which will automatically prepopulate the DB command with the current transaction, e.g:
using (var trans = db.OpenTransaction())
using (var command = db.OpenCommand())
{
command.CommandText = "Delete from SomeTable where ...";
}
Manually assign underlying DB Transaction
When using the underlying ADO.NET IDbCommand you will need to also manually assign the Transaction to the command yourself, i.e:
using (var trans = db.OpenTransaction())
using (var command = db.CreateCommand())
{
command.Transaction = trans.ToDbTransaction();
command.CommandText = "Delete from SomeTable where ...";
}
The ToDbTransaction() extension method lets you access the underlying ADO.NET IDbTransaction which is required when using the underlying ADO.NET IDbCommand.
Both of these new API's are available from v4.0.34+ that's now available on MyGet.
Here is my suggestion that works.It is based on previous answers
IDbConnection conn = DB.Connection;
IDbCommand cmd = conn.CreateCommand();
using (IDbTransaction transaction = conn.OpenTransaction())
{
//ADO.NET code
cmd.Transaction = transaction.ToDbTransaction();
cmd.CommandText = "...Some sql text";
cmd.executeNonQuery();
// Some ORMLite code
conn.Delete<SomeType>();
}
I have a stored procedure
CREATE procedure [dbo].[get_unique_identifier]
AS
DECLARE #ret_val INT
UPDATE seq SET #ret_val = id = id + 1
RETURN #ret_val
that queries a table (seq) that has a single int column and single row, increments the value and then then returns it. Don't ask why I'm doing this, but in short, the idea is to simulate a PostgreSQL sequence and no, an identity column would not do the job. Anyway, this works fine in SQL Management Studio, where sequential executions of
DECLARE #returned INT
EXEC #returned = get_unique_identifier
SELECT #returned
produce the expected output. Unfortunately, I can't seem to get the returned value in the application I'm working on
OdbcCommand command = new OdbcCommand("get_unique_identifier");
command.CommandType = CommandType.StoredProcedure;
OdbcParameter return_param = new OdbcParameter("#RETURN_VALUE", OdbcType.BigInt);
return_param.Direction = ParameterDirection.ReturnValue;
command.Parameters.Add(return_param);
Util.SQLExecuteParameterizedNonQuery(command);
Console.WriteLine(command.Parameters["#RETURN_VALUE"].Value.ToString());
The output is an empty string, but the value itself is DBNull.Value. The OdbcType.BigInt is left over from some testing I was doing. It was initially Int.
Edit: This is clearly a bug with ODBC. A workaround is posted below. Don't use ODBC if you don't have do.
I am sorry to have to say, but this might be due to using the Odbc
I tried this using out SqlClient database object and it returned the values as expected.
SqlCommand command = new SqlCommand("ZZZ_get_unique_identifier");
command.CommandType = CommandType.StoredProcedure;
SqlParameter return_param = new SqlParameter("#RETURN_VALUE",SqlDbType.BigInt);
return_param.Direction = ParameterDirection.ReturnValue;
command.Parameters.Add(return_param);
SqlConnection con = new SqlConnection(dbM.DefaultConnectionString);
con.Open();
command.Connection = con;
command.ExecuteNonQuery();
int i = Convert.ToInt32(command.Parameters["#RETURN_VALUE"].Value.ToString());
con.Close();
The return value of stored procedures is Int (32-bit), not BigInt (64-bit), but I'm not sure that this would be the problem with your code.
Have you not tried using an output variable instead...I am jogging my memory and think that the line command.Parameters["#RETURN_VALUE"].Value.ToString() could be this...The reason it worked in SQL Management Studio because you declared a variable and used it in an TSQL EXEC statement. I hope this answer somehow gives you a hint..
Hope this helps,
Best regards,
Tom.