I have for-each loop container in my SSIS master package
There is 'Execute Package Task'
under this 'Execute SQL Server Agent Job Task' is there.(It has 35 steps)
Problem:
SSIS master package is starting next iteration without completion of all the steps in 'Execute SQL Server Agent Job Task'.
Please suggest an approach to start the next iteration after completion of all the steps only.
Thanks in advance :)
I know I keep throwing C# solutions at SSIS problems but this is exactly what I do to prevent a job/process to run while a job is running.
I have a function that does this check and returns a boolean true/false. Use that result to determine whether to start the job or not.
public static bool isDatabaseUpdating()
{
List<string> jobs = new List<string>();
jobs.Add("[Insert name of Job here]");
jobs.Add("[Insert another job and so on here]");
string sql = "exec msdb.dbo.sp_help_job #execution_status = 1";
using (OleDbConnection conn = new OleDbConnection(cstr))
{
using (OleDbCommand cmd = new OleDbCommand(sql, conn))
{
cmd.CommandType = CommandType.Text;
cmd.Connection.Open();
OleDbDataReader rdr = cmd.ExecuteReader();
while (rdr.Read())
{
if (jobs.Contains(rdr["name"].ToString())) return true;
}
}
}
return false;
}
To use it set a SSIS Variable like this:
Dts.Variables["#variableName"].Value = isDatabaseUpdating();
And then in control flow set expression on path appropriately.
The real key to understanding this function is the SQL.
exec msdb.dbo.sp_help_job #execution_status = 1
That returns a dataset of jobs that are currently running.
Enhancement to your application
This is what your control flow will look like:
Related
I have procedure in Snowflake and would like to call it from my Timer Triggered Azure Function App.
That procedure expects a parameter which is of type string. Following is my code snippet to connect to Snowflake and calling that procedure with parameter.
using (IDbConnection conn = new SnowflakeDbConnection())
{
//Connect to Snowflake
conn.ConnectionString = Environment.GetEnvironmentVariable("SnowflakeConnection");
conn.Open();
using (IDbCommand cmd = conn.CreateCommand())
{
if (conn.State == ConnectionState.Open)
{
cmd.CommandText = "SP_Snowflake_Procedure";
//cmd.CommandType = CommandType.StoredProcedure;
var date = cmd.CreateParameter();
date.ParameterName = "RUNDATE";
date.DbType = DbType.String;
date.Value = "2018-01-01";
cmd.Parameters.Add(date);
using (IDataReader dr = cmd.ExecuteReader())
{
/****************
Logic to work on data
received from SP
*****************/
}
}
}
}
When control comes to cmd.ExecuteReader(), it's failing with error:
Snowflake.Data: SQL compilation error: syntax error line 1 at position 0 unexpected 'SP_Snowflake_Procedure'.
I don't understand this Snowflake, how to call a procedure. I had a thought of, it is way similar to MS SQL. But I am wrong. I couldn't even find proper related documents.
I could use same code without procedure call but simple SELECT statement and worked fine.
Suggest me any changes here.
I can't tell from the code if you're using the ODBC driver for Snowflake or the .NET driver for Snowflake. The ODBC driver supports more features than the .NET driver, but I think executing SPs should be supported in both.
You'll need to make the call using a SQL statement that executes a query (as opposed to methods that execute non-query). It will return a table with a single row with the return from the SP. It will contain a single column with the name of the SP and the scalar value of the SP (basically what would be returned to the SQL worksheet if run in the web UI).
Here's a sample SP to test in case you need a simple one:
create or replace procedure EchoString(stringValue string)
returns VARCHAR
language JavaScript
as
$$
// Note that variables passed to Snowflake stored procedures
// muat be all CAPITAL letters when used in the body of the
// procedure code.
return STRINGVALUE
$$;
--Run the stored procedure to echo the value.
call EchoString('Echo this string.');
Here's how to call the SP from a C# project using an ODBC connection:
OdbcConnection DbConnection = new OdbcConnection("DSN=Snowflake;pwd=******");
OdbcCommand DbCommandSetup = DbConnection.CreateCommand();
DbConnection.Open();
// These two lines are only required if you get a message about no running warehouse.
// It will depend on how your calling user is set up in Snowflake.
DbCommandSetup.CommandText = "use warehouse TEST;";
DbCommandSetup.ExecuteNonQuery();
OdbcCommand DbCommand = DbConnection.CreateCommand();
DbCommand.CommandText = "call TEST.PUBLIC.ECHOSTRING('Echo this string.')";
OdbcDataReader DbReader = DbCommand.ExecuteReader();
// Note: If you define a Snowflake SP, DB, or schema in mixed case without double quoting
// the name, Snowflake will uppercase it in the catalog. You can call it from here without
// converting to upper case as long as it's not double quoted (escaped \") in the string.
I have a C# ASP.NET MVC program which runs a stored procedure while loading a page. If I run the stored procedure from Microsoft SQL Server Management Studio, it takes 1 minute. However, if I try to run the same stored procedure from code, it times out.
I have added Connection Timeout=0 in web.config, but sometimes it works, sometimes not.
You can set the Timeout Command to 0 when you are calling stored procedure.
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
SqlCommand cmd= new SqlCommand(queryString, connection);
// Setting command timeout to 0 second
cmd.CommandTimeout = 0;
try
{
cmd.ExecuteNonQuery();
}
catch(Exception ex)
{
// log ex here
}
}
What is the best way to update a table from a List inside an SSIS Script Task?
We have a shared class library. I have used the dll in the script task to do most of the necessary work. The dll method then returns a List which contains data related to the processes that it ran. It is my job to write this list to a table.
I'm thinking I will loop through each item in the List and and run the update SQL statement.
For brevity, I did not paste the SQL statement, but it is actually an Upsert using MERGE.
Actually, I wish there were a way to output the List to the input of an Execute SQL Task, but I'm unsure if that is possible.
Here is what I have so far. As you can see it is unfinished.
private void UpdateEtlData(List<ProcessStatitics> statistics)
{
var connection = GetOhioConnectionString();
// will loop thru each item in statistics and run the
// following sequence. This code is unfinished, but
// I will use properties inside each statistic to form the
// query
foreach(statistic in statistics)
{
SqlCommand command = new SqlCommand();
command.Connection = connection;
command.CommandText = ""
}
}
The easiest thing for you to do here is to create your SQLCommand outside of the loop, and set it up with parameters to write your data. This blog post covers it well: http://csharp-station.com/Tutorial/AdoDotNet/Lesson06
Steps are:
// 1. declare command object with parameter
SqlCommand cmd = new SqlCommand(
"Insert into CityList (CityName) values (#City)", conn);
// 2. define parameters used in command object
SqlParameter param = new SqlParameter();
param.ParameterName = "#City";
// 3. add new parameter to command object
cmd.Parameters.Add(param);
// When you want to assign the value
param.Value = inputCity;
Then in your loop you can assign your value from the list to param.Value, and call command.ExecuteNonQuery
Currently I am using the SQLAdapter Update command to update changes from a DataTable to a SQL database. I want to know how to cache the changes for later if the server is not available. Is there a way to extract the SQL from the Update command and execute them later on the database?
sqlConn = New SqlConnection(AppConnection)
sqlConn.Open()
sqlAdapter = New SqlDataAdapter
sqlAdapter.SelectCommand = New SqlCommand
sqlAdapter.SelectCommand.Connection = sqlConn
sqlAdapter.SelectCommand.CommandText = "SELECT * FROM SampleTable"
Dim objSQLBuilder As SqlCommandBuilder = New SqlCommandBuilder(sqlAdapter)
'THIS IS NOT RETURNING THE COMPLETE SQL STATEMENT
sqlAdapter.UpdateCommand = objSQLBuilder.GetUpdateCommand()
'IF THIS FAILS, I WANT TO CACHE THE SQL AND EXECUTE THE SQL LATER...
sqlAdapter.Update(dtDataTable)
Thank you for any help!
If I restore from Sql Server no problem, but if I do it through my application, the database is stuck in "restoring state".
I found some advice saying to put noRecovery = false, but this didn't change anything.
If I remove the "with move" option, it works: after the restore the DB is in a normal state.
The thing that I would like to understand is: does "with move" modify a sql server table?
Because if I launch the restore the first time without "with move" it says that he could not find the specified path. Otherwise, if I launch the restore with this option, and then one second time without it, it works. So there must be some tables that sql server uses to map the logical name with a physical path, how can I modify this table?
Here is the code:
SqlConnection sqlConnection = new SqlConnection(string.Format("Data Source={0};Initial Catalog={1};Integrated Security=True", database.SqlServerId, database.Name));
ServerConnection connection = new ServerConnection(sqlConnection);
Server sqlServer = new Server(connection);
Restore rstDatabase = new Restore();
rstDatabase.Action = RestoreActionType.Database;
rstDatabase.Database = backupFile.Name;
BackupDeviceItem bkpDevice = new BackupDeviceItem(backupFile.FileName, DeviceType.File);
rstDatabase.Devices.Add(bkpDevice);
rstDatabase.ReplaceDatabase = true;
rstDatabase.NoRecovery = false;
string dbLogicalName = "";
string logLogicalName = "";
sqlConnection.Open();
SqlCommand command = new SqlCommand(string.Format("RESTORE FILELISTONLY FROM DISK = '{0}'", backupFile.FileName), sqlConnection);
SqlDataReader reader = command.ExecuteReader();
if (reader.HasRows)
{
while (reader.Read())
{
if (reader.GetString(2) == "D")
dbLogicalName = reader.GetString(0);
if (reader.GetString(2) == "L")
logLogicalName = reader.GetString(0);
}
}
reader.Close();
rstDatabase.RelocateFiles.Add(new RelocateFile(dbLogicalName, backupFile.DatabaseFile));
rstDatabase.RelocateFiles.Add(new RelocateFile(logLogicalName, backupFile.LogsFile));
//Restore
rstDatabase.SqlRestore(sqlServer);
rstDatabase.Devices.Remove(bkpDevice);
sqlConnection.Close();
connection.Disconnect();
The thing that I would like to understand is: does "with move" modify a sql server table?
Not necessarily, unless you modify the default databases path in the instance, which you could do in SSMS following these steps:
https://learn.microsoft.com/en-us/sql/database-engine/configure-windows/view-or-change-the-default-locations-for-data-and-log-files
But as someone said, try to fix that on your own application. Do not attempt to modify this on the system database. It could bring unexpected results if you do that.
I think you know that, but WITH MOVE tells the SQL Server to reallocate the restored database in different path than the default path.