I have multiple SSIS projects, but some of the packages inside them are the same.
I would like to create a project with all the generic packages and keep the others projects with theirs specific packages.
So my question is : is it possible to have a master package that can execute and pass parent variables to packages from another project ?
I'm new to SSIS so sorry if it's an obvious question or if i'm not specific enough
When you configure an Integration Services catalog you can execute the packages from another project.
After deploying your all projects in the SSIS Catalog. Go to SSIS Catalog and browse to the package you want to execute.
Right click and select execute
It will pop up a window and ask for required paramter to fill up then
Click on the script menu and select New query editor window
Extra steps - Delete the Select #execution_id (not neccessary)
Extra steps - Delete the DECLARE #var0 smallint = 1 line and replace #var0 with 1 (in the line it will be #parameter_value=1 )
Do not close the sql query window as you need to copy the generated script
Now in your master package:
Add a Execute SQL Task
Add OLE DB connection manager to SSISDB database
In the Execute SQL Task Editor--> SQLstatement: paste here the generated package script
An example of the script:
Declare #execution_id bigint
EXEC [SSISDB].[catalog].[create_execution] #package_name=N'testpackage.dtsx', #execution_id=#execution_id OUTPUT, #folder_name=N'TestFolder', #project_name=N'TestProject', #use32bitruntime=False, #reference_id=Null
Select #execution_id --delete this line
DECLARE #var0 smallint = 1 `-- delete this line`
EXEC [SSISDB].[catalog].[set_execution_parameter_value] #execution_id, #object_type=50, #parameter_name=N'LOGGING_LEVEL', #parameter_value=#var0 `--(replace this #var0 with 1)`
EXEC [SSISDB].[catalog].[start_execution] #execution_id
GO
Passing parameter: Add the following two line to add pass a paremeter
DECLARE #ReportDate nvarchar(100) = `?`
EXEC [SSISDB].[catalog].[set_execution_parameter_value] #execution_id, #object_type=30, #parameter_name=N'ReportDate', #parameter_value=#ReportDate
Now from "Parameter Mapping" tab on the Execute SQL Task Editor add your variable you want to pass as a parameter.
Remember that variable data type has to be same as the parameter data type.
If you are leaving the Logging level as Basic (1), which is the default, you do not need to set this every time. Also, there is a bug in the SSISDB.[catalog].[set_execution_parameter_value] sp if you run a number of these steps in parallel. This can cause a DEADLOCK on the execution_parameter_values. I've raised this issue with MS.
I would have added as a comment but I don't have enough rep.
Related
** Edit: Executing the package with T-SQL script causes an authentication problem:
"Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'." **
** Edit 2: This error is caused by Windows credentials performing a double hop. Here is a solution to this problem **
I'm trying to execute an SSIS package that was already deployed to an SQL Server instance before. This SQL Server instance's version is 14.0.3281 (SQL Server 2017).
I can't use project reference as the wanted package is in another project, and I prefer not to upload it to the server's file system.
Is it really a version problem?
How can I execute a deployed package from another project?
I know I can run the a job with this package with T-SQL, but I want the task to continue only after this package has successfully completed its execution.
This is the error I get:
The attempted operation is not supported with this database version.
------------------------------ ADDITIONAL INFORMATION:
The attempted operation is not supported with this database version.
There is a way to run a package via T-SQL and wait for it to finish using built-in procedures.
-- Create the execution object
DECLARE #execution_id BIGINT
EXEC [SSISDB].[catalog].[create_execution]
#package_name = #PackageName
, #project_name = #ProjectName
, #folder_name = #FolderName
, #use32bitruntime = False
, #reference_id = NULL
, #execution_id = #execution_id OUTPUT
-- Configure execution
EXEC [SSISDB].[catalog].[set_execution_parameter_value]
#execution_id
, #object_type = 50 -- System parameter
, #parameter_name = N'SYNCHRONIZED'
, #parameter_value = 1 -- set this to 1 to wait for completion
-- Start the execution
EXEC [SSISDB].[catalog].[start_execution] #execution_id
You could wrap this into a stored procedure like "startPackage" and also include checking for the result using the execution id:
-- Check package status, and fail script if the package failed
IF 7 <> (SELECT [status] FROM [SSISDB].[catalog].[executions] WHERE execution_id = #execution_id) AND #Syncronized = 1
RAISERROR('The package failed!', 16, 1)
Is there a way, to update the connection manager information from the ssis catalog after deploying with an sql code ?
I'd like to deploy the project without sensitive data first :
exec catalog.deploy_project ..
then add username and password to the SSIS Catalog Project via SQL...
Is there a way, to update the connection manager information from the
ssis catalog after deploying with an sql code ?
It is possible since all changed in SSISDB catalog are via stored procedure.
This is a SQL script generated by SSMS during connection string change:
DECLARE #var SQL_VARIANT
= N'Data Source=ServerName;Initial Catalog=dbName;
Provider=SQLNCLI11.1;Integrated Security=SSPI;Auto Translate=False;';
EXEC [SSISDB].[catalog].[set_object_parameter_value] #object_type = 20,
#parameter_name = N'_ConnectionStringParam',
#object_name = N'ProjectName',
#folder_name = N'FolderName',
#project_name = N'ProjectName',
#value_type = V,
#parameter_value = #var;
References:
catalog.set_object_parameter_value (SSISDB Database)
How can I automate setting SSIS Project Parameters in SSIS 2012?
Since you're asking how to change the username and password once a package is deployed, I assume you're already familiar with the SSIS Catalog, if not the documentation outlines this further. To set the username and passwords via T-SQL use environment variables, with the variable holding the password marked as sensitive. The SSISDB.CATALOG.SET_ENVIRONMENT_VARIABLE_VALUE stored procedure is used to update the values of environment variables, including those marked as sensitive, and can be used as follows.
Start off by creating an environment. To do this, right-click SSISDB and create a new folder. Then in this folder, right-click and create a new environment.
After this right-click the environment and select Properties then go to the Variables pane. Create a string variable for both the username and password, with the variable storing the password set as Sensitive.
Next, right click the package or project depending on the scope that the connection manager was defined in, and press Configure and go to the References page. On this add the newly created environment with the Add button.
Still in the properties window of the project/package, go to the Parameters page and then Connection Managers tab. Find the connection manager that will use the environment variables, right-click the ellipsis on both the username and password properties, change the radio button to "Use Environment Variable", and choose the corresponding variables.
Before executing the package, run the SSISDB.CATALOG.SET_ENVIRONMENT_VARIABLE_VALUE stored procedure to update the environment variables. An example of this follows.
If this package is executed using T-SQL from a SQL Agent job or another method, linked the environment reference to it as done below.
Example:
DECLARE #usernameVar SQL_VARIANT = N'UsernameValue'
EXEC SSISDB.[CATALOG].SET_ENVIRONMENT_VARIABLE_VALUE #variable_name=N'Username',
#environment_name=N'Environment Name', #folder_name=N'Environment folder', #value=#usernameVar
DECLARE #passwordVar SQL_VARIANT = N'PasswordValue'
EXEC SSISDB.[CATALOG].SET_ENVIRONMENT_VARIABLE_VALUE #variable_name=N'Password',
#environment_name=N'Environment Name', #folder_name=N'Environment folder', #value=#passwordVar
--make sure environment mapped with #reference_id
DECLARE #execution_id bigint
EXEC SSISDB.[CATALOG].CREATE_EXECUTION #package_name=N'Package.dtsx', #execution_id=#execution_id OUTPUT,
#folder_name=N'Project Folder', #project_name=N'PackageProject', #use32bitruntime=False, #reference_id=99
DECLARE #var0 smallint = 1
EXEC SSISDB.[CATALOG].SET_EXECUTION_PARAMETER_VALUE #execution_id, #object_type=50,
#parameter_name=N'LOGGING_LEVEL', #parameter_value=#var0
--execute package
EXEC SSISDB.[CATALOG].START_EXECUTION #execution_id
I am using a FileTable in SQL Server 2014 and need to run an executable that parses the file name of any inserted/updated/deleted file and then in turn the executable inserts into other tables on the database the information that was parsed from the name. I do not expect the .exe to run long at all but if it runs into issues, I do not want to lock it for an extended period of time.
For instance:
CREATE PROCEDURE filename_parser
#name nvarchar(255)
AS
BEGIN
DECLARE #exe nvarchar(255)
SET #exe = 'c:\test\my.exe "' + #name + '"'
EXEC master..xp_cmdshell #exe
END
GO
If I run the stored procedure from an INSERT or UPDATE trigger, for instance:
USE [db_1]
GO
CREATE TRIGGER [dbo].[i_table_a]
ON
[dbo].[table_a]
AFTER
INSERT
AS
DECLARE #file nvarchar(255)
SELECT TOP 1
#file = name
FROM
inserted
EXEC filename_parser #name = #file
will I end up locking table_a until the executable completes? Sorry, if the answer is obvious. I have not found a straight forward answer. Any help/pointing in the appropriate direction is appreciated.
Related links:
Do stored procedures lock tables/rows?
SQL Server - How to lock a table until a stored procedure finishes
Microsoft docs say xp_cmdshell will run synchronously. Triggers run synchronously too. So, if your exe gets stuck, it will hang the trigger, which will hang the insert, and other stuff. msdn.microsoft.com/en-us/library/ms175046.aspx#remarks
I use SQL Server Agent to fill tables in DataWarehouse. I have about 50 steps in the job and every step is run with a proxy account. And every step works correctly besides one:
SSIS package which contains about 20 Execute SQL Tasks which execute procedure. This is what I have in the Execute SQL Task:
DECLARE #RC int
DECLARE #fordate datetime = null
DECLARE #tablename sysname = 'D_ENTITY'
DECLARE #dataFolder varchar(1024) = 'C:\MountPoints1\src_etl\'
DECLARE #logFolder varchar(1024) = 'C:\MountPoints1\src_etl\'
DECLARE #debug bit = 0
EXECUTE #RC = [dbo].[ETL1A_HR]
#fordate
,#tablename
,#dataFolder
,#logFolder
,#debug
GO
The thing is, that if I execute the package from the SSIS catalog, it works ok. But if it is run by job, it succeeds, but only deletes from tables, but doesn't fill it. It seems like the procedure stops somewhere in the middle.
Any ideas?
Please advise, it took me days trying to solve this...
I think it maybe related to permissions. Executing the SSIS package will use your security context but running it from the agent impersonates the credentials defined in the proxy, and then runs the job step by using that security context.
I have the following script that I am trying to run through a winforms program that just executes all the scripts in a directory:
USE [master]
GO
EXEC('ALTER DATABASE [AT100Reporting] MODIFY Name = [IngeniumDynamics];')
GO
However, when I try to run this I get the following error:
ALTER DATABASE statement not allowed within multi-statement transaction.
I googled this and the main thing I could find about it was that alter db statements should run in autocommit mode. A further search made me change my script to
USE [master]
GO
SET IMPLICIT_TRANSACTIONS OFF
EXEC('ALTER DATABASE [AT100Reporting] MODIFY Name = [IngeniumDynamics];')
GO
SET IMPLICIT_TRANSACTIONS ON
Yet I still have the same error. Does anyone know what I need to do to make this script run properly
I have also tried this without the EXEC
As Joachim Isaksson has pointed out, the problem was not with the script but with the winforms program that was running the scripts - I had my TransactionScope options set to TransactionScopeOption.Required but if I changed this to TransactionScopeOption.Suppress the query worked fine
You can use,
sp_renamedb OldDbName,NewDbName to rename the database :
exec ('sp_renamedb questionoverflow, stackoverflow')
Dynamic Way :
declare #OldName varchar(50)='questionoverflow'
declare #NewName varchar(50)='stackoverflow'
exec('sp_renamedb '+#OldName+','+#NewName)
Then It should work :
First Set an Single User Mode : and Rename it.
declare #OldName varchar(50)='stackoverflow'
declare #NewName varchar(50)='a'
EXEC('ALTER DATABASE '+#OLDNAME + ' SET SINGLE_USER')
exec('sp_renamedb '+#OldName+','+#NewName)
//And again put it in Multi-User Mode ::
EXEC('ALTER DATABASE '+#NewName+ ' SET MULTI_USER')