Is there a way, to update the connection manager information from the ssis catalog after deploying with an sql code ?
I'd like to deploy the project without sensitive data first :
exec catalog.deploy_project ..
then add username and password to the SSIS Catalog Project via SQL...
Is there a way, to update the connection manager information from the
ssis catalog after deploying with an sql code ?
It is possible since all changed in SSISDB catalog are via stored procedure.
This is a SQL script generated by SSMS during connection string change:
DECLARE #var SQL_VARIANT
= N'Data Source=ServerName;Initial Catalog=dbName;
Provider=SQLNCLI11.1;Integrated Security=SSPI;Auto Translate=False;';
EXEC [SSISDB].[catalog].[set_object_parameter_value] #object_type = 20,
#parameter_name = N'_ConnectionStringParam',
#object_name = N'ProjectName',
#folder_name = N'FolderName',
#project_name = N'ProjectName',
#value_type = V,
#parameter_value = #var;
References:
catalog.set_object_parameter_value (SSISDB Database)
How can I automate setting SSIS Project Parameters in SSIS 2012?
Since you're asking how to change the username and password once a package is deployed, I assume you're already familiar with the SSIS Catalog, if not the documentation outlines this further. To set the username and passwords via T-SQL use environment variables, with the variable holding the password marked as sensitive. The SSISDB.CATALOG.SET_ENVIRONMENT_VARIABLE_VALUE stored procedure is used to update the values of environment variables, including those marked as sensitive, and can be used as follows.
Start off by creating an environment. To do this, right-click SSISDB and create a new folder. Then in this folder, right-click and create a new environment.
After this right-click the environment and select Properties then go to the Variables pane. Create a string variable for both the username and password, with the variable storing the password set as Sensitive.
Next, right click the package or project depending on the scope that the connection manager was defined in, and press Configure and go to the References page. On this add the newly created environment with the Add button.
Still in the properties window of the project/package, go to the Parameters page and then Connection Managers tab. Find the connection manager that will use the environment variables, right-click the ellipsis on both the username and password properties, change the radio button to "Use Environment Variable", and choose the corresponding variables.
Before executing the package, run the SSISDB.CATALOG.SET_ENVIRONMENT_VARIABLE_VALUE stored procedure to update the environment variables. An example of this follows.
If this package is executed using T-SQL from a SQL Agent job or another method, linked the environment reference to it as done below.
Example:
DECLARE #usernameVar SQL_VARIANT = N'UsernameValue'
EXEC SSISDB.[CATALOG].SET_ENVIRONMENT_VARIABLE_VALUE #variable_name=N'Username',
#environment_name=N'Environment Name', #folder_name=N'Environment folder', #value=#usernameVar
DECLARE #passwordVar SQL_VARIANT = N'PasswordValue'
EXEC SSISDB.[CATALOG].SET_ENVIRONMENT_VARIABLE_VALUE #variable_name=N'Password',
#environment_name=N'Environment Name', #folder_name=N'Environment folder', #value=#passwordVar
--make sure environment mapped with #reference_id
DECLARE #execution_id bigint
EXEC SSISDB.[CATALOG].CREATE_EXECUTION #package_name=N'Package.dtsx', #execution_id=#execution_id OUTPUT,
#folder_name=N'Project Folder', #project_name=N'PackageProject', #use32bitruntime=False, #reference_id=99
DECLARE #var0 smallint = 1
EXEC SSISDB.[CATALOG].SET_EXECUTION_PARAMETER_VALUE #execution_id, #object_type=50,
#parameter_name=N'LOGGING_LEVEL', #parameter_value=#var0
--execute package
EXEC SSISDB.[CATALOG].START_EXECUTION #execution_id
Related
I'm trying to create a stored procedure that creates a dB-snapshot for a non privileged user.
The idea is to provide to a normal user a way to create a dB snapshot in order to run queries against it and delete the snapshot when it is done with it.
I thought it would be possible to use the 'with execute as owner" in the procedure declaration. However, I always get the following error:
CREATE DATABASE permission denied in database 'master'.
Here is my code:
-- The user that create the sp has sysadmin right
CREATE OR ALTER PROCEDURE [dbo].[makeSnapshot] WITH EXECUTE AS OWNER
AS
-- just an extract of the code (should test if exist...)
DECLARE #exec NVARCHAR(2000)
set #exec = 'CREATE DATABASE test_dbss1900 ON ( NAME = test, FILENAME =
''C:\\Program Files\\Microsoft SQL Server\\MSSQL14.SQLSERVER2017\\MSSQL\\Data\\test_1900.ss'' ) AS SNAPSHOT OF test';
EXEC (#exec)
GO
-- try to execute it (with any user)
EXEC dbo.[makeSnapshot]
Has anyone an idea how I can come up with a stored proc that will allow a normal user to create a db snapshot?
Thank for any help!
José
I actually found a solution by looking at - http://www.sommarskog.se/grantperm.html#serverlevel (chapter 5.3)
The way was to use certificates
Can we create database scoped credential for shared access signature inside a T-SQL procedure on Azure SQL ?
We are trying to bulk insert data from csv file stored on Azure blob
Below are the steps
Create SAS credentials using JAVA
Java will call stored procedure to create/alter external data source and pass SAS credential to this procedure as a parameter
Above procedure will internally call another procedure to do BULK INSERT
We are facing challenges in creation of database scoped credential via T-SQL procedure , can someone confirm if this approach is actually possible
It would be great help if we can get some sample code for the same .
I've seen the syntax to create database scoped credential on Microsoft site , it works when we run scripts as standalone statements but we are unable to compile the code using a procedure
We tried below code
CREATE procedure prc_create_external_data_source #SAS VARCHAR(100)
AS
BEGIN
SET nocount ON
DECLARE #command varchar(MAX);
PRINT 'Shared Access Key Received Key: '+#SAS;
SET #command =
'CREATE DATABASE SCOPED CREDENTIAL MyAzureBlobStorageCredential
WITH IDENTITY = ''SHARED ACCESS SIGNATURE'',
SECRET = '+#SAS
EXECUTE (#command);
PRINT 'Database Scoped Credential Created';
END
We get below Error message after executing this procedure
EXECUTE prc_create_external_data_source '1245632512344'
Error Number : 102 Error Message : Incorrect syntax near '1245632512344'. Line Number : 3 Procedure Name : (SQL State: S0001 - Error Code: 0)
Thanks
Your secret probably needs to be surrounded by quotes. Also this all needs to be on one line or concatenated with +
If you printed the string it might be obvious. Try this:
SET #command =
'CREATE DATABASE SCOPED CREDENTIAL MyAzureBlobStorageCredential ' +
'WITH IDENTITY = ''SHARED ACCESS SIGNATURE''' +
',SECRET = '''+ #SAS + ''''
I have multiple SSIS projects, but some of the packages inside them are the same.
I would like to create a project with all the generic packages and keep the others projects with theirs specific packages.
So my question is : is it possible to have a master package that can execute and pass parent variables to packages from another project ?
I'm new to SSIS so sorry if it's an obvious question or if i'm not specific enough
When you configure an Integration Services catalog you can execute the packages from another project.
After deploying your all projects in the SSIS Catalog. Go to SSIS Catalog and browse to the package you want to execute.
Right click and select execute
It will pop up a window and ask for required paramter to fill up then
Click on the script menu and select New query editor window
Extra steps - Delete the Select #execution_id (not neccessary)
Extra steps - Delete the DECLARE #var0 smallint = 1 line and replace #var0 with 1 (in the line it will be #parameter_value=1 )
Do not close the sql query window as you need to copy the generated script
Now in your master package:
Add a Execute SQL Task
Add OLE DB connection manager to SSISDB database
In the Execute SQL Task Editor--> SQLstatement: paste here the generated package script
An example of the script:
Declare #execution_id bigint
EXEC [SSISDB].[catalog].[create_execution] #package_name=N'testpackage.dtsx', #execution_id=#execution_id OUTPUT, #folder_name=N'TestFolder', #project_name=N'TestProject', #use32bitruntime=False, #reference_id=Null
Select #execution_id --delete this line
DECLARE #var0 smallint = 1 `-- delete this line`
EXEC [SSISDB].[catalog].[set_execution_parameter_value] #execution_id, #object_type=50, #parameter_name=N'LOGGING_LEVEL', #parameter_value=#var0 `--(replace this #var0 with 1)`
EXEC [SSISDB].[catalog].[start_execution] #execution_id
GO
Passing parameter: Add the following two line to add pass a paremeter
DECLARE #ReportDate nvarchar(100) = `?`
EXEC [SSISDB].[catalog].[set_execution_parameter_value] #execution_id, #object_type=30, #parameter_name=N'ReportDate', #parameter_value=#ReportDate
Now from "Parameter Mapping" tab on the Execute SQL Task Editor add your variable you want to pass as a parameter.
Remember that variable data type has to be same as the parameter data type.
If you are leaving the Logging level as Basic (1), which is the default, you do not need to set this every time. Also, there is a bug in the SSISDB.[catalog].[set_execution_parameter_value] sp if you run a number of these steps in parallel. This can cause a DEADLOCK on the execution_parameter_values. I've raised this issue with MS.
I would have added as a comment but I don't have enough rep.
I found samples where the code:
SELECT * FROM [legacyserver].[database].[schema].[table]
was expressed as:
SELECT * FROM [legacyserver]...[table]
but isn't working for me.
It gives me the error:
An invalid schema or catalog was specified for the provider "MSDASQL" for linked server "legacyserver".
I'm using for legacy server SQL SERVER 2000 and for the new server SQL SERVER 2012.
I tried creating the linked server using:
EXEC sp_addlinkedserver
#server = 'legacyserver',
#srvproduct = '',
#provider = 'MSDASQL',
#provstr = 'DRIVER={SQL Server};SERVER=legacyserver;DATABASE=database;Trusted_Connection=Yes;',
and:
EXEC sp_addlinkedserver
#server = 'legacyserver',
#srvproduct = '',
#provider = 'MSDASQL',
#provstr = 'DRIVER={SQL Server};SERVER=legacyserver;Trusted_Connection=Yes;',
#catalog = 'database';
Is it possible to create the script without hard coding the database name?
I need to create a really big migration script and it need to use a development, acceptance and production databases, and it will be to much work and error prone to change it using replace in the text editor.
UPDATE:
The legacy development, acceptance and production databases are exactly the same (except for the data) and each one have it's own server/instance and database name.
Due to segregation of duties, I can't develop something and deploy it, so I have no way to touch this script after acceptance. I will need to instruct someone else to do so, and if he/she will need to replace every occurrence of [legacyserver].[database], the chances of mistakes are very high.
You can create a synonym
CREATE SYNONYM [table] FOR [legacyserver].[database].[schema].[table]
When you query
SELECT * FROM table
it's actually fetching data from linked server instead of local database.
If want to change database, just drop synonym and create a new one with new database name.
DROP SYNONYM table;
CREATE SYNONYM [table] FOR [legacyserver].[anotherdatabase].[schema].[table]
Your query statement is unchanged.
EDIT: DROP and CREATE SYNONYM statement is a little misleading. You don't need to do it yourself. It's one time job in deployment. Just create a Post-Deployment script that creates all synonyms and parametrize linked server name and database names. Code like:
IF EXISTS (SELECT * FROM sys.synonyms WHERE name = 'table1')
BEGIN
DROP SYNONYM table1
END
EXEC('CREATE SYNONYM table1 FOR ' + '$(LinkedServerName).$(DBName).[dbo].[Table1]')
Note, it use SQLCMD syntax.
Ask operations to change parameter in different environments.
I have a stored procedure that makes bulk insert from some file:
CREATE PROCEDURE [dbo].[SP_BulkInsert] #FileName NVARCHAR(200) AS
BEGIN
DECLARE #bulkinsert NVARCHAR(1000)
SET #bulkinsert = N'BULK INSERT TblTemp FROM ''' + #FileName +
N''' WITH (FIELDTERMINATOR = '','', ROWTERMINATOR = ''\n'')'
EXEC sp_executesql #bulkinsert
RETURN ##ROWCOUNT
END
This stored procedure runs fine when I run it from SQL Server Management Studio, but when I try to run it with ExecuteNonQuery of ADO.NET I get the following error:
"The INSERT permission was denied on the object 'TblTemp', database
'TempDB', schema 'dbo'."
Important: all other stored procedures (that make SELECT/INSERT/DELETE/UPDATE) run fine from ADO.NET.
The user under which all the things are run is a member of the bulkadmin role, and also a member of a custom db_executer role (that has just EXECUTE permission).
The code runs fine for a lot of stored procedures, it's a first time that it fails..
This is the function
public static int BulkInsert(string fileName)
{
SqlParameter paramFileName = new SqlParameter("FileName", fileName);
SqlParameter paramRetValue = new SqlParameter();
paramRetValue.Direction = ParameterDirection.ReturnValue;
SqlParameter[] #parameters = { paramFileName, paramRetValue };
SqlHelper.ExecuteNonQuery(ConnectionSettings.ConnectionString,
CommandType.StoredProcedure, "SP_BulkInsert", parameters, true);
return (int)paramRetValue.Value;
}
I logging into the SSMS with the same username/password that are in the connection string on ADO.NET side..
The bottom question is, why in Management Studio the stored procedure succeed, while via ADO.NET it fails (with the above error message).
Running SQL via the sp_executesql uses different permissions than directly in the stored procedure. I would advise checking that the user which you are running the stored procedure as has (in this instance) INSERT permissions against the table "TblTemp".
To do this in Sql Server Management Studio...
expand the list of tables
right click on the appropriate one and select properties
on the "permissions" tab, click the "Add..." button.
Either type in the user or role, or "Browse..." for it.
with the user or role selected in the top table, tick "Grant" in the appropriate permissions in the "Explicit permissions for {username/role}"
click OK
The Management Studio must be running on the sa account which has all privileges, but the user that you are connecting through ADO.NET may not have the correct privileges set. You need to give the users writing permissions through the Management studio. Look in the users list.