I have deployed a SSIS Package to the Integration Services Catalogs. The issue that I am having is that the flat file source does not have access to the network drive.
How do I configure the package to run under another account.
The reason I chose this route is I wanted to be able to run the SSIS package from a WebAPI and just running a stored procedure.
EXECUTE AS LOGIN = 'Domain\User'
Declare #execution_id bigint
EXEC [SSISDB].[catalog].[create_execution] #package_name=N'Package.dtsx', #execution_id=#execution_id OUTPUT, #folder_name=N'CDMS', #project_name=N'Package Import', #use32bitruntime=False, #reference_id=Null
Select #execution_id
DECLARE #var0 smallint = 1
EXEC [SSISDB].[catalog].[set_execution_parameter_value] #execution_id, #object_type=50, #parameter_name=N'LOGGING_LEVEL', #parameter_value=#var0
EXEC [SSISDB].[catalog].[start_execution] #execution_id <----Fails
Update:
I tried 2 scenarios in SQL Management studio:
When logged in as a SQL Account: The current security context cannot be reverted. Please switch to the original database where 'Execute As' was called and try it again. ( Btw I am not even calling REVERT)
Logged in as Windows Auth and removed the Login statement: Cannot open the datafile...
Question in scenario 2, It is clear SQL is not accessing the network drive under my account. But when using EXECUTE as LOGIN..Does that then access the network drive under the login specified?
UPDATE 2
I have changed the permissions of the share to Modified for Everyone and I am still getting this error
Related
I'm trying to execute a package that among other thing after processing an xlsx in a folder, it deletes it. for this, when executing from visual studio it works fine, the problem comes when executing from sql server (by using script to run it):
Declare #execution_id bigint
EXEC [SSISDB].[catalog].[create_execution] #package_name=N'ImportAllowanceWood.dtsx', #execution_id=#execution_id OUTPUT, #folder_name=N'Packages', #project_name=N'JSTEMS_Payroll_Import', #use32bitruntime=False, #reference_id=Null
Select #execution_id
DECLARE #var0 smallint = 1
EXEC [SSISDB].[catalog].[set_execution_parameter_value] #execution_id, #object_type=50, #parameter_name=N'LOGGING_LEVEL', #parameter_value=#var0
EXEC [SSISDB].[catalog].[set_execution_parameter_value] #execution_id, #object_type=30, #parameter_name=N'Payroll', #parameter_value=N'B00014552'
EXEC [SSISDB].[catalog].[start_execution] #execution_id
GO
the error I'm getting is
and I'm think it might be related to permission on the folder in which I want to operate.
in the folder I gave full permission to the sql server agent, the my domain user (which is used when running the query), "NETWORK SERVICE" and "everyone".
but I still get the error, is there any specific user I should give permission?
I've seen around to grant permission to
NT SERVICE\SQLSERVERAGENT
NT SERVICE\MSSQLSERVER
but I don't know how to do it, since when looking for them in the permission folder they are not found
I need to run SSIS2016 package from older SSIS (another server) because of extending legacy system. The package runs fine from SQL Agent, but I am unable to run it using tsql or DTEXEC. The problem is that package is unable to access network even when run in elevated cmd (under sysadmin)
I am able to access network drive from account I am running package (admin)
I know that package runs fine when it is run from Sql agent under same account.
The package runs fine when file is saved to local temp folder (both DTEXEC and TSQL)
The package fails to save file to network when run using execute is SSMS, TSQL or DTEXEC.
DTEXEC /ISSERVER "\SSISDB\XXXX\Exports\package.dtsx" /SERVER "." /Par
"$Package::FilePath(String)";"C:\TEMP"
From all what I have read on internet, it seems that the only way to run SSIS2016 package is by using SQL Agent and jobs.
I am surprised by the fact that DTEXEC ignores the security context of account which execute the command line.
Is there a way to force it to use proxy account?
I find it hard to believe, that only way to run package with permission to network is to use Sql agent and jobs.
Thank you.
p.s. Here is good article about running packages, but it completely omits network access problem.
EDIT 2019-06-10
I applied MSSQL2016 CU7 and restarted the server.
After this,
Iam able to save file to network using DTEXEC command ran on local server (logged as sysadmin)
Iam able to save file to network using sp ran from local server (logged as sysadmin)
I cant save file to network drive when logged as sysadmin on different machine. The error is
Flat File Destination failed the pre-execute phase and returned error
code 0xC020200E.
Cannot open the datafile ..
there is also warning "Access is denied" in data flow which should save the file.
Source code of stored procedure
DECLARE #execution_id BIGINT;
EXEC [SSISDB].[catalog].[create_execution]
#package_name = N'package.dtsx',
#execution_id = #execution_id OUTPUT,
#folder_name = N'XXXX',
#project_name = N'Exports',
#use32bitruntime = False,
#reference_id = NULL;
SELECT #execution_id;
DECLARE #var0 SQL_VARIANT = N'\\NETWORK\PATH\Export\test_files';
EXEC [SSISDB].[catalog].[set_execution_parameter_value]
#execution_id,
#object_type = 30,
#parameter_name = N'FilePath',
#parameter_value = #var0;
DECLARE #var1 SMALLINT = 1;
EXEC [SSISDB].[catalog].[set_execution_parameter_value]
#execution_id,
#object_type = 50,
#parameter_name = N'LOGGING_LEVEL',
#parameter_value = #var1;
EXEC [SSISDB].[catalog].[start_execution]
#execution_id;
I also added System::UserName to log on start of package and I see correct account.
I assume, that problem might be in double hop as #Piotr mentions.
Iam digging the fileserver logs to see.
EDIT2 2019-06-10
Ok. After much dancing around I can narrow problem down to delegation executing user rights to CIFS.
What i did
Setup a share on dev computer
Create package with scriptask which tries to create test file and get security context of calling user WindowsIdentity.GetCurrent().Name
Then I ran stored procedure with code above in scenario that always ended by access denied.
Until I enabled anonymous access (using this article), the package was not able to save file.
AFAIK we have enabled only MSSQLSvc kerberos delegation.
Closure:
The problem is caused by behaviour of SSIS catalog, when called from TSQL, which gets users security context, but in order to work this properly, when accessing netwrok shares, you need to setup Kerberos delegation for CIFS also.
I was not able to do this in our environment, so I reverted back to xp_cmdshell.
Thanks to all for support.
Earlier we were in SSSIS 2008, we have 3 projects under a solution, Project A, Project B, Project C. I have Master Package in Project B from which I am calling packages in ProjectA and ProjectC in execute package task using external reference as File System, everything worked well till here. We recently moved from SSIS 2008 to SSIS 2012 and used Project deployment model to deploy packages to SSISDB instead of File System using Package Deployment Model, so I replaced Execute Package task with Execute SQL task in my master package to call packages from Project A and C.
I created Connection to SSISDB and used it in Execute SQL task and placed the below T-SQL code in it, it worked fine here also.
Declare #execution_id bigint
EXEC [SSISDB].[catalog].[create_execution]
#package_name=N'Child_Package.dtsx',
#execution_id=#execution_id OUTPUT,
#folder_name=N'Test',
#project_name=N'ProjectA',
#use32bitruntime=False, #reference_id=Null
Select #execution_id
DECLARE #var0 smallint = 1
EXEC [SSISDB].[catalog].[set_execution_parameter_value]
#execution_id,
#object_type=50,
#parameter_name=N'LOGGING_LEVEL',
#parameter_value=#var0
EXEC [SSISDB].[catalog].[start_execution] #execution_id
GO
I have 2 question here
1) If I deploy my packages from development to Production server, will this T-SQL code inside Execute SQL task works there without any issues (I read in one post that it will have some issues when deployed to diff environment) I am not hard-coding any environment parameter value inside my T-SQL code, will this work in another environment as well?
2) I am trying to implement rollback transaction support in my master package, so if any child package fails, everything needs to be rolled back, so I have Changed the package TransactionOption property to required and all other tasks(Execute SQL TASK) as supported(Default), I am ending up with an error like
"[Execute SQL Task] Error: Executing the query "Declare #execution_id bigint
EXEC [SSISDB].[catalo..." failed with the following error: "Cannot use SAVE TRANSACTION within a distributed transaction.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly"
The package is working fine if the TransactionOption property of the package is supported, the problem comes only when I use TransactionOption property of package as Required.
Any help is much appreciated.
DECLARE #PATH NVARCHAR(1000) = N'\\MY-SERVER\C$\Folder\\'
DECLARE #TABLE NVARCHAR(50) = SUBSTRING(#FILENAME,0,CHARINDEX('.',#FILENAME))
DECLARE #SQL NVARCHAR(4000) =
N'IF OBJECT_ID(''dbo.' + #TABLE + ''' , ''U'') IS NOT NULL
DROP TABLE dbo.[' + #TABLE + ']
SELECT * INTO [' + #TABLE + ']
FROM OPENROWSET(''Microsoft.ACE.OLEDB.12.0''
,''Text; Database='+#PATH+';''
,''SELECT * FROM [' + #FILENAME + ']'')'
EXEC(#SQL)
Today I have come across an issue with Microsoft.ACE.OLEDB.12.0 driver in SSIS 2012. The script above sits in a stored procedure which dynamically loads the .csv's into the database based on the current file which is supplied by the SSIS loop in which stored procedure sits. There are files in the directory.
The stored procedure runs correctly when run directly in SQL Server Management Studio.
This has been working fine up until today. Today I am getting the following error:
Executing the query "EXEC [dbo].[CreateAndImportCSVs] ?" failed with the following error: "Cannot initialize the data source object of OLE DB provider "Microsoft.ACE.OLEDB.12.0" for linked server "(null)".".
Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Any help on this issue would be great!
Edit
So looking into what's changed I hear alarm bells when I look at the windows updates installed on the server yesterday! The following two were installed:
Microsoft Office Access Runtime and Data Connectivity 2007 (SP3)
http://support.microsoft.com/kb/2526310
Update for the 2007 Microsoft Office System (KB967642)
http://www.microsoft.com/downloads/details.aspx?FamilyId=E93AB1BE-ADE6-4FF8-8637-DBD3EBE3C5C5&displaylang=en
Many things to Try:
Check the In Process and Dynamic Provider options for the ACE provider
Check the permissions on the Temp folder
Check the MemToLeave memory area allocated
Make sure the EXCEL process is not running in background
Made sure 'ad hoc distributed queries' was enabled (1)
USE [master]
GO
EXEC sp_configure 'Show Advanced Options', 1
RECONFIGURE
GO
EXEC sp_configure 'Ad Hoc Distributed Queries', 1
RECONFIGURE
GO
EXEC sp_MSSet_oledb_prop N'Microsoft.ACE.OLEDB.12.0', N'AllowInProcess', 1
GO
EXEC sp_MSSet_oledb_prop N'Microsoft.ACE.OLEDB.12.0', N'DynamicParameters', 1
GO
Read more at How to solve Microsoft.ACE.OLEDB.12.0 error "Unspecified error"
Also i found an interesting suggested solution in the following link, take a look:
The OLE DB provider Microsoft.Ace.OLEDB.12.0 for linked server (null)
This problem resolved itself. There were further updates installed, one of these must have fixed the issue introduced by the previous updates:
Update for Microsoft Office 2010 (KB4011188) 64-Bit Edition
https://support.microsoft.com/kb/4011188
Update for Microsoft Office 2010 (KB2553347) 64-Bit Edition
http://support.microsoft.com/kb/2553347
Security Update for Microsoft Office 2010 (KB2553338) 64-Bit Edition
https://support.microsoft.com/kb/2553338
Thanks for all the help with this!
The final piece that solved this for me was moving the spreadsheet to a windows directory that SQL Server was able to access. I moved my spreadsheet to a new subdirectory, within the directly where my sql database was located, and the problem was resolved.
SQL Server is not able to access the Access provider, if the installation of office was Click to Run installation.
The issue can be resolved by any of the following approaches.
OLEDB driver issues resolution
Resolution
Beginning with Microsoft 365 Apps for Enterprise Version 2009, work
has been completed to break ACE out of the C2R virtualization bubble
so that applications outside of Office are able to locate the ODBC,
OLEDB and DAO interfaces provided by the Access Database Engine within
the C2R installation.
Use the following table to understand if additional components are
necessary to access these intefaces within your environment:
Probably not a solution, but you should execute SQL with:
sp_executesql #SQL
It protects from injection and wotnot.
The requirement is to call a web service through SSIS and calling the SSIS from a SQL Server Service Broker activated stored procedure.
Here is what I have currently doing:
Queue
CREATE QUEUE [schema].[ProccessingQueue] WITH STATUS = ON , RETENTION = OFF , ACTIVATION ( STATUS = ON , PROCEDURE_NAME = [schema].[usp_ProccessingQueueActivation] , MAX_QUEUE_READERS = 10 , EXECUTE AS N'dbo' ), POISON_MESSAGE_HANDLING (STATUS = ON)
My stored procedure:
ALTER PROCEDURE [schema].[usp_ProccessingQueueActivation]
WITH EXECUTE AS CALLER
AS
BEGIN
SET NOCOUNT ON;
<snip declaration>
BEGIN
BEGIN TRANSACTION;
WAITFOR
(
RECEIVE TOP (1)
#ConversationHandle = conversation_handle,
#MessageBody = CAST(message_body AS XML),
#MessageTypeName = message_type_name
FROM [schema].[ProccessingQueue]
), TIMEOUT 5000;
<snip awasome stuff>
EXEC dbo.RunSSIS <param>
DECLARE #ReplyMessageBody XML = #MessageBody;
SEND ON CONVERSATION #ConversationHandle MESSAGE TYPE [type] (#ReplyMessageBody);
END
<handle error>
COMMIT TRANSACTION;
END
END
Now here is what RunSSIS stored procedure looks like
ALTER PROCEDURE [dbo].[RunSSIS]
<params>
AS
BEGIN
DECLARE #exec_id BIGINT
EXEC [SSISDB].[catalog].[create_execution]
#package_name=N'<SSIS_package>',
#folder_name=N'<folder>',
#project_name=N'<projectName>',
#use32bitruntime=FALSE,
#reference_id=NULL,
#execution_id=#exec_id OUTPUT
EXEC [SSISDB].[catalog].[set_execution_parameter_value]
#exec_id,
#object_type=30,
#parameter_name=N'<param_Name>',
#parameter_value=<param>
SELECT #exec_id
EXEC [SSISDB].[catalog].[start_execution] #exec_id
END
Now this will throws the below exception in event-viewer as the Sql service broker activation security context isn't recognized in SSISDB environment.
The activated proc
'[schema].[usp_ProccessingQueueActivation]' running on
queue '' output the
following: 'The current security context cannot be reverted. Please
switch to the original database where 'Execute As' was called and try
it again.'
To resolve the problem I have tried those following approach
So I follow this link
http://www.databasejournal.com/features/mssql/article.php/3800181/Security-Context-of-Service-Broker-Internal-Activation.htm
and created a User with a self signed certificate (thinking that it
is user that doesn't has permission). But it is returning same error,
digging deeper I found that [internal].[prepare_execution] in
SSISDB has "REVERT" statement in line no 36 that throws the error as
it doesn't like Impersonation at all.
I tried to move the RunSSIS stored procedure to SSISDB and try to call it from activation stored procedure, it was shoot down as SSISDB it doesn't allow any user with SQL Server auth, It needs to have a Windows auth and User created by Certificate obviously doesn't has windows credential.
My question is
Am I on the correct path? I certainly doesn't anticipate using 2 component of SQL server together would be that difficult.
If not in correct approach what would be best approach to call a service from Service broker? I have seen "External Activation" for SQL Server Service broker but haven't explored is yet. But I would try to stick to something that lives inside server environment and scale-able, and don't like the idea of installing different component in prod environment (it is always a overhead for support personal,as there is one more point which can fail)
I am using Windows auth and my credential has sys_Admin access.
I think you can take out the "WITH EXECUTE AS CALLER" and everything (the proc and then the package that ends up getting called) will be run under the security context of the Service Broker. As long as that context has the permissions to do what you want to do, you should be fine.
I have not used a Service Broker in this way, but I do the same thing with jobs fired off by the SQL Agent. As long as the Agent's security context has the permissions needed in the procs/packages everything runs fine. We use network accounts for our services so it all works between servers as well.
This has a code smell of tight coupling and my first instinct is to decouple the queue, the DB that houses the proc, and the SSIS execution into a PowerShell script. Have the script get the messages from service broker then call SSISDB on a different connection without wrapping [catalog].[create_execution] and [catalog].[set_execution_parameter_value] in a stored proc. You can still run this script directly from Agent.
This approach gives you the most flexibility with regard to security contexts, if one of the components moves to a different server, if something is named differently in dev/QA, or technologies change (Azure ServiceBus instead of Broker for instance). You can also get creative with logging/monitoring.