I have a package on SSIS that executes a .BAT file that consequently executes a PowerShell file, which brings the file metadata from a folder, generating a CSV file.
I already granted access to the folder for the SQL Server Agent user, but it didn't work.
When I run the package on SSIS, it works correctly! However, if I execute the same package through a JOB in SSMS, it ends the step, but the .BAT file was not executed, or if it was, the Powershell file did not, because after the execution the CSV file is "1kb", no data.
Am I forgetting something?
SSIS package (portuguese language):
[EDIT 1]
A strange thing that I noticed, was that we have 3 packages scheduled by SSMS, but only one of them that doesn't work, which is the one mentioned in the question. But I can't find out why or what the difference is to others.
2 packages extract information from Sharepoint and work on SSMS, but this other package that extracts metadata from files does not.
Check the job step's "Run As" value, which is a "Proxy" (visible in the Object Explorer in SSMS.
Open the Proxy (SQL Server Agent >> Proxies >> SSIS Package Execution). It will list a Credential Name.
Open the Credential (Security >> Credentials). It will list an Identity. That's the user whose security needs to be configured.
Open a cmd window and run runas /user:<userGoesHere> cmd and enter the password when prompted. See the documentation of runas here
Now, it opens a second cmd window which runs under this user used in the SQL Server Agent. Execute your batch file and see if you'll catch any error.
Also, the process you're running may require the user to have Log on as a batch job permission. Run secpol.msc from an admin account and add this user to this permission.
Related
I've inherited an SSIS package which loads CSV files into a SQL database.
The first thing the package does is call a .BAT file which maps a network drive. The .BAT file contains the username and password used to map the drive in plain text, so urgently needs to be replaced.
I've written a solution which uses New-PSDrive in a Powershell script and creates a credentials XML file with the password encrypted. If I execute the ps1 script it works Ok.
If I call the ps1 script from within SSIS and run it through VS then it also works fine.
When I call the SSIS package that calls the script task through a SQL Agent job (executed as the SQL Server Agent user account) the drive mapping doesn't seem to work, and the package complains it cannot access the file from the file share.
I presume this is no working because the SQL Server Agent user account can't run Powershell queries? The package only errors because it cannot access the share, not during the execution of the Powershell script.
So, two questions:
Is there an obvious issue with the SQL Agent deployment idea
or
Is there a different way of mapping a network drive from within SSIS without storing the password in plain text anywhere?
This is the pattern I suggest using for accessing files on a remote server:
Access the files via unc path: \\MyServer\Files
This means that the package needs to run under a context that has access to the path. There are a couple of options:
Run the package under a proxy, i.e. the account credentials that were stored in the bat file. This means that you have to also grant access to the account on the sql instance as well
Use a group managed service account (gMSA). In this case, the sql agent service account is replaced with the gMSA and the job runs under sql agent. The gMSA needs to be granted access to the remote share as well as the sql instance. This is a much more secure way of addressing the whole process because there is no password to manage (AD takes care of that) and the account cannot be used to log in anywhere. But there is set up work to do to get it created so it's not the fastest option.
I have an SSIS package that is supposed to read a folder and if the folder contains any text files, it will read them and store the data into a database. The package works flawlessly from the file system, however when I deployed it it fails.
The package uses a project level variable that remains consistent that acts as a folder path to see which folder to look in for files.
The SQL Server Agent gives me this error:
After looking at the execution report I see following error messages if I execute it with the service account through SQL Server Agent:
"The file name property is not valid. The file name is a device or contains invalid characters"
"The file name '(path here)' specified in the connection was not valid"
"The Flat File Connection manager failed validation"
However, if I run it myself by directly right clicking on the package in the catalog and executing, the execution report will come back as successful but it will contain a warning that says: "Read Files From File Input Folder Warning: The For Each File enumerator is empty. The For Each File enumerator did not find any files that matched the file pattern, or the specified directory was empty"
I suspect it is a permissions error, but I haven't dealt much with the permissions in Microsoft SQL Server and the service account that I am using was not created by me. I have tried changing the security of both package and project from the default "Encrypt sensitive data with user key" to "Do not save sensitive data", rebuilding and redeploying but that didn't help at all.
It is most likely a permissions error but I've been puzzled on how to go about it. Any help is much appreciated!
EDIT: After setting the "Delay Validation" on tasks in the control flow and executing the package with SQL Server Agent it now comes back with 2 less errors
For filesystem task to work the executing account need permission to the folder in question. When you execute the package via the SQL-agent the package is executed under the account running the SQL-agent account, when you right click the package and press execute the package is executed as your account.
One problem could be that the SQL-agent account doesn't have permission to the folder, and can't enumerate the content. You can verify this by changing the SQL-server account to your own account temporary and running the job again:
Open Sql Server Configuraiton Manager
Under SQL Server Services double click SQL Server Aggent
Change the account to your own account
Restart the service
If this resolvs the problem, change back to the SQL-server account and check the folder permissions on the folder you are trying to enumerate. Add the SQL-agent account or group the account is included in to have permission to read the folder. Obviously you can also just check the folder permissions to begin with, and make sure the executing account have permission to it.
I have an SSIS package that runs a backup in a T-SQL task and then uploads the file to google drive from an execute process task. The package runs perfectly from the catalog. When I try to run it from sql-agent job, the backup runs ok but the upload to google drive does not. There are no error messages in the job history or in the package history.
The sql agent job is set to run from a proxy account with the necessary credentials.
The issue turned out to be that the command file was trying to find another file it needed to run execute the code, but since the windows user changed it was looking in a different users folder. We solved it by passing the filepath as a param so it doesn't default to the user accounts folder its executed from.
I have a SSRS solution that contains 20 reports all of which utilise a shared data source called "DataWarehouse". This datasource has been configured to use Windows Authentication. These reports have been deployed to a server.
I have a requirement where I need to be able to automatically run these reports once a data warehouse has completed loading overnight. Sometimes the warehouse load might take 5 hrs, sometimes it might take 10 and so I don't want schedule these reports at a particular time using reporting services, I want the warehouse load to determine when its done and trigger the reports.
Now, I have a mechanism to do this:
I have written a c# console application that accepts a report name and a file path which runs the report and saves it do wherever it needs to.
I have an SSIS package that executes the console application from (i). I use an SSIS package because the DBA where I work will not enable xp_cmdshell.
I have an agent job that runs the SSIS package when the warehouse load completes.
Now if I run the SSIS package on my local machine (as me) the package executes successfully and the report is generated and saved. When I deploy the SSIS package to the server and try and run it through the agent job it fails with the following error:
System.Web.Services.Protocols.SoapException: The permissions granted to user 'NT SERVICE\SQLSERVERAGENT' are insufficient for performing this operation. ---> Microsoft.ReportingServices.Diagnostics.Utilities.AccessDeniedException: The permissions granted to user 'NT SERVICE\SQLSERVERAGENT' are insufficient for performing this operation.
at Microsoft.ReportingServices.Library.ReportExecution2005Impl.LoadReport (String Report, String HistoryID, ExecutionInfo2& executionInfo)
at Microsoft.ReportingServices.WebServer.ReportExecutionService.LoadReport (String Report, String HistoryID, ExecutionInfo& executionInfo)
My initial guess was that this means the SSIS job gets executed as 'NT SERVICE\SQLSERVERAGENT' and this user does not have access to run reports on the report server. So I jumped on to the report server and tried adding NT SERVICE\SQLSERVERAGENT as a user who could run reports but I was still getting errors. Can anybody help me figure how I can get this working?
Thanks
This will help you execute the SSIS package in the command prompt, and you can run the package on the server in scheduled tasks from the batch file (you may need special permissions to run a batch file in scheduled tasks). What you will want to do is create a windows batch file to allow you to automatically run the SSIS package via Command Prompt and you will need to use Dtexec. Here is what your batch file will look like:
#ECHO OFF
Some comment about what the package is doing
Dtexec /f "C:\some\file\path\YourPackage.dtsx
Then once you save the notepad as a windows batch file you will be able to open the batch file and it will run your package in the Command Prompt. I hope this helps
I edited the console application to connect to the SSRS server as a specific user. I've added what I did below incase its useful to somebody else.
To do this I created a file called account.config which just contained the username on the first line and password on the secondline:
MyUser
MyPassword
Then in my console application I have the following code:
// Read in the config file and add the lines to a list
List<string> accountDetails = File.ReadAllLines(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "account.config")).ToList();
ReportExecutionService myReportExecutionService = new ReportExecutionService();
// Add the credentials
// accountDetails[0] is the username, accountDetails[1] is the password
myReportExecutionService.Credentials = new NetworkCredential(accountDetails[0], accountDetails[1]);
What is the default SSIS Account that is used in SSIS packages.
In one of my packages I have a Data Flow task that creates a flat file. A BAT file later runs and creates a file based on information in that .txt file. If I execute the .BAT in Windows Explorer it runs fine. When SSIS tries to execute it I can see the CMD window open and it tries to access the txt file and isnt able to. Says "Unable to access nameoffile.txt"
I assume the issue is that the permissions are probably inherited by the SSIS user account so I am trying to figure out which account that is.
If you are running it from the IDE then it uses your permissions. Most likely the problem is that your file is locked by some other process in your SSIS package. To verify:
Add a PAUSE to your batch file to make it wait for any key.
Run your package. While the dos prompt is waiting, go into windows explorer and run your batch file again - you'll find the same error.
I changed the attrib value in dos for that specific folder by attrib -r -s and it worked. It thought it was read-only.
I am using Windows Server 2012 server, with a SQL Server 2012 database.
Adding Modify, Read, Write permissions to the folder containing the required file to the NETWORK SERVICE user seems to work for me.