Moving Files with SSIS package, scheduled by a SQL Server Agent Job - sql-server

I developed quite a complex SSIS package. Part of it Moves Files from the "c:\Flags\" folder to the "c:\Flags\Busy\" folder.
I deploy the package under the SSIS Catalog.
When i right click and start the task manually, under my own credentials, it works fine. (i am administrator)
When i schedule the task (to run say every 30 mins) using the SQL Server Agent, it it fails in the part where the files gets moved. The SSIS Package runs under the "SQL SERVER AGENT ACCOUNT", by default.
Any suggestions or pointers how i can resolve the issue? (i did google the issue)
Maybe change the account it runs under (no options here from the GUI) or somehow give the "SQL SERVER AGENT ACCOUNT" rights to move files?

The agent account will almost always have insufficient rights to this type of task depending on where exactly the files are located. You could give more rights to the account but this is not recommended. I suggest a proxy.
http://sqlmag.com/blog/sql-server-dba-tip-credentials-and-proxies

I added the "NT SERVICE\SQLSERVERAGENT" to my folder where I move the files, and granted Modify rights. This resolved the issue for me. Kind Regards.

Related

Is there anyway developers of SSIS packages can use generic Windows domain account, instead of their personal login

We recently created a windows login (domain\etluser) with read-only access to different SQL Server databases. Is there anyway developers can use that generic account to develop SSIS packages using Visual Studio?
It require us to create OLEDB connections using that new account, but couldn't find an option for that
I heard that it is possible to run packages as another user or proxy from SQL Server integration services, once the development and testing is over: How to schedule SSIS package to run as something other than SQL Agent Service Account
In windows, I am aware of two mechanisms to start a process as another user. From the command line, you can use RunAs
As a consultant, I often bring my own device to clients and then use Runas to impersonate my client-domain account without having to get my laptop's domain trusted by their corporate domain. I have a folder filled with batch files that launch a program with their credentials. For example, this is my script to start Visual Studio 2017 Enterprise Edition as my "other" self.
runas /netonly /user:domain.com\billinkc "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE /nosplash"
I've not done SSRS development in a long while, but 2014?ish at least this approach would allow me to develop reports but I could not preview the reports within VS as the preview process would get launched as a "new" process which inherits my current credentials instead of the ones I was started the parent process under. SSIS dev though, worked just fine with this approach.
The biggest complain I have with either approach is that I have to type the password every time. You can't redirect the input from elsewhere and you don't see the typing so you can't tell how many characters you've typed if you screwed up the password. Couple that with an aggressive lockout policy and I spent a lot of time at one client twiddling my thumbs waiting for my account to unlock.
The other approach is a GUI approach. In Windows Explorer, shift right-click on the executable and you should be presented with a context menu like the following.
Pick "Run as different user" and you'll be prompted with a more friendly window to enter credentials
With Runas, you could at least prepopulate the user to save a few keystrokes but with the GUI approach, you have to enter everything every time.
If you run your SSIS jobs through the SQL Server Agent, you can use Proxies: https://learn.microsoft.com/en-us/sql/ssms/agent/create-a-sql-server-agent-proxy?view=sql-server-ver15
There's a couple steps involved:
Create a user for your dom\etluser on the SQL Server, in Security \ Credentials
Create a proxy in the SQL Agent using those credentials, and authorize it to run SSIS jobs
Create a job (or edit an existing job), with the SSIS package as a step and on that step set 'Execute As' to the proxy.

SSIS Package with SharePoint Doesn't Execute In Agent Job (Fine when ran manually from SSIS Store)

So I've created a neat SSIS package which essentially reads a SharePoint 2010 list, moves the data into a 'Delete' list, (thus deleting the contents), then populates the list with fresh data pulled from a SQL Database.
It works in Visual Studio just fine, it works if I'm logged into Integration Services and run the package manually fine, but when I place it into a SQL job I get nothing.
At first I thought maybe it was that I didn't install the SharePoint destination connection manager thing on the server, so I did, only quickly then realizing that if that was the cause, running the package manually from SSIS would have failed.
Then I figured it was the account it was being executed under (SSIS Service Account), so I created a proxy (SharePoint Admin) and added the credential, still no go. Then I added my own credentials, just to be sure, and still no go. Just to be super sure, I also added the SQL Agent account as a SharePoint Site Collection Administrator, which means the account now runs the package fine by right clicking on the package from Integration Services, but put it into a SQL Agent Job as a step, no love.
The job doesn't fail either, it just sits there spinning and spinning. I'm stumped. I've tried turning on logging as far as the options allow me under the advanced tab on the job step, but as the job doesn't fail, nothing is logged.
There's no question that the job can be run, using either my account or the SQL Agent account, but place it into a SQL job, nothing! I'm going crazy here. Any ideas?
I've found that that your SQL Agent account MUST be an SA on the server in order for it to be able to run SSIS packages as part of a job step when the 'Bulk Insert' task type is employed by the package...
https://msdn.microsoft.com/en-us/library/ms141239.aspx
N03L

Windows Event Log Access from SSIS package run in SQL Agent Job

I have created an SSIS package that will be deployed to client SQL installations (2005, 2008 or 2008 R2) to perform data extracts which provide a support tool for our product. The deployment process requires that a Windows AD account (normal user, no elevated privileges) is created as this is used as a service account to execute the SSIS package in a SQL Agent job by way of a credential and a proxy account. This all works perfectly and means I can restrict the privileges required to perform this job.
However, I wanted to include error logging in the SSIS package to the Windows Event Log. When I run the package in BIDS (which of course uses my own credentials) and force the failure of the package, it logs just fine. When I force the package to fail (by putting a duff connection string into the config file) whilst being run by the SQL Agent job, nothing is logged. The service account is being used and it is an authenticated user on my SQL Server host machine but it will not write to the event log. If I add the service account to the local administrators group, it writes to the log just fine, but I thought the idea of the Windows event log was that you did not need elevated privileges to write to it?
Our support teams are keen to use the Windows Event Log but I can see no way of doing so without granting high privileges to a service account which I would rather not do. Am I missing something? The Logging tab in the SSIS job step page doesn't seem to do a lot but perhaps that's what I'm missing?
Apologies if this is more suited to ServerFault, but I couldn't quite decide which side of the line this fell as it is a problem encountered during development. If it is then I'll relocate it.
Many thanks
Steve
If OS is 2003, check the SDDL syntax on who has access to write to the log with this: http://support.microsoft.com/kb/323076
If 2008, you can use wevtutil instead of manually typing in SDDL:
http://support.microsoft.com/kb/2028427
The service account can be given the permissions using the above.

SQL Agent Run SSIS package as Administrator

I'm trying to run a SQL Agent Job with a step that is a SSIS File, and I need this step to be run as administrator.
My Package uses a Script task to download a file, as a Browser i Use WATIN.
I'm using a thread to start this browser because this browser control requires the thread to be set as Single-Threaded Apartment.
This browser control is requiring to be run as administrator.
I've already created a Credential for a user that is Windows Admin, a Proxy SSIS account. (SQL Agent user is not windows admin).
But the SSIS package is not run as administrator yet.
I'm suspecting this is related to UAC.
Some details:
SQL Agent Account is not Windows Administrator
Using Windows Server 2008 R2
My Package is run from the File System.
The Package only works on BIDS if I run it as an Administrator (if not admin doesn't work)
The Proxy account the job step is configured to run is windows admin.
Any help is appreciated!
Take a look at SSIS runs in BIDS but not with SQL Agent for some ideas
I've solved this in another way, The problem wasn't related to running the package as administrator, but creating a windows and setting focus to it, however I had the option to see the result file on the web page and I managed to use it, without downloading it, thus not needing to set focus on the download windows.
Thanks for all the help.
Try to add the user account in SSIS administrator group which can solve your problem.

locating a flat file from SQL server 2005

I have built an SSIS job in visual studio 2005 that accesses a flat file and updates a table in SQL Server 2005. This works perfectly. I then store a copy on SQL Server by using the server storage option in the protection level property. When I try to run this it cannot locate the flat file. The flat file is located on a server and I use the IP address to access it. I have noticed that if I remove my maping to this server and run the job from visual studio, it cannot find it either. Hoe do I access the file from SQL Server 2005 ? I intend to schedule this job to run automatically on SQL Server.
Many Thanks for any help you can give.
Ronnie Carlin
Just to avoid anything more complex being the issue, check what account the job is being run under (e.g. has a procy and credential) or whether it is using the base SQL Agent account, and then check whether that account has access rights to the folder \ file you are trying to import.
Had that catch us out more than once.
Edited to include the additional information in the main answer.
When SQL Agent runs a job it does not use your account, even if you initiated the job execution - it uses the SQL Agent account. The account the SQl Agent is set up under (by default) or a named account you specidy with a proxy / credential. You might be able to access the folder, but you need to check what account SQL Agent is running under and whether it can access it.
To find out which account the job is running under by default you can access it via
Start -> Program files ->SQL Server -> Configuration Tools -> SQL Server Configuration Manager. In the 2005 Services SQL Server Agent will be listed, right click properties, first tab shows what account the Agent is being run under.
However, if you need a job running under a specific account I would advise you create a proxy / credential and not alter the agent account - since that is the account all the other jobs are using.
Credential and proxy are different objects within SQL, Credentials are stored under the security folder / credentials - that is where you create a credential and place your AD username / pwd.
The proxy then asks you to select a credential from the list of credentials already stored on the system - not your domain\user at that point.

Resources