SQL Server Agent Job Can't Read Shared File - sql-server

I've been stuck on this for sometime now. I have an SSIS package thats supposed to read a file and populate a database. I need to run it from a SQL Server Agent Job and the source files to read are located on a folder in another server that I have shared with this server.
The shared path to the folder looks like like this: \\server\D\folder\folder
However when I run agent job through a service account it tells me File name property is not valid. Filename is a device or contains invalid characters
The SQL Server Agent uses a service account to run this job. It runs just fine if the source path is located somewhere on the machine where the database instance lives, however I can't get it to run from a shared folder. If I run it myself by right clicking on the SSIS catalog I can run it just fine. I am aware that it is most likely a credentials issue, but all of these servers and accounts were not set up by me. Can someone help me explain how I should go about adding appropriate permissions to the said SA account so it can read the files successfully? Some examples/references would be greatly appreciated!
Things I've tried: Going to the folder security tab and adding all permissions to everyone on both the server where the folder originally is from and the server that the folder is being shared from. I can confirm everyone has the permissions with the windows PowerShell Get-Acl command.
Switching owner of the job task in SQL Server Agent to my account (I don't think its supposed to work ever to begin with) - this makes Agent complain about being "Unable to determine if user has server acces" with SA account it does have server access, it just can't read the folder.
I saw a post where someone suggests to change the SQL Agent Job advanced step option to "execute as user" and change the user with appropriate credentials, but I don't even see that option in my MSSQL.
I have stumbled upon this thread here , it was never really solved it seems but it looks like the 3 steps given should help me:
Assume that we need to write \serv\share\dir1..\dirN\targetDir\somefile.txt using SSIS throught SQL Agent Job and nonadmin proxy account MyDomain\TestAccount
MyDomain\TestAccount need read/write access to share \serv\share
MyDomain\TestAccount needed at least FILE_READ_DATA permission for all folders (share,dir1,..dirN)
MyDomain\TestAccount needed the CHANGE rights + FILE_DELETE_CHILD permission for folder targetDir
However, me being new to this, I have no idea how to properly check whether or not all these 3 conditions are true and if they are even completely relevant to the problem
EDIT:
There is a project-level variable in SSIS that determines where to read from (in this case set to \\server\d\folder\folder)
This variable is passed into forEach file enumerator in a for loop.
There is also a fileName variable used to check if file name was already loaded in the db as I store them in the table. The variable goes like this:
DECLARE #FileName VARCHAR(50) SET #FileName='' IF EXISTS (SELECT 1 FROM FileLoadStatus WHERE fileName = #FileName) BEGIN SELECT 1 AS FileExistsFlg END ELSE BEGIN SELECT 0 as FileExistsFlg END
If variables are at fault, I still don't know why it works if I execute it manually through catalog myself, but SQL Server Agent is unable to execute it through an SA account
EDIT 2: Exact errors say the following:
EDIT 3: Now that I have set a windows system task to execute the SSIS package instead of a SQL Server Agent Job it just tells me that the "for each file enumerator is empty" basically meaning it can't find any files in the destination to read, even though files are there

it might be a late respond, for all who come to check for an answer to this issue:
the main thing is to be sure that the SQL agent has the authority to read from the shared folder:
1- hold down the Windows key and press R on your keyboard to open the Run command in windows.
2- type services.
3- search for SQL Server Agent.
4- as in the screenshot shows on the logon option you will find which user the agent is using, be sure that this user has the authority to read from the shared folder.
or change the user to another one with the right credentials.
5- you can check the users of the shared folder by right clicking on it and choosing properties --> security. From this window you can change the credentials of the users.

Related

SQL Server Deployed SSIS package won't open files despite SA account having access to folder

I posted a similar question before, but I have now came back to the point where I need to deal with this and after some tuning I have managed to get rid of all the errors, except one warning that basically tells me that the path provided to the package not finding any files in the directory (which is false). This almost makes me want to believe this is again a permissions issue. As when I run the package locally it all works fine. Here is the warning:
Same message appears when I right click SSIS package under the catalogue and directly try to execute the job.
I made sure that the caller has full permissions to the folder in question
SQL Server Agent Job calls the job like so
With a single step in it to execute the SSIS package from the catalogue:
The history of the executions is all successful and the history of SQL Agent job is all green
One of the suggestions I got was to open SQL Server Configuration manager but I don't appear to have access to that. I am new to this whole process so I am not sure if authentication has something to do with it or the sa account's permissions.
Note: I am working on this on a remote dev server and not directly on my pc.
Any help would be greatly appreciated.
Since you're running the package as the Agent Service Account:
The permissions need to be applied to the SQL Server Agent Service Account or its per-service SID.
You can see the service account with PowerShell like this:
PS C:\Users\david> (Get-WmiObject win32_service | where Name -eq "SQLSERVERAGENT").StartName
NT Service\SQLSERVERAGENT
Turns out the database instance was on a whole different server that I wasn't even given access to. That server obviously didn't have the path specified in the SSIS package, so what I had to do is create a folder for files in the correct server, reroute my files there, and change the SSIS package path after obtaining the access to the server where DB instance lives. Me being new to all this, it was absolute frustration especially since our team is small and I am new in it I can't just ask someone questions about this all the time. Hopefully this will save someone a lot of time.

Deployed SSIS package will not read a file

I have an SSIS package that is supposed to read a folder and if the folder contains any text files, it will read them and store the data into a database. The package works flawlessly from the file system, however when I deployed it it fails.
The package uses a project level variable that remains consistent that acts as a folder path to see which folder to look in for files.
The SQL Server Agent gives me this error:
After looking at the execution report I see following error messages if I execute it with the service account through SQL Server Agent:
"The file name property is not valid. The file name is a device or contains invalid characters"
"The file name '(path here)' specified in the connection was not valid"
"The Flat File Connection manager failed validation"
However, if I run it myself by directly right clicking on the package in the catalog and executing, the execution report will come back as successful but it will contain a warning that says: "Read Files From File Input Folder Warning: The For Each File enumerator is empty. The For Each File enumerator did not find any files that matched the file pattern, or the specified directory was empty"
I suspect it is a permissions error, but I haven't dealt much with the permissions in Microsoft SQL Server and the service account that I am using was not created by me. I have tried changing the security of both package and project from the default "Encrypt sensitive data with user key" to "Do not save sensitive data", rebuilding and redeploying but that didn't help at all.
It is most likely a permissions error but I've been puzzled on how to go about it. Any help is much appreciated!
EDIT: After setting the "Delay Validation" on tasks in the control flow and executing the package with SQL Server Agent it now comes back with 2 less errors
For filesystem task to work the executing account need permission to the folder in question. When you execute the package via the SQL-agent the package is executed under the account running the SQL-agent account, when you right click the package and press execute the package is executed as your account.
One problem could be that the SQL-agent account doesn't have permission to the folder, and can't enumerate the content. You can verify this by changing the SQL-server account to your own account temporary and running the job again:
Open Sql Server Configuraiton Manager
Under SQL Server Services double click SQL Server Aggent
Change the account to your own account
Restart the service
If this resolvs the problem, change back to the SQL-server account and check the folder permissions on the folder you are trying to enumerate. Add the SQL-agent account or group the account is included in to have permission to read the folder. Obviously you can also just check the folder permissions to begin with, and make sure the executing account have permission to it.

Executing an SSIS package from SQL job

I have created an SSIS package that simply replicates 1 table to another (on 2 different servers). When I run the SSIS package from the designer, it works perfectly.
However, when I create an SQL Job as it needs to be run automatically. but it fails with a permissions issue.
I am logged onto the to the server with an administrator user.
I have googled this and have tried a host of different things, encrypting the password etc and trying to execute with a proxy user but nothing seems to work.
I would like to know what this user refers to here:
Its the domain name and server name, but I have no idea where this user exists or even if it is an legit user?

Cannot bulk load because the file could not be opened. Operating System Error Code 3

I'm trying to set up a Stored Procedure as a SQL Server Agent Job and it's giving me the following error,
Cannot bulk load because the file "P:\file.csv" could not be opened. Operating system error code 3(failed to retrieve text for this error. Reason: 15105). [SQLSTATE 42000] (Error 4861)
Funny thing is the Stored Procedure works just fine when I execute it manually.
The drive P: is a shared drive on Windows SQL Server from LINUX via Samba Share and it was set up by executing the following command,
EXEC xp_cmdshell 'net use P: "\lnxusanfsd01\Data" Password /user:username /Persistent:Yes'
Any help on this would be highly appreciated
I do not know if you solved this issue, but I ran into the same issue. If the instance is local you must check the permission to access the file, but if you are accessing from your computer to a server (remote access) you have to specify the path in the server, so that means to include the file in a server directory, that solved my case.
example:
BULK INSERT Table
FROM 'C:\bulk\usuarios_prueba.csv' -- This is server path not local
WITH
(
FIELDTERMINATOR =',',
ROWTERMINATOR ='\n'
);
To keep this simple, I just changed the directory from which I was importing the data to a local folder on the server.
I had the file located on a shared folder, I just copied my files to "c:\TEMP\Reports" on my server (updated the query to BULK INSERT from the new folder). The Agent task completed successfully :)
Finally after a long time I'm able to BULK Insert automatically via agent job.
Best regards.
I have solved this issue,
login to server computer where SQL Server is installed get you csv
file on server computer and execute your query it will insert the
records.
If you will give datatype compatibility issue change the datatype for that column
Using SQL connection via Windows Authentication:
A "Kerberos double hop" is happening: one hop is your client application connecting to the SQL Server, a second hop is the SQL Server connecting to the remote "\\NETWORK_MACHINE\". Such a double hop falls under the restrictions of Constrained Delegation and you end up accessing the share as Anonymous Login and hence the Access Denied.
To resolve the issue you need to enable constrained delegation for the SQL Server service account. See here for a good post that explains it quite well
SQL Server using SQL Authentication
You need to create a credential for your SQL login and use that to access that particular network resource. See here
I would suggest the P: drive is not mapped for the account that sql server has started as.
It's probably a permissions issue but you need to make sure to try these steps to troubleshoot:
Put the file on a local drive and see if the job works (you don't necessarily need RDP access if you can map a drive letter on your local workstation to a directory on the database server)
Put the file on a remote directory that doesn't require username and password (allows Everyone to read) and use the UNC path (\server\directory\file.csv)
Configure the SQL job to run as your own username
Configure the SQL job to run as sa and add the net use and net use /delete commands before and after
Remember to undo any changes (especially running as sa). If nothing else works, you can try to change the bulk load into a scheduled task, running on the database server or another server that has bcp installed.
I did try giving access to the folders but that did not help.
My solution was to make the below highlighted options in red selected for the logged in user

Cannot bulk load. Operating system error code 5 (Access is denied.)

For some weird reason I'm having problems executing a bulk insert.
BULK INSERT customer_stg
FROM 'C:\Users\Michael\workspace\pydb\data\andrew.out.txt'
WITH
(
FIRSTROW=0,
FIELDTERMINATOR='\t',
ROWTERMINATOR='\n'
)
I'm confident after reading this that I've setup my user role correctly, as it states...
Members of the bulkadmin fixed server role can run the BULK INSERT statement.
I have set the Login Properties for the Windows Authentication correctly (as seen below).. to grant server-wide permissions on bulkadmin
(source: iforce.co.nz)
And the command EXEC sp_helpsrvrolemember 'bulkadmin' tells me that the information above was successful, and the current user Michael-PC\Michael has bulkadmin permissions.
(source: iforce.co.nz)
But even though I've set everything up correctly as far as I know, I'm still getting the error. executing the bulk insert directly from SQL Server Management Studio.
Msg 4861, Level 16, State 1, Line 2
Cannot bulk load because the file "C:\Users\Michael\workspace\pydb\data\andrew.out.txt" could not be opened. Operating system error code 5(Access is denied.).
which doesn't make sense because apparently bulkadmins can run the statement, am I meant to reconfigure how the bulkadmin works? (I'm so lost). Any ideas on how to fix it?
This error appears when you are using SQL Server Authentication and SQL Server is not allowed to access the bulk load folder.
So giving SQL server access to the folder will solve the issue.
Here is how to:
Go to the folder right click ->properties->Security tab->Edit->Add(on the new window) ->Advanced -> Find Now. Under the users list in the search results, find something like SQLServerMSSQLUser$UserName$SQLExpress and click ok, to all the dialogs opened.
I don't think reinstalling SQL Server is going to fix this, it's just going to kill some time.
Confirm that your user account has read privileges to the folder in question.
Use a tool like Process Monitor to see what user is actually trying to access the file.
My guess is that it is not Michael-PC\Michael that is trying to access the file, but rather the SQL Server service account. If this is the case, then you have at least three options (but probably others):
a. Set the SQL Server service to run as you.
b. Grant the SQL Server service account explicit access to that folder.
c. Put the files somewhere more logical where SQL Server has access, or can be made to have access (e.g. C:\bulk\).
I suggest these things assuming that this is a contained, local workstation. There are definitely more serious security concerns around local filesystem access from SQL Server when we're talking about a production machine, of course this can still be largely mitigated by using c. above - and only giving the service account access to the folders you want it to be able to touch.
I had the same problem SSIS 2012 and the solution was to use Windows Authentication. I was using SQL authentication with the sa user.
Go to start run=>services.msc=>SQL SERVER(MSSQLSERVER) stop the service
Right click on SQL SERVER(MSSQLSERVER)=> properties=>LogOn Tab=>Local System Account=>OK
Restart the SQL server Management Studio.
Try giving the folder(s) containing the CSV and Format File read permissions for ‘MSSQLSERVER’ user (or whatever user the SQL Server service is set to Log On As in Windows Services)
This is what worked for me:
Log on SSIS with Windows authentication.
1. Open services and find MSSQL NT Service account name and copy it:
2. Open folder from which SQL server should read from. Security - Group or user names tab - Add and paste there copied account:**
You will probably get "Multiple names found error", just select MSSQL user:
Your BULK INSERT query should run fine now.
If problem persists try adding SQL Server Agent account to folder permissions in same way.
Make sure you restart MSSQL server in services after you are done.
This is quite simple the way I resolved this problem:
open SQL Server
right click on database (you want to be backup)
select properties
select permissions
select your database role (local or cloud)
in the you bottom you will see explicit permissions table
find " backup database " permission and click Grant permission .
your problem is resolved .
sometimes this can be a bogus error message, tried opening the file with the same account that it is running the process. I had the same issue in my environment and when I did open the file (with the same credentials running the process), it said that it must be associated with a known program, after I did that I was able to open it and run the process without any errors.
Make sure the file you're using ('C:\Users\Michael\workspace\pydb\data\andrew.out.txt') is on the SQL server machine and not the client machine running MSSMS.
1) Open SQL
2) In Task Manager, you can check which account is running the SQL - it is probably not Michael-PC\Michael as Jan wrote.
The account that runs SQL need access to the shared folder.
I have come to similar question when I execute the bulk insert in SSMS it's working but it failed and returned with "Operation system failure code 5" when converting the task into SQL Server Agent.
After browsing lots of solutions posted previously, this way solved my problem by granting the NT SERVER/SQLSERVERAGENT with the 'full control" access right to the source folder.
Hope it would bring some light to these people who are still struggling with the error message.
In our case it ended up being a Kerberos issue. I followed the steps in this article to resolve the issue: https://techcommunity.microsoft.com/t5/SQL-Server-Support/Bulk-Insert-and-Kerberos/ba-p/317304.
It came down to configuring delegation on the machine account of the SQL Server where the BULK INSERT statement is running. The machine account needs to be able to delegate via the "cifs" service to the file server where the files are located. If you are using constrained delegation make sure to specify "Use any authenication protocol".
If DFS is involved you can execute the following Powershell command to get the name of the file server:
Get-DfsnFolderTarget -Path "\\dfsnamespace\share"

Resources