We've run into a strange issue. We have some SSIS 2008 packages that had been working fine for quite a while. As part of the work flow they copy files to a backup folder, and then delete the original file. (No, I don't know why they didn't just make it a File Move task.) Suddenly, the delete task has stopped working. It's not just our packages either, it's the packages of a different group on a different SSIS server. Same target folders though.
The server team has gone over the folder permissions, the account the jobs run under have full access. And, if it was an access issue, how could we move the file but not delete it?
We finally found the answer. The file server we're using is Linux based. It is case sensitive, but doesn't throw an error if you use the wrong case. Was a very tough issue for us to track down.
Related
this is one that has me stumped and Ive been doing this a long while.
Migrating to SQL server 2016, large number of ETL. Easy enough.
One of the ETL packages has a simple script task to take a table of files, run a file exists foreach loop.
it uses a project parameter to create the unc ( \servername\share) and then binds that to the file name in the script task.
use an environment config setup in SSISDB
execute in SSDT works fine, deploy to catalog and it cant see the file. i know youll say permissions, but ive permissioned everyone group to share and drive in case its that. SSISDB execution means it should be running under my security context and im domain admin, local admin and creator owner of the share.
even strangeR, i have created simple package to grab the contents of one of the files and import into a dump table in case permissions or pathway were duff ( even though they work in SSDT might be the enviroNment config in SSISDB). THIS WORKS FINE, therefore it cant be the envrionment setup of SSISDB being referenced.
please note this is not running from an agent job yet so wont be due to agent server account issue. need to get it running from ssisdb first then ill create an agent job
So -- script task cant see unc share, built from two variables, that works in ssdt and its running under same credentials...
Go
For what its work the script task code is
Dts.Variables("BolFileExists").Value = File.Exists(Dts.Variables("StrLoadFileLocation").Value.ToString & Dts.Variables("StrCurrentFile").Value.ToString)
This is a slightly different answer as it shows a different approach and removes the script task. I use a foreach to check if the file exists using GUI tools provided by SSIS:
Well I found the answer and I deserve to punch myself in the face.
Tried everything, it was a file variable and path variable being pulled together in the script task so tried concatenation that before the script task, pumped this into a table to ensure it was going to write table.
Literally everything was fine and still didn’t work.
The issue....
Building it as a 2017 package onto a 2016 Sql server.
I’ve not found what was missing dll wise but it must have been one of those that meant the script task couldn’t find the files but weird it didn’t break and just said the files weren’t there!
Thanks all for input, I’m going to go put my head in the door and slam it
Modifying this post as a friend has helped me figure it out.
The culprit was that SQL was not able to map drive tag to network shared folder, so the deployed SSIS package was not able to write. The execution report showed all green and success, so I was confused as a beginner. See also the comments below.
Backup original post below:
SSIS package text file write works in visual studio not when deployed on sql server
I have narrowed down the issue to the text file writing action in script task (C#), the experiment simply writes the current time stamp into a text file.
It works in Visual Studio 2015, both with (F5) and without (Cntl+F5) Debugger. The project is in package deployment mode. When deployed to a database server of SQL Server 2016 and manually trigger execute with Administrator login, the writing action never happens although execution report shows all success, and Windows system log shows no clue to me either.
I am a beginner on SSIS and hints and tips will be highly appreciated.
Yes, SSIS doesn't like mapping shared folder to a letter drive. Thanks #Nick.McDermaid .
Never use mapped drives. Use UNC instead i.e. \server\share\folder –
Nick.McDermaid Nov 8 '18 at 23:40
Noone on my company's IT is proficient with SQL and servers and an issue has arisen, which no one seems to be able to fix nor is particular eager to. I know a bit but I'm not in IT hence trying to figure out the issue myself.
Anyway, there is a sql server agent job which is supposed to take a daily backup file of a database and delete the previous backup file. The part of creating the .bak file is working but the deletion of the previous day's file is not.
I've found the code, i think, which refers to this delete bit, or rather the parameters (again I think):
EXECUTE master.dbo.xp_sqlmaint N''-D CompanyLive -WriteHistory -VrfyBackup -BkUpDB "J:\Backups" -BkExt "BAK" -BkUpMedia DISK -DelBkUps 1days''',
#database_name=N'CompanyLive',
#flags=4
Shot in the dark, but any glaring errors in this??
Thanks
Each time I run the Microsoft Windows Backup and Restore App that is left over since the Microsoft Windows 7 Operating System, I get an error that some Files are missing and the Backup Process fails.
The Files are actually Folders. I have uninstalled some Apps in the meantime and now there is only one missing Folder that the Backup App does not find.
I have tried to run a Batch File within the CMD.EXE Command-Line Processor App with System Administrator Rights:
#ECHO OFF
SET DIR1="C:\Windows\System32\config\systemprofile\OneDrive\Pictures\Saved Pictures"
MKDIR %DIR1%
PAUSE
The Folder does get created well and nice, but the Backup App is still failing.
Could it be a Rights Dead-Lock?
I am creating the Folder using System Administrator Privileges because it is not possible otherwise.
I suspect that the Backup App is run with Normal Rights. However, the User Account that I am using is also part of the Administrators Group.
Please advise.
I could not reproduce this Issue.
The reason why I guess that this is happening is the following one:
The Microsoft Windows Insider Program is constantly rewriting the whole C:\Windows Folder on each Update, therefore the Folders that are missing have to be constantly recreated.
Earlier, I might have manually started the Microsoft Windows Backup and Restore Application and forgot to run the Batch File. The Application might have started to work on the Files and Folders to back up. Then, I might have manually run the Batch File that correctly created the Files and Folders, but that might have been too late - that is, after the Application already considered them as missing. Therefore, the error was happening.
I do not know for sure whether this is the cause for this error since I have encountered it a number of times, not only once, and I do not feel that it was possible to have manually run the Batch File later than needed each time.
Anyway, a possible workaround for this Issue might be the following one:
Create a Scheduled Task that first runs the Batch File and then runs the Microsoft Windows Backup and Restore Application. I do not know yet how to tell the Scheduler to automatically run the Application, but I can imagine that it might not be difficult to achieve this goal.
Then, whenever the manual Backup is needed to be performed, one can simply manually run the Scheduled Task. This way, this Issue might not reoccur, at least because the previously suspected behavior should be avoided.
I need to perform the backup manually because I am using Removable Disks as a Third Backup Solution. The First One is the ASUS Web Storage Cloud Provider and Synchronizer Application and the Second One is the File History Application run on an External Winchester Hard Disk Drive.
If anybody has a better solution for this Issue, then please let me know.
I need to import a flat file daily. The file changes its name every day. After the file is processed, it needs to be moved to another folder.
I noticed I can schedule jobs in the SQL Server Agent, and that I can tell it to run every hour or so and that I am able to add CMD commands to it.
The solution I found was to run a script to check if the file exists, since the folder should be empty or have at least one file.
If the file exists, the script renames the file to one used in the SSIS package and then it runs the SSIS package.
After the whole thing is done, it should rename the file again based on today's date and move it to another folder.
If the file does not exist, then it should do nothing and wait another hour or so to run again.
What's the best solution to this scenario? Is the script a good idea? Maybe is it possible to add the if/else -for the file exists- into the SSIS package? Or even make the script run from the SSIS package itself instead of adding it to the Server Agent?
EDIT:
It seems I was a little naïve, it's possible to run VB scripts from the server. Would that be the recommended solution? It does solve my problem, but I'm just wondering if it's a good idea.
This solves all my questions:
http://www.sqlservercentral.com/articles/Integration+Services+%28SSIS%29/90571/