SSIS Flat File In Use By Another Process - file

Question

I've got an SSIS package that is responsible for picking up and loading files to SQL tables, and then routing the files to various archive locations depending upon the content of the file. Prior to doing any loading, however, I have some scripts that check for various file-level errors, one of which is if the file is empty or not. I am using the following script to check if a file is empty:
Dts.Variables["blnEmptyFile"].Value = false;
Dts.Variables["User::strFileTimeStamp"].Value = "_" + DateTime.Now.ToString("yyyyMMdd") + "_" + DateTime.Now.ToString("hhmmss");
if (Dts.Variables["User::strFileRecordType"].Value.ToString() == "AF")
{
Dts.Variables["User::strFileOutToGeBBs"].Value = "AF_" + Dts.Variables["User::strCurrentFileIn"].Value.ToString().Substring(0, Dts.Variables["User::strCurrentFileIn"].Value.ToString().Length - 7) + Dts.Variables["User::strFileTimeStamp"].Value.ToString() + ".txt";
}
else if (Dts.Variables["User::strFileRecordType"].Value.ToString() == "AS")
{
Dts.Variables["User::strFileOutToGeBBs"].Value = "AS_" + Dts.Variables["User::strCurrentFileIn"].Value.ToString().Substring(0, Dts.Variables["User::strCurrentFileIn"].Value.ToString().Length - 7) + Dts.Variables["User::strFileTimeStamp"].Value.ToString() + ".txt";
}
else if (Dts.Variables["User::strFileRecordType"].Value.ToString() == "R")
{
Dts.Variables["User::strFileOutToGeBBs"].Value = "R_" + Dts.Variables["User::strCurrentFileIn"].Value.ToString().Substring(0, Dts.Variables["User::strCurrentFileIn"].Value.ToString().Length - 6) + Dts.Variables["User::strFileTimeStamp"].Value.ToString() + ".txt";
}
else
{
Dts.Variables["User::strFileOutToGeBBs"].Value = Dts.Variables["User::strCurrentFileIn"].Value.ToString().Substring(0, Dts.Variables["User::strCurrentFileIn"].Value.ToString().Length - 4) + Dts.Variables["User::strFileTimeStamp"].Value.ToString() + ".txt";
}
int nonDataRows = 1;
string ffConnection;
ffConnection = (string)(Dts.Connections["CDF_Format"].AcquireConnection(null) as string);
FileInfo flatFileInfo = new FileInfo(ffConnection);
long fileSize = flatFileInfo.Length;
if (fileSize > 0)
{
int lineCount = 0;
StreamReader fsFlatFile = new StreamReader(ffConnection);
while (!(fsFlatFile.EndOfStream))
{
Console.WriteLine(fsFlatFile.ReadLine());
lineCount += 1;
if (lineCount > nonDataRows)
{
Dts.Variables["User::blnEmptyFile"].Value = false;
break;
}
else
{
Dts.Variables["User::blnEmptyFile"].Value = true;
}
}
fsFlatFile.Close();
fsFlatFile = null;
}
else
{
Dts.Variables["User::blnEmptyFile"].Value = true;
}
Dts.TaskResult = (int)ScriptResults.Success;
After this script task, I check the blnEmptyFile variable, and if it is True, I send the empty file to a failed files folder location so that our end users can deal with it. And when I run this script inside the editor (VS 2017/2019), it runs to successful completion every single time. However, when this is run on the ETL servers, it intermittently fails with the following error:
2022-01-18 07:15:14.27 fst_ArchFailedFile:Error: An error occurred with the following error message: "The process cannot access the file because it is being used by another process.".
This only ever happens when a file is empty, and even then, it is only happening with a specific set of files that are empty. And it doesn't happen all the time; it is intermittent, and seems to want to happen on Monday (or, in the case of long weekends, Tuesday) mornings.
As you can see in the code, I am closing fsFlatFile, and I'm setting it to null. ffConnection is a string, so I can't close/kill it...although setting it to NULL has no effect either. I've tried setting a PAUSE, and that also has no effect/bearing on what is happening.
Anybody have any ideas why this would happen, and how I can resolve this? We aren't talking about Excel here (I've seen all of those posts on the internet), but simple TXT files that have only but a header and nothing else in them. Any help here would be keen.

Answer 1

StreamReader still has it. Try using a using block, everything inside is properly closed, disposed, etc:
using(StreamReader fsFlatFile = new StreamReader(ffConnection))
{
All the code you want to do while your stream is open.
}