Powershell Script Taking Long Time when seems like it shouldnt? - arrays

Question

So I have a script that is meant to send some pretty huge (700mb) txt files to an FTP of a service we use that automatically set's our prices across a fleet of domains.
I'm using a bitearray (as found here) to upload it to the site, and I have some very elementary built in error handling as well as some database tie-ins.
If we run these seperately, they move pretty fast. For whatever reason, after the completion of one block, the time until starting another block is CRAZY volatile. Sometimes it's 2 minutes, sometimes the script just sits around for a good 40 minutes before moving to the next block.
I'm sort of assuming the issue is I'm giving the thing more housekeeping than I should be? It's also worth noting that even stopping the script can sometimes take 15 minutes. (Like, If I just hit break on the script in the middle of it running, it can take a good 15-20 minutes to stop)
Also, for what it's worth, the script got MUCH worse in terms of runtime in the last few days. I have no idea what we could have changed to make it start taking so much longer but here we are.
Anyway, any insights would be appreciated.
Notes: I don't clear the content variable when I clear variables, should I?
Should I keep the rs open? I don't think I CAN because I'm connecting to the FTP with different usernames.
Here's the code (I actually have about 12 of the *.txt Blocks, but They're identicle so I've kept it down to three here):
#================================== SETUP BLOCK ==================================#
#get dat email ready
$strpasswd = Get-Content "PASSWORDFILE" | ConvertTo-SecureString
$mycreds = New-Object System.Management.Automation.PSCredential ("EXCHANGEUSER",$strpasswd)
$EmailTo = 'some#email','goes#here'
$EmailFrom = 'EXCHANGEUSER'
$EmailSubject = "CA Feed Issue Undefined Subject"
$emailbody = "Body Not Yet Defined"
$SmtpServer = 'MUHSERVER'
#Opens up database session so we can send queries
$strserver = "Server\MUHDB"
$strdatabase = "logs"
$strusername = "EXCHANGEUSER"
#createsdatabaseconnection
$sqlConnection = new-object System.Data.SqlClient.SqlConnection "server='$strserver';database='$strdatabase';Integrated Security=SSPI; User ID='$strusername'; password='$strpassword'"
$sqlConnection.Open()
#define the defaultquery
$strQuery =
"
INSERT INTO [logs].[dbo].[EventLog] (SourceID, Started, Completed, Result, Context, Machine)
values (50,1500,1500,'NOTEDEFINED','NOTDEFINED','Server\MUHCLIENTMACHINE-CAFeed')
"
#this is how I execute the command
#$sqlCommand = $sqlConnection.CreateCommand()
#$sqlCommand.CommandText = $strquery
#$sqlCommand.ExecuteReader()
#==================================Luna.txt ==================================#
##DEFINE THESE TO CREATE NEW FEEDS
$strFilename = "\\PATH\Luna.txt"
$ftp = [System.Net.FtpWebRequest]::Create("FTPLINK1")
$user = "USERNAME1"
$password = "PASSWORDREDACTED"
# create the FtpWebRequest and configure it
$ftp = [System.Net.FtpWebRequest]$ftp
# build authentication and connection
$ftp.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftp.Credentials = new-object System.Net.NetworkCredential($user,$password)
$ftp.UseBinary = $true
$ftp.UsePassive = $true
$ftp.timeout = -1
#start a timer and error handling
$starttime = (get-date).ToString()
$error.Clear()
# read in the file to upload as a byte array
$content = [System.IO.File]::ReadAllBytes("$strfilename")
$ftp.ContentLength = $content.Length
# get the request stream, and write the bytes into it
$rs = $ftp.GetRequestStream()
$rs.Write($content, 0, $content.Length)
$endtime = (get-date).ToString()
#error handle
if ($error)
{
#Assemble the Query
$sqlresult = "THERE IS AN ERROR, Check the error email for details"
$sqlcontext = ($strfilename + '|' + $content.length + ' bytes')
$strquery =
"INSERT INTO [logs].[dbo].[EventLog] (SourceID, Started, Completed, Result, Context, Machine)
values (50,'$starttime','$endtime','$sqlresult','$sqlcontext','Server\MUHCLIENTMACHINE-CAFEEDSCRIPT')"
#Create Command and Execute.
$sqlCommand = $sqlConnection.CreateCommand()
$sqlCommand.CommandText = $strQuery
$sqlCommand.ExecuteNonQuery()
#Send dem emails
$emailbody = "A file for the CA Feed failed on $strfilename at " + (get-date).ToString() + " with the error '$error[0]'"
$emailsubject = "CA Feed Failed File"
Send-MailMessage -SmtpServer $SmtpServer -to $EmailTo -from $EmailFrom -subject $EmailSubject -Body $emailbody
}
else
{
write-host ("$strfilename" + ' Ran Without Errors')
$sqlresult = "RAN WITHOUT ERRORS"
$sqlcontext = ($strfilename + '|' + $content.length + ' bytes')
$strquery =
"INSERT INTO [logs].[dbo].[EventLog] (SourceID, Started, Completed, Result, Context, Machine)
values (50,'$starttime','$endtime','$sqlresult','$sqlcontext','Server\MUHCLIENTMACHINE-CAFEEDSCRIPT')"
#Create a command object.
$sqlCommand = $sqlConnection.CreateCommand()
$sqlCommand.CommandText = $strQuery
$sqlCommand.ExecuteNonQuery()
}
# be sure to clean up after ourselves and get ready for next block
Clear-Variable -Name starttime,endtime,strfilename,sqlresult,sqlcontext,ftp
$rs.Close()
$rs.Dispose()
#==================================LDE.txt ==================================#
##DEFINE THESE TO CREATE NEW FEEDS
$strFilename = "\\PATH\LDE.txt"
$ftp = [System.Net.FtpWebRequest]::Create("FTPLINK2")
$user = "USERNAME2"
$password = "PASSWORDREDACTED"
# create the FtpWebRequest and configure it
$ftp = [System.Net.FtpWebRequest]$ftp
# build authentication and connection
$ftp.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftp.Credentials = new-object System.Net.NetworkCredential($user,$password)
$ftp.UseBinary = $true
$ftp.UsePassive = $true
$ftp.timeout = -1
#start a timer and error handling
$starttime = (get-date).ToString()
$error.Clear()
# read in the file to upload as a byte array
$content = [System.IO.File]::ReadAllBytes("$strfilename")
$ftp.ContentLength = $content.Length
# get the request stream, and write the bytes into it
$rs = $ftp.GetRequestStream()
$rs.Write($content, 0, $content.Length)
$endtime = (get-date).ToString()
#error handle
if ($error)
{
#Assemble the Query
$sqlresult = "THERE IS AN ERROR, Check the error email for details"
$sqlcontext = ($strfilename + '|' + $content.length + ' bytes')
$strquery =
"INSERT INTO [logs].[dbo].[EventLog] (SourceID, Started, Completed, Result, Context, Machine)
values (50,'$starttime','$endtime','$sqlresult','$sqlcontext','Server\MUHCLIENTMACHINE-CAFEEDSCRIPT')"
#Create Command and Execute.
$sqlCommand = $sqlConnection.CreateCommand()
$sqlCommand.CommandText = $strQuery
$sqlCommand.ExecuteNonQuery()
#Send dem emails
$emailbody = "A file for the CA Feed failed on $strfilename at " + (get-date).ToString() + " with the error '$error[0]'"
$emailsubject = "CA Feed Failed File"
Send-MailMessage -SmtpServer $SmtpServer -to $EmailTo -from $EmailFrom -subject $EmailSubject -Body $emailbody
}
else
{
write-host ("$strfilename" + ' Ran Without Errors')
$sqlresult = "RAN WITHOUT ERRORS"
$sqlcontext = ($strfilename + '|' + $content.length + ' bytes')
$strquery =
"INSERT INTO [logs].[dbo].[EventLog] (SourceID, Started, Completed, Result, Context, Machine)
values (50,'$starttime','$endtime','$sqlresult','$sqlcontext','Server\MUHCLIENTMACHINE-CAFEEDSCRIPT')"
#Create a command object.
$sqlCommand = $sqlConnection.CreateCommand()
$sqlCommand.CommandText = $strQuery
$sqlCommand.ExecuteNonQuery()
}
# be sure to clean up after ourselves and get ready for next block
Clear-Variable -Name starttime,endtime,strfilename,sqlresult,sqlcontext,ftp
$rs.Close()
$rs.Dispose()

Answer 1

I don't think anybody is going to debug that code. Your best bet is to find where your issue is. I use a stopwatch like below. set it up strategically:
$SW = [System.Diagnostics.Stopwatch]::new()
$SW.Start()
#Your code block goes here
Write-Host "End of Code block 1"
$SW.Elapsed.TotalSeconds
#Another code block goes here
Write-Host "End of Code block 2"
$SW.Elapsed.TotalSeconds
Now if you are trying to break out and it is taking 15 mins to respond, it is probably stuck doing an operation. It cannot respond until the operation finishes or fails.