Help with SSIS Package - sql-server

Hi all i've got a complex SSIS package, however i'm hitting my head against a brick wall with a particular part. I've got a maintenance part that will delete files that are older than 3 months old (from today's date). The files all have the date in the filename, for example AA-CDR-20110606030000-2-001A648E6F74-026874.xml
So i've written a task that will loop over all the files in a particular folder using a Foreach loop, then i have a script that will load in the filename using a variable set using the foreach loop. Then delete the file using the script below. This works fine when running in debug mode. However when trying to execute this on the server i get a failure with a "System.FormatException: String was not recognized as a valid DateTime.". I really dont understand why, any ideas?
DateTime deleteDate = DateTime.Today.AddMonths(-3);
String file = Dts.Variables["FileName"].Value.ToString();
String fileDateString = file.Substring(42, 8);
String fileDateYear = fileDateString.Substring(0, 4);
String fileDateMonth = fileDateString.Substring(4, 2);
String fileDateDay = fileDateString.Substring(6, 2);
DateTime fileDateTime = Convert.ToDateTime(fileDateDay + "-" + fileDateMonth + "-" + fileDateYear);
if (fileDateTime < deleteDate)
{
System.IO.File.Delete(file);
}

It seems that there is a file on the production server that does not follow the pattern and the date cannot be extracted from its name.
Try to use something like this:
try
{
DateTime fileDateTime = Convert.ToDateTime(fileDateDay + "-" + fileDateMonth + "-" + fileDateYear);
if (fileDateTime < deleteDate)
{
System.IO.File.Delete(file);
}
}
catch (System.FormatException)
{
// log the Dts.Variables["FileName"] value here to see why it could not be parsed
}

Related

Apache Camel consumer template to copy file, cannot copy one file twice

Hi I am using apache camel 2.15.2. I have got a consumer template so that I can copy file with dynamic file names:
if (fileInfo != null) {
filename = fileInfo.getFileName();
String camelUri = "file://" + fileInfo.getCopyFilePath() + "/?fileName=RAW("
+ filename + ")&noop=false&idempotent=false&readLock=changed";
System.out.println("Camel uri: " + camelUri);
logger.info("Camel uri: " + camelUri);
Exchange ex = consumerTemplate.receive(camelUri);
....
As you can see, I have set noop, and idempotent explicitly to achieve copying same file more than once. But, it does not do that. It hangs on receive method for subsequent tries to copy a file with same name. It can copy that, only if we restart the application. Any suggestions would be much appreciated. It might be something similar to this issue, but I do not have access to that solution. Thanks in advance.
When I debugged through Camel code, it seems, it is calling EventDrivenPollingConsumer's receive method, and hangs when calls queue.take() (line 110, EventDrivenPollingConsumer). And, even inside that, 'count' variable is zero in ArrayBlockingQueue:
while (count == 0)
notEmpty.await();
Added this, just in case it helps anyone having a clue.
Ok, If I call 'consumerTemplate.doneUoW(ex)', it does copy multiple times. But, at the same time it was deleting (actually moving to .camel folder) the source file, which I did not want to! Then, had to set noop=true:
if (fileInfo != null) {
filename = fileInfo.getFileName();
String camelUri = "file://" + fileInfo.getCopyFilePath() + "/?fileName=RAW("
+ filename + ")&noop=true&idempotent=false&readLock=none";
System.out.println("Camel uri: " + camelUri);
logger.info("Camel uri: " + camelUri);
Exchange ex = consumerTemplate.receive(camelUri);
// consumerTemplate.r
logger.info("File received: " + fileInfo.getFileName());
exchange.getOut().setBody(ex.getIn().getBody());
consumerTemplate.doneUoW(ex);
}
Now, it works as expected.

Multiple outputs with a for loop in praat

I have a script where I have multiple folders each with three audio files in them ID#_1, ID#_2, and ID#_3. The user can input a string of different ID#s, one after the other, and then the script recognizes the different IDs and runs the code for each of them.
I have a for loop set up for this -
form Settings
comment Enter the IDs of the different subjects
sentence subjectIDs
endform
numOfSubjects = length(subjectIDs$)/4
for i from 0 to (numOfSubjects - 1)
subjectID$ = mid$(subjectIDs$, 1 + 4*i, 4 + 4*i)
outFile$ = subjectID$ + "/SubjectResponseOnsets" + subjectID$ + ".txt"
path$ = subjectID$ + "/" + subjectID$
#firstOutput
#secondOutput
#thirdOutput'
Each of these procedures is defined previously in the code, and they basically output certain ranges from the audio files out to a text file.
The code seems to work fine and generate the output file correctly when one ID is given, but when I try to run it with more than one ID at a time, only the text file for the first ID is outputted.
The for loop does not seem to be working well, but the code does work fine in the first run.
I would greatly appreciate any help!
I don't know if I understood well what your script was trying to do, since the snippet you pasted was incomplete. It's best if you provide code that is executable as is. In this case, you were missing the closing endfor, and you were calling some procedures that were not defined in your snippet (not even as placeholders). I had to write some dummy procedures just to make it run.
Since you also didn't say how your script was failing, it was unclear what needed to be fixed. So I took a stab at making it work.
It sounded as if your ID splitting code was giving you some problems. I took the split procedure from the utils plugin available through CPrAN, which makes inputting the IDs easier (full disclosure: I wrote that plugin).
form Settings
comment Enter the IDs of the different subjects
sentence subjectIDs 01 02 03
endform
#split: " ", subjectIDs$
numOfSubjects = split.length
for i to numOfSubjects
subjectID$ = split.return$[i]
path$ = subjectID$
outFile$ = path$ + "/SubjectResponseOnsets" + subjectID$ + ".txt"
# Make sure output directory exists
createDirectory: path$
#firstOutput
#secondOutput
#thirdOutput
endfor
procedure firstOutput ()
appendFileLine: outFile$, "First"
endproc
procedure secondOutput ()
appendFileLine: outFile$, "Second"
endproc
procedure thirdOutput ()
appendFileLine: outFile$, "Third"
endproc
# split procedure from the utils CPrAN plugin
# http://cpran.net/plugins/utils
procedure split (.sep$, .str$)
.seplen = length(.sep$)
.length = 0
repeat
.strlen = length(.str$)
.sep = index(.str$, .sep$)
if .sep > 0
.part$ = left$(.str$, .sep-1)
.str$ = mid$(.str$, .sep+.seplen, .strlen)
else
.part$ = .str$
endif
.length = .length+1
.return$[.length] = .part$
until .sep = 0
endproc
If this is not what you are having trouble with, you'll have to be more specific.

Find string in log files and return extra characters

How can I get Python to loop through a directory and find a specific string in each file located within that directory, then output a summary of what it found?
I want to search the long files for the following string:
FIRMWARE_VERSION = "2.15"
Only, the firmware version can be different in each file. So I want the log file to report back with whatever version it finds.
import glob
import os
print("The following list contains the firmware version of each server.\n")
os.chdir( "LOGS\\" )
for file in glob.glob('*.log'):
with open(file) as f:
contents = f.read()
if 'FIRMWARE_VERSION = "' in contents:
print (file + " = ???)
I was thinking I could use something like the following to return the extra characters but it's not working.
file[:+5]
I want the output to look something like this:
server1.web.com = FIRMWARE_VERSION = "2.16"
server2.web.com = FIRMWARE_VERSION = "3.01"
server3.web.com = FIRMWARE_VERSION = "1.26"
server4.web.com = FIRMWARE_VERSION = "4.1"
server5.web.com = FIRMWARE_VERSION = "3.50"
Any suggestions on how I can do this?
You can use regex for grub the text :
import re
for file in glob.glob('*.log'):
with open(file) as f:
contents = f.read()
if 'FIRMWARE_VERSION = "' in contents:
print (file + '='+ re.search(r'FIRMWARE_VERSION ="([\d.]+)"',contents).group(1))
In this case re.search will do the job! with searching the file content based on the following pattern :
r'FIRMWARE_VERSION ="([\d.]+)"'
that find a float number between two double quote!also you can use the following that match anything right after FIRMWARE_VERSIONbetween two double quote.
r'FIRMWARE_VERSION =(".*")'

Python Simple PiggyBank Program

This is my Python Program that I have been having some issues with:
-- coding: cp1252 --
from time import gmtime, strftime
print("Welcome to the PiggyBank version 1.")
num_write = int(input("How much money would you like to store in your PiggyBank?"))
f = open("PiggyBanks_Records.txt", "w")
current_time = strftime("%Y-%m-%d %H:%M:%S", gmtime())
convert_1 = str(current_time)
convert_2 = str(int(num_write))
add_1 = ("\n" + convert_1 + " £" + convert_2)
add_2 = ("\n" + add_1) #Tried to make it so new line is added every time the program is run
final_record = str(add_2)
print("Final file written to the PiggyBank: " + final_record)
#Write to File
f.write(final_record)
f.close()
Right now whenever the program writes to the file it over-writes. I would preferably would like to keep, like a history of the amounts added. If anyone can help so the string that needs to be written to the .txt file goes down by one line and essentially keeps going for ever. I am also open to any suggestion on how I can shorten this code.
You need to open your file with append mode :
f = open("PiggyBanks_Records.txt", "a")
Using the 'w' write option with open automatically looks for the specified file, and deletes its contents if it already exists (which you can read about here) or creates it if it doesn't. Use 'a' instead to add / append to the file.

Groovy - create file issue: The filename, directory name or volume label syntax is incorrect

I'm running a script made in Groovy from Soap UI and the script needs to generate lots of files.
Those files have also in the name two numbers from a list (all the combinations in that list are different), and there are 1303 combinations
available and the script generates just 1235 files.
A part of the code is:
filename = groovyUtils.projectPath + "\\" + "$file"+"_OK.txt";
targetFile = new File(filename);
targetFile.createNewFile();
where $file is actually that part of the file name which include those 2 combinations from that list:
file = "abc" + "-$firstNumer"+"_$secondNumber"
For those file which are not created is a message returned:"The filename, directory name or volume label syntax is incorrect".
I've tried puting another path:
filename = "D:\\rez\\" + "\\" + "$file"+"_OK.txt";
targetFile = new File(filename);
targetFile.createNewFile();
and also:
File parentFolder = new File("D:\\rez\\");
File targetFile = new File(parentFolder, "$file"+"_OK.txt");
targetFile.createNewFile();
(which I've found here: What are possible reasons for java.io.IOException: "The filename, directory name, or volume label syntax is incorrect")
but nothing worked.
I have no ideea where the problem is. Is strange that 1235 files are created ok, and the rest of them, 68 aren't created at all.
Thanks,
My guess is that some of the files have illegal characters in their paths. Exactly which characters are illegal is platform specific, e.g. on Windows they are
\ / : * ? " < > |
Why don't you log the full path of the file before targetFile.createNewFile(); is called and also log whether this method succeeded or not, e.g.
filename = groovyUtils.projectPath + "\\" + "$file"+"_OK.txt";
targetFile = new File(filename);
println "attempting to create file: $targetFile"
if (targetFile.createNewFile()) {
println "Successfully created file $targetFile"
} else {
println "Failed to create file $targetFile"
}
When the process is finished, check the logs and I suspect you'll see a common pattern in the ""Failed to create file...." messages
File.createNewFile() returns false when a file or directory with that name already exists. In all other failure cases (security, I/O) it throws an exception.
Evaluate createNewFile()'s return value or, additionally, use the File.exists() method:
File file = new File("foo")
// works the first time
createNewFile(file)
// prints an error message
createNewFile(file)
void createNewFile(File file) {
if (!file.createNewFile()) {
assert file.exists()
println file.getPath() + " already exists."
}
}

Resources