Ant: How to stop copying on duplicates - file

I have a directory with several sub-directories in it. Inside there are files. Out of those file1.ext is a duplicate present in more than 1 directory. The sub-directories and filenames are obviously unknown at the time of execution.
-> bigdir
--------> subdir1
----------------> file1.ext
--------> subdir2
----------------> file1.ext
--------> subdir3
----------------> file2.ext
I am trying to copy contents of bigdir into smalldir using flatten="true" so that my resultant file structure looks like
-> smalldir
----------> file1.ext
----------> file2.ext
So far, I have the following
<copy todir="${smalldir}" flatten="true" verbose="true">
<fileset dir="${bigdir}">
<include name="**/*.ext"/>
</fileset>
</copy>
This works fine and achieves the result that I want, HOWEVER, it is an error to have duplicate files in bigdir. I want the build to fail if it detects that bigdir has duplicate filenames inside
I was trying to find a way to make the <copy> task fail in such case. It doesn't have to be through this approach. I can happily run some kind of validation on the directory before copying too, if I could figure out how.
Maybe make each copied file read-only, so that subsequent overwrites as a result of copy flatten=true fail to overwrite and produce an error?
No ant-contrib please

You may use a scriptcondition (see ant manual conditions) with builtin javascript engine
(included since Java 1.6.06, so no extra libraries or ant addons needed), like that :
<project>
<fileset dir="C:\tmp\bigdir" id="whatever"/>
<echo>${toString:whatever}</echo>
<fail message="Duplicate Filenames detected !!">
<condition>
<scriptcondition language="javascript">
<![CDATA[
importPackage(java.util);
var input = project.getProperty('toString:whatever').split(';');
// get basenames
for (var i = 0; i < input.length; i++) {
input[i]=input[i].substring(input[i].lastIndexOf("\\")+1);
}
var inputList = Arrays.asList(input);
// no duplicates in Hashset
var inputSet = new HashSet(inputList);
self.setValue(inputSet.size() < inputList.size());
]]>
</scriptcondition>
</condition>
</fail>
</project>
output :
[echo] subdir1\foo.txt;subdir2\foo.txt;subdir3\foobar.txt
BUILD FAILED
C:\area51\AntTest\tryme.xml:7: Duplicate Filenames detected !!
-- EDIT after comment --
To display the duplicates use something like :
<project>
<fileset dir="C:\bigdir" id="whatever"/>
<echo>${toString:whatever}</echo>
<fail message="Duplicate Filenames detected !!">
<condition>
<scriptcondition language="javascript">
<![CDATA[
importPackage(java.util);
// get DirectoryScanner
ds = whatever.getDirectoryScanner(project);
// get the included files => array
checkFiles = ds.getIncludedFiles();
var checkSet = new HashSet();
var s = "";
for (j=0; j < checkFiles.length; j++) {
f = new java.io.File(whatever.getDir(project), checkFiles[j]);
if(!checkSet.add(f.getName()))
s += f + '\n';
}
println(s.substring(0, s.length - 1));
self.setValue(checkSet.size() < checkFiles.length);
]]>
</scriptcondition>
</condition>
</fail>
</project>
output :
[echo] subdir1\foo.txt;subdir2\foo.txt;subdir3\foobar.txt;subdir3\subsubdir1\foo.txt;subdir3\subsubdir2\foobar.txt;subdir4\foobaz.txt
[fail] C:\bigdir\subdir2\foo.txt
[fail] C:\bigdir\subdir3\subsubdir1\foo.txt
[fail] C:\bigdir\subdir3\subsubdir2\foobar.txt
BUILD FAILED
Btw, often it's much simpler to use the ant api :
// get DirectoryScanner
ds = whatever.getDirectoryScanner(project);
// get the included files => array
checkFiles = ds.getIncludedFiles();
instead of :
var input = project.getProperty('toString:whatever').split(';');
// get basenames
for (var i = 0; i < input.length; i++) {
input[i]=input[i].substring(input[i].lastIndexOf("\\")+1);
}
as i used in first snippet.

Related

Error loading physionet ECG database on MATLAB

I'm using this code to load the ECG-ID database into MATLAB:
%% Initialization
clear all; close all; clc
%% read files from folder A
% Specify the folder where the files live.
myFolder = 'Databases\ECG_ID';
% Check to make sure that folder actually exists. Warn user if it doesn't.
if ~isfolder(myFolder)
errorMessage = sprintf('Error: The following folder does not exist:\n%s\nPlease specify a new folder.', myFolder;)
uiwait(warndlg(errorMessage);)
myFolder = uigetdir(; % Ask for a new one.)
if myFolder == 0
% User clicked Cancel
return;
end
end
% Get a list of all files in the folder with the desired file name pattern.
filePattern = fullfile(myFolder, '**/rec_*'; % Change to whatever pattern you need.)
theFiles = dir(filePattern;)
for k = 1 : length(theFiles)
baseFileName = theFiles(k.name;)
fullFileName = fullfile(theFiles(k.folder, baseFileName);)
fprintf(1, 'Now reading %s\n', fullFileName;)
% Now do whatever you want with this file name,
% such as reading it in as an image array with imread()
[sig, Fs, tm] = rdsamp(fullFileName, [1],[],[],[],1;)
end
But I keep getting this error message:
Now reading C:\Users\******\Documents\MATLAB\Databases\ECG_ID\Person_01\rec_1.atr
Error using rdsamp (line 203)
Could not find record: C:\Users\******\Documents\MATLAB\Databases\ECG_ID\Person_01\rec_1.atr. Search path is set to: '.
C:\Users\******\Documents\MATLAB\mcode\..\database\ http://physionet.org/physiobank/database/'
I can successfully load one signal at a time (but I can't load the entire database using the above code) using this command:
[sig, Fs, tm] = rdsamp('Databases\ECG_ID\Person_01\rec_1');
How do I solve this problem? How can I load all the files in MATLAB?
Thanks in advance.

Find string in log files and return extra characters

How can I get Python to loop through a directory and find a specific string in each file located within that directory, then output a summary of what it found?
I want to search the long files for the following string:
FIRMWARE_VERSION = "2.15"
Only, the firmware version can be different in each file. So I want the log file to report back with whatever version it finds.
import glob
import os
print("The following list contains the firmware version of each server.\n")
os.chdir( "LOGS\\" )
for file in glob.glob('*.log'):
with open(file) as f:
contents = f.read()
if 'FIRMWARE_VERSION = "' in contents:
print (file + " = ???)
I was thinking I could use something like the following to return the extra characters but it's not working.
file[:+5]
I want the output to look something like this:
server1.web.com = FIRMWARE_VERSION = "2.16"
server2.web.com = FIRMWARE_VERSION = "3.01"
server3.web.com = FIRMWARE_VERSION = "1.26"
server4.web.com = FIRMWARE_VERSION = "4.1"
server5.web.com = FIRMWARE_VERSION = "3.50"
Any suggestions on how I can do this?
You can use regex for grub the text :
import re
for file in glob.glob('*.log'):
with open(file) as f:
contents = f.read()
if 'FIRMWARE_VERSION = "' in contents:
print (file + '='+ re.search(r'FIRMWARE_VERSION ="([\d.]+)"',contents).group(1))
In this case re.search will do the job! with searching the file content based on the following pattern :
r'FIRMWARE_VERSION ="([\d.]+)"'
that find a float number between two double quote!also you can use the following that match anything right after FIRMWARE_VERSIONbetween two double quote.
r'FIRMWARE_VERSION =(".*")'

Looping through A Folder and all its subdirectories

Ok. Many trouble shooting hours ...and many error "dings" later, I'm still having the same problem. Due to my beginner skills I'm having problems achieving the following segment of my project:
I will be as detailed as possible so I can hopefully nail it this time:
On my computer i have a folder C:\data which contains many different subfolders.
The subfolders are named by dates in a MMDDYY fashion. For example "040312"
In each subfolder are excel files named after Baseball teams. each subfolder may contain a different combination of xls files.
I am trying to write code that achieves the following objectives:
1.) Loops through all the subfolders of the C:\data folder looking for xls files that have the filenames: Angles.xls, Diamondbacks.xls, etc.
2.) If the files are found in each subfolder import the spreadsheet data and generate a plot of the data titled "Score" and "Allow".
3.) If the file is not found any given subfolder skip and continue to the next file to be located.
4.)Then save the generated plot in the same folder that the spreadsheet was imported from as a .fig and a .bmp file.
I've gotten hints to use various functions like: genpath, dir, but the code I've been fumbling through isn't able to achieve my goals.
a) the script doesn't import the excel files from all the subfolders
b) the script wont save the .fig or .bmp file in the associated subfolder
Here is the code I have been fumbling through:
%I know all of this is wrong wrong wrong. Please help to adjust my code to %achieve the objectives outlined above!
addpath(genpath('c:\data'))
folder = 'c:\data';
subdirs = dir(folder);
subdirs(~[subdirs.isdir]) = [] ;
numberOfFolders = length(subdirs);
if numberOfFolders <= 0
uiwait(warndlg('Number of folders = 0!'))
end
wantedfiles = {'Angels' 'Diamondbacks' 'Orioles' 'Royals' 'Yankees' 'Mets' 'Giants'};
for K = 1 : numberOfFolders
thissubdir = subdirs(K).name;
if strcmp(thissubdir, '.') || strcmp(thissubdir, '..')
continue;
end
subdirpath = [folder '\' thissubdir];
for L = 1 : length(wantedfiles)
for wantedfiles = {'Angels' 'Diamondbacks' 'Orioles' 'Royals' 'Yankees' 'Mets' 'Giants'};
folder = '';
fileToRead1 = [wantedfiles{1} '.xls'];
sheetName='Sheet1';
if exist(fileToRead1, 'file') == 0
% File does not exist
% Skip to bottom of loop and continue with the loop
continue;
end
%This is to import the data and organize it
% All of this code I had auto-generated from importing files manually
[numbers, strings, raw] = xlsread(fileToRead1, sheetName);
if ~isempty(numbers)
newData1.data = numbers;
end
if ~isempty(strings) && ~isempty(numbers)
[strRows, strCols] = size(strings);
[numRows, numCols] = size(numbers);
likelyRow = size(raw,1) - numRows;
% Break the data up into a new structure with one field per column.
if strCols == numCols && likelyRow > 0 && strRows >= likelyRow
newData1.colheaders = strings(likelyRow, :);
end
end
% Create new variables in the base workspace from those fields.
for i = 1:size(newData1.colheaders, 2)
assignin('base', genvarname(newData1.colheaders{i}), newData1.data(:,i));
end
% Now I execute the plotting of data
subplot (2,1,1), plot(Score,Allow)
title([wantedfiles{1} 'Testing to see if it works']);
subplot (2,1,2), plot(Allow,Score)
title('Well, did it?');
% here I save the generated plots, but they don't save where I want them to
saveas(gcf,[wantedfiles{1} ' did it work.fig']);
saveas(gcf,[wantedfiles{1} ' did it work.bmp']);
end
end
end
%At the end of the script I still was unable to loop over the files that I wanted
rmpath(genpath('c:\data'));

waf - build works, custom build targets fail

The waf command waf build shows compiler errors (if there are any) while waf debug or waf release does not and always fails, utilizing the following wscript file (or maybe the wscript file has some other shortcomings I am currently not aware of):
APPNAME = 'waftest'
VERSION = '0.0.1'
def configure(ctx):
ctx.load('compiler_c')
ctx.define('VERSION', VERSION)
ctx.define('GETTEXT_PACKAGE', APPNAME)
ctx.check_cfg(atleast_pkgconfig_version='0.1.1')
ctx.check_cfg(package='glib-2.0', uselib_store='GLIB', args=['--cflags', '--libs'], mandatory=True)
ctx.check_cfg(package='gobject-2.0', uselib_store='GOBJECT', args=['--cflags', '--libs'], mandatory=True)
ctx.check_cfg(package='gtk+-3.0', uselib_store='GTK3', args=['--cflags', '--libs'], mandatory=True)
ctx.check_cfg(package='libxml-2.0', uselib_store='XML', args=['--cflags', '--libs'], mandatory=True)
ctx.check_large_file(mandatory=False)
ctx.check_endianness(mandatory=False)
ctx.check_inline(mandatory=False)
ctx.setenv('debug')
ctx.env.CFLAGS = ['-g', '-Wall']
ctx.define('DEBUG',1)
ctx.setenv('release')
ctx.env.CFLAGS = ['-O2', '-Wall']
ctx.define('RELEASE',1)
def pre(ctx):
print ('Building [[[' + ctx.variant + ']]] ...')
def post(ctx):
print ('Building is complete.')
def build(ctx):
ctx.add_pre_fun(pre)
ctx.add_post_fun(post)
# if not ctx.variant:
# ctx.fatal('Do "waf debug" or "waf release"')
exe = ctx.program(
features = ['c', 'cprogram'],
target = APPNAME+'.bin',
source = ctx.path.ant_glob(['src/*.c']),
includes = ['src/'],
export_includes = ['src/'],
uselib = 'GOBJECT GLIB GTK3 XML'
)
# for item in exe.includes:
# print(item)
from waflib.Build import BuildContext
class release(BuildContext):
cmd = 'release'
variant = 'release'
class debug(BuildContext):
cmd = 'debug'
variant = 'debug'
Error resulting from waf debug :
Build failed
-> task in 'waftest.bin' failed (exit status -1):
{task 46697488: c qqq.c -> qqq.c.1.o}
[useless filepaths]
I had a look at the waf demos, read the wafbook at section 6.2.2 but those did not supply me with valuable information in order to fix this issue.
What's wrong, and how do I fix it?
You need to do at least the following:
def configure(ctx):
...
ctx.setenv('debug')
ctx.load('compiler_c')
...
Since the cfg.setenv function resets whole previous environment. If you want to save previous environment, you can do cfg.setenv('debug', env=cfg.env.derive()).
Also, you don't need to explicitly specify the features = ['c', 'cprogram'], since, it's redundant, when you call bld.program(...).
P.S. Don't forget to reconfigure after modifying wscript file.

Groovy - create file issue: The filename, directory name or volume label syntax is incorrect

I'm running a script made in Groovy from Soap UI and the script needs to generate lots of files.
Those files have also in the name two numbers from a list (all the combinations in that list are different), and there are 1303 combinations
available and the script generates just 1235 files.
A part of the code is:
filename = groovyUtils.projectPath + "\\" + "$file"+"_OK.txt";
targetFile = new File(filename);
targetFile.createNewFile();
where $file is actually that part of the file name which include those 2 combinations from that list:
file = "abc" + "-$firstNumer"+"_$secondNumber"
For those file which are not created is a message returned:"The filename, directory name or volume label syntax is incorrect".
I've tried puting another path:
filename = "D:\\rez\\" + "\\" + "$file"+"_OK.txt";
targetFile = new File(filename);
targetFile.createNewFile();
and also:
File parentFolder = new File("D:\\rez\\");
File targetFile = new File(parentFolder, "$file"+"_OK.txt");
targetFile.createNewFile();
(which I've found here: What are possible reasons for java.io.IOException: "The filename, directory name, or volume label syntax is incorrect")
but nothing worked.
I have no ideea where the problem is. Is strange that 1235 files are created ok, and the rest of them, 68 aren't created at all.
Thanks,
My guess is that some of the files have illegal characters in their paths. Exactly which characters are illegal is platform specific, e.g. on Windows they are
\ / : * ? " < > |
Why don't you log the full path of the file before targetFile.createNewFile(); is called and also log whether this method succeeded or not, e.g.
filename = groovyUtils.projectPath + "\\" + "$file"+"_OK.txt";
targetFile = new File(filename);
println "attempting to create file: $targetFile"
if (targetFile.createNewFile()) {
println "Successfully created file $targetFile"
} else {
println "Failed to create file $targetFile"
}
When the process is finished, check the logs and I suspect you'll see a common pattern in the ""Failed to create file...." messages
File.createNewFile() returns false when a file or directory with that name already exists. In all other failure cases (security, I/O) it throws an exception.
Evaluate createNewFile()'s return value or, additionally, use the File.exists() method:
File file = new File("foo")
// works the first time
createNewFile(file)
// prints an error message
createNewFile(file)
void createNewFile(File file) {
if (!file.createNewFile()) {
assert file.exists()
println file.getPath() + " already exists."
}
}

Resources