trying to look up files in directories with wildcard *, and put the names into an array
the files have similar names (MATCHr1, MATCHr2 ... )
the problem arises when the file does not exist (which is a possibility)
set command returns a "no match" error and terminates the loop
if this happens how can i get it handle the error by jumping to the next iteration?
set SUBIDS = (10003 10005 10006)
foreach SUBID ($SUBIDS)
foreach SEQR ( MATCH ENC NBACK SIMON FACE )
ls -l *${SEQR}*.nii.gz
set Array = *${SEQR}*.nii.gz
echo $Array[*]
....rest of code works to use contents of Array to create text files
and works great when the *${SEQR}*.nii.gz returns a file name
but fails when the no matches are made with the wildcard
Any help would be well apreciated
thanks!
you can do you ls and check the $status (or $STATUS, depends on your system) flag:
ls -l *${SEQR}*.nii.gz >> /dev/null ; if !$status then ...
Related
I am writing a little script that outputs a list of duplicate files in the directory, ie. pairs of XXX.jpg and XXX (1).jpg. I want to use the output of this script as an argument to a command, namely ql (quicklook) so I can look through all such images (to verify they are indeed duplicate images, or just filenames). For instance, I can do `ql (' which will allow me to look through all the files 'XXX (1).jpg'; but I want to include in that list also the original 'XXX.jpg' files.
Here is my script so far:
dups=()
for file in *\(*; do
dups+=( "${file}" )
breakdown=( $file )
dupfile="${breakdown[0]}.jpg"
if [ -e "$dupfile" ]; then
dups+=( "$dupfile" )
fi
done
echo ${dups[#]}
As far as building an array of the required filenames goes, it works. But when it comes to invoking something like ql $(./printdups.sh), the command gets confused by the filenames with spaces. It will attempt to open 'XXX' as a file, and then '(1).jpg' as another file. So the question is, how can I echo this array such that filenames with spaces are recognised as such by the command I pass it to?
I have tried changing line 3 to:
dups+=( "'$file'" )
And:
dups+=( "${file/ /\ }" )
Both to no avail.
You can't pass arrays from one process to another. All you are doing is writing a space-separated sequence of file names to standard output, and the unquoted command substitution in ql $(./printdups.sh) fails for the same reason you need an array in the first place: word-splitting does not distinguish between spaces in file names and spaces between file names.
I would recommend defining a function, rather than a script, and have that function populate a global array that you can access directly after the function has been called.
get_dups () {
dups=()
for file in *\(*; do
dups+=( "$file" )
read -a breakdown <<< "$file" # safer way to split the name into parts
dupfile="${breakdown[0]}.jpg"
if [ -e "$dupfile" ]; then
dups+=( "$dupfile" )
fi
done
}
get_dups
ql "${dups[#]}"
After struggling with this issue for several hours and searching here and failing to come up with a matching solution, it's time to ask:
In bash (4.3) I'm attempting to do a combination of the following:
Create an array
For loop through the values of the array with a command that isn't super fast (curl to a web server to get a value), so we background each loop to parallelize everything to speed it up.
Set the names of the values in the array to variables assigned to values redirected to it from a command via "read"
Background each loop and get their PID into a regular array, and associate each PID with the related array value in an associative array so I have key=value pairs of array value name to PID
Use "wait" to wait for each PID to exit 0 or throw an error telling us which value name(s) in the array failed to exit with 0 by referencing the associative array
I need to be able export all of the VAR names in the original array and their now-associated values (from the curl command results) because I'm sourcing this script from another bash script that will use the resulting exported VARs/values.
The reason I'm using "read" instead of just "export" with "export var=$(command)" or similar, is because when I background and get the PID to use "wait" with in the next for loop, I actually (incorrectly) get the PID of the "export" command which always exits 0, so I don't detect an error. When I use read with the redirect to set the value of the VAR (from name in the array) and background, it actually gets the PID of the command and I catch any errors in the next loop with the "wait" command.
So, basically, this mostly appears to work, except I realized the "read" command doesn't actually appear to be substituting the variable to the array name value properly in a way that the redirected command sends its output to that name in order to set the substituted VAR name to a value. Or, maybe the command is just entirely wrong so I'm not correctly redirecting the result of my command to a VAR name I'm attempting to set.
For what it's worth, when I run the curl | python command by hand (to pull the value and then parse the JSON output) it is definitely succeeding, so I know that's working, I just can't get the redirect to send the resulting output to the VAR name.
Here's a example of what I'm trying to do:
In parent script:
# Source the child script that has the functions I need
source functions.sh
# Create the array
VALUES=(
VALUE_A
VALUE_B
VALUE_C
)
# Call the function sourced from the script above, which will use the above defined array
function_getvalues
In child (sourced) script:
function_getvalues()
{
curl_pids=( )
declare -A value_pids
for value in "${VALUES[#]}"; do
read ${value} < <(curl -f -s -X GET http://path/to/json/value | python3 -c "import sys, json; print(json.load(sys.stdin)['data']['value'])") & curl_pids+=( $! ) value_pids+=([$!]=${value})
done
for pid in "${curl_pids[#]}"; do
wait "$pid" && echo "Successfully retrieved value ${value_pids[$pid]} from Webserver." || { echo "Something went wrong retrieving value ${value_pids[$pid]}, so we couldn't get the output data needed from Webserver. Exiting." ; exit 1 ; }
done
}
The problem is that read, when run in the background, isn't connected to a standard in.[details] Consider this simplified, working example with comment how to cripple it:
VALUES=( VALUE_A VALUE_B )
for value in "${VALUES[#]}"; do
read ${value} < <(echo ${RANDOM}) # add "&" and it stops working
done
echo "VALUE_A=${VALUE_A}"
echo "VALUE_B=${VALUE_B}"
You might be able to do this with coproc, or using read -u with automatic file descriptor allocation, but really this is a job for temporary files:
tmpdir=$(mktemp -d)
VALUES=( VALUE_A VALUE_B )
for value in "${VALUES[#]}"; do
(sleep 1; echo ${RANDOM} > "${tmpdir}"/"${value}") &
done
for value in "${VALUES[#]}"; do
wait_file "${tmpdir}"/"${value}" && {
read -r ${value} < "${tmpdir}"/"${value}";
}
done
echo "VALUE_A=${VALUE_A}"
echo "VALUE_B=${VALUE_B}"
rm -r "${tmpdir}"
This example uses wait_file helper, but you might use inotifywait if you don't mind some dependencies on OS.
I am looking for a script to rename files and directories that have special characters in them.
My files:
?rip?ev <- Directory
- Juhendid ?rip?evaks.doc <- Document
- ?rip?ev 2 <- Subdirectory
-- t?ts?.xml <- Subdirectory file
They need to be like this:
ripev <- Directory
- Juhendid ripevaks.doc <- Document
- ripev 2 <- Subdirectory
-- tts.xml <- Subdirectory file
I need to change the files and the folders so that the filetype stays the same as it is for example .doc and .xml wont be lost. Last time I did it with rename it lost every filetype and the files were moved to mother directory in this case ?rip?ev directory and subdirectories were empty. Everything was located under the mother directory /home/samba/.
So in this case I need just to rename the question mark in the file name and directory name, but not to move it anywhere else or lose any other character or the filetype. I have been looking around google for a answer but haven't found one. I know it can be done with find and rename, but haven't been able to over come the complexity of the script. Can anyone help me please?
You can just do something like this
find -name '*\?*' -exec bash -c 'echo mv -iv "$0" "${0//\?/}"' {} \;
Note the echo before the mv so you can see what it does before actually changing anything. Otherwise above:
searches for ? in the name (? is equivalent to a single char version of * so needs to be escaped)
executes a bash command passing the {} as the first argument (since there is no script name it's $0 instead of $1)
${0//\?/} performs parameter expansion in bash replacing all occurrences of ? with nothing.
Note also that file types do not depend on the name in linux, but this should not change any file extension unless they contain ?.
Also this will rename symlinks as well if they contain ? (not clear whether or not that was expected from question).
I usually do this kind of thing in Perl:
#!/usr/bin/perl
sub procdir {
chdir #_[0];
for (<*>) {
my $oldname = $_;
rename($oldname, $_) if s/\?//g;
procdir($_) if -d;
}
chdir "..";
}
procdir("top_directory");
The runtime arguments are as follows: $1 is the path to the file containing the list of files
$2 is the path to the directory containing the files
What I want to do is check that each file listed in $1 exists in the $2 directory
I'm thinking something like:
for f in 'cat $1'
do
if (FILEEXISTSIN$2DIRECTORY)
then echo '$f exists in $2'
else echo '$f is missing in $2' sleep 5 exit
fi
done
As you can see, I want it so that if any of the files listed in $1 don't exist in $2 directory, the script states this then closes. The only part I can't get my head around is the (FILEEXISTSIN$2DIRECTORY) part. I know that you can do [ -e $f ], but I don't know how you can make sure its checking that it exists in the $2 directory.
Edit: Thinking further upon this, perhaps I could use nested for loops?
If your specified input file contains a newline-separated list of files to check, then the following solution (using a while read loop) is robust enough to handle file names with spaces properly.
Generally, you should never make use of a loop of the form for i in $(command), and instead opt for a while loop. See http://mywiki.wooledge.org/DontReadLinesWithFor for more details.
while read -r file; do
if [[ -e "$2/$file" ]]; then
echo "$f exists in $2"
else
echo "$f does not exist in $2"
sleep 5
exit 1
fi
done < "$1"
Since you're dealing with a list of file names without spaces in the names (because the $(cat $1) notation will split things up like that), it is relatively straight forward:
for file in $(cat $1)
do
if [ -e "$2/$file" ]
then echo "$file exists in $2"
else echo "$file is missing in $2"; sleep 5; exit 1
fi
done
Basically, use the built-in string concatenation facilities to build the full path to the file, and use the test or [ operator to check the files existence.
The complexities arise if you have to deal with arbitrary file names, especially if one of the arbitrary characters in an arbitrary file name can be the newline character. Suffice to say, they complicate the issue sufficiently that I won't deal with it unless you say you need it dealt with, and even then, I'll negotiate on whether newlines in names need to be handled. The double-quoted variable expansion is a key part of the strategy for dealing with it. The other part of the problem is how to get the file names accurately into a variable.
I have few files in a folder with name pattern in which one of the section is variable.
file1.abc.12.xyz
file2.abc.14.xyz
file3.abc.98.xyz
So the third section (numeric) in above three file names changes everyday.
Now, I have a script which does some tasks on the file data. However, before doing the work, I want to check whether the file exists or not and then do the task:
if(file exist) then
//do this
fi
I wrote the below code using wildcard '*' in numeric section:
export mydir=/myprog/mydata
if[find $mydir/file1.abc.*.xyz]; then
# my tasks here
fi
However, it is not working and giving below error:
[find: not found [No such file or directory]
Using -f instead of find does not work as well:
if[-f $mydir/file1.abc.*.xyz]; then
# my tasks here
fi
What am I doing wrong here ? I am using korn shell.
Thanks for reading!
for i in file1.abc.*.xyz ; do
# use $i here ...
done
I was not using spaces before the unix keywords...
For e.g. "if[-f" should actually be " if [ -f" with spaces before and after the bracket.