working with arrays in bash and do stuff - arrays

I am writing a bash script and need help. This is what I tried:
With the help of #merlin2011
#!/bin/bash
if [ $# -ne 2 ]; then
echo "Usage: `basename $0` <absolute-path> <number>"
exit 1
fi
if [ "$(id -u)" != "0" ]; then
echo "This script must be run as root" 1>&2
exit 1
fi
#find . -name "$2" -exec mv {} /some/path/here \;
find $1 >> /tmp/test
for line in $(cat `/tmp/test`); do
echo $line | mv $2 awk -F"/" '{for (i = 1; i < NF; i++) if ($i == "$2") print $(i-1)}'
done
Now I want to check the result of find command from array and then if there were a directory named 2010 then get the absolute path of it. For ecxample:
arr[1]="/path/to/2010/file.db"
Then I want to rename 2010 to parent directory to. My pattern is:
arr[1]="/path/to/2010/file.db"
arr[2]="/path/test/2010/fileee.db"
arr[3]="/path/tt/2010/fileeeee.db"
.
.
.
arr[100]="/path/last/2010/fileeeeeee.db"
Result should be:
mv /path/to/2010/ to
mv /path/test/2010 test
mv /path/tt/2010/ tt
.
.
.
mv /path/last/2010 last
UPDATE:
Totally I want to know how to get a variable inversely in awk...
/path/to/dir1/2010/file.db
I want to search in absolute path then find 2010 and rename it in previous path with / pattern like : awk -F"/" {print [what?]}
tell awk my state is 2010 then print one variable before it by knowing splitter is /
UPDATE
The files dirs and subdirs pattern are:
/path/to/file/efsef/2010/1.db
/path/to/file/hfjh/sdfsf/2010/2.db
/path/to/file/dsf/sdhher/aqwe/sfrt/2010/3.db
.
.
.
/path/to/file/kldf/2010/100.db
I want to rename all 2010 dirs to their parent then tar all .db
This is what exactly I want :)

This answer addresses only the OP's update. My best interpretation is that you are trying to get awk to print the value dir1 inside the string /path/to/dir1/2010/file.db. The following line will achieve it.
awk -F"/" '{for (i = 1; i < NF; i++) if ($i == "2010") print $(i-1)}'
I tested using the following command, which will output dir1.
echo /path/to/dir1/2010/file.db | awk -F"/" '{for (i = 1; i < NF; i++) if ($i == "2010") print $(i-1)}'
Based on your update, you should surround the awk command with the backtic operator.
mv $2 `awk -F"/" '{for (i = 1; i < NF; i++) if ($i == "$2") print $(i-1)}'`

To implement, we have to do the recursive folder search.
It should be combination of two commands like find and mv
find . -name "2010" -exec mv {} /some/path/here \;
Other way shared by merlin2011
mv $2 awk -F"/" `'{for (i = 1; i < NF; i++) if ($i == "$2") print $(i-1)}'`

Here is awk command:
awk -F"/$2/" '{split($1, a, "/"); system("echo mv " $0 " " a[length(a)]);}' <<< "$1"
mv /path/to/dir1/2010/file.db dir1
Once you're satisfied remove echo in system command.

Related

bash script shuf and save in list

I have A numbers and I want to chose B of them without repeting and save into a list.
Like this:
A="100"
B=5
a=$(gshuf -i 1-$B -n $A)
for i in ${a}
do
echo $i
done
How Can I do it?
A is a string in my code and I am using gshuf instead of shuf because I am in a mac
You may use process substitution:
A="100"
B=5
while read -r i; do
echo "$i"
done < <(gshuf -i 1-$B -n $A)
If you want to save generated numbers in array then use:
arr=()
while read -r i; do
arr+=("$i")
done < <(gshuf -i 1-$B -n $A)
Check content:
declare -p arr

Using mapfile to save output to associative arrays

In practicing bash, I tried writing a script that searches the home directory for duplicate files in the home directory and deletes them. Here's what my script looks like now.
#!/bin/bash
# create-list: create a list of regular files in a directory
declare -A arr1 sumray origray
if [[ -d "$HOME/$1" && -n "$1" ]]; then
echo "$1 is a directory"
else
echo "Usage: create-list Directory | options" >&2
exit 1
fi
for i in $HOME/$1/*; do
[[ -f $i ]] || continue
arr1[$i]="$i"
done
for i in "${arr1[#]}"; do
Name=$(sed 's/[][?*]/\\&/g' <<< "$i")
dupe=$(find ~ -name "${Name##*/}" ! -wholename "$Name")
if [[ $(find ~ -name "${Name##*/}" ! -wholename "$Name") ]]; then
mapfile -t sumray["$i"] < <(find ~ -name "${Name##*/}" ! -wholename "$Name")
origray[$i]=$(md5sum "$i" | cut -c 1-32)
fi
done
for i in "${!sumray[#]}"; do
poten=$(md5sum "$i" | cut -c 1-32)
for i in "${!origray[#]}"; do
if [[ "$poten" = "${origray[$i]}" ]]; then
echo "${sumray[$i]} is a duplicate of $i"
fi
done
done
Originally, where mapfile -t sumray["$i"] < <(find ~ -name "${Name##*/}" ! -wholename "$Name") is now, my line was the following:
sumray["$i"]=$(find ~ -name "${Name##*/}" ! -wholename "$Name")
This saved the output of find to the array. But I had an issue. If a single file had multiple duplicates, then all locations found by find would be saved to a single value. I figured I could use the mapfile command to fix this, but now it's not saving anything to my array at all. Does it have to do with the fact that I'm using an associative array? Or did I just mess up elsewhere?
I'm not sure if I'm allowed to answer my own question, but I figured that I should post how I solved my problem.
As it turns out, the mapfile command does not work on associative arrays at all. So my fix was to save the output of find to a text file and then store that information in an indexed array. I tested this a few times and I haven't seemed to encounter any errors yet.
Here's my finished script.
#!/bin/bash
# create-list: create a list of regular files in a directory
declare -A arr1 origray
declare indexray
#Verify that Parameter is a directory.
if [[ -d "$HOME/$1/" && -n "$1" ]]; then
echo "Searching for duplicates of files in $1"
else
echo "Usage: create-list Directory | options" >&2
exit 1
fi
#create list of files in specified directory
for i in $HOME/${1%/}/*; do
[[ -f $i ]] || continue
arr1[$i]="$i"
done
#search for all duplicate files in the home directory
#by name
#find checksum of files in specified directory
for i in "${arr1[#]}"; do
Name=$(sed 's/[][?*]/\\&/g' <<< "$i")
if [[ $(find ~ -name "${Name##*/}" ! -wholename "$Name") ]]; then
find ~ -name "${Name##*/}" ! -wholename "$Name" >> temp.txt
origray[$i]=$(md5sum "$i" | cut -c 1-32)
fi
done
#create list of duplicate file locations.
if [[ -f temp.txt ]]; then
mapfile -t indexray < temp.txt
else
echo "No duplicates were found."
exit 0
fi
#compare similarly named files by checksum and delete duplicates
count=0
for i in "${!indexray[#]}"; do
poten=$(md5sum "${indexray[$i]}" | cut -c 1-32)
for i in "${!origray[#]}"; do
if [[ "$poten" = "${origray[$i]}" ]]; then
echo "${indexray[$count]} is a duplicate of a file in $1."
fi
done
count=$((count+1))
done
rm temp.txt
This is kind of sloppy but it does what it's supposed to do. md5sum may not be the optimal way to check for file duplicates but it works. All I have to do is replace echo "${indexray[$count]} is a duplicate of a file in $1." with rm -i ${indexray[$count]} and it's good to go.
So my next question would have to be...why doesn't mapfile work with associative arrays?

How to find non-printable characters in the file?

I tried to find out the unprintable characters in data filein unix.
Code :
#!/bin/ksh
export SRCFILE='/data/temp1.dat'
while read line
do
len=lenght($line)
for( $i = 0; $i < $len; $i++ ) {
if( ord(substr($line, $i, 1)) > 127 )
{
print "$line\n";
last;
}
done < $SRCFILE
The code is not working , please help me in getting a solution for the above query.
You can use grep for finding non-printable characters in a file, something like the following, which finds all non-printable-ASCII and all non-ASCII:
grep -P -n "[\x00-\x1F\x7F-\xFF]" input_file
-P gives you the more powerful Perl regular expressions (PCREs) and -n shows line numbers.
If your grep doesn't support PCREs, I'd just use Perl for this directly:
perl -ne '$x++;if($_=~/[\x00-\x1F\x7F-\xFF]/){print"$x:$_"}' input_file
You may try something like this :
grep '[^[:print:]]' filePath
This sounds pretty trite but I was not sure how to do it just now.
I have become fond of "od" depending on what you are doing you may want something suited to printing arbitrary characters. The awk code is not very elegant but it is flexible if you are looking for specifics, the point is just to show the use of od however. Note the problems with awk compares and the spaces etc,
cat filename | od -A n -t x1z | awk '{ p=0; i=1; if ( NF>16) { while (i<17) {if ( $i!="0d"){ if ( $i!="0a") {if ( $i" " < "20 " ) {print $i ; p=1;} if ( $i" "> "7f "){print $i; p=1;}}} i=i+1} if (p==1) print $0; }}' | more

rename specific pattern of files in bash

I have the following files and directories:
/tmp/jj/
/tmp/jj/ese
/tmp/jj/ese/2010
/tmp/jj/ese/2010/test.db
/tmp/jj/dfhdh
/tmp/jj/dfhdh/2010
/tmp/jj/dfhdh/2010/rfdf.db
/tmp/jj/ddfxcg
/tmp/jj/ddfxcg/2010
/tmp/jj/ddfxcg/2010/df.db
/tmp/jj/ddfnghmnhm
/tmp/jj/ddfnghmnhm/2010
/tmp/jj/ddfnghmnhm/2010/sdfs.db
I want to rename all 2010 directories to their parent directories then tar all .db files...
What I tried is:
#!/bin/bash
if [ $# -ne 1 ]; then
echo "Usage: `basename $0` <absolute-path>"
exit 1
fi
if [ "$(id -u)" != "0" ]; then
echo "This script must be run as root" 1>&2
exit 1
fi
rm /tmp/test
find $1 >> /tmp/test
for line in $(cat /tmp/test)
do
arr=$( (echo $line | awk -F"/" '{for (i = 1; i < NF; i++) if ($i == "2010") print $(i-1)}') )
for index in "${arr[#]}"
do
echo $index #HOW TO WRITE MV COMMAND RATHER THAN ECHO COMMAND?
done
done
1) The result is:
ese
dfhdh
ddfxcg
ddfnghmnhm
But it should be:
ese
dfhdh
ddfxcg
ddfnghmnhm
2) How can I rename all 2010 directories to their parent directory?
I mean how to do (I want to do it in loop because of larg numbers of dirs):
mv /tmp/jj/ese/2010 /tmp/jj/ese/ese
mv /tmp/jj/dfhdh/2010 /tmp/jj/dfhdh/dfhdh
mv /tmp/jj/ddfxcg/2010 /tmp/jj/ddfxcg/ddfxcg
mv /tmp/jj/ddfnghmnhm/2010 /tmp/jj/ddfnghmnhm/ddfnghmnhm
You could instead use find in order to determine if a directory contains a subdirectory named 2010 and perform the mv:
find /tmp -type d -exec sh -c '[ -d "{}"/2010 ] && mv "{}"/2010 "{}"/$(basename "{}")' -- {} \;
I'm not sure if you have any other question here but this would do what you've listed at the end of the question, i.e. it would:
mv /tmp/jj/ese/2010 /tmp/jj/ese/ese
and so on...
Can be done using grep -P:
grep -oP '[^/]+(?=/2010)' file
ese
ese
dfhdh
dfhdh
ddfxcg
ddfxcg
ddfnghmnhm
ddfnghmnhm
This should be close:
find "$1" -type d -name 2010 -print |
while IFS= read -r dir
do
parentPath=$(dirname "$dir")
parentDir=$(basename "$parentPath")
echo mv "$dir" "$parentPath/$parentDir"
done
Remove the echo after testing. If your dir names can contain newlines then look into the -print0 option for find, and the -0 option for xargs.
First, only iterate through the dirs you're interested in, and avoid temporary files:
for d in $(find $1 -type d -name '2010') ; do
Then you can use basename and dirname to extract parts of that directory name and reconstruct the desired one. Something like:
b="$(dirname $d)"
p="$(basename $b)"
echo mv "$d" "$b/$p"
You could use shell string replace operations instead of basename/dirname.

Is there a way to search an entire array inside of an argument?

Posted my code below, wondering if I can search one array for a match... or if theres a way I can search a unix file inside of an argument.
#!/bin/bash
# store words in file
cat $1 | ispell -l > file
# move words in file into array
array=($(< file))
# remove temp file
rm file
# move already checked words into array
checked=($(< .spelled))
# print out words & ask for corrections
for ((i=0; i<${#array[#]}; i++ ))
do
if [[ ! ${array[i]} = ${checked[#]} ]]; then
read -p "' ${array[i]} ' is mispelled. Press "Enter" to keep
this spelling, or type a correction here: " input
if [[ ! $input = "" ]]; then
correction[i]=$input
else
echo ${array[i]} >> .spelled
fi
fi
done
echo "MISPELLED: CORRECTIONS:"
for ((i=0; i<${#correction[#]}; i++ ))
do
echo ${array[i]} ${correction[i]}
done
otherwise, i would need to write a for loop to check each array indice, and then somehow make a decision statement whether to go through the loop and print/take input
The ususal shell incantation to do this is:
cat $1 | ispell -l |while read -r ln
do
read -p "$ln is misspelled. Enter correction" corrected
if [ ! x$corrected = x ] ; then
ln=$corrected
fi
echo $ln
done >correctedwords.txt
The while;do;done is kind of like a function and you can pipe data into and out of it.
P.S. I didn't test the above code so there may be syntax errors

Resources