sh bothworkfile.txt >> cleanup_process.log
while [[ `ps -eaf | grep -c "cleanx"` -ge 2 ]]
do
sleep 10
echo "Please be patient, we are still cleaning stuffs ! "
done
echo " Clean up is done."
Hi I am using ksh.
The bothworkfile.txt contains few lines to run the process in the background.
cat bothworkfile.txt
cleanx data01.xsd*.* &
cleanx data01.xsd*.* &
Cleanx is an utility in my local shell. By default there is always a cleanx running in the background. So,
ps -eaf | grep -c "cleanx"
gives 1 always, but when a script calls it the number increases.
But now the issue is it seems i am not able to enter the while loop at all. And after running the script it prints " Clean up is done."
Please help me to understand why it is not entering while loop. Again, its ksh shell.
To make it easier to debug and see that the result is really what you are expecting, assign the result of ps -eaf | grep -c "cleanx" to a variable, then test the variable. I suspect it is evaluating to null or less than 2 right off the bat, which would cause the test to return false, thus skipping the loop. So, print it out before testing it so you know for sure. Something like this:
cleanx_count=$(ps -eaf | grep -c "cleanx")
printf "Cleanx_count: %d/n" ${cleanx_count}
while [[ ${cleanx_count} -ge 2 ]]
Related
This is a follow-up to my question here: How to write bash function to print and run command when the command has arguments with spaces or things to be expanded
Suppose I have this function to print and run a command stored in an array:
# Print and run the cmd stored in the passed-in array
print_and_run() {
echo "Running cmd: $*"
# run the command by calling all elements of the command array at once
"$#"
}
This works fine:
cmd_array=(ls -a /)
print_and_run "${cmd_array[#]}"
But this does NOT work:
cmd_array=(ls -a / | grep "home")
print_and_run "${cmd_array[#]}"
Error: syntax error near unexpected token `|':
eRCaGuy_hello_world/bash$ ./print_and_run.sh
./print_and_run.sh: line 55: syntax error near unexpected token `|'
./print_and_run.sh: line 55: `cmd_array=(ls -a / | grep "home")'
How can I get this concept to work with the pipe operator (|) in the command?
If you want to treat an array element containing only | as an instruction to generate a pipeline, you can do that. I don't recommend it -- it means you have security risk if you don't verify that variables into your string can't consist only of a single pipe character -- but it's possible.
Below, we create a random single-use "$pipe" sigil to make that attack harder. If you're unwilling to do that, change [[ $arg = "$pipe" ]] to [[ $arg = "|" ]].
# generate something random to make an attacker's job harder
pipe=$(uuidgen)
# use that randomly-generated sigil in place of | in our array
cmd_array=(
ls -a /
"$pipe" grep "home"
)
exec_array_pipe() {
local arg cmd_q
local -a cmd=( )
while (( $# )); do
arg=$1; shift
if [[ $arg = "$pipe" ]]; then
# log an eval-safe copy of what we're about to run
printf -v cmd_q '%q ' "${cmd[#]}"
echo "Starting pipeline component: $cmd_q" >&2
# Recurse into a new copy of ourselves as a child process
"${cmd[#]}" | exec_array_pipe "$#"
return
fi
cmd+=( "$arg" )
done
printf -v cmd_q '%q ' "${cmd[#]}"
echo "Starting pipeline component: $cmd_q" >&2
"${cmd[#]}"
}
exec_array_pipe "${cmd_array[#]}"
See this running in an online sandbox at https://ideone.com/IWOTfO
Do this instead. It works.
print_and_run() {
echo "Running cmd: $1"
eval "$1"
}
Example usage:
cmd='ls -a / | grep -C 9999 --color=always "home"'
print_and_run "$cmd"
Output:
Running cmd: ls -a / | grep -C 9999 --color=always "home"
(rest of output here, with the word "home" highlighted in red)
The general direction is that you don't. You do not store the whole command line to be printed later, and this is not the direction you should take.
The "bad" solution is to use eval.
The "good" solution is to store the literal '|' character inside the array (or some better representation of it) and parse the array, extract the pipe parts and execute them. This is presented by Charles in the other amazing answer. It is just rewriting the parser that already exists in the shell. It requires significant work, and expanding it will require significant work.
The end result is, is that you are reimplementing parts of shell inside shell. Basically writing a shell interpreter in shell. At this point, you can just consider taking Bash sources and implementing a new shopt -o print_the_command_before_executing option in the sources, which might just be simpler.
However, I believe the end goal is to give users a way to see what is being executed. I would propose to approach it like .gitlab-ci.yml does with script: statements. If you want to invent your own language with "debug" support, do just that instead of half-measures. Consider the following YAML file:
- ls -a / | grep "home"
- echo other commands
- for i in "stuff"; do
echo "$i";
done
- |
for i in "stuff"; do
echo "$i"
done
Then the following "runner":
import yaml
import shlex
import os
import sys
script = []
input = yaml.safe_load(open(sys.argv[1], "r"))
for line in input:
script += [
"echo + " + shlex.quote(line).replace("\n", "<newline>"), # some unicode like  would look nice
line,
]
os.execvp("bash", ["bash", "-c", "\n".join(script)])
Executing the runner results in:
+ ls -a / | grep "home"
home
+ echo other commands
other commands
+ for i in "stuff"; do echo "$i"; done
stuff
+ for i in "stuff"; do<newline> echo "$i"<newline>done<newline>
stuff
This offers greater flexibility and is rather simple, supports any shell construct with ease. You can try gitlab-ci/cd on their repository and read the docs.
The YAML format is only an example of the input format. Using special comments like # --- cut --- between parts and extracting each part with the parser will allow running shellcheck over the script. Instead of generating a script with echo statements, you could run Bash interactively, print the part to be executed and then "feed" the part to be executed to interactive Bash. This will alow to preserve $?.
Either way - with a "good" solution, you end up with a custom parser.
Instead of passing an array, you can pass the whole function and use the output of declare -f with some custom parsing:
print_and_run() {
echo "+ $(
declare -f "$1" |
# Remove `f() {` and `}`. Remove indentation.
sed '1d;2d;$d;s/^ *//' |
# Replace newlines with <newline>.
sed -z 's/\n*$//;s/\n/<newline>/'
)"
"$#"
}
cmd() { ls -a / | grep "home"; }
print_and_run cmd
Results in:
+ ls --color -F -a / | grep "home"
home/
It will allow for supporting any shell construct and still allow you to check it with shellcheck and doesn't require that much work.
I am trying to figure out how to get my bash script to work. I have a the following command:
curl http://192.168.1.2/api/queue | grep -q test
I need it to repeat until the first command in the pipline succeeds (meaning the server responds) and the second command fails (meaning the pattern is not found or the queue is empty). I've tried a number of combinations but just can't seem to get it. I looked at using $PIPESTATUS but can't get it to function in a loop how I want. I've tried all kind of variations but can't get it to work. This is what I am currently trying:
while [[ "${PIPESTATUS[0]}" -eq 0 && "${PIPESTATUS[1]}" -eq 1 ]]
do curl http://192.168.1.2 | grep -q regular
echo "The exit status of first command ${PIPESTATUS[0]}, and the second command ${PIPESTATUS[1]}"
sleep 5
done
Although it's not really clear what kind of output is returned by the curl call, maybe you are looking for something like this:
curl --silent http://192.168.1.2 |while read line; do
echo $line |grep -q regular || { err="true"; break }
done
if [ -z "$err" ]; then
echo "..All lines OK"
else
echo "..Abend on line: '$line'" >&2
fi
Figured it out. Just had to re-conceptualize it. I couldn't figure it out strictly with a while or until loop but creating an infinite loop and breaking out of it when the condition is met worked.
while true
do curl http://192.168.1.2/api/queue | grep -q test
case ${PIPESTATUS[*]} in
"0 1")
echo "Item is no longer in the queue."
break;;
"0 0")
echo "Item is still in the queue. Trying again in 5 minutes."
sleep 5m;;
"7 1")
echo "Server is unreachable. Trying again in 5 minutes."
sleep 5m;;
esac
done
I am using htop so see what processes are taking up a lot of memory so I can kill them. I have a lot of tmux sessions and lots of similar processes. How can I check which tmux pane a PID is in so I can be sure I am killing stuff I want to kill?
Given that PID in the below line is the target pid number:
$ tmux list-panes -a -F "#{pane_pid} #{pane_id}" | grep ^PID
The above will identify the pane where the PID is running. The output will be two strings. The first number should be the same as PID and the second one (with a percent sign) is "tmux pane id". Example output:
2345 %30
Now, you can use "tmux pane id" to kill the pane without "manually" searching for it:
$ tmux kill-pane -t %30
To answer your question completely, in order to find *tmux session* that a PID belongs to, this command can be used:
$ tmux list-panes -a -F "#{pane_pid} #{session_name}" | grep ^PID
# example output: 2345 development
Here's another possibly useful "line":
$ tmux list-panes -a -F "#{pane_pid} #{session_name}:#{window_index}:#{pane_index}" | grep ^PID
# example output: 2345 development:2:0
The descriptions for all of the interpolation strings (example #{pane_pid}) can be looked up in tmux man page in the FORMATS section.
The answers above give you the pids of the shells running in the panes, you'll be out of luck if you want to find something running in the shells.
try:
https://gist.github.com/nkh/0dfa8bf165a53832a4b5b17ee0d7ab12
This scrip gives you all the pids as well as the files the processes have opened. I never know in which session, window, pane, attached or not, I have a file open, this helps.
I haven't tried it on another machine, tell me if you encounter any problem.
lsof needs to be installed.
if you just want pids, pstree is useful, you can modity the script to use it (it's already there commented)
The following script displays the tree of processes in each window (or pane). It takes list of PIDs as one parameter (one PID per line). Specified processes are underlined. It automatically pipes to less unless is a part of some other pipe. Example:
$ ./tmux-processes.sh "$(pgrep ruby)"
-- session-name-1 window-index-1 window-name-1
7184 7170 bash bash --rcfile /dev/fd/63 -i
7204 7184 vim vim ...
-- session-name-2 window-index-2 window-name-2
7186 7170 bash bash --rcfile /dev/fd/63 -i
10771 7186 bash bash ./manage.sh runserver
10775 10771 django-admi /srv/www/s1/env/bin/python /srv/www/s1/env/bin/...
5761 10775 python /srv/www/s1/env/bin/python /srv/www/s1/env/bin/...
...
tmux-processes.sh:
#!/usr/bin/env bash
set -eu
pids=$1
my_pid=$$
subtree_pids() {
local pid=$1 level=${2:-0}
if [ "$pid" = "$my_pid" ]; then
return
fi
echo "$pid"
ps --ppid "$pid" -o pid= | while read -r pid; do
subtree_pids "$pid" $((level + 1))
done
}
# server_pid=$(tmux display-message -p '#{pid}')
underline=$(tput smul)
# reset=$(tput sgr0) # produces extra symbols in less (^O), TERM=screen-256color (under tmux)
reset=$(echo -e '\033[m')
re=$(echo "$pids" | paste -sd'|')
tmux list-panes -aF '#{session_name} #{window_index} #{window_name} #{pane_pid}' \
| while read -r session_name window_index window_name pane_pid; do
echo "-- $session_name $window_index $window_name"
ps -p "$(subtree_pids "$pane_pid" | paste -sd,)" -Ho pid=,ppid=,comm=,args= \
| sed -E 's/^/ /' \
| awk \
-v re="$re" -v underline="$underline" -v reset="$reset" '
$1 ~ re {print underline $0 reset}
$1 !~ re {print $0}
'
done | {
[ -t 1 ] && less -S || cat
}
Details regarding listing tmux processes you can find here.
To underline lines I use ANSI escape sequences. To show the idea separately, here's a script that displays list of processes and underlines some of them (having PIDs passed as an argument):
#!/usr/bin/env bash
set -eu
pids=$1
bold=$(tput bold)
# reset=$(tput sgr0) # produces extra symbols in less (^O), TERM=xterm-256color
reset=$(echo -e '\033[m')
underline=$(tput smul)
re=$(echo "$pids" | paste -sd'|')
ps -eHo pid,ppid,comm,args | awk \
-v re="$re" -v bold="$bold" -v reset="$reset" -v underline="$underline" '
$1 ~ re {print underline $0 reset}
$1 !~ re {print $0}
'
Usage:
$ ./ps.sh "$(pgrep ruby)"
Details regarding less and $(tput sgr0) can be found here.
I am using tcsh. I want to output:
mkdir dir1 dir2 dir3 dir4
Where of course the maximum number of dirs is variable. The point is they contain the incrementing variable from a loop in their names. I have a feeling I should use xargs. I just want to be able to use a simple loop, and do it all in one line if possible. Is this possible? My feeling is it will be something like:
loop here | xargs mkdir
but I am just not able to make my syntax work.
edit: I figured out how to do it with multiple lines of input. Anyone know how to make the following into a single line input?
for i in {1..5}
do
echo -n " dir$i"
done \
| xargs mkdir
Code:
foreach a ( `seq 1 1 10` )
mkdir dir$a
end
or
seq -f 'dir%.0f' -s ' ' 1 1 10 | xargs mkdir
I'm very new in shell scripting, and I encountered a problem that is quite wired. The program is rather simple so I just post it here:
#!/bin/bash
list=""
list=`mtx -f /dev/sg2 status | while read status
do
result=$(echo ${status} | grep "Full")
if [ -z "$result" ]; then
continue
else
echo $(echo ${result} | cut -f3 -d' ' | tr -d [:alpha:] | tr -d [:punct:])
fi
done`
echo ${list}
for haha in ${list}
do
printf "current slot is:%s \n" ${haha}
done
What the program does is that it executes mtx -f /dev/sg2 status and goes to each line and see if there's a full disk. If it has "Full" in that line, I'll record the slot number in that line, and put in the list.
Notice that I put a back quote after list= at line 6, and it covers the whole "while" loop after that. The reason is unclear to me, but I got this usage by just googling it. It is said that the while loop will open up a separate shell or something like that, so when the while loop is done, whatever you concatenated in the loop will get lost, so in my initial implementation, list is still empty after the while loop.
My question is: even if the code above works fine, it looks pretty tricky to others, and what's worse, I can only make only ONE list after the loop is done. Is there a better way to fix this so that I can pull out more information from the loop? Like what if I need list2 to store other values? Thanks.
Your shell script does work. If you wanted to get two pieces of info per line insteal of one, you would have to change this line
echo $(echo ${result} | cut -f3 -d' ' | tr -d [:alpha:] | tr -d [:punct:])
to concatenate the desired values separated by a comma or any other "special" character. Then you could parse your list this way :
for haha in ${list}
do
printf "current slot is:%s, secondary info:%s \n" $(echo ${haha} | cut -f1 -d',') $(echo ${haha} | cut -f2 -d',')
done
See this explanation. As a pipe is involved, the while read... code isn't executed in your current shell, but in a subshell (A child process which can't update your current process' (environment/shell) variables).
Choose on of the listed workarounds to make the while read... loop execute in your current shell.