Is there anyone who has installed opentsdb on Ubuntu 15.04 version? If so please share the steps to be followed. I tried number of times but I am not able to install it properly.
You need to write tcollector, for example:
Step 1: create metrics:
./tsdb mkmetric proc.loadavg.1m proc.loadavg.5m
Step 2: create collector in shell script or command line.
cat >loadavg-collector.sh <<\EOF
#!/bin/bash set -e
while true; do
awk -v now=`date +%s` -v host=`hostname` \ '{ print "put proc.loadavg.1m " now " " $1 " host=" host;
print "put proc.loadavg.5m " now " " $2 " host=" host }'
/proc/loadavg sleep 15 done | nc -w 30 host.name.of.tsd PORT EOF
Then:
chmod +x loadavg-collector.sh
nohup ./loadavg-collector.sh &
It will collect data every 15 second on metrics proc.loadavg.1m and proc.loadavg.5m. Now you will be able to see graph in webinterface of opentsdb.
For detail please check the link below:
http://opentsdb.net/docs/build/html/user_guide/quickstart.html
Related
#!/bin/bash
count2=1
declare -a input
input=( "$#" )
echo " "
echo " Hostname passed by user is " ${input[0]}
HOST="${input[0]}"
sshpass -p '<pass>' ssh -o StrictHostKeyChecking=no user#$HOST /bin/bash << ENDSSH
echo " Connected "
echo $count2
echo $input
pwd
echo $count2: ${input[$count2]}
nic=${input[$count2]}
echo $nic
echo $(ethtool "${nic}" |& grep 'Link' | awk '{print $3}')
ENDSSH
So Actually want to pass variable 'count2' and 'input' to remote SSH and execute.
But unfortunately it is not getting passed. It is not echoing anything after SSH.
Need help in this.!!
I have sshpass installed in sever.
code output:
[user#l07 ~]$ ./check.sh <hostname> eno6
Hostname passed by user is <hostname>
Connected
After SSH it only echos "Connected". I'm not sure why $count2 and $input is not echoing.
I tired with backlash '\$count2' but that is also not working. All possible combination tried even with quote and unquote of ENDSSH. Pls help
Any help will be really appreciated!!
You basically want to supply to your remote bash a HERE-document to be executed. This is tricky, since you need to "compose" the full text of this document before you can supply it to ssh. I would therefore separate the task into two parts:
Creating the HERE-document
Running it on ssh
This makes it easy for debugging to output the document between steps 1 and 2 and to visually inspect its contents for correctness. Don't forget that once this code runs on the remote host, it can't access any of your variables anymore, unless you have "promoted" them to the remote side using the means provided by ssh.
Hence you could start like this:
# Create the parameters you want to use
nic=${input[$count2]}
# Create a variable holding the content of the remote script,
# which interpolates your parameters
read -r -d '' remote_script << ENDSSH
echo "Connected to host \$(hostname)"
echo "Running bash version: \$BASH_VERSION"
....
ethtool "$nic" |& grep Link | awk '{ print $3 }'
ENDSSH
# Print your script for verification
echo "$remote_script"
# Submit it to the host
sshpass -p '<pass>' ssh -o StrictHostKeyChecking=no "user#$HOST" /bin/bash <<<"$remote_script"
You have to add escapes(\) here:
...
echo \$nic
...
echo \$(ethtool "\${nic}" |& grep 'Link' | awk '{print \$3}')
...
But why echoing this? Try it without echo
...
ethtool "\${nic}" |& grep -i 'Link' | awk '{print \$3}'
...
#!/bin/bash
count2=1
declare -a input
input=( "$#" )
echo " Hostname passed by user is " "${input[0]}"
HOST="${input[0]}"
while [ $# -gt $count2 ]
do
sed -i 's/VALUE/'"${input[$count2]}"'/g' ./check.sh
sshpass -p '<pass>' scp ./check.sh user#"$HOST":/home/user/check.sh
sshpass -p '<pass>' ssh -o StrictHostKeyChecking=no user#"$HOST" "sh /home/user/check.sh && rm -rf /home/user/check.sh"
sed -i 's/'"${input[$count2]}"'/VALUE/g' ./check.sh
((count2++))
done
Found the another solution of this issue: It is working for me now !!!!
I wrote my entire logic which needs to be executed remotely in check.sh file and now replacing or storing the user input into this check.sh file and copying this file into remote server via scp and executing it over remotely and after successful execution removing this file from remote server and after ssh , again changing the user input to it's original value in local server using sed command.
Made this as dynamic script to work for multiple servers.
I'm trying to take a PostgreSQL backup with pg_dump. But I'm not able to take it due to the following error.
I have successfully taken backups for different IP addresses without the special character # in it.
command used and working
sudo /usr/bin/pg_dump --file "/home/myusername/my_first_db.backup" \
--verbose --format=t --blobs -v \
--dbname postgresql://postgres:myfirstpassowrd#112.112.113.114:5432/my_first_db
command used and not working
sudo /usr/bin/pg_dump --file "/home/myuser/xyz_db/DB_BACKUP/db_file.backup" \
--verbose --format=t --blobs -v \
--dbname postgresql://111.222.333.444:5432/prod_live?password=123th#123th4&user=postgres
sudo /usr/bin/pg_dump --file "/home/myuser/xyz_db/DB_BACKUP/db_file.backup" \
--verbose --format=t --blobs -v \
--dbname postgresql://111.222.333.444:5432/prod_live?password=123th%40123th4&user=postgres
Error I'm getting:
[4] 8555
myuser#myuser:~$ pg_dump: [archiver (db)] connection to database "prod_live" failed: FATAL: password authentication failed for user "root"
FATAL: password authentication failed for user "root"
I cannot change the password, because it is production.
As I can see...
Unquoted character & in your command line sends the task to background as described, for example, here: Linux: Start Command In Background. So anything after & character ignored (or interpreted as separate command) by *nix shell.
Solution
Just try to quote the whole string like this:
sudo /usr/bin/pg_dump --file "/home/myuser/xyz_db/DB_BACKUP/db_file.backup" \
--verbose --format=t --blobs -v \
--dbname 'postgresql://111.222.333.444:5432/prod_live?password=123th#123th4&user=postgres'
Explanation
In the output provided by you the line [4] 8555 means Background job #4 with process ID 8555 was started
And single quotes around the string allows to interpret it "as-is", without parameters substitution and other special characters interpreting.
PS: Use $'...' syntax to translate special escaped characters like \n \t \uxxxx and others.
There is several examples:
$ echo abc&defgh
[1] 3426
abc
defgh: command not found
[1]+ Done echo abc
As you can see the output is like to provided by you in the part [x] xxxx
$ echo 'abc&defgh'
abc&defgh
In this case command echo prints exactly what you want
And last but not least:
$ echo '1: abc&\ndefgh'; echo $'2: abc&\ndefgh'
1: abc&\ndefgh
2: abc&
defgh
How to pull all the android archive file at same time by android debug bridge?
this is used for one android archive file:
adb pull /data/app/com.imo.android.imoim-1/base.apk E:\APK
but i want all the file same time.
To copy pull all the files from the directory, you can use the following command if you use adb with root permission:
adb pull /data/app/.
If you are not using adb with root permission, you need to copy all the files first to a "non-root" location and pull them from there:
c:\> adb shell su
root#device: / # mkdir /sdcard/data/copy_apks
root#device: / # cp -R /data/app/. /sdcard/data/copy_apks/
root#device: / # exit
c:\> adb pull /sdcard/data/copy_apks/. c:\to\your\location
c:\> adb shell rm -R /sdcard/data/copy_apks
Note: you need to have a rooted device or you need to be in custom recovery
Linux and non-root solution.
Create a shell script pm_apk.sh with content:
adb shell pm list packages -f | cut -d':' -f2- | rev | cut -d'=' -f2- | rev | awk '{print "adb pull "$0}' > adb_pm.list
Run command bash pm_apk.sh to generate adb_pm.list file(Assume your shell is bash). Example:
$ cat adb_pm.list
adb pull /data/app/~~FOOBAR==/com.azure.FOOBAR-FOOBAR==/base.apk
adb pull /data/app/~~FOOBAR==/com.adobe.FOOBAR_FOOBAR==/base.apk
....
Then create this python script adb_pm_unique_output.py with content (the commented part is to debug to ensure no duplicated output filename when pull):
pkg_names = []
out_names = []
pkg_name = ''
open('adb_pull.sh', 'w').close()
with open('adb_pull.sh', 'a') as f_run:
with open('adb_pm.list') as f:
for lines in f.readlines():
apks = lines.rstrip()
apk_name = apks.split('/')[-1]
pkg_name_raw = apks.split('/')[-2]
if '-' in pkg_name_raw:
pkg_name = pkg_name_raw.split('-')[0]
out_name = pkg_name + '_' + apk_name
else: # overlay
out_name = apk_name
#print(apk_name)
if pkg_name_raw != 'overlay':
#print('what is this no overlay and no - ?' + repr(pkg_name_raw)) # DMService ...etc
pkg_name = pkg_name_raw
if apk_name != (pkg_name_raw + '.apk'):
pass # print(apk_name, '##', pkg_name_raw)
#if pkg_name in pkg_names:
#print(pkg_name)
# print('WARNING. Duplicated pkg name!')
if out_name in out_names:
#print(out_name)
print('WARNING. Duplicated out name!')
pkg_names.append(pkg_name)
out_names.append(out_name)
#print(lines.strip() + " '" + out_name + "'")
f_run.write(lines.strip() + " '" + out_name + "'\n")
Run command python3 adb_pm_unique_output.py to generate adb_pull.sh script. Example:
$ cat adb_pull.sh
adb pull /data/app/~~FOOBAR==/com.azure.FOOBAR-FOOBAR==/base.apk 'com.azure.FOOBAR_base.apk'
adb pull /data/app/~~FOOBAR==/com.adobe.FOOBAR_FOOBAR==/base.apk 'com.adobe.FOOBAR_base.apk'
...
Then run that script with command time parallel :::: adb_pull.sh to pull apks.
time for output spent time. parallel speed up the time from ~3 minutes(non-parallel) to ~2 minutes 34 seconds(parallel) in my case.
Note that few apks such as /vendor/overlay/ get Permission denied because no root.
I am using htop so see what processes are taking up a lot of memory so I can kill them. I have a lot of tmux sessions and lots of similar processes. How can I check which tmux pane a PID is in so I can be sure I am killing stuff I want to kill?
Given that PID in the below line is the target pid number:
$ tmux list-panes -a -F "#{pane_pid} #{pane_id}" | grep ^PID
The above will identify the pane where the PID is running. The output will be two strings. The first number should be the same as PID and the second one (with a percent sign) is "tmux pane id". Example output:
2345 %30
Now, you can use "tmux pane id" to kill the pane without "manually" searching for it:
$ tmux kill-pane -t %30
To answer your question completely, in order to find *tmux session* that a PID belongs to, this command can be used:
$ tmux list-panes -a -F "#{pane_pid} #{session_name}" | grep ^PID
# example output: 2345 development
Here's another possibly useful "line":
$ tmux list-panes -a -F "#{pane_pid} #{session_name}:#{window_index}:#{pane_index}" | grep ^PID
# example output: 2345 development:2:0
The descriptions for all of the interpolation strings (example #{pane_pid}) can be looked up in tmux man page in the FORMATS section.
The answers above give you the pids of the shells running in the panes, you'll be out of luck if you want to find something running in the shells.
try:
https://gist.github.com/nkh/0dfa8bf165a53832a4b5b17ee0d7ab12
This scrip gives you all the pids as well as the files the processes have opened. I never know in which session, window, pane, attached or not, I have a file open, this helps.
I haven't tried it on another machine, tell me if you encounter any problem.
lsof needs to be installed.
if you just want pids, pstree is useful, you can modity the script to use it (it's already there commented)
The following script displays the tree of processes in each window (or pane). It takes list of PIDs as one parameter (one PID per line). Specified processes are underlined. It automatically pipes to less unless is a part of some other pipe. Example:
$ ./tmux-processes.sh "$(pgrep ruby)"
-- session-name-1 window-index-1 window-name-1
7184 7170 bash bash --rcfile /dev/fd/63 -i
7204 7184 vim vim ...
-- session-name-2 window-index-2 window-name-2
7186 7170 bash bash --rcfile /dev/fd/63 -i
10771 7186 bash bash ./manage.sh runserver
10775 10771 django-admi /srv/www/s1/env/bin/python /srv/www/s1/env/bin/...
5761 10775 python /srv/www/s1/env/bin/python /srv/www/s1/env/bin/...
...
tmux-processes.sh:
#!/usr/bin/env bash
set -eu
pids=$1
my_pid=$$
subtree_pids() {
local pid=$1 level=${2:-0}
if [ "$pid" = "$my_pid" ]; then
return
fi
echo "$pid"
ps --ppid "$pid" -o pid= | while read -r pid; do
subtree_pids "$pid" $((level + 1))
done
}
# server_pid=$(tmux display-message -p '#{pid}')
underline=$(tput smul)
# reset=$(tput sgr0) # produces extra symbols in less (^O), TERM=screen-256color (under tmux)
reset=$(echo -e '\033[m')
re=$(echo "$pids" | paste -sd'|')
tmux list-panes -aF '#{session_name} #{window_index} #{window_name} #{pane_pid}' \
| while read -r session_name window_index window_name pane_pid; do
echo "-- $session_name $window_index $window_name"
ps -p "$(subtree_pids "$pane_pid" | paste -sd,)" -Ho pid=,ppid=,comm=,args= \
| sed -E 's/^/ /' \
| awk \
-v re="$re" -v underline="$underline" -v reset="$reset" '
$1 ~ re {print underline $0 reset}
$1 !~ re {print $0}
'
done | {
[ -t 1 ] && less -S || cat
}
Details regarding listing tmux processes you can find here.
To underline lines I use ANSI escape sequences. To show the idea separately, here's a script that displays list of processes and underlines some of them (having PIDs passed as an argument):
#!/usr/bin/env bash
set -eu
pids=$1
bold=$(tput bold)
# reset=$(tput sgr0) # produces extra symbols in less (^O), TERM=xterm-256color
reset=$(echo -e '\033[m')
underline=$(tput smul)
re=$(echo "$pids" | paste -sd'|')
ps -eHo pid,ppid,comm,args | awk \
-v re="$re" -v bold="$bold" -v reset="$reset" -v underline="$underline" '
$1 ~ re {print underline $0 reset}
$1 !~ re {print $0}
'
Usage:
$ ./ps.sh "$(pgrep ruby)"
Details regarding less and $(tput sgr0) can be found here.
I'm writing a command-line program in C, and I'd like to implement a --help option to show the usual stuff like available options and what they do, and usage examples.
Is there a proper method for formatting the text that is displayed in the help? Or do I just do my best to make it look nice?
I looked at some random programs on SourceForge to see how they did it, and most just used a bunch of printf()s to output pre-formatted (as far as spacing and indentation) text.
Is groff or troff appropriate here? I have come across those applications in my Googlings, which seem to be some kind of typesetting programs, but I am unfamiliar with them.
Generally speaking more people are concerned that the program works as stated than how well the help is displayed. Do your best (effort is always appreciated) with printf and get on with it and your life. You have bigger fish to fry.
Don't sweat the format that much. On the other hand, you yourself will be using it from time to time. Include such pertinent details as the program's purpose, relevant options, and some examples. A blank line to clearly delineate them should be good.
On an unrelated note, if your doing command line options in c, use the gnu getopts c library. It makes it much, much easier. And as a bonus, you don't get mugged by the details of parsing the command line.
Using groff or troff is overkill. Decently laid out printf() formatting is sufficient. If you layout your help messages systematically, it is easy enough to manage. The most complex set of options I have in any of my programs has:
static const char optlist[] = "a:cd:e:f:ghi:o:p:st:u:vxyz:A:BCD:E:F:G:HIJL:M:N:O:PQ:RS:TUVX:YZ:";
static const char usestr[] =
"[-cghsvxyY] [-d dbase] [-f file] [-e 'SQL stmt'] ['SQL stmt' ...]\n"
"Other options: [-CJPRU][-BHITV][-D delim][-E escape][-Q quote][-F format]\n"
" [-A date][-i file][-o file][-L qlimit][-N number][-M FIFO]\n"
" [-Z debug][-G eor][-t table][-p password][-u username]\n"
" [-a ibase][-X record=rectag,recset=settag,header=hdrtag]\n"
" [-O orderby][-S skip][-z number]\n"
"NB: -h gives more help!";
static const char fullhelp[] =
"\nOption summary:\n"
" -a ibase - input base (default 0; set to 10 for fields with leading zeroes)\n"
" -c - continue after errors\n"
" -d dbase - select database\n"
" -e 'SQL stmt' - execute SQL command\n"
" -f file - input file for SQLCMD\n"
" -g - debugging mode (single-step commands)\n"
" -h - print this message\n"
" -i file - input file for SQLRELOAD\n"
" -o file - output file for SQLUNLOAD\n"
" -p password - password for connection (beware security implications!)\n"
" -s - silent mode\n"
" -t table - name of table (for SQLRELOAD or SQLUNLOAD)\n"
" -u username - username for connection (beware security implications!)\n"
" -v - verbose mode\n"
" -x - trace mode\n"
" -y - enable history\n"
" -z number - set debugging level for syntax and lexical analyzers\n"
" -A date - set date format (eg dd-mm-yyyy for $DBDATE=dmy4-)\n"
" -B - operate in benchmark mode (implies -x and times SQL)\n"
" -C - operate as SQLCMD\n"
" -D delim - set field delimiter (default $DBDELIMITER or pipe)\n"
" -E escape - set escape character (default $DBESCAPE or backslash)\n"
" -F format - set output format (default SELECT; alternatives include:\n"
" UNLOAD, FIXED, FIXSEP, FIXDEL, CSV, XML, HTML)\n"
" -G eor - set EOR (end of record) character string (default $DBNEWLINE or newline)\n"
" -H - include column headings in output\n"
" -I - interactive mode (after executing command line)\n"
" -J - simulate isql/dbaccess; accept [database|-] [script|-]\n"
" -L qlimit - limit on number of rows returned by query\n"
" -M FIFO - monitor FIFO for input commands\n"
" -N number - size of transaction under RELOAD\n"
" (default 1024; 0 means no sub-transactions)\n"
" -O col1,... - order by these columns (SQLUNLOAD)\n"
" -P - operate as SQLUPLOAD (not implemented yet)\n"
" -Q quote - set quote character (default $DBQUOTE or double quote)\n"
" -R - operate as SQLRELOAD\n"
" -S skip - initial rows to skip (SQLRELOAD)\n"
" -T - include column types in output\n"
" -U - operate as SQLUNLOAD\n"
" -V - print version and exit\n"
" -X xmloptions - configure the XML output\n"
" recset='recsettag',record='recordtag',header='headertag'\n"
" -Y - do not enable history, even in interactive mode\n"
" -Z debug - set debugging level (SQLCMD must be compiled with debugging enabled)\n"
;
Yes, it needs to go to long options before long.