I have a program in C that runs well when running it directly from the comand line but fails when running it with systemd:
Core was generated by `/usr/local/bin/midnite-modbusd'.
Program terminated with signal SIGFPE, Arithmetic exception.
#0 0x0000000000401308 in main (argc=1, argv=0x7ffeae390268) at src/midnite-modbusd.c:139
139 slen= interval - (millis % interval);
The code in question:
//wait for start of each sample interval
gettimeofday(&tv,NULL);
millis= (long long unsigned)tv.tv_sec*1000 + (tv.tv_usec/1000);
slen= interval - (millis % interval);
i= (millis+slen) % 1000;
usleep (slen*1000);
The full code is available on github.
The systyemd unit:
[Unit]
Description=Midnite Classic modbus data polling
After=network.target
[Service]
Type=simple
User=midnite-modbusd
ExecStart=/usr/local/bin/midnite-modbusd
Restart=on-failure
[Install]
WantedBy=multi-user.target
What can be so different when a program runs with systemd ?
Edit 1
It seems that my program has major issues that only happen when running with systemd:
it won't read my configuration file, which should throw an error message and exit(1) because of invalid values
journactl doesn't get filled in real time. Using journactl -f I have to wait a couple of minutes before seeing a bunch of logs that appear suddenly
As a side note for my tests using the command line I run: sudo -H -u midnite-modbusd /usr/local/bin/midnite-modbusd
A defined value of sample_interval from configuration file will initialize the interval, please check if the file is correct and sample_interval is present. An uninitialized value of interval might cause the divide by zero exception
I found the issue in this code:
if (getppid()==1) {
sprintf(str, "Daemon aready running");
log_message(log_file_path,(char*)str);
return;
}
This code is here for when the program was intended to fork itself to run as an "old style" daemon.
I didn't realize that, as systemd is forking it, then the program have a parent process (thus getppid() returning 1 when running with systemd but not from the command line)
Anyway it is badly written: this test should stop the script.
Related
for a project I am working on, I have to regularly send probe requests (~ 10 every 10 ms) for a chosen SSID.
Currently I am doing this with the following code:
char *args[] = {"iwlist", INTERFACE, "scan", "essid", ssid, (char *) NULL};
runQuietlyNonblocking(args);
where runQuietlyNonblocking runs the command in args with execvp and fork without waiting for the new thread to finish.
But since iwlist scan waits for probe answers and this takes more time than I need to create new probe requests, the probe requests get send but I got about 10000 threads after about half an hour and the program crashes due to memory shortage.
I am working on a raspberry pi with raspbian lite installed.
Is there a better way to send probe requests without the need to wait for an answer? Can i send probe requests manually or tell iwlist not to look for responses?
There are a few options. As far as I know popen() with "w" parameter does not wait for command execution, though it forks the process and pclose() will wait until command executes. system() just waits for command execution, but you might try feeding it to shell like this
system("iwlist wlan0 scan &")
That is, if you are stuck with iwlist.
As an alternative you can try using wpa_supplicant. It has a "scan" interface and "scan_results", first one sends request while the later gives out results.
for (;;)
{
FILE *fp = popen ("wpa_cli -i wlan0 scan", "w");
if (fp)
pclose (fp);
sleep (1);
}
(implying wpa_supplicant is up and running).
Last but not least, there are drivers which allow doing background scanning with their own interfaces, have seen it on proprietary Redpine driver.
I am new in C programming and I have been trying hard to customize an opensource tool written in C according to my organizational needs.
IDE: Eclipse,
Debugger: GDB,
OS: RHEL
The tool is multi-process in nature (main process executes first time and spawns several child processes using fork() ) and they share values in run time.
While debugging in Eclipse (using GDB), I find that the process being debugged is only running while other processes are in suspended mode. Thus, the only running process is not able to do its intended job because the other processes are suspended.
I saw somewhere that using MI command in GDB as "set non-stop on" could make other processes running. I used the same command in the gdbinit file shown below:
Note: I have overridden above .gdbinit file with an another gdbinit because the .gdbinit is not letting me to debug child processes as debugger terminates after the execution of main process.
But unfortunately debugger stops responding after using this command.
Please see below commands I am using in the gdbinit file:
Commenting non-stop enables Eclipse to continue usual debugging of the current process.
Adding: You can see in below image that only one process is running while others are suspended.
Can anyone please help me to configure GDB according to my requirement?
Thanks in advance.
OK #n.m.: Actually, You were right. I should have given more time to understand the flow of the code.
The tool creates 3 processes first and then the third process creates 5 threads and keeps on wait() for any child thread to terminate.
Top 5 threads (highlighted in blue) shown in the below image are threads and they are children of Process ID: 17991
The first two processes are intended to initiate basic functionality of the tool and hence they just wait to get exit(0). You can see below.
if (0 != (pid = zbx_fork()))
exit(0);
setsid();
signal(SIGHUP, SIG_IGN);
if (0 != (pid = zbx_fork()))
exit(0);
That was the reason I was not actually able to step in these 3 processes. Whenever, I tried to do so, the whole main process terminated immediately and consequently leaded to terminate all other processes.
So, I learned that I was supposed to "step-into" threads only. And yes, actually I can now debug :)
And this could be achieved because I had to remove the MI command "set follow-fork-mode child". So, I just used the default " .gdbinit" file with enabled "Automatically debug forked process".
Thanks everyone for your input. Stackoverflow is an awesome place to learn and share. :)
I have a complex cgi executable written in C, I configured in Apache2 and now it is running succesfully. How can I debug this program in the source code, such as set break points and inspect variables? Any tools like gdb or eclipse? Any tutorial of how to set up the debugging environment?
Thanks in advance!!
The CGI interface basically consists in passing the HTTP request to the executable's standard input and getting the response on the standard output. Therefore you can write test requests to files and manually execute your CGI without having to use Apache. The debugging can then be done with GDB :
gdb ./my_cgi
>> break some_func
>> run < my_req.txt
with my_req.txt containing the full request:
GET /some/func HTTP/1.0
Host: myhost
If you absolutely need the CGI to be run by Apache it may become tricky to attach GDB to the right process. You can for example configure Apache to have only one worker process, attach to it with gdb -p and use set follow-fork-mode child to make sure it switches to the CGI process when a request arrives.
I did this: in cgi main i added code to look for an existing file, like /var/tmp/flag. While existing, i run in a loop. Time enough to attach to cgi process via gdb. After then i delete /var/tmp/flag and from now I can debug my cgi code.
bool file_exists(const char *filename)
{
ifstream ifile(filename);
return ifile;
}
int cgiMain()
{
while (file_exists ("/var/tmp/flag"))
sleep (1);
...
your code
Unless FastCGI or SCGI is used, the CGI process is short-lived and you need to delay its exit to have enough time to attach the debugger while the process is still running. For casual debugging the easiest option is to simply use sleep() in an endless loop at the breakpoint location and exit the loop with the debugger once it is attached to the program.
Here's a small example CGI program:
#include <stdio.h>
#include <unistd.h>
void wait_for_gdb_to_attach() {
int is_waiting = 1;
while (is_waiting) {
sleep(1); // sleep for 1 second
}
}
int main(void) {
wait_for_gdb_to_attach();
printf("Content-Type: text/plain;charset=us-ascii\n\n");
printf("Hello!");
return 0;
}
Suppose it is compiled into cgi-debugging-example, this is how you would attach the debugger once the application enters the endless loop:
sudo cgdb cgi-debugging-example $(pgrep cgi-debugging)
Next you need to exit the infinite loop and wait_for_gdb_to_attach() function to reach the "breakpoint" in your application. The trick here is to step out of sleep functions until you reach wait_for_gdb_to_attach() and set the value of the variable is_waiting with the debugger so that while (is_waiting) exits:
(gdb) finish
Run till exit from 0x8a0920 __nanosleep_nocancel () at syscall-template.S:81
0x8a07d4 in __sleep (seconds=0) at sleep.c:137
(gdb) finish
Run till exit from 0x8a07d4 in __sleep (seconds=0) at sleep.c:137
wait_for_gdb_to_attach () at cgi-debugging-example.c:6
Value returned is $1 = 0
(gdb) set is_waiting = 0 # <<<<<< to exit while
(gdb) finish
Run till exit from wait_for_gdb_to_attach () cgi-debugging-example.c:6
main () at cgi-debugging-example.c:13
Once you are out of wait_for_gdb_to_attach(), you can continue debugging the program or let it run to completion.
Full example with detailed instructions here.
I'm not sure how to use gdb or other frontends in eclipse, but I just debugged my CGI program with gdb. I'd like to share something that other answers didn't mention, that CGIs usually need to read request meta-variables defined in RFC 3875#4.1 with getenv(3). Popular request variables in my mind are:
SCRIPT_NAME
QUERY_STRING
CONTENT_LENGTH
CONTENT_TYPE
REMOTE_ADDR
There variables are provided by http servers such as Apache. When debugging with gdb, we need to set these values by our own with set environment. In my case, there're only a few variables neededa(and the source code is very old, it still uses SCRIPT_URL instead of SCRIPT_NAME), so here's my example:
gdb cgi_name
set environment SCRIPT_URL /path/to/sub/cgis
set environment QUERY_STRING p1=v1&p2=v2
break foo.c:42
run
For me both solutions for debugging the CGI in gdb without web server presented above didn't work.
Maybe the second solution works for a GET Request.
I needed a combination of both, first setting the environment variables from rfc3875 (not sure if all of them are really neded).
Then I was able to pass only the params (not the compltete request) via STDIN from a file.
gdb cgi_name
set environment REQUEST_METHOD=POST
set environment CONTENT_LENGTH=1337
set environment CONTENT_TYPE=application/json
set environment SCRIPT_NAME=my.cgi
set environment REMOTE_ADDR=127.0.0.1
run < ./params.txt
With params.txt:
{"user":"admin","pass":"admin"}
When I run program via:
myshell$] mpirun --hosts localhost,192.168.1.4 ./a.out
the program executes successfully. Now when I try to run:
myshell$] mpirun --hosts localhost,myac#192.168.1.4 ./a.out
openssh prompts for password. I get:
Fatal error in MPI_Init: Other MPI error, error stack:
MPIR_Init_thread(433)..............:
MPID_Init(176).....................: channel initialization failed
MPIDI_CH3_Init(70).................:
MPID_nem_init(286).................:
MPID_nem_tcp_init(108).............:
MPID_nem_tcp_get_business_card(354):
MPID_nem_tcp_init(313).............: gethostbyname failed, myac#192.168.1.4 (errno 1)
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= EXIT CODE: 1
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
[proxy:0:0#myac] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed
[proxy:0:0#myac] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status
[proxy:0:0#myac] main (./pm/pmiserv/pmip.c:206): demux engine error waiting for event
[mpiexec#myac] HYDT_bscu_wait_for_completion (./tools/bootstrap/utils/bscu_wait.c:76): one of the processes terminated badly; aborting
[mpiexec#myac] HYDT_bsci_wait_for_completion (./tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting for completion
[mpiexec#myac] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:217): launcher returned error waiting for completion
[mpiexec#myac] main (./ui/mpich/mpiexec.c:331): process manager error waiting for completion
Why am I getting error when I am providing the username?
You could try specifying a username in your ssh config file (http://www.cyberciti.biz/faq/create-ssh-config-file-on-linux-unix/) instead of on the mpirun command line. That way perhaps mpirun would not be confused by the extra username part, which as far as I can see from the documentation it does not support. But ssh could, behind the scenes, use the username you specify in your ssh config file. And of course you'll want to set up SSH keys so you don't have to type a password.
I don't believe MPICH supports providing usernames in the --hosts value on the command line. You should try the host file based method described on the wiki. http://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager#Using_Hydra_on_Machines_with_Different_User_Names
For example:
shell$ cat hosts
donner user=foo
foo user=bar
shakey user=bar
terra user=foo
I use a debugging script that runs several related processes in succession with the debugger. I'm currently using -x to execute several commands automatically (such as run). How can I make gdb quit automatically when the debugged process successfully terminates? Adding a quit command to the command file will cause that command to be handled not just on successful termination, but when errors occur also (when I'd rather take over at that point).
Here's an extract of what's going on:
+ gdb -return-child-result -x gdbbatch --args ./mkfs.cpfs /dev/loop0
GNU gdb (GDB) 7.1-ubuntu
Reading symbols from /home/matt/cpfs/mkfs.cpfs...done.
Program exited normally.
Breakpoint 2 at 0x805224f: file log.c, line 32.
(gdb)
Contents of gdbbatch:
start
b cpfs_log if level >= WARNING
I think I have found a complete solution to your question in connection to looking for something similar in How to make gdb send an external notification on receiving a signal?. None of the other guys here seem to have mentioned or discovered gdb hooks.
Based on Matthew's tip about $_exitcode, this is now my app/.gdbinit that achieves exactly the behavior wanted; normal quit on successful termination and drop to gdb prompt, send email, whatnot on everything else:
set $_exitcode = -999
set height 0
handle SIGTERM nostop print pass
handle SIGPIPE nostop
define hook-stop
if $_exitcode != -999
quit
else
shell echo | mail -s "NOTICE: app has stopped on unhandled signal" root
end
end
echo .gdbinit: running app\n
run
gdb sets $_exitcode when the program successfully terminates. You can make use of that - set it to an unlikely value at the start of your script, and only quit at the end if it has changed:
set $_exitcode = -999
# ...
run
# ...
if $_exitcode != -999
quit
end
(Setting $_exitcode to an unlikely value is a bit ugly, but it will otherwise not be defined at all if the program doesn't terminate, and there doesn't seem to be any obvious way of asking "is this variable defined?" in a conditional.)
GDB has a different "language" for interacting with automated programs called GDB/MI (detailed here), but unfortunately, it doesn't look like it supports conditionals, and is expected to run from other programs with parsing and branching. So, it looks like Expect is the easiest (or at least a working) solution:
$ cat gdbrunner
#!/usr/bin/expect -f
#spawn gdb -return-child-result --args ./mkfs.cpfs /dev/loop0
spawn gdb -return-child-result --args [lindex $argv 0]
#send "start\n"
#send "b cpfs_log if level >= WARNING"
send "run\n"
expect {
normally\. { send "quit\n" }
"exited with code" { interact -nobuffer }
}
I tested this with the simple programs:
$ cat prog1.c
int main(void) { return 0; }
$ cat prog2.c
int main(void) { return 1; }
With the following results:
$ ./gdbrunner ./prog1
spawn gdb -return-child-result --args ./prog1
run
(gdb) run
Starting program: /home/foo/prog1
Program exited normally.
(gdb) quit
$ ./gdbrunner ./prog2
spawn gdb -return-child-result --args ./prog2
run
(gdb) run
Starting program: /home/foo/prog2
Program exited with code 01.
(gdb)
Essentially, you have to parse the output and branch using something else. This would of course work with any other program capable of handling input/output of another process, but the above expect script should get you started, if you don't mind Tcl. It should be a little better, and expect the first (gdb) prompt, but works due to stdin buffering.
You can also modify it to use that GDB/MI interface with the -i command-line argument to GDB; its commands and output are a bit more readily parsable, if you will expand to need more advanced features, as you can see in the previously linked documentation.