I compiled and install a new version of the kernel and now I have to make the new kernel support two new primitives, I explain more below
I have this file, mysystemcalls-test.c where I get the process and do forks in a for iteration. This is my test file.
//mysyscalls-test.c
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <mysyscalls.h>
#define NCHILDREN 10
int main()
{
int pids[NCHILDREN], i;
int original_parent = getpid();
printf ("before fork: current number of children: %ld\n", getnchildren(1));
for (i=0; i<NCHILDREN; i++) {
pids[i] = fork();
if (pids[i] == 0) {
while (getppid() == original_parent);
exit(0);
}
}
printf("after fork: current number of children: %ld\n", getnchildren(1));
for(i=0; i<NCHILDREN; i++)
printf("pids[%d]:%d\tgetcpid(%d,1):%ld\n" ,i,pids[i],i,getcpid(i,1));
return 0;
}
The real problem is in the second file, mysyscalls.h I can't make the right call for the child process.
//mysyscalls.h
long getnchildren (int debug){
int children;
children = pids[i];
return children;
}
long getcpid (int order, int debug){
int childrenid;
childrenid = getpid();
return childrenid;
}
I don't know how to call these pids in the mysyscalls.h file. I searched and tested some code that I found here but I don't have an exact answer for my problem. I wrote that code but isn't working, it return the same pid.
It's supposed that long getnchildren returns the number of child processes that the calling process has, at the moment the primitive is called.
long getcpid returns the pid of the child of the calling process, at position order in the list of its children.
Get your own pid with getpid.
Iterate through all directories in /proc that start with a digit.
For each such directory, open the status file.
Read all the lines until you find the PPid line.
If the PPid matches the value from getpid, add this directory name to your array.
That will get you the IDs of every process whose parent process is this process.
This outlines a simple way to do some common functions involving for PIDs. If I were you, I would circumvent trying to do this in C. More trouble than its worth, IMO.
#!/bin/bash
# getting children generally resolves nicely at some point
get_child() {
echo $(pgrep -laP $1 | awk '{print $1}')
}
# recursively getting parents isn't that useful,
# so single use function
get_parent() {
echo $(ps -o ppid= -p 36700)
}
get_children() {
__RET=$(get_child $1)
__CHILDREN=
while [ -n "$__RET" ]; do
__CHILDREN+="$__RET "
__RET=$(get_child $__RET)
done
__CHILDREN=$(echo "${__CHILDREN}" | xargs | sort)
echo "${__CHILDREN}"
}
get_children $1
FILE *fp;
char path[1024];
char command[1024];
sprintf(command, "/bin/bash /get_pids.sh %d", getpid());
fp = popen(command, "r");
while (fgets(path, sizeof(path), fp) != NULL) {
// children are listed here
printf("%s", path);
}
pclose(fp);
I've been trying to cook up an unholy one-liner for David Schwartz's method, but I think it is a fully cursed solution. Some other bash-abuser can take a crack at it.
I would like to get the number of ALL running processes and ALL running threads of my Linux system. I need this information IN a C application. From the terminal, I can get his info with
$ ps -A --no-headers | wc -l
for processes and
$ ps -AL --no-headers | wc -l
for processes including tasks.
I didn't find this info in /proc.
Any suggestions?
The ps examples in your question don't really give you the information you're asking about: the first lists all the processes on your system, not just those spawned by a particular program, and similarly the second lists the number of threads in every process.
If you want to find information about threads spawned by a particular process, you can look in /proc under /proc/<pid>/task. For example, here is a process with a single thread:
bash-4.4$ ls /proc/15355/task/
15355
And here's one that has three threads (in addition to the main thread):
bash-4.4$ ls /proc/15295/task/
15295 15296 15297 15298
The corresponding ps -L output for that process looks like:
bash-4.4$ ps -L -p 15295
PID LWP TTY TIME CMD
15295 15295 pts/4 00:00:00 python
15295 15296 pts/4 00:00:00 python
15295 15297 pts/4 00:00:00 python
15295 15298 pts/4 00:00:00 python
Getting the number of running processes from /proc takes a little more work, since Linux only maintains information about a process' parent, rather than it's children. That means you would need to scan through /proc and find every process for which the parent is your target process...and then repeat recursively for each of those processes.
You can use something like pstree to get this information, of course, but that output isn't really designed to be machine parseable.
Every running process has a corresponding directory /proc/<pid>. You can use that to count the number of running processes (by counting all sub-directories of /proc that are numeric).
Inside each of these directories you can check /proc/<pid>/status to get information about the process. Specifically, the line Threads: <cnt> gives you the number of threads for that process.
Refer to man proc for more info on the /proc (pseudo) file system.
The simplest is to parse the output of commands using popen.
The following:
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
void parse_output(char *buf, size_t bufsize, const char cmd[])
{
assert(buf != NULL);
assert(cmd != NULL);
FILE *fp;
// add dynamic allocation here
memset(buf, 0, bufsize);
if ((fp = popen(cmd, "r")) == NULL) {
printf("Error opening pipe!\n");
exit(-__LINE__);
}
// worst speed ever. And strlen is called twice...
while (fgets(&buf[strlen(buf)], bufsize - strlen(buf), fp) != NULL);
if(pclose(fp)) {
printf("Command not found or exited with error status\n");
exit(-__LINE__);
}
}
int main() {
char buf[256];
long num;
parse_output(buf, sizeof(buf), "ps -A --no-headers | wc -l");
if (sscanf(buf, "%ld", &num) != 1) {
exit(-__LINE__);
}
printf("Number of processes: %ld\n", num);
parse_output(buf, sizeof(buf), "ps -AL --no-headers | wc -l");
if (sscanf(buf, "%ld", &num) != 1) {
exit(-__LINE__);
}
printf("Number of processes including tasks: %ld\n", num);
}
will output on my system:
$ gcc 1.c && ./a.out
Number of processes: 241
Number of processes includeing tasks: 867
I am trying to figure out a way to profile in C a process via its child and after a moment the parent process kills its child to stop profiling. I am using perf to profile my application. perf is going to output its result in a file when killed. It looks like this in a bash script :
./run &
perf stat -o perf.data -p <pid_run> &
kill -9 <pid_perf>
What I have done so far :
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <sys/types.h>
#include <unistd.h>
#include <fcntl.h>
static pid_t perf_id;
void start() {
char *filename="test_data";
char ppid_str[24];
pid_t pid = fork();
if (pid == 0){
pid_t ppid = getppid();
sprintf(ppid_str, "%d",ppid);
char *args[] = {"/usr/bin/perf", "stat","-p",ppid_str,"-o", filename, NULL};
execvp(args[0], args);
}
else {
perf_id = pid
}
}
void stop() {
kill(perf_id,SIGKILL);
}
I have an issue getting the output of perf.
This is an example of code that could run the parent process :
int main() {
start();
int a = 0;
a+=1;
stop();
// ... // There are other instructions after the stop
return 0;
}
I am not getting any output from perf when running this code. I have to kill the parent process to get an output.
If I put a sleep call before killing the child process, then the program will output an empty file.
EDIT :
stat argument is an example in my command, I want also to use the record argument
As mentioned by Zulan, if I use SIGINT instead of SIGKILL, I will get an output, but I can get one only if the main process sleeps for 1 second.
You should send a SIGINT instead of a SIGKILL in order to allow perf to shutdown cleanly and produce a valid output file. The synchronization between the perf child process and the main process will still be imperfect - so if the main process doesn't take significant time as in your example, it is easily possible that no output file is generated at all. This also affects the accuracy of collected data. With the setup of using perf as a child process rather than vice-versa, you cannot really improve it.
the problem is that perf attaches itself to the process and then waits for process termination to print counters. try adding for example the
-I msec
option to perf like -I 1000 to print counters every 1s.
changing your args to execvp to
char *args[] = {"/usr/bin/perf", "stat", "-p",ppid_str,"-o", filename, "-I", "1000", NULL};
and your inc to a loop of something like
while (a < 10) {
a += 1;
sleep(1);
}
while yield results although the file is not properly closed() in this approach.
I would create a small binary that execs perf with a timeout and gracefully closes the file and run that from the child.
I've managed to make a daemon using the following code. My question is I want to create a script to start this daemon and store the daemons PID in /var/run/mydaemon.pid. Moreover, a second script to stop the daemon by accessing the stored mydaemon.pid file.
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#define EXIT_SUCCESS 0
#define EXIT_FAILURE 1
static void daemonize(void)
{
pid_t pid, sid;
/* already a daemon */
if ( getppid() == 1 ) return;
/* Fork off the parent process */
pid = fork();
if (pid < 0) {
exit(EXIT_FAILURE);
}
/* If we got a good PID, then we can exit the parent process. */
if (pid > 0) {
exit(EXIT_SUCCESS);
}
/* At this point we are executing as the child process */
/* Change the file mode mask */
umask(0);
/* Create a new SID for the child process */
sid = setsid();
if (sid < 0) {
exit(EXIT_FAILURE);
}
/* Change the current working directory. This prevents the current
directory from being locked; hence not being able to remove it. */
if ((chdir("/")) < 0) {
exit(EXIT_FAILURE);
}
/* Redirect standard files to /dev/null */
freopen( "/dev/null", "r", stdin);
freopen( "/dev/null", "w", stdout);
freopen( "/dev/null", "w", stderr);
}
int main( int argc, char *argv[] ) {
daemonize();
/* Now we are a daemon -- do the work for which we were paid */
return 0;
}
I've look around and can't seem to find example code to help me. The closest thing I've got is something you see below. But it's not working.
#!/bin/sh
set -e
# Must be a valid filename
NAME=mydaemon
PIDFILE=/var/run/$NAME.pid
DAEMON=/home/me/mydaemon/mydaemon/a.out
export PATH="${PATH:+$PATH:}/usr/sbin:/sbin"
case "$1" in
start)
echo -n "Starting daemon: "$NAME
start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON
echo "."
;;
*)
echo "Usage: "$1" {start}"
exit 1
esac
exit 0
With the daemonizing code as written, you cannot determine the PID of the daemonized process you just started because that information isn't made available. The parent process information would be available if you ran the program in the background ($! would report the PID if you use ./mydaemon &), but that process just exited, leaving another process to run.
You need the help of the daemonizing code; the parent code should report the PID of the child process before exiting.
/* If we got a good PID, report child PID and exit the parent process. */
if (pid > 0) {
printf("%d\n", (int)pid);
exit(EXIT_SUCCESS);
}
Now you can use:
NAME=mydaemon
# PIDFILE=/var/run/$NAME.pid
# DAEMON=/home/me/mydaemon/mydaemon/a.out
pidfile="/var/run/mydaemon.pid"
pid=$($NAME)
if [ -n "$pid" ]
then
echo "$pid" > "$pidfile"
else
echo "$0: failed to launch daemonized process '$NAME'" >&2
exit 1
fi
This relies on the code (in the daemon) not writing to standard output unless it is successful in forking. It can write to standard error if it needs to report any errors.
You could return child pid on parent C program exiting return call
and read value from $? variable on calling script.
./myDeamon
childPid=$?
So I'm making a UNIX minishell, and am trying to add pipelines, so I can do things like this:
ps aux | grep dh | grep -v grep | cut -c1-5
However I'm having trouble wrapping my head around the piping part. I replace all the "|" characters with 0, and then run each line as a normal line. However, I am trying to divert the output and input. The input of a command needs to be the output of the previous command, and the output of a command needs to be the input of the next command.
I'm doing this using pipes, however I can't figure out where to call pipe() and where to close them. From the main processing function, processline(), I have this code:
if((pix = findUnquotChar(line_itr, '|')))
{
line_itr[pix++] = 0;
if(pipe (fd) < 0) perror("pipe");
processline(line_itr, inFD, fd[1], pl_flags);
line_itr = &(line_itr[pix]);
while((pix = findUnquotChar(line_itr, '|')) && pix < line_len)
{
line_itr[pix++] = 0;
//? if(pipe (fd) < 0) perror("pipe");
processline(line_itr, fd[0], fd[1] pl_flags);
line_itr = &(line_itr[pix]);
//? close(fd[0]);
//? close(fd[1]);
}
return;
}
So, I'm recursively(the code above is in processline) sending the commands in between the "|" to be processed by processline. You can see where I commented out the code above, I'm not sure how to make it work. The 2nd and 3rd parameter of processline are the inputFD and outputFD respectively, so I need to process a command, write the output to a pipe, and then call processline again on the next command, however this time the output of the previous command is the input. This just doesn't seem like it can work though, because each time I close fd[0] I'm losing the previous output. Do I need two separate pipes, that I can flip flop back and forth with?
I'm just having trouble seeing how this is possible with a single pipe, if you guys need any additional info just ask. Here's the entire processline function in case you want to take a look:
http://pastebin.com/YiEdaYdj
EDIT: If anybody has an example of a shell that implements pipelines I would love a link to the source, I haven't been able to find one on google so far.
EDIT2: Here's an example of my predicament:
echo a | echo b | echo c
So first I would call the shell like this:
processline("echo a", 0, fd[1], flags);
....
processline("echo b", fd[0], NOT_SURE_GOES_HERE[1], flags);
....
processline("echo c", NOT_SURE_GOES_HERE[0], NOT_SURE_EITHER[1], flags);
Each of these occurs once per iteration, and as you can see I can't figure out what to pass for the input-file-descriptors and the output-file-descriptors for the 2nd and 3rd(and so on) iteration.
Here's some moderately generic but simple code to execute pipelines, a program I'm calling pipeline. It's an SSCCE in a single file as presented, though I'd have the files stderr.h and stderr.c as separate files in a library to be linked with all my programs. (Actually, I have a more complex set of functions in my 'real' stderr.c and stderr.h, but this is a good starting point.)
The code operates in two ways. If you supply no arguments, then it runs a built-in pipeline:
who | awk '{print $1}' | sort | uniq -c | sort -n
This counts the number of times each person is logged in on the system, presenting the list in order of increasing number of sessions. Alternatively, you can invoke with a sequence of arguments that are the command line you want invoked, use a quoted pipe '|' (or "|") to separate commands:
Valid:
pipeline
pipeline ls '|' wc
pipeline who '|' awk '{print $1}' '|' sort '|' uniq -c '|' sort -n
pipeline ls
Invalid:
pipeline '|' wc -l
pipeline ls '|' '|' wc -l
pipeline ls '|' wc -l '|'
The last three invocations enforce 'pipes as separators'. The code does not error check every system call; it does error check fork(), execvp() and pipe(), but skips checking on dup2() and close(). It doesn't include diagnostic printing for the commands that are generated; a -x option to pipeline would be a sensible addition, causing it to print out a trace of what it does. It also does not exit with the exit status of the last command in the pipeline.
Note that the code starts with a child being forked. The child will become the last process in the pipeline, but first creates a pipe and forks another process to run the earlier processes in the pipeline. The mutually recursive functions are unlikely to be the only way of sorting things out, but they do leave minimal code repetition (earlier drafts of the code had the content of exec_nth_command() largely repeated in exec_pipeline() and exec_pipe_command()).
The process structure here is such that the original process only knows about the last process in the pipeline. It is possible to redesign things in such a way that the original process is the parent of every process in the pipeline, so the original process can report separately on the status of each command in the pipeline. I've not yet modified the code to allow for that structure; it will be a little more complex, though not hideously so.
/* One way to create a pipeline of N processes */
/* stderr.h */
#ifndef STDERR_H_INCLUDED
#define STDERR_H_INCLUDED
static void err_setarg0(const char *argv0);
static void err_sysexit(char const *fmt, ...);
static void err_syswarn(char const *fmt, ...);
#endif /* STDERR_H_INCLUDED */
/* pipeline.c */
#include <assert.h>
#include <stdio.h>
#include <string.h>
#include <sys/wait.h>
#include <unistd.h>
/*#include "stderr.h"*/
typedef int Pipe[2];
/* exec_nth_command() and exec_pipe_command() are mutually recursive */
static void exec_pipe_command(int ncmds, char ***cmds, Pipe output);
/* With the standard output plumbing sorted, execute Nth command */
static void exec_nth_command(int ncmds, char ***cmds)
{
assert(ncmds >= 1);
if (ncmds > 1)
{
pid_t pid;
Pipe input;
if (pipe(input) != 0)
err_sysexit("Failed to create pipe");
if ((pid = fork()) < 0)
err_sysexit("Failed to fork");
if (pid == 0)
{
/* Child */
exec_pipe_command(ncmds-1, cmds, input);
}
/* Fix standard input to read end of pipe */
dup2(input[0], 0);
close(input[0]);
close(input[1]);
}
execvp(cmds[ncmds-1][0], cmds[ncmds-1]);
err_sysexit("Failed to exec %s", cmds[ncmds-1][0]);
/*NOTREACHED*/
}
/* Given pipe, plumb it to standard output, then execute Nth command */
static void exec_pipe_command(int ncmds, char ***cmds, Pipe output)
{
assert(ncmds >= 1);
/* Fix stdout to write end of pipe */
dup2(output[1], 1);
close(output[0]);
close(output[1]);
exec_nth_command(ncmds, cmds);
}
/* Execute the N commands in the pipeline */
static void exec_pipeline(int ncmds, char ***cmds)
{
assert(ncmds >= 1);
pid_t pid;
if ((pid = fork()) < 0)
err_syswarn("Failed to fork");
if (pid != 0)
return;
exec_nth_command(ncmds, cmds);
}
/* Collect dead children until there are none left */
static void corpse_collector(void)
{
pid_t parent = getpid();
pid_t corpse;
int status;
while ((corpse = waitpid(0, &status, 0)) != -1)
{
fprintf(stderr, "%d: child %d status 0x%.4X\n",
(int)parent, (int)corpse, status);
}
}
/* who | awk '{print $1}' | sort | uniq -c | sort -n */
static char *cmd0[] = { "who", 0 };
static char *cmd1[] = { "awk", "{print $1}", 0 };
static char *cmd2[] = { "sort", 0 };
static char *cmd3[] = { "uniq", "-c", 0 };
static char *cmd4[] = { "sort", "-n", 0 };
static char **cmds[] = { cmd0, cmd1, cmd2, cmd3, cmd4 };
static int ncmds = sizeof(cmds) / sizeof(cmds[0]);
static void exec_arguments(int argc, char **argv)
{
/* Split the command line into sequences of arguments */
/* Break at pipe symbols as arguments on their own */
char **cmdv[argc/2]; // Way too many
char *args[argc+1];
int cmdn = 0;
int argn = 0;
cmdv[cmdn++] = &args[argn];
for (int i = 1; i < argc; i++)
{
char *arg = argv[i];
if (strcmp(arg, "|") == 0)
{
if (i == 1)
err_sysexit("Syntax error: pipe before any command");
if (args[argn-1] == 0)
err_sysexit("Syntax error: two pipes with no command between");
arg = 0;
}
args[argn++] = arg;
if (arg == 0)
cmdv[cmdn++] = &args[argn];
}
if (args[argn-1] == 0)
err_sysexit("Syntax error: pipe with no command following");
args[argn] = 0;
exec_pipeline(cmdn, cmdv);
}
int main(int argc, char **argv)
{
err_setarg0(argv[0]);
if (argc == 1)
{
/* Run the built in pipe-line */
exec_pipeline(ncmds, cmds);
}
else
{
/* Run command line specified by user */
exec_arguments(argc, argv);
}
corpse_collector();
return(0);
}
/* stderr.c */
/*#include "stderr.h"*/
#include <stdio.h>
#include <stdarg.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
static const char *arg0 = "<undefined>";
static void err_setarg0(const char *argv0)
{
arg0 = argv0;
}
static void err_vsyswarn(char const *fmt, va_list args)
{
int errnum = errno;
fprintf(stderr, "%s:%d: ", arg0, (int)getpid());
vfprintf(stderr, fmt, args);
if (errnum != 0)
fprintf(stderr, " (%d: %s)", errnum, strerror(errnum));
putc('\n', stderr);
}
static void err_syswarn(char const *fmt, ...)
{
va_list args;
va_start(args, fmt);
err_vsyswarn(fmt, args);
va_end(args);
}
static void err_sysexit(char const *fmt, ...)
{
va_list args;
va_start(args, fmt);
err_vsyswarn(fmt, args);
va_end(args);
exit(1);
}
Signals and SIGCHLD
The POSIX Signal Concepts section discusses SIGCHLD:
Under SIG_DFL:
If the default action is to ignore the signal, delivery of the signal shall have no effect on the process.
Under SIG_IGN:
If the action for the SIGCHLD signal is set to SIG_IGN, child processes of the calling processes shall not be transformed into zombie processes when they terminate. If the calling process subsequently waits for its children, and the process has no unwaited-for children that were transformed into zombie processes, it shall block until all of its children terminate, and wait(), waitid(), and waitpid() shall fail and set errno to [ECHILD].
The description of <signal.h> has a table of default dispositions for signals, and for SIGCHLD, the default is I (SIG_IGN).
I added another function to the code above:
#include <signal.h>
typedef void (*SigHandler)(int signum);
static void sigchld_status(void)
{
const char *handling = "Handler";
SigHandler sigchld = signal(SIGCHLD, SIG_IGN);
signal(SIGCHLD, sigchld);
if (sigchld == SIG_IGN)
handling = "Ignored";
else if (sigchld == SIG_DFL)
handling = "Default";
printf("SIGCHLD set to %s\n", handling);
}
I called it immediately after the call to err_setarg0(), and it reports 'Default' on both Mac OS X 10.7.5 and Linux (RHEL 5, x86/64). I validated its operation by running:
(trap '' CHLD; pipeline)
On both platforms, that reported 'Ignored', and the pipeline command no longer reported the exit status of the child; it didn't get it.
So, if the program is ignoring SIGCHLD, it does not generate any zombies, but does wait until 'all' of its children terminate. That is, until all of its direct children terminate; a process cannot wait on its grandchildren or more distant progeny, nor on its siblings, nor on its ancestors.
On the other hand, if the setting for SIGCHLD is the default, the signal is ignored, and zombies are created.
That's the most convenient behaviour for this program as written. The corpse_collector() function has a loop that collects the status information from any children. There's only one child at a time with this code; the rest of the pipeline is run as a child (of the child, of the child, ...) of the last process in the pipeline.
However I'm having trouble with zombies/corpses. My teacher had me implement it the same way you did, as cmd1 isn't the parent of cmd2 in the case of: "cmd1 | cmd2 | cmd3". Unless I tell my shell to wait on each process (cmd1, cmd2, and cmd3), rather than just waiting on the last process (cmd3), the entire pipeline shuts down before the output can reach the end. I'm having trouble figuring out a good way to wait on them; my teacher said to use WNOHANG.
I'm not sure I understand the problem. With the code I provided, cmd3 is the parent of cmd2, and cmd2 is the parent of cmd1 in a 3-command pipeline (and the shell is the parent of cmd3), so the shell can only wait on cmd3. I did state originally:
The process structure here is such that the original process only knows about the last process in the pipeline. It is possible to redesign things in such a way that the original process is the parent of every process in the pipeline, so the original process can report separately on the status of each command in the pipeline. I've not yet modified the code to allow for that structure; it will be a little more complex, though not hideously so.
If you've got your shell able to wait on all three commands in the pipeline, you must be using the alternative organization.
The waitpid() description includes:
The pid argument specifies a set of child processes for which status is requested. The waitpid() function shall only return the status of a child process from this set:
If pid is equal to (pid_t)-1, status is requested for any child process. In this respect, waitpid() is then equivalent to wait().
If pid is greater than 0, it specifies the process ID of a single child process for which status is requested.
If pid is 0, status is requested for any child process whose process group ID is equal to that of the calling process.
If pid is less than (pid_t)-1, status is requested for any child process whose process group ID is equal to the absolute value of pid.
The options argument is constructed from the bitwise-inclusive OR of zero or more of the following flags, defined in the header:
...
WNOHANG
The waitpid() function shall not suspend execution of the calling thread if status is not immediately available for one of the child processes specified by pid.
...
This means that if you're using process groups and the shell knows which process group the pipeline is running in (for example, because the pipeline is put into its own process group by the first process), then the parent can wait for the appropriate children to terminate.
...rambling... I think there's some useful information here; there probably should be more that I'm writing, but my mind's gone blank.