I'm creating a simple network scanning function using nmap and C. I want to use popen() to execute nmap, but nmap takes close to 30 seconds to complete because I'm scanning a wide range of IPs.
So, is there a way to check when the command has finished executing? I don't want my C code to hang at the system call, instead I would like to just check a flag or something in a loop that will know when popen/nmap has finished so other parts of my C program don't come to an halt. Is this possible??
Thanks!
I can think of 2 direct ways to do it
You could fork() directly and then establish a way to communicate the two processes, this would be very difficult because IPC is not an easy matter.
You could create a new thread with pthread_create() and call popen() there, it would be a lot easier and you could share data between threads by using an appropriate locking mechanism to prevent race conditions.
You can either use multi processing with fork (hardmode)
Or you can use multithreading using pthread (easymode)
Either one allows you to do 2 things at once. Multiprocessing is hard because you must worry about innerproccess communications (pipes) and the 2 tasks you're trying to do can not share memory.
Multithreading is a much more easy because all you need is to include the libraries (-lpthread) and then specify what function is on the seperate thread.
Related
I'm not sure if the title accurately describes what I want to do but here's the rub:
We have a large and hairy codebase (not-invented-here courtesy of Elbonian Code Slaves) which currently compiles as one big binary which internally creates several pthreads for various specific tasks, communicating through IPC messages.
It's not ideal for a number of reasons, and several of the threads would be better as independent autonomous processes as they are all individual specific "workers" rather than multiple instances of the same piece of code.
I feel a bit like I'm missing some trick, is our only option to split off the various thread code and compile each as a standalone executable invoked using system() or exec() from the main blob of code? It feels clunky somehow.
If you want to take a part of your program that currently runs as a thread, and instead run it as a separate process launched by your main program, then you have two main options:
Instead of calling pthread_create(), fork() and in the child process call the thread-start function directly (do not use any of the exec-family functions).
Compile the code that the the thread executes as a separate executable. Launch that executable at need by the standard fork / exec sequence. (Or you could use system() instead of fork/exec, but don't. Doing so needlessly brings the shell into it, and also gives you much less control.)
The former has the disadvantage that each process image contains a lot of code that it will never use, since each is a complete copy of everything. Inasmuch as under Linux fork() uses copy-on-write, however, that's mostly an address-space issue, not a resource-wastage issue.
The latter has the disadvantage that the main program needs to be able to find the child programs on the file system. That's not necessarily a hard problem, mind you, but it is substantially different from already having the needed code at hand. If there is any way that any of the child programs would be independently useful, however, then breaking them out as separate programs makes a fair amount of sense.
Do note, by the way, that I do not in general accept your premise that it is inappropriate to implement specific for-purpose workers as threads. If you want to break out such tasks, however, then the above are your available alternatives.
Edited to add:
As #EOF pointed out, if you intend that after the revamp your main process will still be multi-threaded (that is, if you intend to convert only some threads to child processes) then you need to be aware of a significant restriction placed by POSIX:
If a multi-threaded process calls fork(), [...] to avoid errors, the child process may only execute async-signal-safe operations until such time as one of the exec functions is called.
On the other hand, I'm pretty sure the relevant definition of "multi-threaded" is that the process has multiple live threads at the time fork() is called. It should not present a problem if the child processes are all forked off before any additional threads are created, or after all but one thread is joined.
I've been asked to create a dbus interface to a program that uses SocketCAN to communicate to a set of control units on a car (that's just background info and not relevant to the question). The original concept was to use a listener interface that spawned a new "node" for each control unit being interrogated in the car. This would lead to something like the following objects on the com.acme.myservice bus:
/com/acme/listener
/com/acme/node1
/com/acme/node2
...
/com/acme/nodeN
The idea is to take over the code from a C program that already communicates from the command line with a single ECU. That code depends on fork()ing in order to run parts of the communication cyclically and other parts in parallel for timing reasons.
I had planned on using the GDBusObjectManager model using new code for the listener and reusing the fork()ing code for each node instance. Unfortunately I ran into trouble because I still need to process DBus messages and relay the information to the (now forked) children and that's not supported:
On UNIX, the GLib mainloop is incompatible with fork(). Any program using the mainloop must either exec() or exit() from the child without returning to the mainloop.
I could restructure the forking code so that it executes in a single process in between iterations of the main loop instead of using g_main_loop_run but this will probably cause too much latency if there's a significant amount of data and multiple nodes running.
What's the best way around this? Is it possible to somehow handle these parallel items with a built-in glib mechanism or is there a way around the mainloop/forking issue? Is the only solution to use a different IPC mechanism between a single process using DBus and forked children doing the actual work? (This would probably be as much code as needed for the gdbus interface so it makes it sort of redundant)
I am implementing an application which executes two programs in lockstep. Each system call is a synchronization point. An application might have more than one thread, thus I need to identify unequivocally each of them in order to synchronize the execution of a thread from the first application with the execution of the same thread in the second application.
Is there a way to identify if two remote threads are executing the same code or function?
Every suggestion is welcomed!! :D
Well it's hard to say not knowing how you plan to do this synchronization. Are the two program communicating to each other and/or a third monitoring pgm?
In any case there are at least 3 possibilities:
Use an associative container like a map in the two programs (or the 3rd) that matches up
the pthread thread ids from the two programs (e.g. pthread_self() to get the tids)
the linux thread ids (e.g. gettid())
Or you can possibly make use of pthread_setname_np() and pthread_getname_np(). You can use these to give each thread in both programs the same name and maybe that becomes useful in some messaging scenario. You might also make use of the __FILE__, __LINE__ and __FUNCTION__ (__func__ in c99) macros in conjuction with the thread name if you are messaging.
That's my (black box) suggestions!
As you might know, all threads in the application die in a forked process, other than the thread doing the fork. However, I plan to ressurrect those threads in the forked process by calling pthread_create and using pthread_attr_setstack, so as to assign the newly created threads the same stack as the dead threads. Something like as follows.
// stackAddr and stacksize taken from the dead thread
pthread_attr_setstack(&attr, stackAddr, stacksize);
rc = pthread_create(&thread, &attr, threadRoutine, NULL);
However, I would still need to get the CPU register values, such as stack pointer, base pointer, instruction pointer etc, to restart threads from the same point. How can I do that? And what else do I need to do to successfully achieve my goal?
Also note that I'm using a 64-bit architecture. What additional difficulties would it have as compared to 32-bit one?
I see two possible ways to shoot yourself in the foot and lose hair^W^W^W^W^W^W^W^Wtry to do this:
Try to force each thread into calling getcontext() before the fork(), and then restore the context of each thread via setcontext(). Probably won't work, but you can try for fun.
Save ptrace(PTRACE_GETREGS), ptrace(PTRACE_GETFPREGS), and restore with ptrace(PTRACE_SETREGS), ptrace(PTRACE_SETFPREGS).
The other threads in the current process aren't killed by a fork -- they're still there and running in the parent. The problem you seem to have is that fork only forks a SINGLE thread in the current procces, creating a new process running one thread with a copy of all non-thread resources in the parent.
What you apparently want is a way of duplicating an entire multithreaded task, forking all the threads in it and creating a new process/task with the same number of threads.
In order to do THAT, you would need to find and pause all the other threads in the process, dump their current state (including all locks they hold), fork a new process, and then (re)create each of those other threads in the child, rewiring the lock state to refer to the new child threads where needed.
Unfortunately, the POSIX pthread interface is hopelessly underspecified, and provides no way of doing that. In particular, it lacks any sort of reflective interface allowing you to figure out what threads are actually running.
If you want to try to do this anyway, I can see two ways of trying to approach this:
poke around in /proc/self/task to figure out what threads are running in your process, effectively getting that reflective interface in a highly non-portable way. You'll likely end up having to ptrace(2) the other threads to get their internal state. This will be very difficult.
wrap the pthreads library -- instead of using library directly, intercept every call and keep track of all the threads/mutexes/locks that get created, so that you have that information available when you want to fork. This will work fine as long as you don't want to use any third-party libraries that use pthreads
The second option is much easier (and somewhat portable), but only works well if you have access to all the source code of your entire application, and can modify it to use your wrappers properly.
Just googling around I found that solaris has a forkall() call that does exactly what you want, see the documentation here:
http://download.oracle.com/docs/cd/E19963-01/html/821-1601/gen-1.html
I assume you're running on linux, but it is possible to run solaris on x86 hardware. So maybe that is an option for you.
There is a C process, I need to call one function which will modify one C variables by Ruby program, I want to know how to do this, thanks!
Why don't you use sockets?
You could even use threads or forks to avoid locking.
If you are on linux you should check out D-Bus, it allows communication through sockets too, but with some extra goodies ;)