I'm working on a filesystem project based on FUSE. And I want to add some sort of read ahead to it. So I create a thread to process such tasks, but It seems that I made it really slower than I thought.(Even if I just add a idle thread, it makes my program become much more slower than without that, but that do not happened when I added this function to my server program, which do not use fuse)
I did not simply use fuse_main function, instead I read the sshfs's code and try to initialize it by myself with the following functions,
fuse_parse_cmdline
fuse_mount
fcntl
fuse_new
fuse_daemonize
fuse_set_signal_handlers
fuse_loop_mt
and without add the thread, it runs pretty well, but after I add this thread in
pthread_create(&tid, NULL, test, NULL); // function test is just a while(1){}
it get slower(Read a 100M file, without this thread it is 40s, and with that it is nearly 100s)
Is this something to do about schedparam or something else?
Hope you guys could give me some advice, like what things I need to check.
Thanks again.
Your thread is busy waiting, which means it will use as much CPU power as it can. You might want to add a little delay in your thread to let other threads and processes run too:
while (1)
{
usleep(1000); /* Sleep for one millisecond */
}
Related
Situation:
I have a thread pool of 64 threads. I have a shared buffer, in which I put and get "works" in a synchronous and organized way. As a normal thread pool should work. It all works fine.
Problem:
Inside main(), I want to put a work in the shared buffer and I want a thread in the thread pool to execute it. However, I need main() to wait before that thread finishes that work. Normally I would pthread_join() it, but i don't know which thread is running the work.
Sorry if I didn't make myself very clear, but I hope you can help me.
Thanks in advance!
Include a synchro object,(eg. semaphore), in the work struct. Issue the work and wait on the synchro. Have the pool threads signal the synchro when they have finished. Don't go near join() unless at gunpoint.
Basically, I just want to create a pthread for each number of threads available on the machine, without doing any work. I want to somehow block each one at the start and add it to a threadpool so I can pull an available pthread from the pool when I need work to be done.
How do I go about blocking without having a pthread doing any work? I've tried a dummy function but that doesn't seem to work.
Thanks
Just have it start off by waiting on the work queue. This would normally involve pthread_cond_wait or sem_wait or similar depending on what type of synchronization primitives you're using to manage the work queue.
As you might know, all threads in the application die in a forked process, other than the thread doing the fork. However, I plan to ressurrect those threads in the forked process by calling pthread_create and using pthread_attr_setstack, so as to assign the newly created threads the same stack as the dead threads. Something like as follows.
// stackAddr and stacksize taken from the dead thread
pthread_attr_setstack(&attr, stackAddr, stacksize);
rc = pthread_create(&thread, &attr, threadRoutine, NULL);
However, I would still need to get the CPU register values, such as stack pointer, base pointer, instruction pointer etc, to restart threads from the same point. How can I do that? And what else do I need to do to successfully achieve my goal?
Also note that I'm using a 64-bit architecture. What additional difficulties would it have as compared to 32-bit one?
I see two possible ways to shoot yourself in the foot and lose hair^W^W^W^W^W^W^W^Wtry to do this:
Try to force each thread into calling getcontext() before the fork(), and then restore the context of each thread via setcontext(). Probably won't work, but you can try for fun.
Save ptrace(PTRACE_GETREGS), ptrace(PTRACE_GETFPREGS), and restore with ptrace(PTRACE_SETREGS), ptrace(PTRACE_SETFPREGS).
The other threads in the current process aren't killed by a fork -- they're still there and running in the parent. The problem you seem to have is that fork only forks a SINGLE thread in the current procces, creating a new process running one thread with a copy of all non-thread resources in the parent.
What you apparently want is a way of duplicating an entire multithreaded task, forking all the threads in it and creating a new process/task with the same number of threads.
In order to do THAT, you would need to find and pause all the other threads in the process, dump their current state (including all locks they hold), fork a new process, and then (re)create each of those other threads in the child, rewiring the lock state to refer to the new child threads where needed.
Unfortunately, the POSIX pthread interface is hopelessly underspecified, and provides no way of doing that. In particular, it lacks any sort of reflective interface allowing you to figure out what threads are actually running.
If you want to try to do this anyway, I can see two ways of trying to approach this:
poke around in /proc/self/task to figure out what threads are running in your process, effectively getting that reflective interface in a highly non-portable way. You'll likely end up having to ptrace(2) the other threads to get their internal state. This will be very difficult.
wrap the pthreads library -- instead of using library directly, intercept every call and keep track of all the threads/mutexes/locks that get created, so that you have that information available when you want to fork. This will work fine as long as you don't want to use any third-party libraries that use pthreads
The second option is much easier (and somewhat portable), but only works well if you have access to all the source code of your entire application, and can modify it to use your wrappers properly.
Just googling around I found that solaris has a forkall() call that does exactly what you want, see the documentation here:
http://download.oracle.com/docs/cd/E19963-01/html/821-1601/gen-1.html
I assume you're running on linux, but it is possible to run solaris on x86 hardware. So maybe that is an option for you.
I'm writing a linux daemon in C which gets values from an ADC by SPI interface (ioctl). The SPI (spidev - userland) seems to be a bit unstable and freezes the daemon at random times.
I need to have some better control of the calls to the functions getting the values, and I was thinking of making it as a thread which I could wait for to finish and get the return value and if it times out assume that it froze and kill it without this new thread taking down the daemon itself. Then I could apply measures like resetting the ADC before restarting. Is this possible?
Pseudo example of what I want to achieve:
(function int get_adc_value(int adc_channel, float *value) )
pid = thread( get_adc_value(1,&value); //makes thread calling the function
wait_until_finish(pid, timeout); //waits until function finishes/timesout
if(timeout) kill pid, start over //if thread do not return in given time, kill it (it is frozen)
else if return value sane, continue //if successful, handle return variable value and continue
Thanks for any input on the matter, examples highly appreciated!
I would try looking at using the pthreads library. I have used it for some of my c projects with good success and it gives you pretty good control over what is running and when.
A pretty good tutorial can be found here:
http://www.yolinux.com/TUTORIALS/LinuxTutorialPosixThreads.html
In glib there is too a way to check the threads, using GCond (look for it in the glib help).
In resume you should periodically set a GCond in the child thread and check it in the main thread with a g_cond_timed_wait. It's the same with the glib or the pthread.
Here is an example with the pthread:
http://koders.com/c/fidA03D565734AE2AD9F5B42AFC740B9C17D75A33E3.aspx?s=%22pthread_cond_timedwait%22#L46
I'd recommend a different approach.
Write a program that takes samples and writes them to standard output. It simply need have alarm(TIMEOUT); before every sample collection, and should it hang the program will exit automatically.
Write another program that runs that first program. If it exits, it runs it again. It looks something like this:
main(){for(;;){system("sampler");sleep(1);}}
Then in your other program, use FILE*fp=popen("supervise_sampler","r"); and read the samples from fp. Better still: Have the program simply read the samples from stdin and insist users start your program like this:
(while true;do sampler;sleep 1; done)|program
Splitting up the task like this makes it easier to develop and easier to test, for example, you can collect samples and save them to a file and then run your program on that file:
sampler > data
program < data
Then, as you make changes to program, you can simply run it again on the same data over and over again.
It's also trivial to enable data logging- so should you find a serious issue you can run all your data through your program again to find the bugs.
Something very interesting happens to a thread when it executes an ioctl(), it goes into a very special kind of sleep known as disk sleep where it can not be interrupted or killed until the call returns. This is by design and prevents the kernel from rotting from the inside out.
If your daemon is getting stuck in ioctl(), its conceivable that it may stay that way forever (at least till the ADC is re-set).
I'd advise dropping something, like a file with a timestamp prior to calling ioctl() on a known buggy interface. If your thread does not unlink that file in xx amount of seconds, something else needs to re-start the ADC.
I also agree with the use of pthreads, if you need example code, just update your question.
Is there a way to create a timer (say, to 10 seconds) on a different thread?
I mean, I know how to use CreateThread() and I know how to create/use timers. The problem I have is that the new thread cannot receive a callback function.
For those that will inevitably ask "why do you want to do this?" the answer is because i have to do it this way. it is part of a bigger program that can't at this specific part of the code use callback functions. that's all.
Is there any way to achieve this?
code is appreciated.
Thanks!
EDIT:
A better explanation of the problem:
My application consist of two separate programs. The main program (visible, interface for the user) and another doing the hard work in the background (sort of like a daemon).
The background process need to finishing writing to the DB and closing a lot of little files before exiting.
The main application send a "we're done" message to that background process. Upon receiving this the background process returns the current status and exists.
Now, I need to add the following: upon receiving the message it returns a status and triggers a timer that will wait X amount of time on another thread, in the meantime the background process closes all the DB connections and files. If the timer reached 0 then and the background process is still alive then it terminates it. If the background process closed all the db and files then the thread (and timer) will die before reaching 0 as the application terminates normally.
Is this better?
So, you need a watchdog inside the DB process (I misread again, didn't I). ThreadProc like this will probably suffice, since all threads terminates when main thread terminates:
DWORD WINAPI TerminateAfter10s(LPVOID param) {
Sleep(10000);
ExitProcess(0);
}
If you use the multimedia timer function timeSetEvent, it can be configured to pulse an event rather than use the normal callback. Does that satisfy the requirement ?
I'm more interested in knowing why you have this requirement to avoid the use of a callback. Callbacks would seem to be entirely appropriate to use in a worker thread.