I have a C program creating a detached thread as a child.
Inside of the function I pass to pthread_create I use pthread_detach to detach the thread. At the end I call pthread_exit((void *) 0)
I would like to know if it is normal behaviour that the memory consumption increases after the thread is created.
I did a valgrind check and there are no leaks just 4 suppressed errors.
I would like to know if it is normal behaviour that the memory consumption increases after the thread is created.
Yes, as
each thread gets its own stack assigned. The size is OS setting dependend and could be around 1M.
some system resource will be used to manage each thread itself.
Both will be released if the thread ends for a detached thread or if the thread was joined for a joinable thread.
Related
On linux, pthread (linux threads),
what does happen to the running threads when returning from main (before the threads are finished)?
When returning from main, the memory is dis-allocated so the threads should access unallocated memory. Does this cause the threads to exit?
I'm sure the threads are killed, but how does this actually happen?
I'm sure the threads are killed, but how does this actually happen?
Returning from main is the same as calling exit(). This means handlers established by atexit(), and any system cleanup handlers are run. Finally the kernel is asked to terminate the entire process(i.e. all threads).
(Note that this might cause issues if you have other threads running at that point, e.g. another thread accessing a global C++ objects right after the runtime calls their destructors.)
Well, threads operate under the process of main application (or other process but I assume you do not create another process, just threads). They share memory with it, and are the same process, so is system kills the process it automatically kills all threads. There is nothing more to it. A thread cannot exists without a process, so there is no option of accessing some disallocated memory, it just stops executing, and the memory is cleaned up on a process clean-up level.
And how it happens is obviously system dependent. E.g. Windows 95 did not free memory after a process ended, so if application had a memory leak, killing it didn't help. This had changed since then. Every system can handle it differently.
I am wondering, what can happen if we do a pthread_create without a pthread_join?
Who will "clean" all the memory of the "non-joined" thread.
When the process terminates, all resources associated with the process cease to exist. (This of course does not include shared resources the process created, like files in the filesystem, shared memory segments, etc.) Until then, unjoined threads will continue to consume resources, potentially calling future calls to pthread_create or even malloc to fail.
Well, assuming that it's an app-lifetime thread that does not need or try to explicitly terminate, the OS will do it when its process is terminated, (on all non-trivial OS).
If the thread is created without using pthread_join then when the main thread completes execution all other threads created in main function will be stopped and hence will not complete executing the whole statements in it.
Look at the documentation of Pthread_join.
It will make the main thread to suspend until the spawned thread completes execution.
In man page of pthread_detach i read that when any thread is detached then at a time of thread termination it release its resources back to system.
What are thread resources?Is that it is a part of memory used by that thread, if it is so then that memory is a part of a process address space. I am trying to understand this but i dint got it.
And what about the joinable thread, when does thread release its resources? at a time of pthread_join or at a time of termination or process?
When resources are released in pthread_cancel command.
Every thread consumes some amount of bookkeeping resources in the operating system, as well as its own execution stack in userspace memory. Those resources are freed when the thread is destroyed, which can occur under several conditions, such as:
pthread_join returns when called on a joinable thread,
a detached thread's entry function returns,
main returns normally or exit is called,
the process is terminated due to receiving an unhandled signal,
exec is called successfully.
It is possible, however, to exit the thread that is running main and leave other, detached threads running. To do so, you have to call pthread_exit in the main thread.
I create more than 100 threads from my main() so I just wanted to know that do I need to call pthread_join() before I exit my main().
Also, I do not need the data generated by these threads, basically, all the threads are doing some job independent from main() and other threads.
pthread_join does two things:
Wait for the thread to finish.
Clean up any resources associated with the thread.
If you exit the process without joining, then (2) will be done for you by the OS (although it won't do thread cancellation cleanup, just nuke the thread from orbit), and (1) will not. So whether you need to call pthread_join depends whether you need (1) to happen.
If you don't need the thread to run, then as everyone else is saying you may as well detach it. A detached thread cannot be joined (so you can't wait on its completion), but its resources are freed automatically if it does complete.
Yes if thread is attachable then pthread_join is must otherwise it creates a Zombie thread.
Agree with answers above, just sharing a note from man page of pthread_join.
NOTES
After a successful call to pthread_join(), the caller is guaranteed that the target thread has terminated.
Joining with a thread that has previously been joined results in undefined behavior.
Failure to join with a thread that is joinable (i.e., one that is not detached), produces a "zombie thread". Avoid doing this, since each zombie thread consumes some system resources, and when
enough zombie threads have accumulated, it will no longer be possible to create new threads (or processes).
When you exit, you do not need to join because all other threads and resources will be automatically cleaned up. This assumes that you actually want all the threads to be killed when main exits.
If you don't need to join with a thread, you can create it as a "detached" thread by using pthread_attr_setdetachstate on the attributes before creating the thread. Detached threads cannot be joined, but they don't need to be joined either.
So,
If you want all threads to complete before the program finishes, joining from the main thread makes this work.
As an alternative, you can create the threads as detached, and return from main after all threads exit, coordinating using a semaphore or mutex+condition variable.
If you don't need all threads to complete, simply return from main. All other threads will be destroyed. You may also create the threads as detached threads, which may reduce resource consumption.
By default threads in pthreads library are created as joinable.
Threads may, however, detach, rendering them no longer joinable. Because threads consume system resources until joined, just as processes consume resources until their parent calls wait(), threads that you do not intend to join must be detached, which is a good programming practice.
Of course once the main routine exits, all threading resources are freed.
If we fail to do that(detaching), then, when the thread terminates it produces the thread equivalent of a zombie process. Aside from wasting system resources, if enough thread zombies accumulate, we won't be able to create additional threads.
Per default a thread runs attached, that means the resources it needs are kept in use until the thread is joined.
As from your description noone but the thread itself needs the thread's resources, so you might create the thread detached or detach the thread prior to having it started.
To detach a thread after its creation call pthread_detach().
Anyhow if you want to make sure all threads are gone before the program ends, you should run the threads attached and join them before leaving the main thread (the program).
If you want to be sure that your thread have actually finished, you want to call pthread_join.
If you don't, then terminating your program will terminate all the unfinished thread abruptly.
That said, your main can wait a sufficiently long time until it exits. But then, how can you be sure that it is suffucient?
If your main ends your application ends and your threads die... So you do need to use thread join (or use fork instead).
I'm having a problem with Valgrind telling me I have some memory possible lost:
==23205== 544 bytes in 2 blocks are possibly lost in loss record 156 of 265
==23205== at 0x6022879: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==23205== by 0x540E209: allocate_dtv (in /lib/ld-2.12.1.so)
==23205== by 0x540E91D: _dl_allocate_tls (in /lib/ld-2.12.1.so)
==23205== by 0x623068D: pthread_create##GLIBC_2.2.5 (in /lib/libpthread-2.12.1.so)
==23205== by 0x758D66: MTPCreateThreadPool (MTP.c:290)
==23205== by 0x405787: main (MServer.c:317)
The code that creates these threads (MTPCreateThreadPool) basically gets an index into a block of waiting pthread_t slots, and creates a thread with that. TI becomes a pointer to a struct that has a thread index and a pthread_t. (simplified/sanitized):
for (tindex = 0; tindex < NumThreads; tindex++)
{
int rc;
TI = &TP->ThreadInfo[tindex];
TI->ThreadID = tindex;
rc = pthread_create(&TI->ThreadHandle,NULL,MTPHandleRequestsLoop,TI);
/* check for non-success that I've omitted */
pthread_detach(&TI->ThreadHandle);
}
Then we have a function MTPDestroyThreadPool that loops through all the threads we created and cancels them (since the MTPHandleRequestsLoop doesn't exit).
for (tindex = 0; tindex < NumThreads; tindex++)
{
pthread_cancel(TP->ThreadInfo[tindex].ThreadHandle);
}
I've read elsewhere (including other questions here on SO) that detaching a thread explicitly would prevent this possibly lost error, but it clearly isn't. Any thoughts?
glibc's threads implementation intentionally leaks memory. It keeps the memory allocated to a thread context cached to reuse the next time a thread is created. I did some benchmarking versus an implementation without the caching, and it seems the caching shaves 50% off the otherwise-optimal time for pthread_create, but drastically slows down pthread_join, for a net loss. Of course it's still a (small) gain if what you care about is thread creation latency and not throughput.
Also note that it's very difficult for a detached thread to deallocate its context, even if it wanted to. With a joinable thread, the thread that calls pthread_join can deallocate the context, but a detached thread would have to be able to operate with no stack during the interval between deallocating its context and terminating itself. This can only be achieved by writing that small piece of code in pure asm.
Wondering how detached threads' contexts get returned to the cache without a similar race condition? Linux has a feature to zero-out an int at a particular address (registered by the userspace threads library) when the thread terminates. So the thread can add its own context to the cache safely, since until it terminates, other threads will still see a nonzero value (usually its thread-id) at this address and interpret that to mean that the context is still in use.
One reason could be that pthread_cancel doesn't actually cancel the thread - it's not guaranteed to. Thread cancellation is asynchronous; pthread_cancel returns immediately, but cancellation may be deferred until the next cancellation point. In that case, the threads may still be around when Valgrind collects statistics.
Creating a thread joinable to detach it immediately makes not much sense to me. This creates nothing but overhead for the system.
I'd start the threads detached from the start, you have your shared ThreadInfo data structure anyhow to control your threads.
Also I'd have a flag or something like that in your per thread argument ThreadInfo that tells the thread to shutdown in a controlled way.