I have a C app that listens to a TCP port and creates a new thread each time it accepts a new connection. It works OK initially, but after a while I start getting error code 11 from pthread_create.
There are no thread-related function calls inside the body of the thread function, and the log shows that there is a matching 'out' for every 'in'.
When it fails, I call the thread function that directly, and it works fine indefinitely on the main thread, so it seems unlikely that I am using up resources within the function. Any suggestions on what causes the error 11, and how to fix it?
This is the thread function:
void * tcp_process_message (void * arg) {
MESSAGE_BUFFER * bp = (MESSAGE_BUFFER *) arg;
USER_LOG (UL_INFO, "tpm in %d", bp - buffers);
...
USER_LOG (UL_INFO, "tpm out %d", bp - buffers);
}
This is the section that creates threads: there is no other code that interacts with the new thread once it is created.
while(!cancel){
connfd = accept(listenfd, (struct sockaddr *) &from_addr, &fromsize);
if (!cancel) {
MESSAGE_BUFFER * bp = allocate_message ();
if (bp == NULL) {
USER_LOG (UL_ERROR, "%s", "allocate_message failed");
close(connfd);
}
else {
bp->connfd = connfd;
strcpy (bp->ip_addr, inet_ntoa(from_addr.sin_addr));
int err = pthread_create (&tid, NULL, &tcp_process_message, (void *) bp);
if (err) {
USER_LOG (UL_ERROR, "thread create failed (%d)", err);
tcp_process_message ((void *) bp);
}
}
}
}
When creating thread, the thread resource still exist even after the thread function exits.
Some piece of code must either wait for the thread using pthread_join(the_thread); or, it must detach the thread and let it die once its function exits: pthread_detach(the_thread);.
If not done, the thread remain in the system and the system will soon run out of resources and won't be able to create new threads anymore.
Related
I'm learning THE LINUX PROGRAMING INTERFACE this book and get confused here.
I invoke the pthread_attr_getdetachstate() and find that thread created by mq_notify() it's JOINABLE, but how to pthread_join() this zoomble(after it terminated) thread?
I saw the man page, but he doesn't use join.
I just looked at the glibc source code for mq_notify.
mq_notify starts a single control thread with PTHREAD_CREATE_DETACHED. Therefore, it is not joinable.
The pthread_t value that gets set by the call from mq_notify to pthread_create is not available to you. So, whatever value you're using in your pthread_attr_getdetachstate call has no relationship to the mq_notify call.
This control thread created by mq_notify may create other threads. But, they also run detached [using pthread_detach(pthread_self())].
UPDATE:
I wonder whether the OP is talking about using SIGEV_THREAD to receive message notifications. It makes some sense that they would want to know about how to manage the thread in which their specified function is called. – John Bollinger
In the glibc wrapper function mq_notify ...
The first time mq_notify is called, it creates a netlink socket for communication and a single control thread.
In the sigevent struct [from caller], if sigev_notify is SIGEV_THREAD, the struct can have a [non-null] pthread_attr_t * value in sigev_notify_attributes. These attributes are copied and passed off to the actual mq_notify syscall via an internal struct
This is because although the syscall could create a thread [via an internal clone call], it could not do this from a signal handler and all the things that pthread_* functions do to (e.g.) set up pthread_t could not be done by the kernel.
So, after that, the [internal] struct is [ultimately] sent to the control thread [from the kernel] via a message on the netlink socket. The thread function for the control thread is [in the code below]: helper_function
The control thread will set the attributes from that message/struct into the pthread_create call it issues to start a [per-message] notification thread.
But ... The start routine specified in the pthread_create is not caller's notification/thread function. It is another wrapper function (e.g. notification_function) that calls pthread_detach as I mentioned above. It then calls user's start function.
I could be misreading the code, but, AFAICT, by the time the user's callback function has been invoked, it is in a separate thread that has already been detached.
So, again, the thread is not joinable. Whether the pthread_detach call updates the attributes of the per-message thread so that the caller can do: pthread_attribute_getdetachstate(pthread_self(),...) and get this is a bit of a moot point. We know it's detached.
So, user/caller's thread function can't pthread_join itself. And, caller's original thread can no longer manage the notification thread [because, again, it's detached], there really isn't anything to do [or anything that can be done].
Here's an excerpt from sysdeps/unix/sysv/linux/mq_notify.c:
/* The function used for the notification. */
static void *
notification_function(void *arg)
{
/* Copy the function and parameter so that the parent thread can go on with its life. */
volatile union notify_data *data = (volatile union notify_data *) arg;
void (*fct) (union sigval) = data->fct;
union sigval param = data->param;
/* Let the parent go. */
(void) __pthread_barrier_wait(¬ify_barrier);
/* Make the thread detached. */
(void) pthread_detach(pthread_self());
/* The parent thread has all signals blocked. This is probably a bit surprising for this thread. So we unblock all of them. */
(void) change_sigmask(SIG_UNBLOCK, NULL);
/* Now run the user code. */
fct(param);
/* And we are done. */
return NULL;
}
/* Helper thread. */
static void *
helper_thread(void *arg)
{
while (1) {
union notify_data data;
ssize_t n = __recv(netlink_socket, &data, sizeof(data),
MSG_NOSIGNAL | MSG_WAITALL);
if (n < NOTIFY_COOKIE_LEN)
continue;
if (data.raw[NOTIFY_COOKIE_LEN - 1] == NOTIFY_WOKENUP) {
/* Just create the thread as instructed. There is no way to report a problem with creating a thread. */
pthread_t th;
if (__builtin_expect(pthread_create(&th, data.attr, notification_function, &data)
== 0, 0))
/* Since we passed a pointer to DATA to the new thread we have to wait until it is done with it. */
(void) __pthread_barrier_wait(¬ify_barrier);
}
else if (data.raw[NOTIFY_COOKIE_LEN - 1] == NOTIFY_REMOVED)
/* The only state we keep is the copy of the thread attributes. */
free(data.attr);
}
return NULL;
}
static void
reset_once(void)
{
once = PTHREAD_ONCE_INIT;
}
static void
init_mq_netlink(void)
{
/* This code might be called a second time after fork(). The file descriptor is inherited from the parent. */
if (netlink_socket == -1) {
/* Just a normal netlink socket, not bound. */
netlink_socket = __socket(AF_NETLINK, SOCK_RAW | SOCK_CLOEXEC, 0);
/* No need to do more if we have no socket. */
if (netlink_socket == -1)
return;
}
int err = 1;
/* Initialize the barrier. */
if (__builtin_expect(__pthread_barrier_init(¬ify_barrier, NULL, 2) == 0, 0)) {
/* Create the helper thread. */
pthread_attr_t attr;
(void) pthread_attr_init(&attr);
(void) pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
/* We do not need much stack space, the bare minimum will be enough. */
(void) pthread_attr_setstacksize(&attr, __pthread_get_minstack(&attr));
/* Temporarily block all signals so that the newly created thread inherits the mask. */
sigset_t oss;
int have_no_oss = change_sigmask(SIG_BLOCK, &oss);
pthread_t th;
err = pthread_create(&th, &attr, helper_thread, NULL);
/* Reset the signal mask. */
if (!have_no_oss)
pthread_sigmask(SIG_SETMASK, &oss, NULL);
(void) pthread_attr_destroy(&attr);
if (err == 0) {
static int added_atfork;
if (added_atfork == 0 && pthread_atfork(NULL, NULL, reset_once) != 0) {
/* The child thread will call recv() which is a cancellation point. */
(void) pthread_cancel(th);
err = 1;
}
else
added_atfork = 1;
}
}
if (err != 0) {
__close_nocancel_nostatus(netlink_socket);
netlink_socket = -1;
}
}
/* Register notification upon message arrival to an empty message queue
MQDES. */
int
mq_notify(mqd_t mqdes, const struct sigevent *notification)
{
/* Make sure the type is correctly defined. */
assert(sizeof(union notify_data) == NOTIFY_COOKIE_LEN);
/* Special treatment needed for SIGEV_THREAD. */
if (notification == NULL || notification->sigev_notify != SIGEV_THREAD)
return INLINE_SYSCALL(mq_notify, 2, mqdes, notification);
/* The kernel cannot directly start threads. This will have to be done at userlevel. Since we cannot start threads from signal handlers we have to create a dedicated thread which waits for notifications for arriving messages and creates threads in response. */
/* Initialize only once. */
pthread_once(&once, init_mq_netlink);
/* If we cannot create the netlink socket we cannot provide SIGEV_THREAD support. */
if (__glibc_unlikely(netlink_socket == -1)) {
__set_errno(ENOSYS);
return -1;
}
/* Create the cookie. It will hold almost all the state. */
union notify_data data;
memset(&data, '\0', sizeof(data));
data.fct = notification->sigev_notify_function;
data.param = notification->sigev_value;
if (notification->sigev_notify_attributes != NULL) {
/* The thread attribute has to be allocated separately. */
data.attr = (pthread_attr_t *) malloc(sizeof(pthread_attr_t));
if (data.attr == NULL)
return -1;
memcpy(data.attr, notification->sigev_notify_attributes, sizeof(pthread_attr_t));
}
/* Construct the new request. */
struct sigevent se;
se.sigev_notify = SIGEV_THREAD;
se.sigev_signo = netlink_socket;
se.sigev_value.sival_ptr = &data;
/* Tell the kernel. */
int retval = INLINE_SYSCALL(mq_notify, 2, mqdes, &se);
/* If it failed, free the allocated memory. */
if (__glibc_unlikely(retval != 0))
free(data.attr);
return retval;
}
I am trying to create a Kthread within the body of the __do_page_fault exception handler. This thread will do some work (print the pc of the user process that caused the exception, and trace it using ptrace).
Currently I am just printing the pc in a loop that does not return, but the system hangs after a while, as if the Kthread should return in order for the user process to continue execution.
This is the code of the Kthread:
int th1( void * data ){
struct task_struct * tsk= (struct task_struct*) data;
int pid = tsk->pid;
int id= current->pid;
printk("thread %d is tracking %d ",id, tsk->pid);
while(true){
mdelay(100000);
printk("%d, %ud\n",id,task_pt_regs(tsk)->ARM_pc);
}
}
And this where it is created:
if (likely(!(fault & (VM_FAULT_ERROR | VM_FAULT_BADMAP | VM_FAULT_BADACCESS)))){
printk("the page fault was made by the procces id %d\n",tsk->pid);
if(tsk->is_tracked == 0) {
tsk->is_tracked=1 ;
struct task_struct * child_debugger;
void * data = (void *) tsk;
child_debugger = kthread_create(th1,data, "os2");
if(child_debugger) {
wake_up_process(child_debugger);
}else{
printk("error\n");
}
printk("thread went out \n");
}
return 0 ;
}
Threads shouldn't be created in interrupt context (like in exception handlers).
For defer work from an exception handler you may use pre-created thread, or a workqueue.
I am doing socket programming in C. In essence, I need a server that runs the indefinite listening loop in a separate thread and whenever it accepts a new connection, it must create a new client thread to handle the client's requests.
Below I have the main function that declares a port number and calls the function createListenThread. This function creates a new thread and invokes the function listenLoop.
int main(int argc, char *argv[])
{
int client_port = 6000;
createListenThread(client_port);
}
void createListenThread(int listen_port)
{
pthread_t listen_tid;
printf("In createListenThread\n");
// listenLoop(&listen_port);
if(pthread_create(&listen_tid, NULL, listenLoop, &listen_port) != 0)
socketError("Could not create thread\n");
}
void *listenLoop(void *arg)
{
pthread_detach(pthread_self());
int listen_socket, listen_port, *client_socket, struct_size;
struct sockaddr_in client_addr;
pthread_t client_tid;
listen_port = *((int *)arg);
listen_socket = createSocket(listen_port);
struct_size = sizeof(struct sockaddr_in);
while(1)
{
printf("In ListenLoop\n");
client_socket = malloc(sizeof(int));
*client_socket = -1;
*client_socket = accept(listen_socket, (struct sockaddr *)&client_addr, &struct_size);
printf("Received connection request from (%s , %d)\n",
inet_ntoa(client_addr.sin_addr),ntohs(client_addr.sin_port));
pthread_create(&client_tid, NULL, starterFunction, client_socket);
}
close(listen_socket);
}
My problem is that whenever I run the server, only "In ListenThread" and "In ListenLoop" is never printed. I have even tried fprintf(stdout, "In ListenLoop") and fflush(stdout) but the statement is still not printed. When I comment out:
if(pthread_create(&listen_tid, NULL, listenLoop, &listen_port) != 0)
socketError("Could not create thread\n");
and simply invoke ListenLoop as follows:
listenLoop(&listen_port);
Both the print statements appear. Is there an obvious mistake in the way I'm creating the thread and invoking the ListenLoop function? Is the function ListenLoop ever executed?
Edit: I ran the program in gdb which printed the following:
In createListenThread
[New Thread 0xb7e30b70 (LWP 10024)]
[Thread 0xb7e30b70 (LWP 10024) exited]
[Inferior 1 (process 10021) exited normally]
Why is the thread exiting??
The problem will be that your main function returns immediately after calling createListenThread. You should wait for your thread function to finish using pthread_join within your createListenThread otherwise the program may terminate before the thread function can finish. Calling pthread_join on your thread will wait until the thread function returns and will therefore definitely get chance to run before main() returns.
I hope that you declare all your functions above main ... But you should really call your listenLoop in main thread. Else, you create a thread to run the loop and main thread passively waits it. And you will not be bothered by argument passing, thread waiting and so on.
listenLoop could then be a true void and no longer a void *, as you do not return anything.
If you were forking a process, it would make sense, since it is the correct way to create a daemon process immediateley adopted by init process, but for threads it is simply useless.
I'm trying to create a concurrent TCP server which accepts multiple(N connections) where N is known.(e.g.N=8) So I'm trying to create a prethreaded TCP server.
.....
for(i=0;i<NUMBER_OF_CONNECTIONS;i++)
{
CreateThreads(i);
}
return 0;
}
//Create threads to handle the connection
void CreateThreads( int i )
{
pthread_create(&thread_tid[i], NULL, thread_function, (void *) i);
return;
}
void* thread_function(void *arg)
{
puts("In thread_function");
int client_soc,clilen;
struct sockaddr_in *clientaddr;
if( (clientaddr = malloc(addrlen)) == NULL)
printf("malloc Error\n");
printf("thread %d starting\n", (int) arg);
while(1)
{
clilen = sizeof(struct sockaddr_in);
pthread_mutex_lock(&mlock);
puts("Calling accept \n");
if ( (client_soc = accept(socket_desc, (struct sockaddr *)&clientaddr,(socklen_t*)&clilen)) < 0)
{
printf("accept error\n");
}
pthread_mutex_unlock(&mlock);
printf("Process Request...calling connection handler \n");
connection_handler(client_soc); /* process request */
close(client_soc);
}
}
//This will handle connection for each client
void *connection_handler(void *socket_desc)
{
//Receive and process.....
return 0;
}
In the above code threads are created and the thread_function() gets called but it doesn't work as expected. The program comes to an end after calling the thread_function() till then a few threads get created.
I actually want the "N" number of threads to get created and wait for clients to connect(using accept()). Once connected i want to receive/collect data (or) sending commands etc. That's the reason i have connection_handler() but i'm stuck before that.
Can any one pls try to correct the thread_function() function ? I'm kind of getting stuck here.Thanks.
UPDATE
The program is based on the
http://www.cse.fau.edu/~jie/research/publications/Publication_files/roussevwu.pdf
Look at section 3.2 for using lock and accept.
The program comes to an end after calling the thread_function()
The problem is that after you create the threads the main thread falls through and ends the program. You should call pthread_join.
There's a problem with your approach. In your code you lock mlock and then you accept(2). You are calling a blocking function in a critical section. This severely limits parallelism, thus defeating much of the purpose of the threads.
You could try an approach where the main thread accepts and dispatches the new sockets to threads.
I am new to both sockets and threads. I have this code:
listen(socket_fd, 20);
/* Looooop */
while (1) {
newsocket_fd = accept(socket_fd,
(struct sockaddr *) &client_addr,
&client_len);
if (newsocket_fd < 0) {
error("ERROR on accept");
}
pthread_t thread;
pthread_create(&thread, NULL, run_thread, (void *) newsocket_fd);
pthread_join(thread, NULL);
}
How can I start a new thread for each new connection, rather than for each request? These threads should be started when a new connection comes in, and these threads should then wait for requests, handle those requests, and finally return when the connection is closed. There should be one thread for each connection. Here is the code for run_thread:
void
*run_thread(void *ptr) {
char buffer[256];
bzero(buffer, 256);
int n;
n = read((int) ptr, buffer, 255);
if (n < 0) error("ERROR Reading from socket");
printf("%s\n\n**********\n\n", buffer);
/* Parse buffer and return result */
char *result;
{
/* First, determine command, 4 characters */
/* (much code) */
}
n = write((int) ptr, result, strlen(result));
if (n < 0) error("ERROR Writing to socket");
}
Can anyone help me? Thanks.
There is also a different critical error.
You cast the int to (void*). This does not make sense. Also, you can't pass the address directly since the variable could be changed on the next accept() call before the thread can copy the variable to its local stack. One way to write it would be something like this:
while (1) {
newsocket_fd = accept(socket_fd,
(struct sockaddr *) &client_addr,
&client_len);
if (newsocket_fd < 0) {
error("ERROR on accept");
}
pthread_t thread;
int *newsock = malloc(sizeof(int));
*newsock = newsocket_fd;
pthread_create(&thread, NULL, run_thread, newsock);
pthread_detach(thread);
}
With this approach, the thread will make sure to free() the newsock. E.g a simple
void *handler(void *thread_data) {
int fd = *(int *) thread_data;
free(thread_data);
....
}
Also, I assume pthread_detach() is okay, if the main program doesn't care about syncing up with the thread later with pthread_join().
You almost got it right. The problem is, however, that you are joining the thread right after creation, and pthread_join is actually a blocking call which is waiting for the thread to finish. It means that you will not be able to accept any more connections while that one thread is running. To solve this problem, you might want to use detached threads. You don't have to join detached threads. For this purpose, you have to create thread attributes using pthread_attr_init function and pass those attributes to pthread_create.
Be aware that if you have too many client connections, your application may run out of resources. So, in real world, you have to manage a pool of threads. But the best case scenario for TCP/IP server applications is to use asynchronous I/O. I do not know about C, but there is a very good library in C++ for asynchronous I/O application called boost::asio.
Vlad has good advice.
Also note that your newsocket_fd variable is being reused for each new connection in your accept loop, and then a pointer to it is passed to every worker thread. This will cause problems when you start having multiple clients connected at the same time.
EDIT: Ignore this comment, I misread the mistake you were making. Others have given proper corrections for your handling of newsocket_fd.