Concurrent TCP server using threads - c

I'm trying to create a concurrent TCP server which accepts multiple(N connections) where N is known.(e.g.N=8) So I'm trying to create a prethreaded TCP server.
.....
for(i=0;i<NUMBER_OF_CONNECTIONS;i++)
{
CreateThreads(i);
}
return 0;
}
//Create threads to handle the connection
void CreateThreads( int i )
{
pthread_create(&thread_tid[i], NULL, thread_function, (void *) i);
return;
}
void* thread_function(void *arg)
{
puts("In thread_function");
int client_soc,clilen;
struct sockaddr_in *clientaddr;
if( (clientaddr = malloc(addrlen)) == NULL)
printf("malloc Error\n");
printf("thread %d starting\n", (int) arg);
while(1)
{
clilen = sizeof(struct sockaddr_in);
pthread_mutex_lock(&mlock);
puts("Calling accept \n");
if ( (client_soc = accept(socket_desc, (struct sockaddr *)&clientaddr,(socklen_t*)&clilen)) < 0)
{
printf("accept error\n");
}
pthread_mutex_unlock(&mlock);
printf("Process Request...calling connection handler \n");
connection_handler(client_soc); /* process request */
close(client_soc);
}
}
//This will handle connection for each client
void *connection_handler(void *socket_desc)
{
//Receive and process.....
return 0;
}
In the above code threads are created and the thread_function() gets called but it doesn't work as expected. The program comes to an end after calling the thread_function() till then a few threads get created.
I actually want the "N" number of threads to get created and wait for clients to connect(using accept()). Once connected i want to receive/collect data (or) sending commands etc. That's the reason i have connection_handler() but i'm stuck before that.
Can any one pls try to correct the thread_function() function ? I'm kind of getting stuck here.Thanks.
UPDATE
The program is based on the
http://www.cse.fau.edu/~jie/research/publications/Publication_files/roussevwu.pdf
Look at section 3.2 for using lock and accept.

The program comes to an end after calling the thread_function()
The problem is that after you create the threads the main thread falls through and ends the program. You should call pthread_join.
There's a problem with your approach. In your code you lock mlock and then you accept(2). You are calling a blocking function in a critical section. This severely limits parallelism, thus defeating much of the purpose of the threads.
You could try an approach where the main thread accepts and dispatches the new sockets to threads.

Related

Error 11 when using pthread_create in TCP server

I have a C app that listens to a TCP port and creates a new thread each time it accepts a new connection. It works OK initially, but after a while I start getting error code 11 from pthread_create.
There are no thread-related function calls inside the body of the thread function, and the log shows that there is a matching 'out' for every 'in'.
When it fails, I call the thread function that directly, and it works fine indefinitely on the main thread, so it seems unlikely that I am using up resources within the function. Any suggestions on what causes the error 11, and how to fix it?
This is the thread function:
void * tcp_process_message (void * arg) {
MESSAGE_BUFFER * bp = (MESSAGE_BUFFER *) arg;
USER_LOG (UL_INFO, "tpm in %d", bp - buffers);
...
USER_LOG (UL_INFO, "tpm out %d", bp - buffers);
}
This is the section that creates threads: there is no other code that interacts with the new thread once it is created.
while(!cancel){
connfd = accept(listenfd, (struct sockaddr *) &from_addr, &fromsize);
if (!cancel) {
MESSAGE_BUFFER * bp = allocate_message ();
if (bp == NULL) {
USER_LOG (UL_ERROR, "%s", "allocate_message failed");
close(connfd);
}
else {
bp->connfd = connfd;
strcpy (bp->ip_addr, inet_ntoa(from_addr.sin_addr));
int err = pthread_create (&tid, NULL, &tcp_process_message, (void *) bp);
if (err) {
USER_LOG (UL_ERROR, "thread create failed (%d)", err);
tcp_process_message ((void *) bp);
}
}
}
}
When creating thread, the thread resource still exist even after the thread function exits.
Some piece of code must either wait for the thread using pthread_join(the_thread); or, it must detach the thread and let it die once its function exits: pthread_detach(the_thread);.
If not done, the thread remain in the system and the system will soon run out of resources and won't be able to create new threads anymore.

Cancel a thread within another thread

I have a multithreaded program. All I am doing in the main method is setting up multiple connections between clients and the server.
Each client runs this thread.
Now this thread has another thread within it.
What I want to do is close the connection at the end of the thread.
This is the code I am using. I have no idea why it does not work. Can anyone help me please?
void *thread_function(void * sock_desc){
//get the socket descriptor
int sock = *(int*)sock_desc;
int n;
int * sock2 = malloc(4);
*sock2 = sock;
pthread_t update_thread;
pthread_create(&update_thread, NULL, updater, (void*)sock2);
//do something
pthread_cancel(update_thread);
close(sock); //close the connection
free(sock_desc);
free(sock2);
return (void *)0;
}//close thread_function()

write()/read() in a Client/Server, pthread application in C returns -1 always when on non-main thread?

So, I have this client/server application, where the server has a producer/consumer architecture. I have 2 functions that handle writting and reading to the socket. The main thread of the server (the Producer) handles connections and passes socket descriptors via a Stack to the second thread, the Consumer, for processing.
The problem is, whenever I try to write() or read() the socket from a different-than-main thread, it always returns -1 and causes a Connection reset by peer error on client and Transport endpoint is not connected error on the server. Surpirsingly, it works perfectly when socket is read/written from the main thread.
Why does this happen? Is this official behaviour? How do I go about replying to the client with the Consumer thread? I don't believe it's because of the code I wrote, since the Consumer thread only calls the read/write-to-socket functions.
If you have any suspicion on which part could be a culprit, ask me to post some of the code.
EDIT:
typedef struct s_stack {
int * c_stack;
int base;
int top;
unsigned char is_full;
unsigned char is_empty;
int max_size;
} s_stack_t;
s_stack_t stack;
void * producer_routine(void * arguments) {
/* socket(), bind(), listen(), etc.,
socket fd on "socket_fd",
new connection fd on "new_fd" */
for(;;) {
new_fd = accept(socket_fd, (struct sockaddr *)&client_addr, &clen);
pthread_mutex_lock(&mutex);
while (stack.is_full) {
pthread_cond_wait(&stack_not_full, &mutex);
}
if (stack.is_full){
push(&stack, new_fd);
pthread_cond_signal(&stack_not_empty);
}
pthread_mutex_unlock(&mutex);
}
close(new_fd);
}
void * consumer_routine(void *args) {
for(;;) {
int socket_fd;
/* same mutex lock as above, just reversed, pop to socket_fd */
write_a_message_to_client(socket_fd);
}
}
int main() {
stack_init(&stack, 1024); // (s_stack_t * stack, int max_size)
pthread_t tidp, tidc;
int prc = pthread_create(&tidp, NULL, producer_routine, NULL);
int crc = pthread_create(&tidc, NULL, consumer_routine, NULL);
stack_destroy(&stack);
return 0;
}
The client just sends a message, and waits to receive one. If write_a_message_to_client() is called within any of those threads, even with the socket_fd passed as a parameter, I get the same errors. If it's called directly in main, it has no problem.
EDIT #2:
I tested this, and found my stack implementation to not work on Cygwin. Cygwin adds gibberish after the 3rd element for some reason, so the socket fds were invalid. Also, I was testing this in a Debian 6 VM and the server was crashing after connection from client. But I tested it in Arch, Kali and my Uni servers (Debian 7) and works as it should have been. A whole lot of trouble for a whole lot of nothing. Thanks Cygwin!
You should not call stack_destroy() until after both threads have completed. I think your entire program is running using a destroyed stack.
Use pthread_join() to wait for the threads to complete before destroying the stack.

Can an application handle N accept connections and each accept uses an independent thread

I have C linux TCP Client/Server application.
I came up with a strange scenario, but I don't know if there are any consequences with this application.
I have a server side that can accept N connections, for example this server will accept 100 connections.
In this scenario I create the listen socket in the main thread, then I create 100 threads and each thread has an independent accept() and a select() iomux, also each thread can only accept one connection.
My concerns here, if two simultaneous accept() want to accept the same socket(connection) because of the select is a ready to read on the same socket, I don't know if the simultaneous accepts are thread safe in kernel and only one accept can handle that incoming connection and the other will wait for another connection?
I tried that on my RedHat machine that works fine, but I don't know If I am a lucky enough to avoid a deadlock!
Thanks
rc = bind(sd, (struct sockaddr_in *)& groupSock, sizeof(struct sockaddr_in));
CHECK_VALUE("Bind address error", rc, 0, goto cleanup);
rc = listen(sd, 10);
CHECK_VALUE("listen", rc, 0, goto cleanup);
for(; count< num_socks; count++){
par_data[count].sd = sd;
par_data[count].thread_num = count;
par_data[count].err_chk = -1;
rc = pthread_create(&thread_id[count], NULL, accept_sock_thread, (void *)& par_data[count]);
CHECK_VALUE("pthread_create", rc, 0, goto cleanup);
}
void * accept_sock_thread(void* atr){
int rc;
int sock = INVALID_SOCKET;
int datalen = config.traffic;
char *databuf = NULL;
struct thread_data *data = NULL;
struct sockaddr_in tcp_remote;
struct timeval t;
socklen_t size;
fd_set socks;
databuf = malloc(sizeof(char) * datalen);
memset(databuf, 0, datalen);
data = (struct thread_data*) atr;
DEBUG(my_debug_flags, ENTER_FUNCT, ("Enter Function accept_sock_thread thread_num %d \n", data->thread_num));
FD_ZERO(&socks);
FD_SET(data->sd, &socks);
t.tv_sec = 25;
t.tv_usec = 0;
rc = select(data->sd + 1 , &socks, NULL, NULL,&t);
if(rc < 0){
VL_MISC_ERR(("Error in select with Errno: %d", errno));
goto cleanup;
}
else if(rc == 0){
VL_MISC_ERR(("Accept Select returned a TIEMOUT."));
goto cleanup;
}
size = sizeof(struct sockaddr_in);
sock = accept(data->sd, (struct sockaddr *)& tcp_remote, &size);
CHECK_NOT_EQUAL("tcp accept error", sock, INVALID_SOCKET, goto cleanup);
cleanup:
// sleep(2); /* avoid EOF */
if(sock != INVALID_SOCKET){
rc = close(sock);
if(rc != 0){
data->err_chk = -1;
}
}
return NULL;
}
accept() is thread-safe and reentrant according to POSIX.
This means that two call of accept on the same descriptor should not give an undefined behaviour. One of the accept will open the socket and the other will return an error.
You can see a little more there :
Is accept() thread-safe?
Are BSD/Posix sockets reentrant?
Only one thread is going to accept the connection. The kernel insures this. It has been this way in the unix/posix world for a very long time now.
Only one thread would accept(), so far so good ... - but before this all threads will be triggered to have select() return, which might not be what you want.
So if you'd had N threads sleeping in select() and one connection comes in all threads would wake up, but only one would be needed, as only one would succeed in accept()ing.
To use accept() on same socket object from multiple threads/processes is not only standard, but also widly used strategy. From my knowledge apache does this. nginx does this too but in slightly different fashion.
Do note: select() can wake up multiple threads, but only one of them will accept connection, while others will either a) hang in accept() or b) return -1 and set errno to EAGAIN in case of non-blocking IO. Since you are using select() before accept() I suppose socket descriptor is in non blocking mode. So this can lead to some threads never serve a connection.
Also I would advise you to never close same socket in multiple threads. This can lead to very nasty and hard to debug consequences. Either use wrapper and shared_ptr or assign to one of threads a role of 'owner of socket'.

Sockets and threads using C

I am new to both sockets and threads. I have this code:
listen(socket_fd, 20);
/* Looooop */
while (1) {
newsocket_fd = accept(socket_fd,
(struct sockaddr *) &client_addr,
&client_len);
if (newsocket_fd < 0) {
error("ERROR on accept");
}
pthread_t thread;
pthread_create(&thread, NULL, run_thread, (void *) newsocket_fd);
pthread_join(thread, NULL);
}
How can I start a new thread for each new connection, rather than for each request? These threads should be started when a new connection comes in, and these threads should then wait for requests, handle those requests, and finally return when the connection is closed. There should be one thread for each connection. Here is the code for run_thread:
void
*run_thread(void *ptr) {
char buffer[256];
bzero(buffer, 256);
int n;
n = read((int) ptr, buffer, 255);
if (n < 0) error("ERROR Reading from socket");
printf("%s\n\n**********\n\n", buffer);
/* Parse buffer and return result */
char *result;
{
/* First, determine command, 4 characters */
/* (much code) */
}
n = write((int) ptr, result, strlen(result));
if (n < 0) error("ERROR Writing to socket");
}
Can anyone help me? Thanks.
There is also a different critical error.
You cast the int to (void*). This does not make sense. Also, you can't pass the address directly since the variable could be changed on the next accept() call before the thread can copy the variable to its local stack. One way to write it would be something like this:
while (1) {
newsocket_fd = accept(socket_fd,
(struct sockaddr *) &client_addr,
&client_len);
if (newsocket_fd < 0) {
error("ERROR on accept");
}
pthread_t thread;
int *newsock = malloc(sizeof(int));
*newsock = newsocket_fd;
pthread_create(&thread, NULL, run_thread, newsock);
pthread_detach(thread);
}
With this approach, the thread will make sure to free() the newsock. E.g a simple
void *handler(void *thread_data) {
int fd = *(int *) thread_data;
free(thread_data);
....
}
Also, I assume pthread_detach() is okay, if the main program doesn't care about syncing up with the thread later with pthread_join().
You almost got it right. The problem is, however, that you are joining the thread right after creation, and pthread_join is actually a blocking call which is waiting for the thread to finish. It means that you will not be able to accept any more connections while that one thread is running. To solve this problem, you might want to use detached threads. You don't have to join detached threads. For this purpose, you have to create thread attributes using pthread_attr_init function and pass those attributes to pthread_create.
Be aware that if you have too many client connections, your application may run out of resources. So, in real world, you have to manage a pool of threads. But the best case scenario for TCP/IP server applications is to use asynchronous I/O. I do not know about C, but there is a very good library in C++ for asynchronous I/O application called boost::asio.
Vlad has good advice.
Also note that your newsocket_fd variable is being reused for each new connection in your accept loop, and then a pointer to it is passed to every worker thread. This will cause problems when you start having multiple clients connected at the same time.
EDIT: Ignore this comment, I misread the mistake you were making. Others have given proper corrections for your handling of newsocket_fd.

Resources