There's something going on in my code that I didn't intend.
I've got a program with several threads. One thread reads a rfid chip and saves the rfid code string into a global struct variable. There are also some other threads that are opened when a client, which is supposed to get the string, connects to the program on a certain socket. They are on a timed waiting (pthread_cond_timedwait) for the reading thread to broadcast a pthread condition variable (thread->done) that signals that a rfid chip was read and there is a new string to be sent to the clients. Now to the problem: the pthread_cond_timedwait in the sending thread shall only be satisfied when a new string is ready to be sent, but it also triggers when a new client thread is opened. The client threads start with the function below. I hope I explained my problem clearly and anyone can help me with the problem.
code of the sending function:
void* send_data(void* arg)
{
int client_id;
int status_timeout;
struct timespec timeout;
struct Thread_parameter *thread = (struct Thread_parameter*)arg;
pthread_condattr_t attr;
pthread_condattr_init(&attr);
pthread_condattr_setclock(&attr, CLOCK_MONOTONIC);
pthread_cond_init(&thread->done, &attr);
// read the client id from struct
pthread_mutex_lock(&thread->mutex);
client_id = thread->t_client_id;
pthread_mutex_unlock(&thread->mutex);
while (1)
{
// wait for string
while (1)
{
pthread_mutex_lock(&thread->mutex);
status_timeout = ETIMEDOUT;
while (1)
{
clock_gettime(CLOCK_MONOTONIC, &timeout);
timeout.tv_sec += 1;
status_timeout = pthread_cond_timedwait(&thread->done, &thread->mutex, &timeout);
// Timeout
if (status_timeout == ETIMEDOUT)
{
if (is_connected(client_id))
continue;
else
{
log_output("Lost connection to client '%i'", client_id);
pthread_mutex_unlock(&thread->mutex);
pthread_exit(NULL);
}
}
else if (status_timeout == 0)
{
// condition satisfied
break;
}
}
pthread_mutex_unlock(&thread->mutex);
break;
}
// send the string to the client when the condition is satisfied
while (1)
{
pthread_mutex_lock(&thread->mutex);
if (send(client_id, thread->t_chip_id, CHIP_ID_LENGTH, MSG_NOSIGNAL) >= 0)
{
pthread_mutex_unlock(&thread->mutex);
log_output("Successfully sent data to client '%i'", client_id);
break;
}
else
{
pthread_mutex_unlock(&thread->mutex);
// sending not successful, check if client is still connected
if (is_connected(client_id))
continue;
else
{
close(client_id);
log_output("Lost connection to client '%i'", client_id);
pthread_exit(NULL);
}
}
}
}
}
Related
I'm having trouble terminating my server in my multithreaded program (one server, multiple clients).
When the variable global_var, which counts the number of currently connected clients, gets set to 0, the server should terminate, but it doesn't.
What I think is happening is since accept() is blocking , the code never reaches the break condition in main loop.
It's breaking correctly out of thread_func but then it blocks inside the while loop, just before the accept() call and after printing "Exiting thread_func".
volatile int finished = 0; // Gets set to 1 by catching SIGINT/SIGSTOP
int global_var = 0; // When it gets to 0, server should terminate
int server_fd;
void * thread_func(void* arg)
{
do_some_pre_stuff();
while(1)
{
if(!global_var)
{
close(server_fd);
finished = 1;
break;
}
if(recv(...) > 0)
{
do_more_stuff()
}
else
{
disconnect_client();
global_var--;
break;
}
}
free_more_ressources();
return NULL;
}
int main()
{
do_initial_stuff();
init_socket();
listen();
while (!finished)
{
if( (fd = accept(server_fd,...)) == -1)
exit(-1);
global_var++;
/* Some intermediate code */
if(!global_var)
break;
// Thread for the newly connected player
if(pthread_create(&thread_id[...], NULL, thread_func, (void*)some_arg)
exit(-1);
}
free_resources();
puts("Exiting thread_func");
}
I tried the advice listed here without success (except the pipe answer, not trying to mess with pipes).
I'm new to socket programming but what I tried so far looked correct but none of the solutions worked (including semaphores, pthread_cancel,etc)
PS: synchronization has been implemented, just omitted here for readability
i have a function like this :
void create_serv_and_init_client(client_t *cl, serv_t *serv)
{
static int i = 0;
pthread_t thread_serv;
if (i == 0) {
*serv = create_serv_socket();
if (pthread_create(&thread_serv, NULL, waiting_connection, \
(void *)serv) < 0) {
perror("could not create thread");
exit(1);
}
pthread_join(thread_serv, NULL);
cl[0] = create_client(0);
printf("OK\n");
i++;
}
}
waiting_connection function :
void *waiting_connection(void *server)
{
serv_t *serv = (serv_t *)server;
serv->newSocket = accept(serv->sockfd, (struct sockaddr*)&serv->newAddr, \
&serv->addr_size);
if (serv->newSocket < 0) {
exit(1);
}
if ((serv->childpid = fork()) == 0) {
close(serv->sockfd);
while (recv(serv->newSocket, serv->buffer, 1024, 0) != 0) {
printf("Client: %s\n", serv->buffer);
send(serv->newSocket, serv->buffer, strlen(serv->buffer), 0);
bzero(serv->buffer, sizeof(serv->buffer));
}
}
}
if i dont pthread_join, i will never receive the sended msg by the client, but however, it will block my program until receiving the message, but i wanna have a unblockant waiting of the client message, so it is possible to do a unblockant waiting, for the reception of the client message ?
because this is a game, so the server is launched when the first client is connected to the game, and if my program always wait others input of others plays, the actual players connected can't play, so this need to be unblockant, asynchronous if you want.
So you want one thread per client connection, and to be able to always accept a new client connection. That's sounds about right.
In that case the management of the messages received from a given client must be done in the corresponding thread, not in the main thread. The main thread only manages the client connections and launch the new threads, it does not have to join the other threads.
So the accept are done in the main thread, not in the separated threads, and it gives the socket with the new client in argument when it launch the new thread for that client, then pthread_detach the new thread.
if ((serv->childpid = fork()) == 0) {
close(serv->sockfd);
all of that does not exist, there is no fork, you use threads.
Of course an other way is to not use threads at all but to fork, anyway the roles are unchanged between the initial process and the child processes.
Your problem is that you didn't assign the right roles to everyone.
I'm new to signals, I'm trying to set SIGALRM on UDP echo service, as a socket programming practice.
So here I have a UDP socket, the client sends a string to server and waits for response (any response, here the string is echoed by server).
The goals is to set SIGALRM and let the client resend the string a few times if no responses were made by server or UDP packets get lost.
Here, I used a small sample and simplified long lines with ..., you can get more details on my github repo (line 51)
sigALRM-Client.c
unsigned int tries = 0;
void CatchAlarm()
{
tries += 1;
}
int main(int argc, char **argv)
{
// SKIPPED
// ...
struct sigaction handler;
handler.sa_handler = CatchAlarm;
handler.sa_flags = 0;
if(sigfillset(&handler.sa_mask) < 0)
return 1;
if(sigaction(SIGALRM, &handler, 0) < 0)
return 2;
ssize_t bytes;
bytes = sendto(servSock,...);
while((bytes = recvfrom(servSock,...)) < 0) {
// alarm went off
if(errno == EINTR) {
// try 5 times
if(tries < 5) {
bytes = sendto(servSock,...);
} else {
fprintf(stdout, "no response, waiting...\n");
}
} else {
fprintf(stdout, "failed to get data\n");
return 3;
}
}
// recvfrom() got something, cancel timeout
alarm(0);
fprintf(stdout, "received %d bytes of data\n", bytes);
close(servSock);
}
When I run the client, it won't receive SIGALRM signal and UDP packets get lost in first attempt?!
Client won't retry sending string then exit after 5 attempts, instead, it waits for server response forever!
What prevents client to get SIGALRM?
Did I miss something here?
Your code in the GitHub repo never calls alarm() with a non-zero number. You'll never get an alarm signal delivered automatically unless you actually request one. Relying on some other process to send your process an alarm signal isn't resilient.
I just wondered about how Instant Messengers and Online Games can accept and deliver messages so fast. (Network programming with sockets)
I read about that this is done with nonblocking sockets.
I tried blocking sockets with pthreads (each client gets its own thread) and nonblocking sockets with kqueue.Then I profiled both servers with a program which made 99 connections (each connection in one thread) and then writes some garbage to it (with a sleep of 1 second). When all threads are set up, I measured in the main thread how long it took to get a connection from the server (with wall clock time) (while "99 users" are writing to it).
threads (avg): 0.000350 // only small difference to kqueue
kqueue (avg): 0.000300 // and this is not even stable (client side)
The problem is, while testing with kqueue I got multiple times a SIGPIPE error (client-side). (With a little timeout usleep(50) this error was fixed). I think this is really bad because a server should be capable to handle thousands of connections. (Or is it my fault on the client side?) The crazy thing about this is the infamous pthread approach did just fine (with and without timeout).
So my question is: how can you build a stable socket server in C which can handle thousands of clients "asynchronously"? I only see the threads approach as a good thing, but this is considered bad practice.
Greetings
EDIT:
My test code:
double get_wall_time(){
struct timeval time;
if (gettimeofday(&time,NULL)){
// Handle error
return 0;
}
return (double)time.tv_sec + (double)time.tv_usec * .000001;
}
#define NTHREADS 100
volatile unsigned n_threads = 0;
volatile unsigned n_writes = 0;
pthread_mutex_t main_ready;
pthread_mutex_t stop_mtx;
volatile bool running = true;
void stop(void)
{
pthread_mutex_lock(&stop_mtx);
running = false;
pthread_mutex_unlock(&stop_mtx);
}
bool shouldRun(void)
{
bool copy;
pthread_mutex_lock(&stop_mtx);
copy = running;
pthread_mutex_unlock(&stop_mtx);
return copy;
}
#define TARGET_HOST "localhost"
#define TARGET_PORT "1336"
void *thread(void *args)
{
char tmp = 0x01;
if (__sync_add_and_fetch(&n_threads, 1) == NTHREADS) {
pthread_mutex_unlock(&main_ready);
fprintf(stderr, "All %u Threads are ready...\n", (unsigned)n_threads);
}
int fd = socket(res->ai_family, SOCK_STREAM, res->ai_protocol);
if (connect(fd, res->ai_addr, res->ai_addrlen) != 0) {
socket_close(fd);
fd = -1;
}
if (fd <= 0) {
fprintf(stderr, "socket_create failed\n");
}
if (write(fd, &tmp, 1) <= 0) {
fprintf(stderr, "pre-write failed\n");
}
do {
/* Write some garbage */
if (write(fd, &tmp, 1) <= 0) {
fprintf(stderr, "in-write failed\n");
break;
}
__sync_add_and_fetch(&n_writes, 1);
/* Wait some time */
usleep(500);
} while (shouldRun());
socket_close(fd);
return NULL;
}
int main(int argc, const char * argv[])
{
pthread_t threads[NTHREADS];
pthread_mutex_init(&main_ready, NULL);
pthread_mutex_lock(&main_ready);
pthread_mutex_init(&stop_mtx, NULL);
bzero((char *)&hint, sizeof(hint));
hint.ai_socktype = SOCK_STREAM;
hint.ai_family = AF_INET;
if (getaddrinfo(TARGET_HOST, TARGET_PORT, &hint, &res) != 0) {
return -1;
}
for (int i = 0; i < NTHREADS; ++i) {
pthread_create(&threads[i], NULL, thread, NULL);
}
/* wait for all threads to be set up */
pthread_mutex_lock(&main_ready);
fprintf(stderr, "Main thread is ready...\n");
{
double start, end;
int fd;
start = get_wall_time();
fd = socket(res->ai_family, SOCK_STREAM, res->ai_protocol);
if (connect(fd, res->ai_addr, res->ai_addrlen) != 0) {
socket_close(fd);
fd = -1;
}
end = get_wall_time();
if (fd > 0) {
fprintf(stderr, "Took %f ms\n", (end - start) * 1000);
socket_close(fd);
}
}
/* Stop all running threads */
stop();
/* Waiting for termination */
for (int i = 0; i < NTHREADS; ++i) {
pthread_join(threads[i], NULL);
}
fprintf(stderr, "Performed %u successfull writes\n", (unsigned)n_writes);
/* Lol.. */
freeaddrinfo(res);
return 0;
}
SIGPIPE comes when I try to connect to the kqueue server (after 10 connections are made, the server is "stuck"?). And when too many users are writing stuff, the server cannot open a new connection. (kqueue server code from http://eradman.com/posts/kqueue-tcp.html)
SIGPIPE means you're trying to write to a socket (or pipe) where the other end has already been closed (so noone will be able to read it). If you don't care about that, you can ignore SIGPIPE signals (call signal(SIGPIPE, SIG_IGN)) and the signals won't be a problem. Of course the write (or send) calls on the sockets will still be failing (with EPIPE), so you need to make you code robust enough to deal with that.
The reason that SIGPIPE normally kills the process is that its too easy to write programs that ignore errors on write/send calls and run amok using up 100% of CPU time otherwise. As long as you carefully always check for errors and deal with them, you can safely ignore SIGPIPEs
Or is it my fault?
It was your fault. TCP works. Most probably you didn't read all the data that was sent.
And when too many users are writing stuff, the server cannot open a new connection
Servers don't open connections. Clients open connections. Servers accept connections. If your server stops doing that, there something wrong with your accept loop. It should only do two things: accept a connection, and start a thread.
I'm writing a web server in C (which I suck with) using Pthreads (which I suck with even more) and I'm stuck at this point. The model for the server is boss-worker so the boss thread instantiates all worker threads at the beginning of the program. There is a global queue that stores the socket of the incoming connection(s). The boss thread is the one that adds all items (sockets) to the queue as the connections are accepted. All of the worker threads then wait for an item to be added to a global queue in order for them to take up the processing.
The server works fine as long as I connect to it less times than the number of worker threads that the server has. Because of that, I think that either something is wrong with my mutexes (maybe the signals are getting lost?) or the threads are being disabled after they run once (which would explain why if there are 8 threads, it can only parse the first 8 http requests).
Here is my global queue variable.
int queue[QUEUE_SIZE];
This is the main thread. It creates a queue struct (defined elsewhere) with methods enqueue, dequeue, empty, etc. When the server accepts a connection, it enqueues the socket that the incoming connection is on. The worker threads which were dispatched at the beginning are constantly checking this queue to see if any jobs have been added, and if there are jobs, then they dequeue the socket, connect to that port, and read/parse/write the incoming http request.
int main(int argc, char* argv[])
{
int hSocket, hServerSocket; /* handle to socket */
struct hostent* pHostInfo; /* holds info about a machine */
struct sockaddr_in Address; /* Internet socket address stuct */
int nAddressSize = sizeof(struct sockaddr_in);
int nHostPort;
int numThreads;
int i;
init(&head,&tail);
//**********************************************
//ALL OF THIS JUST SETS UP SERVER (ADDR STRUCT,PORT,HOST INFO, ETC)
if(argc < 3) {
printf("\nserver-usage port-num num-thread\n");
return 0;
}
else {
nHostPort=atoi(argv[1]);
numThreads=atoi(argv[2]);
}
printf("\nStarting server");
printf("\nMaking socket");
/* make a socket */
hServerSocket=socket(AF_INET,SOCK_STREAM,0);
if(hServerSocket == SOCKET_ERROR)
{
printf("\nCould not make a socket\n");
return 0;
}
/* fill address struct */
Address.sin_addr.s_addr = INADDR_ANY;
Address.sin_port = htons(nHostPort);
Address.sin_family = AF_INET;
printf("\nBinding to port %d\n",nHostPort);
/* bind to a port */
if(bind(hServerSocket,(struct sockaddr*)&Address,sizeof(Address)) == SOCKET_ERROR) {
printf("\nCould not connect to host\n");
return 0;
}
/* get port number */
getsockname(hServerSocket, (struct sockaddr *) &Address,(socklen_t *)&nAddressSize);
printf("Opened socket as fd (%d) on port (%d) for stream i/o\n",hServerSocket, ntohs(Address.sin_port));
printf("Server\n\
sin_family = %d\n\
sin_addr.s_addr = %d\n\
sin_port = %d\n"
, Address.sin_family
, Address.sin_addr.s_addr
, ntohs(Address.sin_port)
);
//Up to this point is boring server set up stuff. I need help below this.
//**********************************************
//instantiate all threads
pthread_t tid[numThreads];
for(i = 0; i < numThreads; i++) {
pthread_create(&tid[i],NULL,worker,NULL);
}
printf("\nMaking a listen queue of %d elements",QUEUE_SIZE);
/* establish listen queue */
if(listen(hServerSocket,QUEUE_SIZE) == SOCKET_ERROR) {
printf("\nCould not listen\n");
return 0;
}
while(1) {
pthread_mutex_lock(&mtx);
printf("\nWaiting for a connection");
while(!empty(head,tail)) {
pthread_cond_wait (&cond2, &mtx);
}
/* get the connected socket */
hSocket = accept(hServerSocket,(struct sockaddr*)&Address,(socklen_t *)&nAddressSize);
printf("\nGot a connection");
enqueue(queue,&tail,hSocket);
pthread_mutex_unlock(&mtx);
pthread_cond_signal(&cond); // wake worker thread
}
}
Here is the worker thread. This should be always running checking for new requests (by seeing if the queue is not empty). At the end of this method, it should be deferring back to the boss thread to wait for the next time it is needed.
void *worker(void *threadarg) {
pthread_mutex_lock(&mtx);
while(empty(head,tail)) {
pthread_cond_wait(&cond, &mtx);
}
int hSocket = dequeue(queue,&head);
unsigned nSendAmount, nRecvAmount;
char line[BUFFER_SIZE];
nRecvAmount = read(hSocket,line,sizeof line);
printf("\nReceived %s from client\n",line);
//***********************************************
//DO ALL HTTP PARSING (Removed for the sake of space; I can add it back if needed)
//***********************************************
nSendAmount = write(hSocket,allText,sizeof(allText));
if(nSendAmount != -1) {
totalBytesSent = totalBytesSent + nSendAmount;
}
printf("\nSending result: \"%s\" back to client\n",allText);
printf("\nClosing the socket");
/* close socket */
if(close(hSocket) == SOCKET_ERROR) {
printf("\nCould not close socket\n");
return 0;
}
pthread_mutex_unlock(&mtx);
pthread_cond_signal(&cond2);
}
Any help would be greatly appreciated. I can post more of the code if anyone needs it, just let me know. I'm not the best with OS stuff, especially in C, but I know the basics of mutexes, cond. variables, semaphores, etc. Like I said, I'll take all the help I can get. (Also, I'm not sure if I posted the code exactly right since this is my first question. Let me know if I should change the formatting at all to make it more readable.)
Thanks!
Time for a workers' revolution.
The work threads seem to be missing a while(true) loop. After the HTTP exchange and closing the socket, they should be looping back to wait on the queue for more sockets/requests.