I'm having trouble terminating my server in my multithreaded program (one server, multiple clients).
When the variable global_var, which counts the number of currently connected clients, gets set to 0, the server should terminate, but it doesn't.
What I think is happening is since accept() is blocking , the code never reaches the break condition in main loop.
It's breaking correctly out of thread_func but then it blocks inside the while loop, just before the accept() call and after printing "Exiting thread_func".
volatile int finished = 0; // Gets set to 1 by catching SIGINT/SIGSTOP
int global_var = 0; // When it gets to 0, server should terminate
int server_fd;
void * thread_func(void* arg)
{
do_some_pre_stuff();
while(1)
{
if(!global_var)
{
close(server_fd);
finished = 1;
break;
}
if(recv(...) > 0)
{
do_more_stuff()
}
else
{
disconnect_client();
global_var--;
break;
}
}
free_more_ressources();
return NULL;
}
int main()
{
do_initial_stuff();
init_socket();
listen();
while (!finished)
{
if( (fd = accept(server_fd,...)) == -1)
exit(-1);
global_var++;
/* Some intermediate code */
if(!global_var)
break;
// Thread for the newly connected player
if(pthread_create(&thread_id[...], NULL, thread_func, (void*)some_arg)
exit(-1);
}
free_resources();
puts("Exiting thread_func");
}
I tried the advice listed here without success (except the pipe answer, not trying to mess with pipes).
I'm new to socket programming but what I tried so far looked correct but none of the solutions worked (including semaphores, pthread_cancel,etc)
PS: synchronization has been implemented, just omitted here for readability
Related
i have a function like this :
void create_serv_and_init_client(client_t *cl, serv_t *serv)
{
static int i = 0;
pthread_t thread_serv;
if (i == 0) {
*serv = create_serv_socket();
if (pthread_create(&thread_serv, NULL, waiting_connection, \
(void *)serv) < 0) {
perror("could not create thread");
exit(1);
}
pthread_join(thread_serv, NULL);
cl[0] = create_client(0);
printf("OK\n");
i++;
}
}
waiting_connection function :
void *waiting_connection(void *server)
{
serv_t *serv = (serv_t *)server;
serv->newSocket = accept(serv->sockfd, (struct sockaddr*)&serv->newAddr, \
&serv->addr_size);
if (serv->newSocket < 0) {
exit(1);
}
if ((serv->childpid = fork()) == 0) {
close(serv->sockfd);
while (recv(serv->newSocket, serv->buffer, 1024, 0) != 0) {
printf("Client: %s\n", serv->buffer);
send(serv->newSocket, serv->buffer, strlen(serv->buffer), 0);
bzero(serv->buffer, sizeof(serv->buffer));
}
}
}
if i dont pthread_join, i will never receive the sended msg by the client, but however, it will block my program until receiving the message, but i wanna have a unblockant waiting of the client message, so it is possible to do a unblockant waiting, for the reception of the client message ?
because this is a game, so the server is launched when the first client is connected to the game, and if my program always wait others input of others plays, the actual players connected can't play, so this need to be unblockant, asynchronous if you want.
So you want one thread per client connection, and to be able to always accept a new client connection. That's sounds about right.
In that case the management of the messages received from a given client must be done in the corresponding thread, not in the main thread. The main thread only manages the client connections and launch the new threads, it does not have to join the other threads.
So the accept are done in the main thread, not in the separated threads, and it gives the socket with the new client in argument when it launch the new thread for that client, then pthread_detach the new thread.
if ((serv->childpid = fork()) == 0) {
close(serv->sockfd);
all of that does not exist, there is no fork, you use threads.
Of course an other way is to not use threads at all but to fork, anyway the roles are unchanged between the initial process and the child processes.
Your problem is that you didn't assign the right roles to everyone.
I am writing a concurrent C program where I want to wait for all threads to finish in the main().
Based on this solution, I wrote the following code in main():
// Create threads
pthread_t cid[num_mappers];
int t_iter;
for (t_iter = 0; t_iter < num_mappers; t_iter++){
pthread_create(&(cid[t_iter]), NULL, &map_consumer, NULL);
}
// Wait for all threads to finish
for (t_iter = 0; t_iter < num_mappers; t_iter++){
printf("Joining %d\n", t_iter);
int result = pthread_join(cid[t_iter], NULL);
}
printf("Done mapping.\n");
The function passed into threads is defined as:
// Consumer function for mapping phase
void *map_consumer(void *arg){
while (1){
pthread_mutex_lock(&g_lock);
if (g_cur >= g_numfull){
// No works to do, just quit
return NULL;
}
// Get the file name
char *filename = g_job_queue[g_cur];
g_cur++;
pthread_mutex_unlock(&g_lock);
// Do the mapping
printf("%s\n", filename);
g_map(filename);
}
}
The threads are all successfully created and executed, but the join loop will never finish if num_mappers >= 2.
You return without unlocking the mutex:
pthread_mutex_lock(&g_lock);
if (g_cur >= g_numfull){
// No works to do, just quit
return NULL; <-- mutex is still locked here
}
// Get the file name
char *filename = g_job_queue[g_cur];
g_cur++;
pthread_mutex_unlock(&g_lock);
So only one thread ever returns and ends - the first one, but since it never unlocks the mutex, the other threads remain blocked.
You need something more like
pthread_mutex_lock(&g_lock);
if (g_cur >= g_numfull){
// No works to do, just quit
pthread_mutex_unlock(&g_lock);
return NULL;
}
// Get the file name
char *filename = g_job_queue[g_cur];
g_cur++;
pthread_mutex_unlock(&g_lock);
I Work with couple of threads. all running as long as an exit_flag is set to false.
I Have specific thread that doesn't recognize the change in the flag, and therefor not ending and freeing up its resources, and i'm trying to understand why.
UPDATE: After debugging a bit with gdb, i can see that given 'enough time' the problematic thread does detects the flag change.
My conclusion from this is that not enough time passes for the thread to detect the change in normal run.
How can i 'delay' my main thread, long enough for all threads to detect the flag change, without having to JOIN them? (the use of exit_flag was in an intention NOT to join the threads, as i don't want to manage all threads id's for that - i'm just detaching each one of them, except the thread that handles input).
I've tried using sleep(5) in close_server() method, after the flag changing, with no luck
Notes:
Other threads that loop on the same flag does terminate succesfully
exit_flag declaration is: static volatile bool exit_flag
All threads are reading the flag, flag value is changed only in close_server() method i have (which does only that)
Data race that may occur when a thread reads the flag just before its changed, doesn't matter to me, as long as in the next iteration of the while loop it will read the correct value.
No error occurs in the thread itself (according to strerr & stdout which are 'clean' from error messages (for the errors i handle in the thread)
Ths situation also occurs even when commenting out the entire while((!exit_flag) && (remain_data > 0)) code block - so this is not a sendfile hanging issure
station_info_t struct:
typedef struct station_info {
int socket_fd;
int station_num;
} station_info_t;
Problematic thread code:
void * station_handler(void * arg_p)
{
status_type_t rs = SUCCESS;
station_info_t * info = (station_info_t *)arg_p;
int remain_data = 0;
int sent_bytes = 0;
int song_fd = 0;
off_t offset = 0;
FILE * fp = NULL;
struct stat file_stat;
/* validate station number for this handler */
if(info->station_num < 0) {
fprintf(stderr, "station_handler() station_num = %d, something's very wrong! exiting\n", info->station_num);
exit(EXIT_FAILURE);
}
/* Open the file to send, and get his stats */
fp = fopen(srv_params.songs_names[info->station_num], "r");
if(NULL == fp) {
close(info->socket_fd);
free(info);
error_and_exit("fopen() failed! errno = ", errno);
}
song_fd = fileno(fp);
if( fstat(song_fd, &file_stat) ) {
close(info->socket_fd);
fclose(fp);
free(info);
error_and_exit("fstat() failed! errno = ", errno);
}
/** Run as long as no exit procedure was initiated */
while( !exit_flag ) {
offset = 0;
remain_data = file_stat.st_size;
while( (!exit_flag) && (remain_data > 0) ) {
sent_bytes = sendfile(info->socket_fd, song_fd, &offset, SEND_BUF);
if(sent_bytes < 0 ) {
error_and_exit("sendfile() failed! errno = ", errno);
}
remain_data = remain_data - sent_bytes;
usleep(USLEEP_TIME);
}
}
printf("Station %d handle exited\n", info->station_num);
/* Free \ close all resources */
close(info->socket_fd);
fclose(fp);
free(info);
return NULL;
}
I'll be glad to get some help.
Thanks guys
Well, as stated by user362924 the main issue is that i don't join the threads in my main thread, therefore not allowing them enough time to exit.
A workaround to the matter, if for some reason one wouldn't want to join all threads and dynamically manage thread id's, is to use sleep command in the end of the main thread, for a couple of seconds.
of course this workaround is not good practice and not recommended (to anyone who gets here by google)
I just wondered about how Instant Messengers and Online Games can accept and deliver messages so fast. (Network programming with sockets)
I read about that this is done with nonblocking sockets.
I tried blocking sockets with pthreads (each client gets its own thread) and nonblocking sockets with kqueue.Then I profiled both servers with a program which made 99 connections (each connection in one thread) and then writes some garbage to it (with a sleep of 1 second). When all threads are set up, I measured in the main thread how long it took to get a connection from the server (with wall clock time) (while "99 users" are writing to it).
threads (avg): 0.000350 // only small difference to kqueue
kqueue (avg): 0.000300 // and this is not even stable (client side)
The problem is, while testing with kqueue I got multiple times a SIGPIPE error (client-side). (With a little timeout usleep(50) this error was fixed). I think this is really bad because a server should be capable to handle thousands of connections. (Or is it my fault on the client side?) The crazy thing about this is the infamous pthread approach did just fine (with and without timeout).
So my question is: how can you build a stable socket server in C which can handle thousands of clients "asynchronously"? I only see the threads approach as a good thing, but this is considered bad practice.
Greetings
EDIT:
My test code:
double get_wall_time(){
struct timeval time;
if (gettimeofday(&time,NULL)){
// Handle error
return 0;
}
return (double)time.tv_sec + (double)time.tv_usec * .000001;
}
#define NTHREADS 100
volatile unsigned n_threads = 0;
volatile unsigned n_writes = 0;
pthread_mutex_t main_ready;
pthread_mutex_t stop_mtx;
volatile bool running = true;
void stop(void)
{
pthread_mutex_lock(&stop_mtx);
running = false;
pthread_mutex_unlock(&stop_mtx);
}
bool shouldRun(void)
{
bool copy;
pthread_mutex_lock(&stop_mtx);
copy = running;
pthread_mutex_unlock(&stop_mtx);
return copy;
}
#define TARGET_HOST "localhost"
#define TARGET_PORT "1336"
void *thread(void *args)
{
char tmp = 0x01;
if (__sync_add_and_fetch(&n_threads, 1) == NTHREADS) {
pthread_mutex_unlock(&main_ready);
fprintf(stderr, "All %u Threads are ready...\n", (unsigned)n_threads);
}
int fd = socket(res->ai_family, SOCK_STREAM, res->ai_protocol);
if (connect(fd, res->ai_addr, res->ai_addrlen) != 0) {
socket_close(fd);
fd = -1;
}
if (fd <= 0) {
fprintf(stderr, "socket_create failed\n");
}
if (write(fd, &tmp, 1) <= 0) {
fprintf(stderr, "pre-write failed\n");
}
do {
/* Write some garbage */
if (write(fd, &tmp, 1) <= 0) {
fprintf(stderr, "in-write failed\n");
break;
}
__sync_add_and_fetch(&n_writes, 1);
/* Wait some time */
usleep(500);
} while (shouldRun());
socket_close(fd);
return NULL;
}
int main(int argc, const char * argv[])
{
pthread_t threads[NTHREADS];
pthread_mutex_init(&main_ready, NULL);
pthread_mutex_lock(&main_ready);
pthread_mutex_init(&stop_mtx, NULL);
bzero((char *)&hint, sizeof(hint));
hint.ai_socktype = SOCK_STREAM;
hint.ai_family = AF_INET;
if (getaddrinfo(TARGET_HOST, TARGET_PORT, &hint, &res) != 0) {
return -1;
}
for (int i = 0; i < NTHREADS; ++i) {
pthread_create(&threads[i], NULL, thread, NULL);
}
/* wait for all threads to be set up */
pthread_mutex_lock(&main_ready);
fprintf(stderr, "Main thread is ready...\n");
{
double start, end;
int fd;
start = get_wall_time();
fd = socket(res->ai_family, SOCK_STREAM, res->ai_protocol);
if (connect(fd, res->ai_addr, res->ai_addrlen) != 0) {
socket_close(fd);
fd = -1;
}
end = get_wall_time();
if (fd > 0) {
fprintf(stderr, "Took %f ms\n", (end - start) * 1000);
socket_close(fd);
}
}
/* Stop all running threads */
stop();
/* Waiting for termination */
for (int i = 0; i < NTHREADS; ++i) {
pthread_join(threads[i], NULL);
}
fprintf(stderr, "Performed %u successfull writes\n", (unsigned)n_writes);
/* Lol.. */
freeaddrinfo(res);
return 0;
}
SIGPIPE comes when I try to connect to the kqueue server (after 10 connections are made, the server is "stuck"?). And when too many users are writing stuff, the server cannot open a new connection. (kqueue server code from http://eradman.com/posts/kqueue-tcp.html)
SIGPIPE means you're trying to write to a socket (or pipe) where the other end has already been closed (so noone will be able to read it). If you don't care about that, you can ignore SIGPIPE signals (call signal(SIGPIPE, SIG_IGN)) and the signals won't be a problem. Of course the write (or send) calls on the sockets will still be failing (with EPIPE), so you need to make you code robust enough to deal with that.
The reason that SIGPIPE normally kills the process is that its too easy to write programs that ignore errors on write/send calls and run amok using up 100% of CPU time otherwise. As long as you carefully always check for errors and deal with them, you can safely ignore SIGPIPEs
Or is it my fault?
It was your fault. TCP works. Most probably you didn't read all the data that was sent.
And when too many users are writing stuff, the server cannot open a new connection
Servers don't open connections. Clients open connections. Servers accept connections. If your server stops doing that, there something wrong with your accept loop. It should only do two things: accept a connection, and start a thread.
Whats the smartest way to run an application continuously so that it doesn't exit after it hits the bottom? Instead it starts again from the top of main and only exits when commanded. (This is in C)
You should always have some way of exiting cleanly. I'd suggest moving the code off to another function that returns a flag to say whether to exit or not.
int main(int argc, char*argv[])
{
// param parsing, init code
while (DoStuff());
// cleanup code
return 0;
}
int DoStuff(void)
{
// code that you would have had in main
if (we_should_exit)
return 0;
return 1;
}
Most applications that don't fall through enter some kind of event processing loop that allows for event-driven programming.
Under Win32 development, for instance, you'd write your WinMain function to continually handle new messages until it receives the WM_QUIT message telling the application to finish. This code typically takes the following form:
// ...meanwhile, somewhere inside WinMain()
MSG msg;
while (GetMessage(&msg, NULL, 0, 0))
{
TranslateMessage(&msg);
DispatchMessage(&msg);
}
If you are writing a game using SDL, you would loop on SDL events until deciding to exit, such as when you detect that the user has hit the Esc key. Some code to do that might resemble the following:
bool done = false;
while (!done)
{
SDL_Event event;
while (SDL_PollEvent(&event))
{
switch (event.type)
{
case SDL_QUIT:
done = true;
break;
case SDL_KEYDOWN:
if (event.key.keysym.sym == SDLK_ESCAPE)
{
done = true;
}
break;
}
}
}
You may also want to read about Unix Daemons and Windows Services.
while (true)
{
....
}
To elaborate a bit more, you want to put something in that loop that allows you to let the user do repeated actions. Whether it's reading key strokes and performing actions based on the keys pressed, or reading data from the socket and sending back a response.
There are a number of ways to "command" your application to exit (such as a global exit flag or return codes). Some have already touched on using an exit code so I'll put forward an easy modification to make to an existing program using an exit flag.
Let's assume your program executes a system call to output a directory listing (full directory or a single file):
int main (int argCount, char *argValue[]) {
char *cmdLine;
if (argCount < 2) {
system ("ls");
} else {
cmdLine = malloc (strlen (argValue[1]) + 4);
sprintf (cmdLine, "ls %s", argValue[1]);
system (cmdLine);
}
}
How do we go about making that loop until an exit condition. The following steps are taken:
Change main() to oldMain().
Add new exitFlag.
Add new main() to continuously call oldMain() until exit flagged.
Change oldMain() to signal exit at some point.
This gives the following code:
static int exitFlag = 0;
int main (int argCount, char *argValue[]) {
int retVal = 0;
while (!exitFlag) {
retVal = oldMain (argCount, argValue);
}
return retVal;
}
static int oldMain (int argCount, char *argValue[]) {
char *cmdLine;
if (argCount < 2) {
system ("ls");
} else {
cmdLine = malloc (strlen (argValue[1]) + 4);
sprintf (cmdLine, "ls %s", argValue[1]);
system (cmdLine);
}
if (someCondition)
exitFlag = 1;
}