I'm having trouble terminating my server in my multithreaded program (one server, multiple clients).
When the variable global_var, which counts the number of currently connected clients, gets set to 0, the server should terminate, but it doesn't.
What I think is happening is since accept() is blocking , the code never reaches the break condition in main loop.
It's breaking correctly out of thread_func but then it blocks inside the while loop, just before the accept() call and after printing "Exiting thread_func".
volatile int finished = 0; // Gets set to 1 by catching SIGINT/SIGSTOP
int global_var = 0; // When it gets to 0, server should terminate
int server_fd;
void * thread_func(void* arg)
{
do_some_pre_stuff();
while(1)
{
if(!global_var)
{
close(server_fd);
finished = 1;
break;
}
if(recv(...) > 0)
{
do_more_stuff()
}
else
{
disconnect_client();
global_var--;
break;
}
}
free_more_ressources();
return NULL;
}
int main()
{
do_initial_stuff();
init_socket();
listen();
while (!finished)
{
if( (fd = accept(server_fd,...)) == -1)
exit(-1);
global_var++;
/* Some intermediate code */
if(!global_var)
break;
// Thread for the newly connected player
if(pthread_create(&thread_id[...], NULL, thread_func, (void*)some_arg)
exit(-1);
}
free_resources();
puts("Exiting thread_func");
}
I tried the advice listed here without success (except the pipe answer, not trying to mess with pipes).
I'm new to socket programming but what I tried so far looked correct but none of the solutions worked (including semaphores, pthread_cancel,etc)
PS: synchronization has been implemented, just omitted here for readability
I have a writer process which outputs its status at regular intervals as a readable chunck of wchar_t.
I would need to ensure the following properties:
When there's and update, the readers shouldn't read partial/corrupted data
The file should be volatile in memory so that when the writer quits, the file is gone
The file content size is variable
Multiple readers could read the file in parallel, doesn't matter if the content is synced, as long as is non partial for each client
If using truncate and then write, clients should only read the full file and not observe such partial operations
How could I implement such /procfs-like file, outside /procfs filesystem?
I was thinking to use classic c Linux file APIs and create something under /dev/shm by default, but I find it hard to implement effectively point 1 and 5 most of all.
How could I expose such file?
Typical solution is to create a new file in the same directory, then rename (hardlink) it over the old one.
This way, processes see either an old one or a new one, never a mix; and it only depends on the moment when they open the file.
The Linux kernel takes care of the caching, so if the file is accessed often, it will be in RAM (page cache). The writer must, however, remember to delete the file when it exits.
A better approach is to use fcntl()-based advisory record locks (typically over the entire file, i.e. .l_whence = SEEK_SET, .l_start = 0, .l_len = 0).
The writer will grab a write/exclusive lock before truncating and rewriting the contents, and readers a read/shared lock before reading the contents.
This requires cooperation, however, and the writer must be prepared to not be able to lock (or grabbing the lock may take undefined amount of time).
A Linux-only scheme would be to use atomic replacement (via rename/hardlinking), and file leases.
(When the writer process has an exclusive lease on an open file, it gets a signal whenever another process wants to open that same file (inode, not file name). It has at least a few seconds to downgrade or release the lease, at which point the opener gets access to the contents.)
Basically, the writer process creates an empty status file, and obtains exclusive lease on it. Whenever the writer receives a signal that a reader wants to access the status file, it writes the current status to the file, releases the lease, creates a new empty file in the same directory (same mount suffices) as the status file, obtains an exclusive lease on that one, and renames/hardlinks it over the status file.
If the status file contents do not change all the time, only periodically, then the writer process creates an empty status file, and obtains exclusive lease on it. Whenever the writer receives a signal that a reader wants to access the (empty) status file, it writes the current status to the file, and releases the lease. Then, when the writer process' status is updated, and there is no lease yet, it creates a new empty file in the status file directory, takes an exclusive lease on it, and renames/hardlinks over the status file.
This way, the status file is always updated just before a reader opens it, and only then. If there are multiple readers at the same time, they can open the status file without interruption when the writer releases the lease.
It is important to note that the status information should be collected in a single structure or similar, so that writing it out to the status file is efficient. Leases are automatically broken if not released soon enough (but there are a few seconds at least to react), and the lease is on the inode – file contents – not the file name, so we still need the atomic replacement.
Here's a crude example implementation:
#define _POSIX_C_SOURCE 200809L
#define _GNU_SOURCE
#include <stdlib.h>
#include <stdarg.h>
#include <inttypes.h>
#include <unistd.h>
#include <fcntl.h>
#include <pthread.h>
#include <signal.h>
#include <limits.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>
#define LEASE_SIGNAL (SIGRTMIN+0)
static pthread_mutex_t status_lock = PTHREAD_MUTEX_INITIALIZER;
static int status_changed = 0;
static size_t status_len = 0;
static char *status = NULL;
static pthread_t status_thread;
static char *status_newpath = NULL;
static char *status_path = NULL;
static int status_fd = -1;
static int status_errno = 0;
char *join2(const char *src1, const char *src2)
{
const size_t len1 = (src1) ? strlen(src1) : 0;
const size_t len2 = (src2) ? strlen(src2) : 0;
char *dst;
dst = malloc(len1 + len2 + 1);
if (!dst) {
errno = ENOMEM;
return NULL;
}
if (len1 > 0)
memcpy(dst, src1, len1);
if (len2 > 0)
memcpy(dst+len1, src2, len2);
dst[len1+len2] = '\0';
return dst;
}
static void *status_worker(void *payload __attribute__((unused)))
{
siginfo_t info;
sigset_t mask;
int err, num;
/* This thread blocks all signals except LEASE_SIGNAL. */
sigfillset(&mask);
sigdelset(&mask, LEASE_SIGNAL);
err = pthread_sigmask(SIG_BLOCK, &mask, NULL);
if (err)
return (void *)(intptr_t)err;
/* Mask for LEASE_SIGNAL. */
sigemptyset(&mask);
sigaddset(&mask, LEASE_SIGNAL);
/* This thread can be canceled at any cancellation point. */
pthread_setcanceltype(PTHREAD_CANCEL_DEFERRED, NULL);
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
while (1) {
num = sigwaitinfo(&mask, &info);
if (num == -1 && errno != EINTR)
return (void *)(intptr_t)errno;
/* Ignore all but the lease signals related to the status file. */
if (num != LEASE_SIGNAL || info.si_signo != LEASE_SIGNAL || info.si_fd != status_fd)
continue;
/* We can be canceled at this point safely. */
pthread_testcancel();
/* Block cancelability for a sec, so that we maintain the mutex correctly. */
pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, NULL);
pthread_mutex_lock(&status_lock);
status_changed = 0;
/* Write the new status to the file. */
if (status && status_len > 0) {
const char *ptr = status;
const char *const end = status + status_len;
ssize_t n;
while (ptr < end) {
n = write(status_fd, ptr, (size_t)(end - ptr));
if (n > 0) {
ptr += n;
} else
if (n != -1) {
if (!status_errno)
status_errno = EIO;
break;
} else
if (errno != EINTR) {
if (!status_errno)
status_errno = errno;
break;
}
}
}
/* Close and release lease. */
close(status_fd);
status_fd = -1;
/* After we release the mutex, we can be safely canceled again. */
pthread_mutex_unlock(&status_lock);
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
pthread_testcancel();
}
}
static int start_status_worker(void)
{
sigset_t mask;
int result;
pthread_attr_t attrs;
/* This thread should block LEASE_SIGNAL signals. */
sigemptyset(&mask);
sigaddset(&mask, LEASE_SIGNAL);
result = pthread_sigmask(SIG_BLOCK, &mask, NULL);
if (result)
return errno = result;
/* Create the worker thread. */
pthread_attr_init(&attrs);
pthread_attr_setstacksize(&attrs, 2*PTHREAD_STACK_MIN);
result = pthread_create(&status_thread, &attrs, status_worker, NULL);
pthread_attr_destroy(&attrs);
/* Ready. */
return 0;
}
int set_status(const char *format, ...)
{
va_list args;
char *new_status = NULL;
int len;
if (!format)
return errno = EINVAL;
va_start(args, format);
len = vasprintf(&new_status, format, args);
va_end(args);
if (len < 0)
return errno = EINVAL;
pthread_mutex_lock(&status_lock);
free(status);
status = new_status;
status_len = len;
status_changed++;
/* Do we already have a status file prepared? */
if (status_fd != -1 || !status_newpath) {
pthread_mutex_unlock(&status_lock);
return 0;
}
/* Prepare the status file. */
do {
status_fd = open(status_newpath, O_WRONLY | O_CREAT | O_CLOEXEC, 0666);
} while (status_fd == -1 && errno == EINTR);
if (status_fd == -1) {
pthread_mutex_unlock(&status_lock);
return 0;
}
/* In case of failure, do cleanup. */
do {
/* Set lease signal. */
if (fcntl(status_fd, F_SETSIG, LEASE_SIGNAL) == -1)
break;
/* Get exclusive lease on the status file. */
if (fcntl(status_fd, F_SETLEASE, F_WRLCK) == -1)
break;
/* Replace status file with the new, leased one. */
if (rename(status_newpath, status_path) == -1)
break;
/* Success. */
pthread_mutex_unlock(&status_lock);
return 0;
} while (0);
if (status_fd != -1) {
close(status_fd);
status_fd = -1;
}
unlink(status_newpath);
pthread_mutex_unlock(&status_lock);
return 0;
}
int main(int argc, char *argv[])
{
char *line = NULL;
size_t size = 0;
ssize_t len;
if (argc != 2 || !strcmp(argv[1], "-h") || !strcmp(argv[1], "--help")) {
const char *argv0 = (argc > 0 && argv[0]) ? argv[0] : "(this)";
fprintf(stderr, "\n");
fprintf(stderr, "Usage: %s [ -h | --help ]\n", argv0);
fprintf(stderr, " %s STATUS-FILE\n", argv0);
fprintf(stderr, "\n");
fprintf(stderr, "This program maintains a pseudofile-like status file,\n");
fprintf(stderr, "using the contents from standard input.\n");
fprintf(stderr, "Supply an empty line to exit.\n");
fprintf(stderr, "\n");
return EXIT_FAILURE;
}
status_path = join2(argv[1], "");
status_newpath = join2(argv[1], ".new");
unlink(status_path);
unlink(status_newpath);
if (start_status_worker()) {
fprintf(stderr, "Cannot start status worker thread: %s.\n", strerror(errno));
return EXIT_FAILURE;
}
if (set_status("Empty\n")) {
fprintf(stderr, "Cannot create initial empty status: %s.\n", strerror(errno));
return EXIT_FAILURE;
}
while (1) {
len = getline(&line, &size, stdin);
if (len < 1)
break;
line[strcspn(line, "\n")] = '\0';
if (line[0] == '\0')
break;
set_status("%s\n", line);
}
pthread_cancel(status_thread);
pthread_join(status_thread, NULL);
if (status_fd != -1)
close(status_fd);
unlink(status_path);
unlink(status_newpath);
return EXIT_SUCCESS;
}
Save the above as server.c, then compile using e.g.
gcc -Wall -Wextra -O2 server.c -lpthread -o server
This implements a status server, storing each line from standard input to the status file if necessary. Supply an empty line to exit. For example, to use the file status in the current directory, just run
./server status
Then, if you use another terminal window to examine the directory, you see it has a file named status (with typically zero size). But, cat status shows you its contents; just like procfs/sysfs pseudofiles.
Note that the status file is only updated if necessary, and only for the first reader/accessor after status changes. This keeps writer/server overhead and I/O low, even if the status changes very often.
The above example program uses a worker thread to catch the lease-break signals. This is because pthread mutexes cannot be locked or released safely in a signal handler (pthread_mutex_lock() etc. are not async-signal safe). The worker thread maintains its cancelability, so that it won't be canceled when it holds the mutex; if canceled during that time, it will be canceled after it releases the mutex. It is careful that way.
Also, the temporary replacement file is not random, it is just the status file name with .new appended at end. Anywhere on the same mount would work fine.
As long as other threads also block the lease break signal, this works fine in multithreaded programs, too. (If you create other threads after the worker thread, they'll inherit the correct signal mask from the main thread; start_status_worker() sets the signal mask for the calling thread.)
I do trust the approach in the program, but there may be bugs (and perhaps even thinkos) in this implementation. If you find any, please comment or edit.
I Work with couple of threads. all running as long as an exit_flag is set to false.
I Have specific thread that doesn't recognize the change in the flag, and therefor not ending and freeing up its resources, and i'm trying to understand why.
UPDATE: After debugging a bit with gdb, i can see that given 'enough time' the problematic thread does detects the flag change.
My conclusion from this is that not enough time passes for the thread to detect the change in normal run.
How can i 'delay' my main thread, long enough for all threads to detect the flag change, without having to JOIN them? (the use of exit_flag was in an intention NOT to join the threads, as i don't want to manage all threads id's for that - i'm just detaching each one of them, except the thread that handles input).
I've tried using sleep(5) in close_server() method, after the flag changing, with no luck
Notes:
Other threads that loop on the same flag does terminate succesfully
exit_flag declaration is: static volatile bool exit_flag
All threads are reading the flag, flag value is changed only in close_server() method i have (which does only that)
Data race that may occur when a thread reads the flag just before its changed, doesn't matter to me, as long as in the next iteration of the while loop it will read the correct value.
No error occurs in the thread itself (according to strerr & stdout which are 'clean' from error messages (for the errors i handle in the thread)
Ths situation also occurs even when commenting out the entire while((!exit_flag) && (remain_data > 0)) code block - so this is not a sendfile hanging issure
station_info_t struct:
typedef struct station_info {
int socket_fd;
int station_num;
} station_info_t;
Problematic thread code:
void * station_handler(void * arg_p)
{
status_type_t rs = SUCCESS;
station_info_t * info = (station_info_t *)arg_p;
int remain_data = 0;
int sent_bytes = 0;
int song_fd = 0;
off_t offset = 0;
FILE * fp = NULL;
struct stat file_stat;
/* validate station number for this handler */
if(info->station_num < 0) {
fprintf(stderr, "station_handler() station_num = %d, something's very wrong! exiting\n", info->station_num);
exit(EXIT_FAILURE);
}
/* Open the file to send, and get his stats */
fp = fopen(srv_params.songs_names[info->station_num], "r");
if(NULL == fp) {
close(info->socket_fd);
free(info);
error_and_exit("fopen() failed! errno = ", errno);
}
song_fd = fileno(fp);
if( fstat(song_fd, &file_stat) ) {
close(info->socket_fd);
fclose(fp);
free(info);
error_and_exit("fstat() failed! errno = ", errno);
}
/** Run as long as no exit procedure was initiated */
while( !exit_flag ) {
offset = 0;
remain_data = file_stat.st_size;
while( (!exit_flag) && (remain_data > 0) ) {
sent_bytes = sendfile(info->socket_fd, song_fd, &offset, SEND_BUF);
if(sent_bytes < 0 ) {
error_and_exit("sendfile() failed! errno = ", errno);
}
remain_data = remain_data - sent_bytes;
usleep(USLEEP_TIME);
}
}
printf("Station %d handle exited\n", info->station_num);
/* Free \ close all resources */
close(info->socket_fd);
fclose(fp);
free(info);
return NULL;
}
I'll be glad to get some help.
Thanks guys
Well, as stated by user362924 the main issue is that i don't join the threads in my main thread, therefore not allowing them enough time to exit.
A workaround to the matter, if for some reason one wouldn't want to join all threads and dynamically manage thread id's, is to use sleep command in the end of the main thread, for a couple of seconds.
of course this workaround is not good practice and not recommended (to anyone who gets here by google)
I am creating a timer which runs approximately every second and which is waiting for a key to be pressed (which i am not doing). While it is running it shows:
select : interrupted system call
select : interrupted system call
select : interrupted system call
select : interrupted system call
Can you tell me why its this is happening:
struct sigaction s1;
static timer_t tid3;
sigfillset(&s1.sa_mask);
s1.sa_flags = SA_SIGINFO;
s1.sa_sigaction = SignalHandler;
if (sigaction(SIGU, &s1, NULL) == -1)
{
perror("s1 failed");
exit( EXIT_FAILURE );
}
printf("\nTimer %d is setting up \n",TimerIdentity);
tid3=SetTimer(SIGU, 1000, 1);
// ---------- SET timer values -------------------
static struct sigevent sigev;
static timer_t tid;
static struct itimerspec itval;
static struct itimerspec oitval;
sigev.sigev_notify = SIGEV_SIGNAL;
sigev.sigev_signo = signo;
sigev.sigev_value.sival_ptr = &tid;
if (timer_create(CLOCK_REALTIME, &sigev, &tid) == 0)
{
itval.it_value.tv_sec = sec/1000;
itval.it_value.tv_nsec = (long)(sec % 1000) * (1000000L);
//itval.it_value.tv_nsec = 0;
if (mode == 1)
{
itval.it_interval.tv_sec = itval.it_value.tv_sec;
itval.it_interval.tv_nsec = itval.it_value.tv_nsec;
}
if (timer_settime(tid, 0, &itval, NULL) == 0)
{
printf("Timer_settime \n");
}
else
{
perror("time_settime error!");
}
}
//---------------- SIGNAL HANDLER ----------------
void SignalHandler(int signo, siginfo_t* info, void* context)
{
else if (signo == SIGU) // for keypad being pressed
{
calltimer3function();
}
}
//-----------------calltimer3function------------------------
unsigned char key5_debounce=0,key5_debounce_count=0;
calltimer3function()
{
if(!key5_debounce)
{
if((GPIORead(INPUT_SW5)==0))
{
key5_debounce=1;
}
}
if(key5_debounce)
{
if((GPIORead(INPUT_SW5)==0))
{
key5_debounce_count++;
}
else
key5_debounce=0;
if(key5_debounce_count>=KEY_DEBOUNCE)
{
printf("key5 pressed\n");
extr_count=1;
printf("\nDisplay menu called");
display_menu();
key5_debounce=0;
key5_debounce_count=0;
}
}
}
It may be worth mentioning two things:
Blocking functions such as select, read, etc.. get interrupted by signals. You may like to set SA_RESTART flag when calling sigaction. man signal(7):
If a signal handler is invoked while a system call or library function call is blocked, then either:
the call is automatically restarted after the signal handler returns; or
the call fails with the error EINTR.
Which of these two behaviors occurs depends on the interface and whether or not the signal handler was established using the SA_RESTART flag (see sigaction(2)). The details vary across UNIX systems; below, the details for Linux.
In the signal handler you should only call async signal safe functions. Or use the self-pipe trick to avoid doing anything in the signal handler at all.
Alternatively, there is a way to have timers without using timer_create and timerfd_create. select accepts a timeout argument which can be used to specify time till the next timer expiry. Then, select returns 0 if the timeout occurred. This method applies to other event demultiplexing APIs, such as poll and epoll.
I'm trying to write a thread-safe read-write lock in C using POSIX semaphores. You can see the current state of the source code here.
I followed this to create a readers-preferred lock.
The problem is I would like to handle the destruction of the lock for any possible state it may be when rwl_destroy() is called.
If destroy is called and no other thread is on the lock then it will lock the wrt (used by writers) to prevent any other thread from accessing the data guarded by the lock. Next the destroy function should destroy the semaphores and free the memory allocated for the ReadWriteLock struct. But what if another thread is now waiting on the lock? According to the documentation this thread will be left in an undefined state.
That's what I'm trying to avoid in order make lock easier to use.
EDIT:
the current code is:
typedef struct ReadWriteLock
{
sem_t wrt;
sem_t mtx;
sem_t delFlag;
int readcount;
int active;
}ReadWriteLock;
//forward declaration
/* This function is used to take the state of the lock.
* Return values:
* [*] 1 is returned when the lock is alive.
* [*] 0 is returned when the lock is marked for delete.
* [*] -1 is returned if an error was encountered.
*/
int isActive(ReadWriteLock*);
int rwl_init(ReadWriteLock* lock)
{
lock = malloc(sizeof(ReadWriteLock));
if (lock == NULL)
{
perror("rwl_init - could not allocate memory for lock\n");
return -1;
}
if (sem_init(&(lock->wrt), 0, 1) == -1)
{
perror("rwl_init - could not allocate wrt semaphore\n");
free(lock);
lock = NULL;
return -1;
}
if (sem_init(&(lock->mtx), 0, 1) == -1)
{
perror("rwl_init - could not allocate mtx semaphore\n");
sem_destroy(&(lock->wrt));
free(lock);
lock = NULL;
return -1;
}
if (sem_init(&(lock->delFlag), 0 , 1) == -1)
{
perror("rwl_init - could not allocate delFlag semaphore\n");
sem_destroy(&(lock->wrt));
sem_destroy(&(lock->mtx));
free(lock);
lock = NULL;
return -1;
}
lock->readcount = 0;
lock->active = 1;
return 0;
}
int rwl_destroy(ReadWriteLock* lock)
{
errno = 0;
if (sem_trywait(&(lock->wrt)) == -1)
perror("rwl_destroy - trywait on wrt failed.");
if ( errno == EAGAIN)
perror("rwl_destroy - wrt is locked, undefined behaviour.");
errno = 0;
if (sem_trywait(&(lock->mtx)) == -1)
perror("rwl_destroy - trywait on mtx failed.");
if ( errno == EAGAIN)
perror("rwl_destroy - mtx is locked, undefined behaviour.");
if (sem_destroy(&(lock->wrt)) == -1)
perror("rwl_destroy - destroy wrt failed");
if (sem_destroy(&(lock->mtx)) == -1)
perror("rwl_destroy - destroy mtx failed");
if (sem_destroy(&(lock->delFlag)) == -1)
perror("rwl_destroy - destroy delFlag failed");
free(lock);
lock = NULL;
return 0;
}
int isActive(ReadWriteLock* lock)
{
errno = 0;
if (sem_trywait(&(lock->delFlag)) == -1)
{
perror("isActive - trywait on delFlag failed.");
return -1;
}
if ( errno == EAGAIN)
{//delFlag is down, lock is marked for delete
perror("isActive - tried to lock but ReadWriteLock was marked for delete");
return 0;
}
return 1;
}
I also have these functions:
int rwl_writeLock(ReadWriteLock*);
int rwl_writeUnlock(ReadWriteLock*);
int rwl_readLock(ReadWriteLock*);
int rwl_readUnlock(ReadWriteLock*);
So my question is how you change these functions in order to avoid the undefined state I described above. Is it even possible or do the user of this code should be responsible for releasing all locks before attempting to destroy the ReadWriteLock?
The isActive() function and the delFlag semaphore are not used currently, they were made in my attempt to solve the problem.
You should implement a "disposed" state of your ReadWriteLock instance (the "active" field looks appropriate, but you don't use it, why?).
Check it twice in rwl_writeLock / rwl_readLock, before and after the sem_wait() call. This trick is well-known as a "double-checking lock pattern". If you find your Lock to be deleted before entering sem_wait, just leave the function.
If you find your Lock to be deleted after entering sem_wait, do sem_post immediately and leave.
In your destroy() routine, set active=0, then sem_post to both semaphores (don't bother if sem_post fails). If you still need the sem_destroy afterwards, usleep a bit (so all readers and writers have their time to receive the signal) and do sem_destroy.
P.S. You actually have no need to call sem_destroy if you are sure that the semaphore is not used anymore.