Are the event_base_loop()(publisher thread) and event_base_dispatch()(subscriber thread) functions thread safe? Recently I checked my application for any synchronization errors using valgrind's helgrind tool. In the report it's mentioned that there exists a race condition between event_base_loop() and event_base_dispatch() even though both these functions are called with different struct event_base variables (struct event_base base_pub(publisher thread) and struct event_base base_sub (subscriber thread)). Below is the sample code
Publisher Thread
struct event_base base_pub = event_base_new();
while (true) {
... // publish data
event_base_loop(base_pub, EVLOOP_NONBLOCK);
sleep(1);
...
}
Subscriber Thread
struct event_base base_sub = event_base_new();
while (true) {
... // register subscriber callback function
event_base_dispatch(base_sub);
...
}
They are thread-safe, if you enable locking/condition variables through:
evthread_use_pthreads();
You also need to link with libevent_pthreads. Note that:
Before you call any other Libevent functions, you need to set up the
library. If you're going to use Libevent from multiple threads in a
multithreaded application, you need to initialize thread support –
typically by using evthread_use_pthreads() or
evthread_use_windows_threads().
Currently, only one thread can be dispatching a given event_base at a
time. If you want to run events in multiple threads at once, you can
either have a single event_base whose events add work to a work queue,
or you can create multiple event_base objects.
Related
Is it possible to run the C socket library in non-blocking mode, and trigger a function on receiving any data? The function will then evaluate the received data and decide the flow of control in the program.
It would be very helpful if you could mention some references.
Yes, using libevent library that is possible.
First create an event base
struct event_base *ev_base = event_base_new();
Create and add that event for anything received on the socket
struct event *read_ev = event_new(ev_base, socket_fd, EV_READ|EV_PERSIST, callback_function_ptr, callback_function_arg);
event_add(read_ev, NULL);
At the end of the function dispatch events
event_base_dispatch(ev_base);
When trying to implement a simple echo server with concurrent support on linux.
Following approaches are used:
Use pthread functions to create a pool of thread, and maintained in a linked list. It's created on process start, and destroy on process termination.
Main thread will accept request, and use a POSIX message queue to store accepted socket file descriptor.
Threads in pool loop to read from message queue, and handle request it gets, when there is no request, it will block.
The program seems working now.
The questions are:
Is it suitable to use message queue in the middle, is it efficient enough?
What is the general approach to accomplish a thread tool that needs to handle concurrent request from multiple clients?
If it's not proper to make threads in pool loop & block to retrieve msg from message queue, then how to deliver requests to threads?
This seems unneccesarily complicated to me. The usual approach for a multithreaded server is:
Create a listen-socket in a thread process
Accept the client-connections in a thread
For each accepted client connection, create a new threads, which receives the corresponding file descriptor and does the work
The worker thread closes the client connection, when it is fully handled
I do not see much benefit in prepopulating a thread-pool here.
If you really want a threadpool:
I would just use a linked list for accepted connections and a pthread_mutex to synchronize access to it:
The listener-process enqueues client fds at the tail of the list.
The clients dequeue it at the head.
If the queue is empty, the thread can wait on a variable (pthread_cond_wait) and are notified by the listener process (pthread_cond_signal) when connections are available.
Another alternative
Depending on the complexity of handling requests, it might be an option to make the server single-threaded, i.e. handle all connections in one thread. This eliminates context-switches altogether and can thus be very performant.
One drawback is, that only one CPU-core is used. To improve that, a hybrid-model can be used:
Create one worker-thread per core.
Each thread handles simultaneously n connections.
You would however have to implement mechanisms to distribute the work fairly amongst the workers.
In addition to using pthread_mutex, you will want to use pthread_cond_t (pthread condition), this will allow you to put the threads in the thread pool to sleep while they are not actually doing work. Otherwise, you will be wasting compute cycles if they are sitting there in a loop checking for something in the work queue.
I would definitely consider using C++ instead of just pure C. The reason I suggest it is that in C++ you are able to use templates. Using a pure virtual base class (lets call it: "vtask"), you can create templated derived classes that accept arguments and insert the arguments when the overloaded operator() is called, allowing for much, much more functionality in your tasks:
//============================================================================//
void* thread_pool::execute_thread()
{
vtask* task = NULL;
while(true)
{
//--------------------------------------------------------------------//
// Try to pick a task
m_task_lock.lock();
//--------------------------------------------------------------------//
// We need to put condition.wait() in a loop for two reasons:
// 1. There can be spurious wake-ups (due to signal/ENITR)
// 2. When mutex is released for waiting, another thread can be waken up
// from a signal/broadcast and that thread can mess up the condition.
// So when the current thread wakes up the condition may no longer be
// actually true!
while ((m_pool_state != state::STOPPED) && (m_main_tasks.empty()))
{
// Wait until there is a task in the queue
// Unlock mutex while wait, then lock it back when signaled
m_task_cond.wait(m_task_lock.base_mutex_ptr());
}
// If the thread was waked to notify process shutdown, return from here
if (m_pool_state == state::STOPPED)
{
//m_has_exited.
m_task_lock.unlock();
//----------------------------------------------------------------//
if(mad::details::allocator_list_tl::get_allocator_list_if_exists() &&
tids.find(CORETHREADSELF()) != tids.end())
mad::details::allocator_list_tl::get_allocator_list()
->Destroy(tids.find(CORETHREADSELF())->second, 1);
//----------------------------------------------------------------//
CORETHREADEXIT(NULL);
}
task = m_main_tasks.front();
m_main_tasks.pop_front();
//--------------------------------------------------------------------//
//run(task);
// Unlock
m_task_lock.unlock();
//--------------------------------------------------------------------//
// execute the task
run(task);
m_task_count -= 1;
m_join_lock.lock();
m_join_cond.signal();
m_join_lock.unlock();
//--------------------------------------------------------------------//
}
return NULL;
}
//============================================================================//
int thread_pool::add_task(vtask* task)
{
#ifndef ENABLE_THREADING
run(task);
return 0;
#endif
if(!is_alive_flag)
{
run(task);
return 0;
}
// do outside of lock because is thread-safe and needs to be updated as
// soon as possible
m_task_count += 1;
m_task_lock.lock();
// if the thread pool hasn't been initialize, initialize it
if(m_pool_state == state::NONINIT)
initialize_threadpool();
// TODO: put a limit on how many tasks can be added at most
m_main_tasks.push_back(task);
// wake up one thread that is waiting for a task to be available
m_task_cond.signal();
m_task_lock.unlock();
return 0;
}
//============================================================================//
void thread_pool::run(vtask*& task)
{
(*task)();
if(task->force_delete())
{
delete task;
task = 0;
} else {
if(task->get() && !task->is_stored_elsewhere())
save_task(task);
else if(!task->is_stored_elsewhere())
{
delete task;
task = 0;
}
}
}
In the above, each created thread runs execute_thread() until the m_pool_state is set to state::STOPPED. You lock the m_task_lock, and if the state is not STOPPED and the list is empty, you pass the m_task_lock to your condition, which puts the thread to sleep and frees the lock. You create the tasks (not shown), add the task (m_task_count is an atomic, by the way, that is why it is thread safe). During the add task, the condition is signaled to wake up a thread, from which the thread proceeds from the m_task_cond.wait(m_task_lock.base_mutex_ptr()) section of execute_thread() after m_task_lock has been acquired and locked.
NOTE: this is a highly customized implementation that wraps most of the pthread functions/objects into C++ classes so copy-and-pasting will not work whatsoever... Sorry. And w.r.t. the thread_pool::run(), unless you are worrying about return values, the (*task)() line is all you need.
I hope this helps.
EDIT: the m_join_* references is for checking whether all the tasks have been completed. The main thread sits in a similar conditioned wait that checks whether all the tasks have been completed as this is necessary for the applications I use this implementation in before proceeding.
in the libev ,I have initilized the io watcher to catch events and this event causes to store certain value in some cache. I have another timer watcher which runs every 10 sec, reads the cache value. In such case I suppose there is a race condition. do I need to use lock in two different libev watcher or libev handles it.
eg:
TCP_CACHE_TIMEOUT = g_hash_table_new_full(g_str_hash, g_int_equal, key_destroy_cb, value_destroy_timeoutcb);
zlog_info(_c,"TCP Server started at _port: %d",_port);
int fd =setup_tcp_socket(_port);
if(fd<0)
{
return NULL;
}
struct ev_loop *loop = EV_DEFAULT;
struct _sock_ev_serv server;
server.fd = fd;
ev_io_init(&server.io, event_server, server.fd, EV_READ);
ev_io_start(EV_A_ &server.io);
ev_timer_init (&timeout_watcher, timeout_idle_fd, 0, 10.);
ev_timer_again (loop,&timeout_watcher);
ev_loop(loop, 0);
here I have loop and initilizes io watcher to accept server event, and timer watcher to look in the cache every 10 sec. In such case Do I need to handle the race condition myself or the two watcher io and timer running time is managed by libev?
Short answer: There is no race condition, you don't need a lock.
Longer answer:
The reason that there is no race condition, is because libev is in a loop checking the io watcher, then the timer, then the io, then the timer.....
Whichever one is triggered first, is run first. There is no overlap between the two callbacks.
If however, you were using a threaded event loop (possible, but unlikely judging by your code), and you were reading from the same file in two different threads, then there would be a race condition and you would need a lock.
Example:
If you get data in the io watcher after 0.9 seconds, and your callback for that watcher takes 0.2 seconds to run, your timer will be invoked after the io callback has already finished (at ~10.1 seconds).
For college, I am developing a local relay chat. I have to program a chat server and client that will only work on sending messages on different terminal windows on the same computer with threads and FIFOs.
The FIFOs part I am having no trouble, the threads part is the one that is giving me some headaches.
The server has one thread for receiving commands from a FIFO (used by all clients) and another thread for each client that is connected.
For each client that is connected I need to know a certain information. Firstly, I was using global variables, which worked as longs as there was only one client connected, which is much of a chat, to chat alone.
So, ideally I would have some data like:
-nickname
-name
-email
-etc...
per client that is connected. However, I don't know how to do that.
I could create a client_data[MAX_NUMBER_OF_THREADS] where client_data was a struct with everything I needed to have access to, but this would require to, in every communication between server and client to ask for the id of the client in the array client_data and that does not seem very practical
I could also instantiate a client_data immediately after creating the thread but it would only be available in that block, and that is not very practical either.
As you can see I am in need of a little guidance here. Any comment, piece of code or link to any relevant information is greatly appreciated.
I don't know what language you're using but here are some basic ideas:
Start your server in a thread (possibly your main thread).
The server's while loop will block on accepting a socket connection.
When the socket connection is accepted it should spawn a new thread to handle the connection.
Start communicating with the client in the new thread.
A simple socket server loop looks like this (in Java):
while(true){
ClientWorker w;
try{
//server.accept returns a client connection
w = new ClientWorker(server.accept(), textArea);
Thread t = new Thread(w);
t.start();
} catch (IOException e) {
// log the exception or something...
}
}
If you're wondering what it does- the ClientWorker is available here. In C# if you're creating a new Thread don't forget to set its IsBackground property to true so the thread closes when your app shuts down, i.e. no hanging threads.
Remember: accepting a socket connection or receiving data from a socket is usually a blocking call, which means that your thread will block until somebody connects to the socket or data comes through the socket.
In C#:
The Chat Client: http://www.geekpedia.com/tutorial239_Csharp-Chat-Part-1---Building-the-Chat-Client.html
The Chat Server: http://www.geekpedia.com/tutorial240_Csharp-Chat-Part-2---Building-the-Chat-Server.html
A Basic Client/Server: http://www.dreamincode.net/forums/topic/33396-basic-clientserver-chat-application-in-c%23/
In Java:
Chat Client/Server: http://pirate.shu.edu/~wachsmut/Teaching/CSAS2214/Virtual/Lectures/chat-client-server.html
Nakov Chat Client/Server: http://inetjava.sourceforge.net/lectures/part1_sockets/InetJava-1.9-Chat-Client-Server-Example.html
In C++
On Code Project: http://www.codeproject.com/KB/cpp/chat_client_server.aspx
Another Code Project TCP/IP chat client/server: http://www.codeproject.com/KB/IP/ipchat.aspx
Update
Instead of doing global variables, just define a struct for the client account and declare an account variable for each user... here is how you can define the account information:
struct account {
char nickname[32];
char first_name[32];
char last_name[32];
char e_mail[32];
char password[32];
};
When the client sends a message it should have a standard format: FROM|TO|CONTENT
struct message{
char nickname_from[32];
char nickname_to[32]; // optional
char msg_content[256];
};
Put each message on the fifo [queue] and you will have all the information you need to identify who sent it.
Here is some psuedo code that might actually almost run. Note, I'm not freeing what I allocate, I'm not checking errors, I'm just trying to demonstrate how to pass a structure to a thread and use a simple mutex.
There is one thing to be careful of, The function pointer for threads specifies a void * argument, which could be literally any kind of type. In the thread, we assume that its safe to cast the thread argument to a type that we've defined for use. If you are passing multiple possible types, you have to be careful :)
I'm not quite sure about the structure of your program, or how you are handling threads, but here's a short approach-agnostic example on how to pass data to them:
typedef struct client {
char *firstname;
char *lastname;
char *email;
char *nickname
} client_t;
pthread_mutex_t client_lock = PTHREAD_MUTEX_INITIALIZER;
void *client_thread(void *threadarg)
{
client_t *client = (client_t *) threadarg;
pthread_mutex_lock(&client_lock);
/* read and write to client structure */
pthread_mutex_unlock(&client_lock);
/* do some other stuff */
pthread_exit(NULL);
}
int main(void)
{
client_t *client;
pthread_t cthread;
client = (client_t *)malloc(sizeof(struct client));
if (client == NULL)
return 1;
client->firstname = strdup("Joe Public");
/* set the rest of the data */
pthread_create(&cthread, NULL, (void *)client_thread, (void *)client);
/* join / detach / etc */
/* Free all the structure members and the structure itself */
return 0;
}
I'm pretty sure this is what you were asking?
g_timeout_add (100, (GSourceFunc) read_next_packets, NULL);
I can feel the GUI response is slow because of the above statement.
How can I make it work asynchronously so that it doesn't affect the GUI response?
Callbacks of these functions are called in the main thread. If read_next_packets is slow or blocks for I/O, you should instead create a separate thread for it that doesn't interfere with GUI. When that thread needs to inform the main thread of something, then it can use g_idle_add to transfer execution to the main thread scope.
In pseudocode:
// In a dedicated thread:
while (...) {
Package* package = do_read (); // This call is slow or blocks.
if (package)
g_idle_add ((GSourceFunc) process_package, package);
}
// This is called in the main thread. Should be fast to not freeze GUI.
gboolean
process_package (Package* package)
{
...
package_free (package);
}