my question is that i am using rawsocket passing high rate ( larger than 50kpps) traffic, two threads, one is to send ( read from buffer), another one is to receive ( write to buffer).
i have to use while(1) loop to make sure an infinite loop, and i cannot use usleep since then i will loose packet ( i have tried that)... now the cpu usage is 100% and i think i am buring my cpu...
here is the code:
while (1)
{
if (sendIndex == PACKET_COUNT_MAX)
{
sendIndex = 0;
}
else if (ringBuffer[sendIndex].drop == 0)
{if(sendtosocket (ringBuffer, sendIndex, rawout) < 0)
a++;
else
sendIndex++;}
else if (ringBuffer[sendIndex].drop == 1) {
ringBuffer[sendIndex].header.free = 1;
memset (ringBuffer[sendIndex].data, 0, sizeof (ringBuffer[sendIndex].data));
sendIndex++;
}
else
{
a++;
}
//nanosleep((struct timespec[]){{0, 5}}, NULL);
}
Thanks in advance!!!!!!!
Lisa
You need to pass the control over to the kernel. The command you may find useful is select. Check out the whole story on http://manpages.courier-mta.org/htmlman2/select.2.html. For more info, http://www.gnu.org/software/libc/manual/html_node/Waiting-for-I_002fO.html.
It's all about knowing you have nothing else to do except wait for input from the network. Or the file system. Or anything else that is a file descriptor (U*ix lingo). So, you let the kernel awake you once you've got something to process.
You can try
Increasing the receive buffer
Slowing down the sendto with a sleep, that should not lose packets
Use the MSG_WAITALL flag with recvfrom to make it a blocking read, and make sure the socket was not opened with SOCK_NONBLOCK or O_NONBLOCK
You need sane synchronization between your threads. This includes:
Using locks of some kind to ensure that a variable isn't read by one thread while it is, or might be, modified in another.
Using some kind of waiting scheme so that the sending thread can wait for there to be work for it do without spinning.
Check out pthread_mutex_lock and pthread_cond_wait (assuming you're using POSIX threads).
Related
I'm using pthreads to solve a producer-consumer problem. Basicaly the producer reads a file to a buffer and the consumers (at least one, but not limited) take entries from the buffer and operate them one by one.
This is the producer:
//...Local stuff...
if(file){
while(fgets(line, 256, file)){
pthread_mutex_lock(&buffer_mutex);
while(data->buffer->buffer_items == data->buffer->buffer_size){
pthread_cond_wait(&buffer_full_cv, &buffer_mutex);}
data->buffer->buffer_items);
reads++;
add_to_head(data->buffer, line);
pthread_cond_broadcast(&buffer_ready_cv);
pthread_mutex_unlock(&buffer_mutex);
}
pthread_mutex_lock(&buffer_mutex);
work = 0;
pthread_mutex_unlock(&buffer_mutex);
fclose(file);
}
And this is the consumer:
//...Local stuff...
while(1){
pthread_mutex_lock(&buffer_mutex);
while(data->buffer->buffer_items == 0){
if(work)
pthread_cond_wait(&buffer_ready_cv, &buffer_mutex);
else if(!work && !data->buffer->buffer_items)
pthread_exit(NULL);
}
remove_from_tail(data->buffer, string_to_check);
data->buffer->buffer_items);
pthread_cond_signal(&buffer_full_cv);
pthread_mutex_unlock(&buffer_mutex);
for(unsigned int i = 0; i < data->num_substrings; i++){
cur_occurrence = strstr(string_to_check, data->substrings[i]);
while(cur_occurrence != NULL){
pthread_mutex_lock(&buffer_mutex);
data->occurrences[i]++;
cur_occurrence++;
cur_occurrence = strstr(cur_occurrence, data->substrings[i]);
pthread_mutex_unlock(&buffer_mutex);
}
}
}
What seems to be happening is the file is completely read and there's still work to be done, but as the producer is not running anymore, the wait in the consumer never finishes.
PS.: I've also tried pthread_cond_signal instead of broadcast, but didn't work either.
Anyway, Is there something I'm missing here?
What seems to be happening is the file is completely read and there's still work to be done, but as the producer is not running anymore, the wait in the consumer never finishes.
Technically, this is not a deadlock. This is a common challenge with producer/consumer thread configurations. There are various ways to deal with this.
You could use a special buffer value (separate from the empty buffer) which signals that the producer has finished. (If you have multiple consumers, this special value has to be left in the buffer.) Such in-band signaling, while sometimes convenient to implement, is typically not a good design, though.
If you have multiple producers, you probably should combine the buffer with a counter of the number of producers running, essentially adding a semaphore to the buffer. If the count of producers reaches zero, the consumers need to exit.
The thread which spawns the producers and consumers could use pthread_cancel after joining all consumers, so that pthread_cond_wait is aborted. This is tricky to get completely right, though, and cancellation is best avoided in general.
Note that if you have multiple consumers, each one needs to broadcast the signal after it observed that the end-of-data state has been reached, so that the other consumers have a chance to observe it, too,
I'm still trying to implement inter-thread communication, and I don't understand how to make threads read a shared chunk of data when it is available. The problem is: I'm listening to a few sockets inside epoll in each thread, and if I receive information form one I need to inform other threads to read it.
I have tried eventfd with EFD_SEMAPHORE, but it doesn't suit because there's one thread that reads all data from it ahead of everybody. I cannot find a use for posix semaphores or condition variables because they can hardly be used with epoll.
Is there any way to implement such a synchronization easily?
UPDATE:
int k = epoll_wait(epollfd, n, events);
for (i = 0; i < k; i++) {
int fd = events[i].data.fd;
if (fd == eventfd) {
if (eventfd_read(eventfd, ..) != -1)
/* for each socket on this thread send latest_message to it */
}
else {
latest_message = read_message(fd);
eventfd_write(eventfd, 1)
}
}
this is the example of what I'm trying to do, assuming recvs are rare.
Actually, I thought about better idea, why do I have to bind sockets to thread when I can have a list of all sockets and just let the fastest threads send the messages, increasing something like last_reciever variable on message structure.
I am working on one project in which i need to read from 80 or more clients and then write their o/p into a file continuously and then read these new data for another task. My question is what should i use select or multithreading?
Also I tried to use multi threading using read/fgets and write/fputs call but as they are blocking calls and one operation can be performed at one time so it is not feasible. Any idea is much appreciated.
update 1: I have tried to implement the same using condition variable. I able to achieve this but it is writing and reading one at a time.When another client tried to write then it cannot able to write unless i quit from the 1st thread. I do not understand this. This should work now. What mistake i am doing?
Update 2: Thanks all .. I am able to succeeded to get this model implemented using mutex condition variable.
updated Code is as below:
**header file*******
char *mailbox ;
pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER ;
pthread_cond_t writer = PTHREAD_COND_INITIALIZER;
int main(int argc,char *argv[])
{
pthread_t t1 , t2;
pthread_attr_t attr;
int fd, sock , *newfd;
struct sockaddr_in cliaddr;
socklen_t clilen;
void *read_file();
void *update_file();
//making a server socket
if((fd=make_server(atoi(argv[1])))==-1)
oops("Unable to make server",1)
//detaching threads
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr,PTHREAD_CREATE_DETACHED);
///opening thread for reading
pthread_create(&t2,&attr,read_file,NULL);
while(1)
{
clilen = sizeof(cliaddr);
//accepting request
sock=accept(fd,(struct sockaddr *)&cliaddr,&clilen);
//error comparison against failire of request and INT
if(sock==-1 && errno != EINTR)
oops("accept",2)
else if ( sock ==-1 && errno == EINTR)
oops("Pressed INT",3)
newfd = (int *)malloc(sizeof(int));
*newfd = sock;
//creating thread per request
pthread_create(&t1,&attr,update_file,(void *)newfd);
}
free(newfd);
return 0;
}
void *read_file(void *m)
{
pthread_mutex_lock(&lock);
while(1)
{
printf("Waiting for lock.\n");
pthread_cond_wait(&writer,&lock);
printf("I am reading here.\n");
printf("%s",mailbox);
mailbox = NULL ;
pthread_cond_signal(&writer);
}
}
void *update_file(int *m)
{
int sock = *m;
int fs ;
int nread;
char buffer[BUFSIZ] ;
if((fs=open("database.txt",O_RDWR))==-1)
oops("Unable to open file",4)
while(1)
{
pthread_mutex_lock(&lock);
write(1,"Waiting to get writer lock.\n",29);
if(mailbox != NULL)
pthread_cond_wait(&writer,&lock);
lseek(fs,0,SEEK_END);
printf("Reading from socket.\n");
nread=read(sock,buffer,BUFSIZ);
printf("Writing in file.\n");
write(fs,buffer,nread);
mailbox = buffer ;
pthread_cond_signal(&writer);
pthread_mutex_unlock(&lock);
}
close(fs);
}
I think for the the networking portion of things, either thread-per-client or multiplexed single-threaded would work fine.
As for the disk I/O, you are right that disk I/O operations are blocking operations, and if your data throughput is high enough (and/or your hard drive is slow enough), they can slow down your network operations if the disk I/O is done synchronously.
If that is an actual problem for you (and you should measure first to verify that it really is a problem; no point complicating things if you don't need to), the first thing I would try to ameliorate the problem would be to make your file's output-buffer larger by calling setbuffer. With a large enough buffer, it may be possible for the C runtime library to hide any latency caused by disk access.
If larger buffers aren't sufficient, the next thing I'd try is creating one or more threads dedicated to reading and/or writing data. That is, when your network thread wants to save data to disk, rather than calling fputs()/write() directly, it allocates a buffer containing the data it wants written, and passes that buffer to the IO-write thread via a (mutex-protected or lockless) FIFO queue. The I/O thread then pops that buffer out of the queue, writes the data to the disk, and frees the buffer. The I/O thread can afford to be occasionally slow in writing because no other threads are blocked waiting for the writes to complete. Threaded reading from disk is a little more complex, but basically the IO-read thread would fill up one or more buffers of in-memory data for the network thread to drain; and whenever the network thread drained some of the data out of the buffer, it would signal the IO-read thread to refill the buffer up to the top again. That way (ideally) there is always plenty of input-data already present in RAM whenever the network thread needs to send some to a client.
Note that the multithreaded method above is a bit tricky to get right, since it involves inter-thread synchronization and communication; so don't do it unless there isn't any simpler alternative that will suffice.
Either select/poll or multithreading is ok if you you program solves the problem.
I' guess your program would be io-bound as the number of clients grows up, as you have disk read/write frequently. So it would not speed up to have multiple threads doing the io operation. Polling may be a better choice then
You can set a socket that you get from accept to be non-blocking. Then it is easy to use select to find out when there is data, read the number of bytes that are available and process them.
With (only) 80 clients, I see no reason to expect any significant difference from using threads unless you get very different amounts of data from different clients.
I am trying to make a simple client-server chat program. On the client side I spin off another thread to read any incomming data from the server. The problem is, I want to gracefully terminate that second thread when a person logs out from the main thread. I was trying to use a shared variable 'running' to terminate, problem is, the socket read() command is a blocking command, so if I do while(running == 1), the server has to send something before the read returns and the while condition can be checked again. I am looking for a method (with common unix sockets only) to do a non-blocking read, basically some form of peek() would work, for I can continually check the loop to see if I'm done.
The reading thread loop is below, right now it does not have any mutex's for the shared variables, but I plan to add that later don't worry! ;)
void *serverlisten(void *vargp)
{
while(running == 1)
{
read(socket, readbuffer, sizeof(readbuffer));
printf("CLIENT RECIEVED: %s\n", readbuffer);
}
pthread_exit(NULL);
}
You can make socket not blockable, as suggested in another post plus use select to wait input with timeout, like this:
fd_set input;
FD_ZERO(&input);
FD_SET(sd, &input);
struct timeval timeout;
timeout.tv_sec = sec;
timeout.tv_usec = msec * 1000;
int n = select(sd + 1, &input, NULL, NULL, &timeout);
if (n == -1) {
//something wrong
} else if (n == 0)
continue;//timeout
if (!FD_ISSET(sd, &input))
;//again something wrong
//here we can call not blockable read
fcntl(socket, F_SETFL, O_NONBLOCK);
or, if you have other flags:
int x;
x=fcntl(socket ,F_GETFL, 0);
fcntl(socket, F_SETFL, x | O_NONBLOCK);
then check the return value of read to see whether there was data available.
note: a bit of googling will yield you lots of full examples.
You can also use blocking sockets, and "peek" with select with a timeout. It seems more appropriate here so you don't do busy wait.
The best thing is likely to get rid of the extra thread and use select() or poll() to handle everything in one thread.
If you want to keep the thread, one thing you can do is call shutdown() on the socket with SHUT_RDWR, which will shut down the connection, wake up all threads blocked on it but keep the file descriptor valid. After you have joined the reader thread, you can then close the socket. Note that this only works on sockets, not on other types of file descriptor.
Look for function setsockopt with option SO_RCVTIMEO.
I have a function say void *WorkerThread ( void *ptr).
The function *WorkerThread( void *ptr) has infinite loop which reads and writes continously from Serial Port
example
void *WorkerThread( void *ptr)
{
while(1)
{
// READS AND WRITE from Serial Port USING MUXTEX_LOCK AND MUTEX_UNLOCK
} //while ends
}
The other function I worte is ThreadTest
example
int ThreadTest()
{
pthread_t Worker;
int iret1;
pthread_mutex_init(&stop_mutex, NULL);
if( iret1 = pthread_create(&Worker, NULL, WorkerThread, NULL) == 0)
{
pthread_mutex_lock(&stop_mutex);
stopThread = true;
pthread_mutex_unlock(&stop_mutex);
}
if (stopThread != false)
stopThread = false;
pthread_mutex_destroy(&stop_mutex);
return 0;
}
In main function
I have something like
int main(int argc, char **argv)
{
fd = OpenSerialPort();
if( ConfigurePort(fd) < 0) return 0;
while (true)
{
ThreadTest();
}
return 0;
}
Now, when I run this sort of code with debug statement it runs fine for few hours and then throw message like "can't able to create thread" and application terminates.
Does anyone have an idea where I am making mistakes.
Also if there is way to run ThreadTest in main with using while(true) as I am already using while(1) in ThreadWorker to read and write infinitely.
All comments and criticism are welcome.
Thanks & regards,
SamPrat.
You are creating threads continually and might be hitting the limit on number of threads.
Pthread_create man page says:
EAGAIN Insufficient resources to create another thread, or a system-imposed
limit on the number of threads was encountered. The latter case may
occur in two ways: the RLIMIT_NPROC soft resource limit (set via
setrlimit(2)), which limits the number of process for a real user ID,
was reached; or the kernel's system-wide limit on the number of
threads, /proc/sys/kernel/threads-max, was reached.
You should rethink of the design of your application. Creating an infinite number of threads is not a god design.
[UPDATE]
you are using lock to set an integer variable:
pthread_mutex_lock(&stop_mutex);
stopThread = true;
pthread_mutex_unlock(&stop_mutex);
However, this is not required as setting an int is atomic (on probably all architectures?). You should use a lock when you are doing not-atomic operations, eg: test and set
take_lock ();
if (a != 1)
a = 1
release_lock ();
You create a new thread each time ThreadTest is called, and never destroy these threads. So eventually you (or the OS) run out of thread handles (a limited resource).
Threads consume resources (memory & processing), and you're creating a thread each time your main loop calls ThreadTest(). And resources are finite, while your loop is not, so this will eventually throw a memory allocation error.
You should get rid of the main loop, and make ThreadTest return the newly created thread (pthread_t). Finally, make main wait for the thread termination using pthread_join.
Your pthreads are zombies and consume system resources. For Linux you can use ulimit -s to check your active upper limits -- but they are not infinite either. Use pthread_join() to let a thread finish and release the resources it consumed.
Do you know that select() is able to read from multiple (device) handles ? You can also define a user defined source to stop select(), or a timeout. With this in mind you are able to start one thread and let it sleeping if nothing occurs. If you intent to stop it, you can send a event (or timeout) to break the select() function call.
An additional design concept you have to consider is message queues to share information between your main application and/or pthread. select() is compatible with this technique so you can use one concept for data sources (devices and message queues).
Here a reference to a good pthread reading and the best pthread book available: Programming with POSIX(R) Threads, ISBN-13:978-0201633924
Looks like you've not called pthread_join() which cleans up state after non-detached threads are finished. I'd speculate that you've hit some per process resource limit here as a result.
As others have noted this is not great design though - why not re-use the thread rather than creating a new one on every loop?