I'm starting my studies with syncronzed threads using semaphore.
I just did a test using binary semaphore (2 threads only) and it's all good.
Imagine a lanhouse, that have 3 computers (threads) and some clients (Threads). If all computers are bussy, the client will wait in a queue with a know limit (e.g 15 clients).
I can't understand how threads will relate to each other.
As far as I know, semaphore is used to control the access of the threads to a certain critical region / memory area / global variable.
1) Create 1 semaphore to control the Clients accessing the computers (but both are threads);
2) Create 1 semaphore to control the clients in queue;
But how relate threads with threads ? How the semaphore will know which thread(s) it should work with.
I don't need a full answer for it. I just need to understand how the Threads will relate to eachother. Some help to understand the situation.
This is my code so far and it's not working ;P can't control the clients to access the 3 computers avaliable.
#include <pthread.h>
#include <semaphore.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#define ROOM_SIZE 15
sem_t queue, pc, mutex;
int room [ROOM_SIZE];
int pcsAvaliable = 3, nAvaliable = 0, roomAvaliable = ROOM_SIZE;
int computers [3]; // 0 -> Empty | 1 -> Ocuppied
void* Lan(void* arg)
{
//Enter the lanhouse
//Verify if there is a computer avaliable
sem_wait(&pc);
if(pcsAvaliable > 0)
{
sem_wait(&mutex);
pcsAvaliable--;
computers[nAvaliable] = 1;
printf("Cliente pegou pc: %d\n", nAvaliable);
nAvaliable++;
sem_post(&mutex);
//Wait for 80~90ms
printf("Client liberou pc: %d\n", nAvaliable);
computers[nAvaliable] = 0;
nAvaliable--;
sem_post(&pc);
}
else
{
printf("No computer avaliable...\n");
//Check the waiting room for avaliable slot
if(roomAvaliable > 0)
{
roomAvaliable--;
printf("Client entered the waiting room.");
}
else
printf("No avaliable space in waiting room..\n");
}
}
int main(int argc, char const *argv[])
{
int i;
if(argc > 1)
{
int numClients = atoi(argv[1]);
sem_init(&pc, 0, 3);
sem_init(&mutex, 0, 1);
pthread_t clients[numClients];
//Create Clients
for(i=0; i< numClients; i++)
{
pthread_create(&clients[i], NULL, Lan, NULL);
}
//Join Clients
for(i=0; i< numClients; i++)
{
pthread_join(clients[i], NULL);
}
}
else
printf("Please, insert a parameter.");
pthread_exit(NULL);
sem_destroy(&pc);
return 0;
}
If you're going to be technical, if you're syncing tasks between threads you should use Semaphore. Example reading input before parsing it.
Here's an answer on semaphores.
But if you're using shared resources, and need to avoid race condition/two threads accesing at the same time, you should use mutexes. Here's a question on what is a mutex.
Also look at the disambiguation by Michael Barr which is a really good.
I would read both question thoroughly and the disambiguation, and you might actually end up not using semaphore and just mutexes since from what you explained you're only controlling a shared resource.
Common semaphore function
int sem_init(sem_t *sem, int pshared, unsigned int value); //Use pshared with 0, starts the semaphore with a given value
int sem_wait(sem_t *sem);//decreases the value of a semaphore, if it's in 0 it waits until it's increased
int sem_post(sem_t *sem);//increases the semaphore by 1
int sem_getvalue(sem_t *sem, int *valp);// returns in valp the value of the semaphore the returned int is error control
int sem_destroy(sem_t *sem);//destroys a semaphore created with sim_init
Common Mutexes functions (for linux not sure what O.S. you're running on)
int pthread_mutex_init(pthread_mutex_t *p_mutex, const pthread_mutexattr_t *attr); //starts mutex pointed by p_mutex, use attr NULL for simple use
int pthread_mutex_lock(pthread_mutex_t *p_mutex); //locks the mutex
int pthread_mutex_unlock(pthread_mutex_t *p_mutex); //unlocks the mutex
int pthread_mutex_destroy(pthread_mutex_t *p_mutex);//destroys the mutex
You can treat computers as resources. The data structure for the resource can be initialized by the main thread. Then, there can be client threads trying to acquire an instance of resource (a computer). You can use a counting semaphore with a value 3 for the number of computers. To acquire a computer, a client thread does
P (computer_sem).
Similarly to release the client thread has to do,
V (computer_sem)
For more information on threads and semaphore usage, refer
POSIX Threads Synchronization in C.
Related
I was asked a question in interview, how synchronization is done in shared memory. I told Take a struct. In that you have a flag and a data. Test the flag and change the data.
I took the following program from internet as below-. Can anyone tell if there is better way of synchronization in shared memory
#define NOT_READY -1
#define FILLED 0
#define TAKEN 1
struct Memory {
int status;
int data[4];
};
Assume that the server and client are in the current directory. The server uses ftok() to generate a key and uses it for requesting a shared memory. Before the shared memory is filled with data, status is set to NOT_READY. After the shared memory is filled, the server sets status to FILLED. Then, the server waits until status becomes TAKEN, meaning that the client has taken the data.
The following is the server program. Click here to download a copy of this server program server.c.
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include "shm-02.h"
void main(int argc, char *argv[])
{
key_t ShmKEY;
int ShmID;
struct Memory *ShmPTR;
if (argc != 5) {
printf("Use: %s #1 #2 #3 #4\n", argv[0]);
exit(1);
}
ShmKEY = ftok(".", 'x');
ShmID = shmget(ShmKEY, sizeof(struct Memory), IPC_CREAT | 0666);
if (ShmID < 0) {
printf("*** shmget error (server) ***\n");
exit(1);
}
printf("Server has received a shared memory of four integers...\n");
ShmPTR = (struct Memory *) shmat(ShmID, NULL, 0);
if ((int) ShmPTR == -1) {
printf("*** shmat error (server) ***\n");
exit(1);
}
printf("Server has attached the shared memory...\n");
ShmPTR->status = NOT_READY;
ShmPTR->data[0] = atoi(argv[1]);
ShmPTR->data[1] = atoi(argv[2]);
ShmPTR->data[2] = atoi(argv[3]);
ShmPTR->data[3] = atoi(argv[4]);
printf("Server has filled %d %d %d %d to shared memory...\n",
ShmPTR->data[0], ShmPTR->data[1],
ShmPTR->data[2], ShmPTR->data[3]);
ShmPTR->status = FILLED;
printf("Please start the client in another window...\n");
while (ShmPTR->status != TAKEN)
sleep(1);
printf("Server has detected the completion of its child...\n");
shmdt((void *) ShmPTR);
printf("Server has detached its shared memory...\n");
shmctl(ShmID, IPC_RMID, NULL);
printf("Server has removed its shared memory...\n");
printf("Server exits...\n");
exit(0);
}
The client part is similar to the server. It waits until status is FILLED. Then, the clients retrieves the data and sets status to TAKEN, informing the server that data have been taken. The following is the client program. Click here to download a copy of this server program client.c.
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include "shm-02.h"
void main(void)
{
key_t ShmKEY;
int ShmID;
struct Memory *ShmPTR;
ShmKEY = ftok(".", 'x');
ShmID = shmget(ShmKEY, sizeof(struct Memory), 0666);
if (ShmID < 0) {
printf("*** shmget error (client) ***\n");
exit(1);
}
printf(" Client has received a shared memory of four integers...\n");
ShmPTR = (struct Memory *) shmat(ShmID, NULL, 0);
if ((int) ShmPTR == -1) {
printf("*** shmat error (client) ***\n");
exit(1);
}
printf(" Client has attached the shared memory...\n");
while (ShmPTR->status != FILLED)
;
printf(" Client found the data is ready...\n");
printf(" Client found %d %d %d %d in shared memory...\n",
ShmPTR->data[0], ShmPTR->data[1],
ShmPTR->data[2], ShmPTR->data[3]);
ShmPTR->status = TAKEN;
printf(" Client has informed server data have been taken...\n");
shmdt((void *) ShmPTR);
printf(" Client has detached its shared memory...\n");
printf(" Client exits...\n");
exit(0);
}
Can anyone tell if there is better way of synchronization in shared memory?
Definitely, yes. I would say the way you waste CPU cycles in busy-wait (while (ShmPTR->status != FILLED) ;) is already a fatal mistake.
Note that POSIX shared memory has a much more sensible interface than the old SysV does. See man 7 shm_overview for details.
There are two distinct purposes for synchronization primitives:
Data synchronization
To protect data against concurrent modification, and to ensure each reader gets a consistent view of the data, there are three basic approaches:
Atomic access
Atomic access requires hardware support, and is typically only supported for native machine word sized units (32 or 64 bits).
Mutexes and condition variables
Mutexes are mutually exclusive locks. The idea is to grab the mutex before examining or modifying the value.
Condition variables are basically unordered queues for threads or processes to wait for a "condition". POSIX pthreads library includes facilities for atomically releasing a mutex and waiting on a condition variable. This makes waiting for a dataset to change trivial to implement, if each modifier signals or broadcasts on the condition variable after each modification.
Read-write locks.
An rwlock is a primitive that allows any number of concurrent "read locks", but only one "write lock" to be held on it at any time. The idea is that each reader grabs a read lock before examining the data, and each writer a write lock before modifying it. This works best when the data is more often examined than modified, and a mechanism for waiting for a change to occur is not needed.
Process synchronization
There are situations where threads and processes should wait (block) until some event has occurred. There are two most common primitives used for this:
Semaphores
A POSIX semaphore is basically an opaque nonnegative counter you initialize to whatever (zero or positive value, within the limits set by the implementation).
sem_wait() checks the counter. If it is nonzero, it decrements the counter and continues execution. If the counter is zero, it blocks until another thread/process calls sem_post() on the counter.
sem_post() increments the counter. It is one of the rare synchronization primitives you can use in a signal handler.
Barriers
A barrier is a synchronization primitive that blocks until there is a specific number of threads or processes blocking in the barrier, then releases them all at once.
Linux does not implement POSIX barriers (pthread_barrier_init(), pthread_barrier_wait(), pthread_barrier_destroy()), but you can easily achieve the same using a mutex, a counter (counting the number of additional processes needed to release all waiters), and a condition variable.
There are many better ways of implementing the said server-client pair (where shared memory contains a flag and some data).
For data integrity and change management, a mutex and one or two condition variables should be used. (If the server may change the data at any time, one condition variable (changed) suffices; if the server must wait until a client has read the data before modifying it, two are needed (changed and observed).)
Here is an example structure you could use to describe the shared memory segment:
#ifndef SHARED_H
#define SHARED_H
#include <stdlib.h>
#include <unistd.h>
#include <pthread.h>
struct shared_data {
/* Shared memory data */
};
struct shared {
pthread_mutex_t lock;
pthread_cond_t change; /* Condition variable for clients waiting on data changes */
pthread_cond_t observe; /* Condition variable for server waiting on data observations */
unsigned long changed; /* Number of times data has been changed */
unsigned long observed; /* Number of times current data has been observed */
struct shared_data data;
};
/* Return the size of 'struct shared', rounded up to a multiple of page size. */
static inline size_t shared_size_page_aligned(void)
{
size_t page, size;
page = (size_t)sysconf(_SC_PAGESIZE);
size = sizeof (struct shared) + page - 1;
return size - (size % page);
}
#endif /* SHARED_H */
The changed and observed fields are counters, that help avoid any time-of-check-to-time-of-use race windows. It is important that before the shared memory is accessed the thread does pthread_mutex_lock(&(shared_memory->lock)), to ensure a consistent view of the data.
If a thread/process examines the data, it should do
shared_memory->observed++;
pthread_cond_broadcast(&(shared_memory->observe));
pthread_mutex_unlock(&(shared_memory->lock));
and if a thread/process modifies the data, it should do
shared_memory->modified++;
shared_memory->observed = 0;
pthread_cond_broadcast(&(shared_memory->change));
pthread_mutex_unlock(&(shared_memory->lock));
to notify any waiters and update the counters, when unlocking the mutex.
I'm new to concept of threading.
I was doing producer consumer problem in C but the consumer thread doesn't run when parallel with producer.
my code is as follows:
#include<stdio.h>
#include<stdlib.h>
#include<pthread.h>
int S;
int E;
int F;
void waitS(){
//printf("hbasd");
while(S<=0);
S--;
}
void signalS(){
S++;
}
void waitE(){
while(E<=0);
E--;
}
void signalE(){
E++;
}
void waitF(){
while(F<=0);
F--;
}
void signalF(){
F++;
}
int p,c;
void* producer(void *n){
int *j = (int *)n;
int i = *j;
while(1){
waitS();
waitE();
printf("Producer %d\n",E);
signalS();
signalF();
p++;
if(p>=i){
printf("Exiting: producer\n");
pthread_exit(0);
}
}
}
void* consumer(void *n){
int *j = (int *)n;
int i = *j;
while(1){
waitS();
waitF();
printf("Consumer %d\n",E);
signalS();
signalE();
c++;
if(c>=i){
printf("Exiting Consumer\n");
pthread_exit(0);
}
}
}
int main(int argc, char* argv[]){
int n = atoi(argv[1]);
E = n;
S = 1;
F = 0;
int pro = atoi(argv[2]);
int con = atoi(argv[3]);
pthread_t pid, cid;
pthread_attr_t attr;
pthread_attr_init(&attr);
pthread_create(&pid,&attr,producer,(void *)&pro);
pthread_create(&cid,&attr,consumer,(void *)&con);
pthread_join(pid,NULL);
pthread_join(cid,NULL);
}
When i give the input as ./a.out 3 4 3
i.e n=3, pro = 4, con = 3
I get no out just an a dead lock kind of situation.
I expect an output like
Producer 2
Producer 1
Producer 0
Consumer 0
Consumer 1
Producer 0
Exiting: producer
Consumer 0
Exiting: consumer
...similar outputs where Producer runs 4 times and consumer thrice
When i give an input like ./a.out 4 4 3
i get the following output
Producer 3
Producer 2
Producer 1
Producer 0
Exiting: producer
Consumer 0
Consumer 1
Consumer 2
Exiting: consumer
from the results i'm getting a conclusion that pthread producer is executing 1st and then is pthread consumer.
I want both of them to execute simultaneously so that i get an answer similar to the first expected output when test cases like 3 4 3 are given.
You are accessing non-atomic variables from different threads without any kind of synchronization; this is a race condition and it leads to undefined behavior.
In particular, modern CPUs provide separate registers and separate caches to each CPU core, which means that if a thread running on CPU core #1 modifies the value of a variable, that modification may remain solely in CPU #1's cache for quite a while, without getting "pushed out" to RAM, and so another thread running on CPU core #2 may not "see" the thread #1's update for a long time (or perhaps never).
The traditional way to deal with this problem is either to serialize accesses to your shared variables with one or more mutexes (see pthread_mutex_init(), pthread_mutex_lock(), pthread_mutex_unlock(), etc), or use atomic variables rather than standard ints for values you want to access from multiple threads simultaneously. Both of those mechanisms have safeguards to ensure that undefined behavior won't occur (if you are using them correctly).
You can not access same memory from two different threads without synchronization. The standard for pthreads spells it out quite clearly here:
Applications shall ensure that access to any memory location by more than one thread of control (threads or processes) is restricted such that no thread of control can read or modify a memory location while another thread of control may be modifying it. Such access is restricted using functions that synchronize thread execution and also synchronize memory with respect to other threads.
Besides, even if we ignore that many CPUs don't synchronise memory unless you explicitly ask them to, your code is still incorrect in normal C because if variables can be changed behind your back they should be volatile. But even though volatile might help on some CPUs, it is incorrect for pthreads.
Just use proper locking, don't spin on global variables, there are methods to heat a room that are much cheaper than using a CPU.
In general, you should use synchronization primitives, but unlike other answerers I do believe we might not need any if we run this program on x86 architecture and prevent compiler to optimize some critical parts in the code.
According to Wikipedia, x86 architecture has almost sequential consistency, which is more than enough to implement a producer-consumer algorithm.
The rules to successfully implement such an producer-consumer algorithm is quite simple:
We must avoid writing the same variable from different threads, i.e. if one thread writes to variable X, another thread just read from X
We must tell the compiler explicitly that our variables might change somewhere, i.e. use volatile keyword on all shared between threads variables.
And here is the working example based on your code. Producer produces numbers from 5 down to 0, consumer consumes them. Please remember, this will work on x86 only due to weaker ordering on other architectures:
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
volatile int P = 0;
volatile int C = 0;
volatile int value = 0;
void produce(int v)
{
value = v;
P++;
}
int consume()
{
int v = value;
C++;
return v;
}
void waitForConsumer()
{
while (C != P)
;
}
void waitForProducer()
{
while (C == P)
;
}
void *producer(void *n)
{
int i = *(int *)n;
while (1) {
waitForConsumer();
printf("Producing %d\n", i);
produce(i);
i--;
if (i < 0) {
printf("Exiting: producer\n");
pthread_exit(0);
}
}
}
void *consumer(void *n)
{
while (1) {
waitForProducer();
int v = consume();
printf("Consumed %d\n", v);
if (v == 0) {
printf("Exiting: consumer\n");
pthread_exit(0);
}
}
}
int main(int argc, char *argv[])
{
int pro = 5;
pthread_t pid, cid;
pthread_attr_t attr;
pthread_attr_init(&attr);
pthread_create(&pid, &attr, producer, (void *)&pro);
pthread_create(&cid, &attr, consumer, NULL);
pthread_join(pid, NULL);
pthread_join(cid, NULL);
}
Produces the following result:
$ ./a.out
Producing 5
Producing 4
Consumed 5
Consumed 4
Producing 3
Producing 2
Consumed 3
Consumed 2
Producing 1
Producing 0
Exiting: producer
Consumed 1
Consumed 0
Exiting: consumer
For more information, I really recommend Herb Sutter's presentation called atomic<> Weapons, which is quite long, but has everything you need to know about ordering and atomics.
Despite the code listed above will work OK on x86, I really encourage you to watch the presentation above and use builtin atomics, like __atomic_load_n(), which will generate the correct assembly code on any platform.
Create new threads for producer and consumer each i.e all producers and consumers have their own threads.
I am writing a program that will transfer 50 integers from one thread to another using shared memory and upon receiving the integers the receiver thread will print them.
The code compiles without any errors or warnings but when I run it I get the following output:
pthread_create() for thread 1 returns: 0
pthread_create() for thread 2 returns: 0
Segmentation fault (core dumped)
Here is my code so far:
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <sys/msg.h>
#include <string.h>
//define declarations
#define SIZE 50
//function declarations
void *send_data();
void *receive_data();
int main()
{
pthread_t thread1;
pthread_t thread2;
int ret_val_t1;
int ret_val_t2;
//create thread1
ret_val_t1 = pthread_create( &thread1, NULL, send_data, NULL);
if(ret_val_t1)
{
fprintf(stderr,"Error - pthread_create() return value: %d\n",ret_val_t1);
exit(EXIT_FAILURE);
}
//create thread2
ret_val_t2 = pthread_create( &thread2, NULL, receive_data, NULL);
if(ret_val_t2)
{
fprintf(stderr,"Error - pthread_create() return value: %d\n",ret_val_t2);
exit(EXIT_FAILURE);
}
printf("pthread_create() for thread 1 returns: %d\n",ret_val_t1);
printf("pthread_create() for thread 2 returns: %d\n",ret_val_t2);
//wait untill threads are done with their routines before continuing with main thread
pthread_join( thread1, NULL);
pthread_join( thread2, NULL);
exit(EXIT_SUCCESS);
}
void *send_data(){
int shmid;
int *array;
int i = 0;
int SizeMem;
key_t key = 12345;
SizeMem = sizeof(*array)*SIZE;
shmid = shmget(key, SIZE*sizeof(int), IPC_CREAT);
array = (int *)shmat(shmid, 0, 0);
for(i=0; i<SIZE; i++)
{
array[i] = i;
}
for(i=0; i<SIZE; i++)
{
printf("\n%d---\n", array[i]);
}
printf("\nWritting to memory succesful--\n");
shmdt((void *) array);
}
void *receive_data(){
int shmid;
int *array;
int i = 0;
key_t key = 12345;
shmid = shmget(key, SIZE*sizeof(int), IPC_EXCL);
array = shmat(shmid, 0, SHM_RDONLY);
for(i=0; i<SIZE; i++)
{
printf("\n%d---\n", array[i]);
}
printf("\nRead to memory succesful--\n");
shmdt((void *) array);
}
Use the ipcs utility and look at the output from ipcs -m
Per the shmget man page:
SYNOPSIS top
#include <sys/ipc.h>
#include <sys/shm.h>
int shmget(key_t key, size_t size, int shmflg);
...
In addition to the above flags, the least significant 9 bits of
shmflg specify the permissions granted to the owner, group, and
others. These bits have the same format, and the same meaning, as
the mode argument of open(2). Presently, execute permissions are not
used by the system.
In line of code
shmid = shmget(key, SIZE*sizeof(int), IPC_CREAT);
the "the least significant 9 bits of shmflg" are set to zero.
No one has permissions to read/write your shared memory segment.
This would be better:
shmid = shmget(key, SIZE*sizeof(int), IPC_CREAT | 0600 );
That would give the owning user read/write permission.
Here in this situation, shared memory is a costly affair. And you are trying to access from the same process, there is no benefit here.
The better way to handle your situation is:
Threads of a parent process, will have access to parent process heap memory. This is simple concept to use. Create a heap object for the parent process and share across your threads.
As said in many comments, shared memory is one of the "Inter Process Communication" communication concept, like pipes or named pipes or message queue etc....
If you are using for practice, then it is OK. But as apart of PROD use, this is a costly affair.
If you do not apply locks and access restriction to your mapped/shared memory properly, and you are not behind a secured firewall, then you are welcoming attackers.
One race condition arises from your code. Your second thread can get cpu first and try to use a non yet created memory segment. As it doesn't exist, and as you don't check for errors, you can get a SIGSEGV from trying to access memory at NULL address.
You have to put synchronization devices that forbid the second thread to get to the shared memory access before it has been created and filled with data. Something like.
pthread_mutex_t lock;
pthread_cond_t shm_created_and_filled;
then in main you initialize the lock and shm_created_and_filled.
in the writing thread you will create the shared memory segment and write the numbers, finally locking and signalling the shm_created_and_filled condition variable. In the reading thread you first lock the lock variable and then wait for the shm_created_and_filled condition. There's still a little race condition in which the first thread runs and signals the shm_created_and_filled condition before the reading thread waits for it (it will lose the condition) but I have left it as an exercise for the reader.
We are developing a simple application in Linux for Desktop computer. The scheme is very simple and as below:
A main process which deals with the external world interface (which gives some data periodically) and spawns and keeps track of child processes.
Various child processes which process this data and report back to main process from time to time.
The data coming in from external world interface is in the chunks of around 240 KBytes and 1 chunk comes in at the rate of roughly once per milliseconds. All the child processes use and work on the same data i.e. the complete chunk is sent to every child process when it arrives.
The number of child processes is not fixed, can vary from 4 to 20. The scheme adopted for inter process communication is as follows:
Shared memory capable of holding multiple data chunks is created by all the processes using common key and using shmget() and shmat() APIs. Main process writes to this shared memory and all the child processes read from this memory.
To inform the child processes that new data chunk have arrived I have used pthread_cond_broadcast() API. The conditional variable and the corresponding mutex used for this purpose reside in a small separate shared memory and are initialized to default attributes in the main process.
So whenever new data chunk arrives (roughly once per 1 ms) main process calls pthread_cond_broadcast() and the child processes which are waiting on pthread_cond_wait() read this data from the shared memory and process it. The problem I am facing is:
Depending on the processor load, sometimes the pthread signals are getting lost i.e. either delivered to only some or none of the waiting child processes. This severly affects the data processing as the data continuity is lost (and the child process is not even aware of it). Processing time of the child process is average 300 micro seconds and this application is running on a multicore processor.
To pin down the problem I even created a dummy application with 1 main process and several dummy child processes who does nothing but to wait on pthread_cond_wait(). From main process I called pthread_cond_broadcast every 1 ms and a count was incremented and printed, similarly every time a pthread signal was received in a child process another count was incremented and printed. When I ran this test program I found after some time the receiver's count began to lag the sender's count and the gap between their count went on increasing. Am I right in my understanding that this was due to some pthread signals not delivered? Are there any other fast plus secure IPC mechanisms.
I even tried the same thing using internet domain sockets using UDP datagrams in broadcast (only for the synchronization purpose while the data was still read from the shared memory). But here also I noticed as the number of child processes increased the synchronization signals were getting lost. Please give your thoughts and ideas.
Consider the test program as below:
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <pthread.h>
#define SHM_KEY 3579
#define NumOfChildProc 20
int Packets_Tx = 0, Packets_Rx = 0;
void ChildProc(void)
{
/* Create the shared memory with same key as SHM_KEY
* Declare the condition and mutex and assign them the shared memory
address */
while(1)
{
pthread_mutex_lock(PTMutex);
pthread_cond_wait(PTCond, PTMutex);
pthread_mutex_unlock(PTMutex);
printf("From CP [%d]: Packets Received = %d\n",getpid(), Packets_Rx++);
}
}
int main(void)
{
int shmid, i;
pid_t l_pid;
char* SigBlock;
pthread_condattr_t condattr;
pthread_mutexattr_t mutexattr;
pthread_cond_t* PTCond;
pthread_mutex_t* PTMutex;
shmid = shmget(SHM_KEY, (sizeof(pthread_cond_t) + sizeof(pthread_mutex_t)), IPC_CREAT | 0666);
if(shmid < 0)
{
perror("shmget");
}
SigBlock = (char *)shmat(shmid, NULL, 0);
if(SigBlock == (char *) -1)
{
perror("shmat");
}
PTCond = (pthread_cond_t*) SigBlock;
PTMutex = (pthread_mutex_t*)(SigBlock + sizeof(pthread_cond_t));
pthread_condattr_init(&condattr);
pthread_condattr_setpshared(&condattr, PTHREAD_PROCESS_SHARED);
pthread_cond_init(PTCond, &condattr);
pthread_condattr_destroy(&condattr);
pthread_mutexattr_init(&mutexattr);
pthread_mutexattr_setpshared(&mutexattr, PTHREAD_PROCESS_SHARED);
pthread_mutex_init(PTMutex, &mutexattr);
pthread_mutexattr_destroy(&mutexattr);
for(i=0; i<NumOfChildProc; i++)
{
l_pid = fork();
if(l_pid == 0)
ChildProc();
}
sleep(1);
while(1)
{
/* Send pthread broadcast and increment the packets count */
printf("From Main Process : Packets Sent = %d\n", Packets_Tx++);
pthread_cond_broadcast(PTCond);
usleep(1000);
}
}
pthread_cond_broadcast() signals do not get "lost". Every thread which is waiting in a pthread_cond_wait() call at the point where the broadcast is sent will be woken - your problem is almost certainly that every thread is not waiting in the pthread_cond_wait() call at the point where pthead_cond_broadcast() is called - some threads may still be processing that last lot of data when the broadcast is sent, in which case they will "miss" the broadcast.
A pthread conditional variable should always be paired with a suitable condition (or predicate) over shared state, and a thread should only call pthread_cond_wait() after checking the state of that predicate.
For example, in your case you might have a shared variable which is the block number of the latest chunk to have arrived. In the main thread, it would increment this (while holding the mutex) before broadcasting the condition variable:
pthread_mutex_lock(&lock);
latest_block++;
pthread_cond_broadcast(&cond);
pthread_mutex_unlock(&lock);
In the worker threads, each thread would keep track of the last block it has processed in a local variable, and check to see if another block has arrived before calling pthread_cond_wait():
pthread_mutex_lock(&lock);
while (latest_block <= my_last_block)
pthread_cond_wait(&cond, &lock);
pthread_mutex_unlock(&lock);
This will cause the worker to wait until the main thread has incremented latest_block to be greater than my_last_block (the last block that was processed by this worker).
Your example test code has the same problem - sooner or later the main thread will call pthread_cond_broadcast() when a child thread is locking or unlocking the mutex, or inside the printf() call.
A version of your example code, updated to use the fix I mentioned, does not show this problem:
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <pthread.h>
#define SHM_KEY 9753
#define NumOfChildProc 20
int Packets_Tx = 0, Packets_Rx = 0;
struct {
pthread_cond_t PTCond;
pthread_mutex_t PTMutex;
int last_packet;
} *shared_data;
void ChildProc(void)
{
int my_last_packet = 0;
/* Create the shared memory with same key as SHM_KEY
* Declare the condition and mutex and assign them the shared memory
address */
while(1)
{
pthread_mutex_lock(&shared_data->PTMutex);
while (shared_data->last_packet <= my_last_packet)
pthread_cond_wait(&shared_data->PTCond, &shared_data->PTMutex);
pthread_mutex_unlock(&shared_data->PTMutex);
printf("From CP [%d]: Packets Received = %d\n",getpid(), Packets_Rx++);
my_last_packet++;
}
}
int main(void)
{
int shmid, i;
pid_t l_pid;
pthread_condattr_t condattr;
pthread_mutexattr_t mutexattr;
shmid = shmget(SHM_KEY, sizeof *shared_data, IPC_CREAT | 0666);
if(shmid < 0)
{
perror("shmget");
}
shared_data = shmat(shmid, NULL, 0);
if(shared_data == (void *) -1)
{
perror("shmat");
}
pthread_condattr_init(&condattr);
pthread_condattr_setpshared(&condattr, PTHREAD_PROCESS_SHARED);
pthread_cond_init(&shared_data->PTCond, &condattr);
pthread_condattr_destroy(&condattr);
pthread_mutexattr_init(&mutexattr);
pthread_mutexattr_setpshared(&mutexattr, PTHREAD_PROCESS_SHARED);
pthread_mutex_init(&shared_data->PTMutex, &mutexattr);
pthread_mutexattr_destroy(&mutexattr);
shared_data->last_packet = 0;
for(i=0; i<NumOfChildProc; i++)
{
l_pid = fork();
if(l_pid == 0)
ChildProc();
}
sleep(1);
while(1)
{
/* Send pthread broadcast and increment the packets count */
printf("From Main Process : Packets Sent = %d\n", Packets_Tx++);
pthread_mutex_lock(&shared_data->PTMutex);
shared_data->last_packet++;
pthread_cond_broadcast(&shared_data->PTCond);
pthread_mutex_unlock(&shared_data->PTMutex);
usleep(30);
}
}
Writing my basic programs on multi threading and I m coming across several difficulties.
In the program below if I give sleep at position 1 then value of shared data being printed is always 10 while keeping sleep at position 2 the value of shared data is always 0.
Why this kind of output is coming ?
How to decide at which place we should give sleep.
Does this mean that if we are placing a sleep inside the mutex then the other thread is not being executed at all thus the shared data being 0.
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include<unistd.h>
pthread_mutex_t lock;
int shared_data = 0;
void * function(void *arg)
{
int i ;
for(i =0; i < 10; i++)
{
pthread_mutex_lock(&lock);
shared_data++;
pthread_mutex_unlock(&lock);
}
pthread_exit(NULL);
}
int main()
{
pthread_t thread;
void * exit_status;
int i;
pthread_mutex_init(&lock, NULL);
i = pthread_create(&thread, NULL, function, NULL);
for(i =0; i < 10; i++)
{
sleep(1); //POSITION 1
pthread_mutex_lock(&lock);
//sleep(1); //POSITION 2
printf("Shared data value is %d\n", shared_data);
pthread_mutex_unlock(&lock);
}
pthread_join(thread, &exit_status);
pthread_mutex_destroy(&lock);
}
When you sleep before you lock the mutex, then you're giving the other thread plenty of time to change the value of the shared variable. That's why you're seeing a value of "10" with the 'sleep' in position #1.
When you grab the mutex first, you're able to lock it fast enough that you can print out the value before the other thread has a chance to modify it. The other thread sits and blocks on the pthread_mutex_lock() call until your main thread has finished sleeping and unlocked it. At that point, the second thread finally gets to run and alter the value. That's why you're seeing a value of "0" with the 'sleep' at position #2.
This is a classic case of a race condition. On a different machine, the same code might not display "0" with the sleep call at position #2. It's entirely possible that the second thread has the opportunity to alter the value of the variable once or twice before your main thread locks the mutex. A mutex can ensure that two threads don't access the same variable at the same time, but it doesn't have any control over the order in which the two threads access it.
I had a full explanation here but ended up deleting it. This is a basic synchronization problem and you should be able to trace and identify it before tackling anything more complicated.
But I'll give you a hint: It's only the sleep() in position 1 that matters; the other one inside the lock is irrelevant as long as it doesn't change the code outside the lock.