I am sorry if this sounds like I am repeating this question, but I have a couple additions that I am hoping someone can explain for me.
I am trying to implement a 'packet queueing system' with pipes. I have 1 thread that has a packet of data that it needs to pass to a second thread (Lets call the threads A and B respectively). Originally I did this with a queueing structure that I implemented using linked lists. I would lock a Mutex, write to the queue, and then unlock the Mutex. On the read side, I would do the same thing, lock, read, unlock. Now I decided to change my implementation and make use of pipes (so that I can make use of blocking when data is not available). Now for my question:
Do I need to use Mutexs to lock the file descriptors of the pipe for read and write operations?
Here is my thinking.
I have a standard message that gets written to the pipe on writes, and it is expected to be read on the read side.
struct pipe_message {
int stuff;
short more_stuff;
char * data;
int length;
};
// This is where I read from the pipe
num_bytes_read = read(read_descriptor, &buffer, sizeof(struct pipe_message));
if(num_bytes_read != sizeof(struct pipe_message)) // If the message isn't full
{
printe("Error: Read did not receive a full message\n");
return NULL;
}
If I do not use Mutexs, could I potentially read only half of my message from the pipe?
This could be bad because I would not have a pointer to the data and I could be left with memory leaks.
But, if I use Mutexs, I would lock the Mutex on the read, attempt to read which would block, and then because the Mutex is locked, the write side would not be able to access the pipe.
Do I need to use Mutexs to lock the file descriptors of the pipe for read and write operations?
It depends on the circumstances. Normally, no.
Normality
If you have a single thread writing into the pipe's write file descriptor, no. Nor does the reader need to use semaphores or mutexes to control reading from the pipe. That's all taken care of by the OS underneath on your behalf. Just go ahead and call write() and read(); nothing else is required.
Less Usual
If you have multiple threads writing into the pipe's write file descriptor, then the answer is maybe.
Under Linux calling write() on the pipe's write file descriptor is an atomic operation provided that the size of data being written is less than a certain amount (this is specified in the man page for pipe(), but I recall that it's 4kbytes). This means that you don't need a mutex or semaphore to control access to the pipe's write file descriptor.
If the size of the data you're writing is too large then then the call to write() on the pipe is not atomic. So if you have multiple threads writing to the pipe and the size is too large then you do need a mutex to control access to the write end of the pipe.
Using a mutex with a blocking pipe is actually dangerous. If the write side takes the mutex, writes to the pipe and blocks because the pipe is full, then the read side can't get the mutex to read the data from the pipe, and you have a deadlock.
To be safe, on the write side you'd probably need to do something like take the mutex, check if the pipe has space for what you want to write, if not then release the mutex, yield and then try again.
Related
Here is process a and b, both of which are multithreaded.
a forks b and b immediatly execs one new program;
a dups and freopens stderr to the logfile (a is defacto apache's httpd2.22)
b inherits the opened stderr from a. ( i am adapting apache httpd, b is my program), and b uses fprintf(stderr....) for logging
so a, b share the same file for logging
there is no lock mechanism for a, b to write log
I found that some log msg are interleaving, and a little bit of log msg got lost.
Can the two writers to the same file implicitly lock each other out?
The more important question is: If we use fprintf only within one single multithreaded process, fprintf is thread safe, i.e. one call of fprintf won't intervene another fprintf call in another thread? Many articles said this, but this is not easy to ensure myself, so I ask for help here.
A: the code for duplicate the fd is like this:
......
rv = apr_file_dup2(stderr_log, s_main->error_log, stderr_p);//dup the stderr to the logfile
apr_file_close(s_main->error_log);//here ,2 fd point to the same file description,so close one of
then
B:apache it self use this manner for logging:
......
if (rv != APR_SUCCESS) {
ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s_main, ".........");
C:for convenience,i logging in this way:
fprintf(stderr,".....\n")
I am quite sure apache and me use the same fd for file writing.
If you're using a single FILE object to perform output on an open file, then whole fprintf calls on that FILE will be atomic, i.e. lock is held on the FILE for the duration of the fprintf call. Since a FILE is local to a single process's address space, this setup is only possible in multi-threaded applications; it does not apply to multi-process setups where several different processes are accessing separate FILE objects referring to the same underlying open file. Even though you're using fprintf here, each process has its own FILE it can lock and unlock without the others seeing the changes, so writes can end up interleaved. There are several ways to prevent this from happening:
Allocate a synchronization object (e.g. a process-shared semaphore or mutex) in shared memory and make each process obtain the lock before writing to the file (so only one process can write at a time); OR
Use filesystem-level advisory locking, e.g. fcntl locks or the (non-POSIX) BSD flock interface; OR
Instead of writing directly to the log file, write to a pipe that another process will feed into the log file. Writes to a pipe are guaranteed (by POSIX) to be atomic as long as they are smaller than PIPE_BUF bytes long. You cannot use fprintf in this case (since it might perform multiple underlying write operations), but you could use snprintf to a PIPE_BUF-sized buffer followed by write.
I have a POSIX thread which reads from the non-blocking anonymous pipe (marked with O_NONBLOCK flag). When thread is stopping (because of errors for example) I want to check if there is something left in pipe (in its internal buffer). If pipe has data - run new thread with the same read descriptor (it is shared between threads) so the new thread can continue reading from pipe. If pipe is empty - close pipe and do nothing.
So I need to check if pipe is empty without removing data from pipe (as regular read will do). Is there any way to do it?
P.S. I think setting count = 0 in read(int fd, void *buf, size_t count); may help but the documentation sais that it is some kind of undefined behavior:
If count is zero, read() may detect the errors described below. In
the absence of any errors, or if read() does not check for errors, a
read() with a count of 0 returns zero and has no other effects.
I believe you want poll or select, called with a zero timeout.
Short description from the select() docs:
select() and pselect() allow a program to monitor multiple file
descriptors, waiting until one or more of the file descriptors become
"ready" for some class of I/O operation (e.g., input possible).
...and the poll() docs:
poll() performs a similar task to select(2): it waits for one of a
set of file descriptors to become ready to perform I/O.
I know for pthreads, if they're modifying the same variables or files, you can use pthread_mutex_lock to prevent simultaneous writes.
If I'm using fork() to have multiple processes, which are editing the same file, how can I make sure they're not writing simultaneously to that file?
Ideally I'd like to lock the file for one writer at a time, and each process would only need to write once (no loops necessary). Do I need to do this manually or will UNIX do it for me?
Short answer: you have to do it manually. There are certain guarantees on the atomicity of each write, but you'll still need to synchronize the processes to avoid interleaving writes. There are a lot of techniques for synchronizing processes. Since all of your writers are descendants of a common process, probably the easiest thing to do is to pass a token on a common pipe. Before you fork, create a pipe and write a single byte into it. Any time a process wants to write to the file, it will do a blocking read on the pipe. If it gets a byte, then it proceeds to write to the file. When it is done, it writes a byte back into the pipe. If any other process wants to access the file, it will block on the pipe read until the other process is done writing. This is often simpler than using a semaphore, which is another excellent technique.
I have multiple processes (and multiple threads within some processes) writing to a single named pipe. The pipe is opened with O_WRONLY for each writer.
I have another process reading from this pipe, blocking with select. The pipe is opened with O_RDONLY | O_NONBLOCK in the reader.
When select in the reader wakes up, will read return at most one chunk of data available, or could it return multiple chunks? If the former, then I expect after I read the first chunk, select will immediately wake up until I finish reading the remaining chunks.
Or could read return less than one of the chunks written by a writer?
I'm writing and reading strings, and they're all less than PIPE_BUF, so I know the writes are atomic. I can easily append a delimiter to check for multiple strings, but I'm just curious how it works on Linux.
read will return all data available in the pipe, it doesn't matter how many writes were used to write that data. The number of bytes returned will be same as the size requested, when there is more data in pipe. In such cases, select will return immediately, indicating that there is some data to read.
You will have to delimit each chuck you write to pipe and separate it after reading.
I'm trying to implement a pipe using shared memory and semaphores (it may be that I need signals also, to complete my implementation)
I encountered the algorithmic problem of how to set the semaphores right.
Lets say I already allocated a piece of shared memory for the pipe buffer,
and a piece of shared memory for the pipe's info (such as how much bytes there are in the pipe, etc...)
I want to create mutual exclusion (only one reader/writer using the pipe at once)
If reader wants to read from an empty pipe, I should block him, till a writer writes something
Same thing like '2', but writer who writes to a full pipe
I tried to search for an answer but I didn't find any even though it seems like a common exercise...
I'm aware of a solution called "Bounded buffer problem" or "consumer producer problem"
which is implemented like this:
There are 3 semaphores:
mutex - initialized to 1
full - initialized to 0
empty - initialized to n (whilst n is the number of, lets say "bytes" I have in the pipe)
Consumer's code:
wait(full)
wait(mutex)
remove a byte from the pipe
signal(mutex)
signal(empty)
Producer's code:
wait(empty)
wait(mutex)
add a byte to the pipe
signal(mutex)
signal(full)
The problem in this solution (to use as a solution to my problem) is that in a given time, only one byte is read from the pipe, or write into it.
In my problem - Implementing a pipe, I don't know for sure how much bytes a writer will write. If he wants to write 'n' bytes, then he will write it only if there is a place in the pipe, and if not, he will write less then 'n' bytes...
That means that a writer must check how much free space there is in the pipe, before writing into it. This is a problem - because the writer will touch a critical section (the pipe's information) without mutual exclusion..
So I thought about putting this part inside the critical section, but then - if a writer wants to write and the pipe is full - how can I let only one reader inside, and then letting the writer to write more?
I've got confused...
Any help will be appreciated, Thanks!
There is no need to have so many mutexes or lock them for that amount of time. In single producer/consumer scenario, the producer never needs to worry about the free space reducing (it is the only one that can use up that space), and similarly for the consumer. Therefore your pseudocode should be:
Producer
while (lock_and_get_free_space() < bytes_to_write)
wait()
unlock()
write(bytes_to_write)
lock_and_update_free_space()
Consumer
while (lock_and_get_data() < bytes_to_read)
wait()
unlock()
read(bytes_to_read)
lock_and_update_free_space()