I have two processes communicating via a pty, nonblocking. Problem is that the fread() on the master fails when there is no data available to process.
How can I ignore the "no reader/data present" case when reading from the unconnected file descriptor on the master side? I suspect there is some flag for open() or fcntl() which I skipped during reading?
// initialization:
int pty_fd = posix_openpt(O_RDWR | O_NOCTTY);
int rc = grantpt(pty_fd);
rc = unlockpt(pty_fd);
fcntl(pty_fd, F_SETFL, O_NONBLOCK);
fd_read = fdopen(pty_fd, "r");
// now, trying to read will fail if no data is present:
char buf[100];
int count = sizeof(buf);
size_t bytesRead = fread(buf, sizeof(char), count, fd_read);
if ((bytesRead == 0) && (ferror(fd_read)) {
printf("fail...\n");
}
Sure, I can ignore the return value of ferror(), but I suppose this is not the right way to use this function.
Ah, one thing: I found the POLLHUP trick on stackoverflowz. It works but is racy and hence not suitable for my case...
Greetings
Related
I have a function that receives a descriptor (you know, one of those things that open() and socket() spit), reads from it and does something with the data:
int do_something(int fd);
I want to test this function. Preferably, the input data should sit right next to the test assert for the sake of easy debugging. (Therefore, actual file reading should be avoided.) In my mind, the ideal would be something like
unsigned char test_input[] = { 1, 2, 3 };
int fd = char_array_to_fd(test_input);
ck_assert(do_something(fd) == 1234);
(ck_assert is from the Check framework. It's just a typical unit test assert.)
Is there a way to implement char_array_to_fd()? I don't mind if I need to NULL-terminate the array or send the length in.
I imagine that I can open a socket to myself and write on one end so the test function receives the data on the other end. I just don't want to write something awkward and find out that Unix had something less contrived all along. The solution should be any-Unix friendly.
(Basically, I'm asking for a C equivalent of ByteArrayInputStream.)
Alternatively: Should I be thinking in some other way to solve this problem?
On Linux, you can use memfd_create() to create a memory-backed temporary file:
unsigned char test_input[] = ...;
int fd = memfd_create( "test_input", 0 );
// write test data to the the "file"
write( fd, test_input, sizeof( test_input );
// reset file descriptor to the start of the "file"
lseek( fd, 0, SEEK_SET );
Note that completely lacks error checking.
You can use mkstemp to make a temporary file and write to it or read from it:
int fd = mkstemp("tempXXXXXX");
If you want something more dynamic, you can use socketpair to create a pair of connected sockets.
int pair[2];
socketpair(AF_UNIX, SOCK_STREAM, 0, pair);
You can then fork process or thread to interact with your program under test.
#ChrisDodd's answer is already accepted, but I wanted to add the pipes solution I had developed (thanks to #Someprogrammerdude's comment) for completeness sake:
struct writer_args {
int fd;
unsigned char *buffer;
size_t size;
};
/* Pipe writer thread function */
void *write_buffer(void *_args)
{
struct writer_args *args = _args;
/*
* error handling omitted.
* Should probably also be thrown into a loop, in case it writes less
* than args->size.
*/
write(args->fd, args->buffer, args->size);
close(args->fd);
return NULL;
}
/*
* Wrapper for quick & easy testing of the do_something() function.
* Replaces the fd parameter for a char array and its size.
*/
static int __do_something(unsigned char *input, size_t size)
{
pthread_t writer_thread;
struct writer_args args;
int fd[2];
int result;
pipe(fd); /* error handling omitted */
/* fd[0] is for reading, fd[1] is for writing */
/*
* We want one thread for reading, another one for writing.
* This is because pipes have a nonstandardized maximum buffer capacity.
* If we write too much without reading, it will block forever.
*/
args.fd = fd[1];
args.buffer = input;
args.size = size;
/* error handling omitted */
pthread_create(&writer_thread, NULL, write_buffer, &args);
result = do_something(fd[0]);
close(fd[0]);
pthread_join(writer_thread, NULL); /* error handling omitted */
return result;
}
Then, I can keep testing do_something as much as I want:
ret = __do_something(input1, input1_size);
if (ret != 1234)
fprintf(stderr, "Fail. Expected:1234 Actual:%d\n", ret);
ret = __do_something(input2, input2_size);
if (ret != 0)
fprintf(stderr, "Fail. Expected:0 Actual:%d\n", ret);
ret = __do_something(input3, input3_size);
if (ret != 555)
fprintf(stderr, "Fail. Expected:555 Actual:%d\n", ret);
...
I have a sample program:
int main()
{
const char* fn = "/tmp/tmpfifo";
int i = mkfifo(fn, 0666);
int fd = open(fn, O_RDONLY | O_NONBLOCK);
int flags = fcntl(fd, F_GETFL);
flags &= ~O_NONBLOCK;
fcntl(fd, F_SETFL, flags);
char buf[1024];
int rd= read(fd, buf, 100);
cout << rd << endl;
remove(fn);
return 0;
}
It seems that after removing the non-blocking flag from the file descriptor, the read call should block until something is written into the FIFO, but my program always runs without blocking and rd=0 result. Can you please explain this behaviour? Thanks!
The behavior you are seeing is expected. You've done the following:
Opened the read end of the FIFO using O_NONBLOCK, so that a writer need not be present on the FIFO. This guarantees that the open() will immediately succeed.
Disabled O_NONBLOCK before subsequent reads. You've now taken yourself back to a position that is equivalent to the standard (blocking) case where a FIFO had a reader and writer, but the writer closed the FIFO. At that point, the reader should see end-of-file, which is what you are seeing.
It's strange! I tried a code which opens the file without O_NONBLOCK and then procedes in 3 stages. The 3rd stage doesn'act correctly althoug the O_NONBLOCK flag results reset!
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
int main()
{
char buf[1024];
int rd;
const char* fn = "prova.txt";
int i = mkfifo(fn, 0666);
int fd = open(fn, O_RDONLY); // | O_NONBLOCK);
int flags = fcntl(fd, F_GETFL);
//flags &= ~O_NONBLOCK;
printf("1) Waits!\t\tflags=0%08o\n",flags);
rd= read(fd, buf, 100);
printf("%d %o\n",rd,flags);
flags |= O_NONBLOCK;
printf("2) Doesn't wait!\tflags=0%08o\n",flags);
fcntl(fd, F_SETFL, flags);
rd= read(fd, buf, 100);
printf("%d %o\n",rd,flags);
//This doen't act the flag ????
flags &= ~O_NONBLOCK;
fcntl(fd, F_SETFL, flags);
flags=fcntl(fd, F_GETFL);
printf("3) Waits!\t\tflags=0%08o\n",flags);
rd= read(fd, buf, 100);
printf("%d %o\n",rd,flags);
puts("End!");
return 0;
}
Here is the command sequence and the output:
sergio#zarathustra:~$ ./a.out &
[2] 6555
sergio#zarathustra:~$ echo xxx >> prova.txt
1) Waits! flags=000100000
4 100000
2) Doesn't wait! flags=000104000
0 104000
3) Waits! flags=000100000
0 100000
End!
sergio#zarathustra:~$
I looked at your code and at first glance it seems like it should work. There are no errors returned, you don't seem to be breaking any rules, but it's just not blocking.
So I went ahead and traced the read call to see what it was doing:
And it goes all the way to the pipe_read function without any attempt to block. Once it's there it realizes there's nobody on the other side of the pipe and returns EOF.
So this is apparently by design, but with a pipe only the open call will try to block if there's no writer, once open returns it's just assumed that there must be a writer at the other end of that pipe or that you're nonblocking and ready to handle that. And it sort of makes sense. If you're trying to read from a pipe but the writer is gone (or was never there in the first place), you don't want to keep waiting there forever.
If you want to wait until a writer opens the pipe, don't use O_NONBLOCK in the open call. If you do use O_NONBLOCK in open, then there might not be anyone at the other end of the pipe and the read calls may just return EOF without blocking.
So in short make sure there's someone at the other end of the pipe when you're reading from it.
I modified a program from APUE, the program first open a file, then mark the fd as non-blocking, then continue write to the fd until write return -1.
I think since disk I/O is slow, when write buffers in OS is nearly full, the write system call will return -1, and the errno should be EAGAIN or EWOULDBLOCK.
But I ran the program for about several minutes and I repeated running the program serveral times, the write system call didn't returned -1 even once! Why?
Here's the code:
#include "apue.h"
#include <errno.h>
#include <fcntl.h>
char buf[4096];
int
main(void)
{
int nwrite;
int fd = open("a.txt", O_RDWR);
if(fd<0){
printf("fd<0\n");
return 0;
}
int i;
for(i = 0; i<sizeof(buf); i++)
buf[i] = i*2;
set_fl(fd, O_NONBLOCK); /* set nonblocking */
while (1) {
nwrite = write(fd, buf, sizeof(buf));
if (nwrite < 0) {
printf("write returned:%d, errno=%d\n", nwrite, errno);
return 0;
}
}
clr_fl(STDOUT_FILENO, O_NONBLOCK); /* clear nonblocking */
exit(0);
}
The O_NONBLOCK flag is primarily meaningful for file descriptors representing streams (e.g, pipes, sockets, and character devices), where it prevents read and write operations from blocking when there is no data waiting to read, or buffers are too full to write anything more at the moment. It has no effect on file descriptors opened to regular files; disk I/O delays are essentially ignored by the system.
If you want to do asynchronous I/O to files, you may want to take a look at the POSIX AIO interface. Be warned that it's rather hairy and infrequently used, though.
I have 10 processes which try open the same file more or less at the same time using open(O_CREAT) call, then delete it. Is there any robust way to find out which process actually did create the file and which did open already create file, for instance, if I want to accurately count how many times that file was opened in such scenario.
I guess I could put a global mutex on file open operation, and do a sequence of open() calls using O_CREAT and O_EXCL flags, but that doesn't fit my definition of "robust".
Use O_EXCL flag with O_CREAT. This will fail if the file exists and errno will be set to EEXIST. If it does fail
then attempt open again without O_CREAT and without O_EXCL modes.
e.g.
int fd = open(path, O_WRONLY | O_CREAT | O_EXCL, 0644);
if ((fd == -1) && (EEXIST == errno))
{
/* open the existing file with write flag */
fd = open(path, O_WRONLY);
}
Based roughly on your comments, you want something along the lines of this function:
/* return the fd or negative on error (check errno);
how is 1 if created, or 0 if opened */
int create_or_open (const char *path, int create_flags, int open_flags,
int *how) {
int fd;
create_flags |= (O_CREAT|O_EXCL);
open_flags &= ~(O_CREAT|O_EXCL);
for (;;) {
*how = 1;
fd = open(path, create_flags);
if (fd >= 0) break;
if (errno != EEXIST) break;
*how = 0;
fd = open(path, open_flags);
if (fd >= 0) break;
if (errno != ENOENT) break;
}
return fd;
}
This solution is not bullet proof. There may be cases (symbolic links maybe?) that would cause it to loop forever. Also, it may live-lock in certain concurrency scenarios. I'll leave resolving such issues as an exercise. :-)
In your edited question, you pose:
I have 10 processes which try open the same file more or less at the same time using open(O_CREAT) call, then delete it.
A hack-ish, but more bullet proof, solution would be to give each process a different user ID. Then, just use the regular open(path, O_CREAT|...) call. You can then query the file with fstat() on the file descriptor, and check the st_uid field of the stat structure. If the field equals the processes' user ID, then it was the creator. Otherwise, it was an opener. This works since each process deletes the file after opening.
I'm not very familiar with C design patterns and searching for the best solution for the following problem. I want to write a little chat client based on libpurple.
While running the program I want to be able to connect and disconnect several instant message accounts. The connect and disconnect calls should be passed over command line, but waiting for input with gets(); is no solution, because the program should run all the time getting new messages from the already connected instant message accounts.
You probably want to use poll (or select) for handling the events. So after establishing the connections, you have the file descriptors, and in addition you have the standard input, which also has a file descriptor from the OS (namely 0), and you can pass all those file descriptors to poll, which notifies you when there is incoming data on any of the file descriptors. Example code:
/* fd1, fd2 are sockets */
while(1) {
pollfd fds[3];
int ret;
fds[0].fd = fd1;
fds[1].fd = fd2;
fds[2].fd = STDIN_FILENO;
fds[0].events = POLLIN;
fds[1].events = POLLIN;
fds[2].events = POLLIN;
ret = poll(fds, 3, -1); /* poll() blocks, but you can set a timeout here */
if(ret < 0) {
perror("poll");
}
else if(ret == 0) {
printf("timeout\n");
}
else {
if(fds[0].revents & POLLIN) {
/* incoming data from fd1 */
}
if(fds[0].revents & (POLLERR | POLLNVAL)) {
/* error on fd1 */
}
if(fds[1].revents & POLLIN) {
/* incoming data from fd2 */
}
if(fds[1].revents & (POLLERR | POLLNVAL)) {
/* error on fd2 */
}
if(fds[2].revents & POLLIN) {
/* incoming data from stdin */
char buf[1024];
int bytes_read = read(STDIN_FILENO, buf, 1024);
/* handle input, which is stored in buf */
}
}
}
You didn't mention the OS. This works for POSIX (OS X, Linux, Windows with mingw). If you need to use the Win32 API, it'll look a bit different but the principle is the same.
Check out select(2). I'm not really sure how libpurple works, but if it allows notification via file-descriptor (like a file or socket), then select is your solution.
You could also try creating a seperate thread with pthread_create(3). That way it can block on gets (or whatever) while the rest of your program does it's thing.