I'm having a difficult time figuring out how select() is suppose to work with pipes in UNIX. I've scanned the man pages several times, and I don't completely understand the given definition.
From reading the man pages, I was under the impression that select() would make the system wait until one of the file descriptors given could make a read (in my case) from a pipe without blocking.
Here's some of my outline code[EDITED]:
int size, size2;
fd_set rfds;
struct timeval tv;
char buffer[100];
char buffer2[100];
int retval;
while(1)
{
FD_ZERO(&rfds);
FD_SET(fd[0], &rfds);
FD_SET(fd2[0], &rfds);
tv.tv_sec = 2;
tv.tv_usec = 0;
retval = select(2, &rfds, NULL, NULL, &tv); //2 seconds before timeout
if(retval == -1)
perror("Select failed.\n");
else if(retval)
{
size = read(fd[0], buffer, sizeof(buffer));
if(size > 0)
printf("Parent received from even: %s\n", buffer);
size2 = read(fd2[READ], buffer2, sizeof(buffer2));
if(size2 > 0)
printf("Parent received from odd: %s\n", buffer2);
}
else
printf("No data written to pipe in 2 last seconds.\n");
}
I have two pipes here. Two children processes are writing to their respective pipes and the parent has to read them both in.
As a test, I write a small string to each pipe. I then attempt to read them in and prevent blocking with select. The only thing that gets printed out is the string from the even pipe. It appears to still be blocking. I am becoming frustrated as I feel like I'm missing something on the man pages. Could someone tell me what I'm doing wrong?
After select() returns, 0 or more of your file descriptors will be in a "ready" state where you can read them without blocking. But if you read one that's not ready, it will still block. Right now you are reading all of them, and since select() only waits until one is ready, it's very likely that another will not be.
What you need to do is figure out which ones are ready, and only read() from them. The return value of select() will tell you how many are ready, and you can ask if a specific one is ready with the ISSET() macro.
You need to use FD_ZERO() - see select before the FD_SET
Set the timeout values just before the select. Those values are changed through the select
Related
I have the following simplified code template:
pid_t pid;
int pipe1[2], pipe2[2];
pid = fork();
pipe(pipe1); pipe(pipe2)
if(pid == 0) //child
{
read(pipe1[0],...);
write(pipe1[1],...);
close(pipe1[0]);
close(pipe1[1]);
close(pipe2[1]);
read(pipe2[0]...);
}
else //parent
{
write(pipe1[1],...);
wait(NULL);
read(pipe1[0]...);
close(pipe1[0]);
close(pipe1[1]);
close(pipe2[0]);
write(pipe2[1]...);
}
If I am not using the pipe2 in parent and child, the code works perfectly, but if I do, it seems like the child has nothing to read(the program does nothing until I intrerrupt it). Also, is there a way to use only one pipe for more than 2 read/writes? I tried using wait(NULL) more than once but that didn't work.
Simply put, your code template is garbage. Let me explain why.
Each pipe is unidirectional.
If you use a pipe to send data from the child to the parent, close the read end in the child, and the write end in the parent. This allows the parent to see when the child (write end) closes the pipe or exits, as read() will then return -1 with errno == EPIPE.
If you use a pipe to send data from the parent to the child, close the read end in the parent, and the write end in the child. This allows the parent to detect if the child exits prematurely, as write() will then return with -1 with errno == EPIPE and a SIGPIPE signal gets raised in the parent.
If you need bidirectional "pipe" between the parent and a child, use an Unix domain stream socket pair via socketpair(AF_UNIX, SOCK_STREAM, 0, fdpair).
Such a socket pair works very much like a pipe, except that the socket pair is bidirectional. You can also use send(descriptor, buffer, length, MSG_NOSIGNAL) instead of write(descriptor, buffer, length); in the former case, no SIGPIPE signal is raised if the other end of the socket is already closed.
Use one of the descriptors in the parent, and the other in the child. Both parent and child should close the other descriptor. Otherwise, one end cannot detect when the other end has closed its descriptor.
In some cases, an Unix Domain datagram socket pair may be preferable. Each send() generates a separate datagram, that is received using a single recv(). (That is, message boundaries are retained.) If the receiving end knows the maximum size of a datagram the sending side might send, this is an extremely robust and simple way to implement bidirectional communications between a parent and a child process; I personally use it a lot.
read() and write() to pipes and sockets may be short.
(POSIX states that you should always be able to stuff at least 512 bytes into a pipe, though; Linux supports that at least up to a full page, if I recall correctly.)
This means that rather than a single call, you need to do a loop until you have as much data as you need.
With sockets, send() either sends all the data, or fails with -1 (with errno == EMSGSIZE, or some other error code).
For datagram sockets (Unix domain datagram sockets, UDP sockets), if the buffer is large enough, recv() either receives the entire datagram, or fails with -1 (with errno set). Receiving zero-length datagrams is iffy, so don't try to do that.
For stream sockets, recv() may return only some of the data (i.e. short, or partial receive).
When two processes both send and receive, or read and write, data to/from each other, deadlock is a serious, common problem.
Simply put, both ends may end up waiting for the other end to read/write at the same time, with nothing at all happening.
There are three typical solutions to avoid a deadlock in such situations:
Use a query - response protocol, so that one endpoint always initiates communications, and then waits for the other endpoint to respond. At most one endpoint is transferring data at any given time.
Use nonblocking/asynchronous I/O. That is, before trying to write()/send(), each endpoint does a read()/recv() to see if the other end has already sent something. This supports full duplex communications (information can flow both ways at the same time).
Use a separate thread to continuously read()/recv(), while another does write()/send(). This essentially separates each socket into two unidirectional "lanes", with one thread handling their direction only. This is useful for protocols where one end produces a lot of data, and the other end sends occasional commands.
Combining all the above, we'll find that there is no single template one should use. There are variants with significant differences, making them better in some use cases but harder/worse in others. One should pick one depending on the situation at hand. If OP wants to see a better example ("template"), they should describe an actual problem case, including the desired behaviour.
you have 2 mistakes in your code.
you created a dead lock with your wait statement. your parents waits on the child while the child waits for the parents to write something.
you did fork before you did pipe. therefore, your child and your parent see different versions of unconnected pipes.
Here is the fixed version of your program:
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <string.h>
#include <sys/wait.h>
int main() {
pid_t pid;
int pipe1[2], pipe2[2];
char *m1 = "hello";
char *m2 = "world";
char *m3 = "bye";
pipe(pipe1);
pipe(pipe2);
pid = fork();
if(pid == 0) { //chil
char buf1[256], buf2[256];
int len;
len = read(pipe1[0], buf1, 255);
buf1[len] = 0;
write(pipe1[1], m2, strlen(m2));
close(pipe1[0]);
close(pipe1[1]);
close(pipe2[1]);
len = read(pipe2[0], buf2, 255);
buf2[len] = 0;
printf("child read %s %s\n", buf1, buf2);
}
else {
char buf[256];
int len;
write(pipe1[1], m1, strlen(m1));
//wait(NULL);
len = read(pipe1[0], buf, 255);
buf[len] = 0;
close(pipe1[0]);
close(pipe1[1]);
close(pipe2[0]);
write(pipe2[1], m3, strlen(m3));
wait(NULL);
printf("Parent read %s\n", buf);
}
return 0;
}
I am trying to understand how epoll() is different from select() and poll(). select() and poll() are pretty similar. select() allows you to monitor multiple file descriptors and it checks if any of those file descriptors are available for an operation (e.g. read, write) without blocking. When the timeout expires, select() returns the file descriptors that are ready and the program can perform the operations on those file descriptors without blocking.
...
FD_ZERO(&rfds);
FD_SET(0, &rfds);
/* Wait up to five seconds. */
tv.tv_sec = 5;
tv.tv_usec = 0;
retval = select(1, &rfds, NULL, NULL, &tv);
/* Don’t rely on the value of tv now! */
if (retval == -1)
perror("select()");
else if (retval)
printf("Data is available now.\n");
/* FD_ISSET(0, &rfds) will be true. */
else
printf("No data within five seconds.\n");
...
poll() is a little more flexible in that it does not rely on bitmap, but array of file descriptors. Also, since poll() uses separate fields for requested (events) and result (revents), you don't have to worry to refill the sets that were overwritten by kernel.
...
struct pollfd fds[2];
fds[0].fd = open("/dev/dev0", ...);
fds[1].fd = open("/dev/dev1", ...);
fds[0].events = POLLOUT | POLLWRBAND;
fds[1].events = POLLOUT | POLLWRBAND;
ret = poll(fds, 2, timeout_msecs);
if (ret > 0) {
for (i=0; i<2; i++) {
if (fds[i].revents & POLLWRBAND) {
...
However, I read that there is an issue with poll() too since both select() and poll() are stateless; the kernel does not internally maintain the requested sets. I read this:
Suppose that there are 10,000 concurrent connections. Typically, only
a small number of file descriptors among them, say 10, are ready to
read. The rest 9,990 file descriptors are copied and scanned for no
reason, for every select()/poll() call. As mentioned earlier, this
problem comes from the fact that those select()/poll() interfaces are
stateless.
I don't understand what is meant by the file descripters are "copied" and "scanned". Copied where? And I don't know what is meant by "stateless". Thanks for clarification.
"Stateless" means "Does not retain anything between two calls". So kernel need to rebuild many things for mainly nothing in the mentioned example.
I am trying to implement a client-server type of communication system using the poll function in C. The flow is as follows:
Main program forks a sub-process
Child process calls the exec function to execute some_binary
Parent and child send messages to each other alternately, each message that is sent depends on the last message received.
I tried to implement this using poll, but ran into problems because the child process buffers its output, causing my poll calls to timeout. Here's my code:
int main() {
char *buffer = (char *) malloc(1000);
int n;
pid_t pid; /* pid of child process */
int rpipe[2]; /* pipe used to read from child process */
int wpipe[2]; /* pipe used to write to child process */
pipe(rpipe);
pipe(wpipe);
pid = fork();
if (pid == (pid_t) 0)
{
/* child */
dup2(wpipe[0], STDIN_FILENO);
dup2(rpipe[1], STDOUT_FILENO);
close(wpipe[0]); close(rpipe[0]);
close(wpipe[1]); close(rpipe[1]);
if (execl("./server", "./server", (char *) NULL) == -1)
{
fprintf(stderr, "exec failed\n");
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
else
{
/* parent */
/* close the other ends */
close(wpipe[0]);
close(rpipe[1]);
/*
poll to check if write is good to go
This poll succeeds, write goes through
*/
struct pollfd pfds[1];
pfds[0].fd = wpipe[1];
pfds[0].events = POLLIN | POLLOUT;
int pres = poll(pfds, (nfds_t) 1, 1000);
if (pres > 0)
{
if (pfds[0].revents & POLLOUT)
{
printf("Writing data...\n");
write(wpipe[1], "hello\n", 6);
}
}
/*
poll to check if there's something to read.
This poll times out because the child buffers its stdout stream.
*/
pfds[0].fd = rpipe[0];
pfds[0].events = POLLIN | POLLOUT;
pres = poll(pfds, (nfds_t) 1, 1000);
if (pres > 0)
{
if (pfds[0].revents & POLLIN)
{
printf("Reading data...\n");
int n = read(rpipe[0], buffer, 1000);
buffer[n] = '\0';
printf("child says:\n%s\n", buffer);
}
}
kill(pid, SIGTERM);
return EXIT_SUCCESS;
}
}
The server code is simply:
int main() {
char *buffer = (char *) malloc(1000);
while (scanf("%s", buffer) != EOF)
{
printf("I received %s\n", buffer);
}
return 0;
}
How do I prevent poll calls from timing out because of buffering?
EDIT:
I would like the program to work even when the execed binary is external, i.e., I have no control over the code - like a unix command, e.g., cat or ls.
There seem to be two problems in your code. "stdout" is by default buffered,
so the server should flush it explicitly:
printf("I received %s\n", buffer);
fflush(stdout);
And the main program should not register for POLLOUT when trying to read
(but you may want register for POLLERR):
pfds[0].fd = rpipe[0];
pfds[0].events = POLLIN | POLLERR;
With these modifications you get the expected output:
$ ./main
Writing data...
Reading data...
child says:
I received hello
Generally, you should also check the return value of poll(), and repeat the call if
necessary (e.g. in the case of an interrupted system call or timeout).
You need, as I answered in a related answer to a previous question by you, to implement an event loop; as it name implies, it is looping, so you should code in the parent process:
while (1) { // simplistic event loop!
int status=0;
if (waitpid(pid, &status, WNOHANG) == pid)
{ // clean up, child process has ended
handle_process_end(status);
break;
};
struct pollpfd pfd[2];
memset (&pfd, 0, sizeof(pfd)); // probably useless but dont harm
pfd[0].fd = rpipe[0];
pfd[0].events = POLL_IN;
pfd[1].fd = wpipe[1];
pfd[0].event = POLL_OUT;
#define DELAY 5000 /* 5 seconds */
if (poll(pfd, 2, DELAY)>0) {
if (pfd[0].revents & POLL_IN) {
/* read something from rpipe[0]; detect end of file;
you probably need to do some buffering, because you may
e.g. read some partial line chunk written by the child, and
you could only handle full lines. */
};
if (pfd[1].revents & POLL_OUT) {
/* write something on wpipe[1] */
};
}
fflush(NULL);
} /* end while(1) */
you cannot predict in which order the pipes are readable or writable, and this can happen many times. Of course, a lot of buffering (in the parent process) is involved, I leave the details to you.... You have no influence on the buffering in the child process (some programs detect that their output is or not a terminal with isatty).
What an event polling loop like above gives you is to avoid the deadlock situation where the child process is blocked because its stdout pipe is full, while the parent is blocked writing (to the child's stdin pipe) because the pipe is full: with an event loop, you read as soon as some data is polled readable on the input pipe (i.e. the stdout of the child process), and you write some data as soon as the output pipe is polled writable (i.e. is not full). You cannot predict in advance in which order these events "output of child is readable by parent" and "input of child is writable by parent" happen.
I recommend reading Advanced Linux Programming which has several chapters explaining these issues!
BTW my simplistic event loop is a bit wrong: if the child process terminated and some data remains in its stdout pipe, its reading is not done. You could move the waitpid test after the poll
Also, don't expect that a single write (from the child process) into a pipe would trigger one single read in the parent process. In other words, there is no notion of message length. However, POSIX knows about PIPE_MAX .... See its write documentation. Probably your buffer passed to read and write should be of PIPE_MAX size.
I repeat: you need to call poll inside your event loop and very probably poll will be called several times (because your loop will be repeated many times!), and will report readable or writable pipe ends in an unpredictable (and non-reproducible) order! A first run of your program could report "rpipe[0] readable", you read 324 bytes from it, you repeat the event loop, poll says you "wpipe[1] writable", you can write 10 bytes to it, you repeat the event loop, poll tells that "rpipe[0] readable", you read 110 bytes from it, you repeat the event loop, poll tells again "rpipe[0] readable", you read 4096 bytes from it, etc etc etc... A second run of the same program in the same environment would give different events, like: poll says that "wpipe[1] writable", you write 1000 bytes to it, you repeat the loop, poll says that "rpipe[0] readable, etc....
NB: your issue is not the buffering in the child ("client") program, which we assume you cannot change. So what matters is not the buffered data in it, but the genuine input and output (that is the only thing your parent process can observe; internal child buffering is irrelevant for the parent), i.e. the data that your child program has been able to really read(2) and write(2). And if going thru a pipe(7), such data will become poll(2)-able in the parent process (and your parent process can read or write some of it after POLL_IN or POLL_OUT in the updated revents field after poll). BTW, if you did code the child, don't forget to call fflush at appropriate places inside it.
I am trying to make a simple client-server chat program. On the client side I spin off another thread to read any incomming data from the server. The problem is, I want to gracefully terminate that second thread when a person logs out from the main thread. I was trying to use a shared variable 'running' to terminate, problem is, the socket read() command is a blocking command, so if I do while(running == 1), the server has to send something before the read returns and the while condition can be checked again. I am looking for a method (with common unix sockets only) to do a non-blocking read, basically some form of peek() would work, for I can continually check the loop to see if I'm done.
The reading thread loop is below, right now it does not have any mutex's for the shared variables, but I plan to add that later don't worry! ;)
void *serverlisten(void *vargp)
{
while(running == 1)
{
read(socket, readbuffer, sizeof(readbuffer));
printf("CLIENT RECIEVED: %s\n", readbuffer);
}
pthread_exit(NULL);
}
You can make socket not blockable, as suggested in another post plus use select to wait input with timeout, like this:
fd_set input;
FD_ZERO(&input);
FD_SET(sd, &input);
struct timeval timeout;
timeout.tv_sec = sec;
timeout.tv_usec = msec * 1000;
int n = select(sd + 1, &input, NULL, NULL, &timeout);
if (n == -1) {
//something wrong
} else if (n == 0)
continue;//timeout
if (!FD_ISSET(sd, &input))
;//again something wrong
//here we can call not blockable read
fcntl(socket, F_SETFL, O_NONBLOCK);
or, if you have other flags:
int x;
x=fcntl(socket ,F_GETFL, 0);
fcntl(socket, F_SETFL, x | O_NONBLOCK);
then check the return value of read to see whether there was data available.
note: a bit of googling will yield you lots of full examples.
You can also use blocking sockets, and "peek" with select with a timeout. It seems more appropriate here so you don't do busy wait.
The best thing is likely to get rid of the extra thread and use select() or poll() to handle everything in one thread.
If you want to keep the thread, one thing you can do is call shutdown() on the socket with SHUT_RDWR, which will shut down the connection, wake up all threads blocked on it but keep the file descriptor valid. After you have joined the reader thread, you can then close the socket. Note that this only works on sockets, not on other types of file descriptor.
Look for function setsockopt with option SO_RCVTIMEO.
Is there a way to timeout a reading from stdin in order for the program not to hang for too long ?
read(0, var, numberofbytes);
You can use ncurses or if you don't want to, you can use select as described in this blog post. Basically, you can use select and specify the timeout. If the stdin FD is set, then you can read from it safely and won't block. If you want more info on select, check this out and of course wikipedia. It's a handy call to know about.
EDIT: I feel compelled to supply code, so here it is, straight from the blog post with some comments.
// if != 0, then there is data to be read on stdin
int kbhit()
{
// timeout structure passed into select
struct timeval tv;
// fd_set passed into select
fd_set fds;
// Set up the timeout. here we can wait for 1 second
tv.tv_sec = 1;
tv.tv_usec = 0;
// Zero out the fd_set - make sure it's pristine
FD_ZERO(&fds);
// Set the FD that we want to read
FD_SET(STDIN_FILENO, &fds); //STDIN_FILENO is 0
// select takes the last file descriptor value + 1 in the fdset to check,
// the fdset for reads, writes, and errors. We are only passing in reads.
// the last parameter is the timeout. select will return if an FD is ready or
// the timeout has occurred
select(STDIN_FILENO+1, &fds, NULL, NULL, &tv);
// return 0 if STDIN is not ready to be read.
return FD_ISSET(STDIN_FILENO, &fds);
}
Use select, poll or any other IO multiplexing facility. They all take a timeout argument.
Note that this will not work if stdin is a regular file, but it will if stdin is a terminal/tty,socket,pipe.
e.g.
fd_set selectset;
struct timeval timeout = {10,0}; //timeout of 10 secs.
int ret;
FD_ZERO(&selectset);
FD_SET(0,&selectset);
ret = select(1,&selectset,NULL,NULL,&timeout);
if(ret == 0)
//timeout
else if(ret == -1)
//error
else
// stdin has data, read it
// (we know stdin is readable, since we only asked for read events
//and stdin is the only fd in our select set.
Call alarm() or ualarm() before calling read(). This will cause a SIGALRM signal to be delivered to the process, interrupting the read(), provided that you haven't told the O/S to restart system calls after interrupts. Be sure to cancel the alarm if the read() returns normally.