In Linux (running on ARM) there is one process that has an open fd to /dev/watchdog/ and is sending an ioctl() each few seconds as a keep alive:
while (1) {
ioctl(fd, WDIOC_KEEPALIVE, 0);
sleep(10);
}
I want to send the keep alive from another process too, but I can't open another fd to /dev/watchdog/: when I tried to echo to /dev/watchdog/ I get the error "Device or resource busy".
Where I can see that the watchdog is defined to work only with 1 process at a time? (I saw in another Linux that some processes can open fd to /dev/watchdog/).
What can I do to feed the watchdog from 2 processes?
Due to the implementation of /dev/watchdog in the kernel, only one process can use it at the same time, so opening /dev/watchdog from two different processes is not possible.
You can see this right in the source code of the Linux kernel, specifically in drivers/watchdog/watchdog_dev.c. Here's the relevant snippet of code:
/*
* watchdog_open: open the /dev/watchdog* devices.
* #inode: inode of device
* #file: file handle to device
*
* When the /dev/watchdog* device gets opened, we start the watchdog.
* Watch out: the /dev/watchdog device is single open, so we make sure
* it can only be opened once.
*/
static int watchdog_open(struct inode *inode, struct file *file)
{
/* ... */
/* the watchdog is single open! */
if (test_and_set_bit(_WDOG_DEV_OPEN, &wd_data->status))
return -EBUSY;
/* ... */
If you want to feed the watchdog from two different processes, you can work around this issue by creating a simple "master" program that talks to the watchdog while orchestrating the two subprocesses as you wish. This can be done in different ways (pipes, sockets, threads, etc). A single popen() per child process seems like a simple solution.
Here's a working example, master.c:
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <linux/watchdog.h>
int main(int argc, char **argv) {
int watchdog_fd;
FILE *child1_fp, *child2_fp;
if (argc != 3 || !argv[1] || !*argv[1] || !argv[2] || !*argv[2]) {
fprintf(stderr, "Usage: %s 'CHILD_1_COMMAND' 'CHILD_2_COMMAND'\n", argv[0]);
return 1;
}
// Open a fd to talk to the watchdog.
watchdog_fd = open("/dev/watchdog", O_RDWR);
if (watchdog_fd == -1) {
perror("open failed");
return 1;
}
// Start the first process.
child1_fp = popen(argv[1], "r");
if (child1_fp == NULL) {
perror("popen (1) failed");
return 1;
}
// Start the second process.
child2_fp = popen(argv[2], "r");
if (child2_fp == NULL) {
perror("popen (2) failed");
return 1;
}
while (1) {
char tmp;
size_t count;
// Get one byte of data from each of the two processes.
count = fread(&tmp, 1, 1, child1_fp);
count += fread(&tmp, 1, 1, child2_fp);
// If both processes provided the data, ping the watchdog.
if (count == 2) {
if (ioctl(watchdog_fd, WDIOC_KEEPALIVE, 0) < 0)
perror("ioctl failed");
}
}
return 0;
}
And two identical programs a.c and b.c just for testing purposes:
#include <stdio.h>
#include <unistd.h>
int main(void) {
setvbuf(stdout, NULL, _IONBF, 0);
while (1) {
putchar('x');
sleep(10);
}
}
Compile and run:
$ gcc -o master master.c
$ gcc -o a a.c
$ gcc -o b b.c
$ ./master ./a ./b
In the above example code, master pings the watchdog if and only if the two children are alive and running: if one of the two hangs or dies, the master will stop pinging the watchdog. However, it's simple to rework the logic to work differently, and it's also simple to make it work with more than two child processes.
Related
In HA proxy and other load balancing systems, we have a process called reload which forks new process and let it take up new connections and the existing connection will stick to the old process and the old process will stay alive until all the connections are drained.
Is there any way to move the existing connections from the old process to newly forked process and close the old one?
Can someone kindly explain why it cannot be done?
Here is an example adapted from something I did a long time ago.
The trick is to rely on the ancillary data of sendmsg()/recvmsg()
system calls on a local socket to pass an open file descriptor.
This example uses regular files to illustrate fd-passing, but of
course TCP sockets can be used instead (in a non-windows environment).
There are many details that are not so obvious without digging into
the documentation, but once encapsulated into some functions like
send_fd()/receive_fd() in this example, it is pretty straightforward
to use.
Note that, when received, the file descriptor does not necessarily have
the same number as when it was sent.
One should think about it as a kind of dup() between different processes:
file descriptors with different numbers actually refer to the same resource.
Note also that, as soon as the file descriptor has been sent to another
process, it can be closed (exactly like with dup()/dup2()) because
the new file descriptor that will be received by the other process still refers to the original resource.
/**
gcc -std=c99 -o prog_c prog_c.c \
-pedantic -Wall -Wextra -Wconversion \
-Wc++-compat -Wwrite-strings -Wold-style-definition -Wvla \
-g -O0 -UNDEBUG -fsanitize=address,undefined
$ ./prog_c
parent-process: using fd 3
child-process: using fd 4
parent-process: using fd 3
child-process: using fd 4
parent-process: using fd 3
child-process: using fd 4
parent-process: using fd 3
child-process: using fd 4
parent-process: using fd 3
child-process: using fd 4
child-process: done
$ cat file_?.txt
file_0.txt written by parent-process when opening
Now child-process uses this open file.
file_1.txt written by parent-process when opening
Now child-process uses this open file.
file_2.txt written by parent-process when opening
Now child-process uses this open file.
file_3.txt written by parent-process when opening
Now child-process uses this open file.
file_4.txt written by parent-process when opening
Now child-process uses this open file.
**/
#include <sys/types.h>
#include <sys/wait.h>
#include <sys/socket.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
int // 0: success -1: error
send_fd(int control_fd,
int fd_to_send)
{
int dummy_msg=1234;
struct iovec vec;
vec.iov_base=&dummy_msg;
vec.iov_len=sizeof(dummy_msg);
struct msghdr msg;
msg.msg_name=NULL;
msg.msg_namelen=0;
msg.msg_iov=&vec;
msg.msg_iovlen=1;
struct cmsghdr * cmsg;
char buffer[CMSG_SPACE(sizeof(int))];
msg.msg_control=buffer;
msg.msg_controllen=CMSG_SPACE(sizeof(int));
cmsg=CMSG_FIRSTHDR(&msg);
cmsg->cmsg_level=SOL_SOCKET;
cmsg->cmsg_type=SCM_RIGHTS; // fd passing
cmsg->cmsg_len=CMSG_LEN(sizeof(int));
*((int *)CMSG_DATA(cmsg))=fd_to_send; // send new file descriptor
msg.msg_flags=0;
return sendmsg(control_fd, &msg, 0)==sizeof(dummy_msg) ? 0 : -1;
}
int // fd or -1
receive_fd(int control_fd)
{
char buffer[CMSG_SPACE(sizeof(int))];
struct msghdr msg;
struct iovec vec;
struct cmsghdr *cmsg;
int dummy_msg;
vec.iov_base=&dummy_msg;
vec.iov_len=sizeof(dummy_msg);
msg.msg_name=NULL;
msg.msg_namelen=0;
msg.msg_iov=&vec;
msg.msg_iovlen=1;
msg.msg_control=buffer;
msg.msg_controllen=CMSG_SPACE(sizeof(int));
msg.msg_flags=0;
if(recvmsg(control_fd, &msg, 0)!=sizeof(dummy_msg))
{
return -1;
}
int fd=-1;
cmsg=CMSG_FIRSTHDR(&msg); // ancillary data?
if(cmsg&&(cmsg->cmsg_len>=(socklen_t)CMSG_LEN(sizeof(int)))&&
(cmsg->cmsg_level==SOL_SOCKET)&&
(cmsg->cmsg_type==SCM_RIGHTS)) // fd passing?
{
fd=*((int *)CMSG_DATA(cmsg)); // store new file descriptor
}
return fd;
}
int
main(void)
{
int control_pair[2];
if(socketpair(PF_LOCAL, SOCK_STREAM, 0, control_pair)==-1)
{
perror("socketpair");
exit(1);
}
pid_t p=fork();
if(p==-1)
{
perror("fork");
exit(1);
}
if(p==0) // child process
{
close(control_pair[1]); // used by parent-process (arbitrary)
for(;;)
{
int fd=receive_fd(control_pair[0]);
if(fd==-1)
{
printf("child-process: done\n");
break;
}
printf("child-process: using fd %d\n", fd);
const char *text="Now child-process uses this open file.\n";
write(fd, text, strlen(text));
close(fd);
}
close(control_pair[0]);
exit(0); // child-process stops here
}
// parent process
close(control_pair[0]); // used by child-process (arbitrary)
for(int i=0; i<5; ++i)
{
char text[100];
sprintf(text, "file_%d.txt", i);
int fd=open(text, O_WRONLY|O_TRUNC|O_CREAT, 0644);
if(fd==-1)
{
perror("open");
break;
}
printf("parent-process: using fd %d\n", fd);
strcat(text, " written by parent-process when opening\n");
write(fd, text, strlen(text));
if(send_fd(control_pair[1], fd)==-1)
{
perror("send_fd");
close(fd);
break;
}
close(fd);
sleep(1);
}
close(control_pair[1]);
if(waitpid(p, NULL, 0)!=p)
{
perror("waitpid");
exit(1);
}
return 0;
}
I'm practicing C code with pipe system call, it works well with small chunks of data. but as the data goes beyond the pipe capacity, dead lock occurs.
My test system is Debian Sid, but i believe it share the common ground with other Linux distributions. This piece of code works well while the input file '/tmp/a.out' is small enough to fit within the pipe, but blocked as the file is up to 1M.
#include <sys/errno.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <stdio.h>
#define CHUNK 2048
int main() {
int fd=open("/tmp/a.out",O_RDONLY);
int pin[2];
int pout[2];
int nread;
char buff[CHUNK];
pipe(pin);
pipe(pout);
int rc;
pid_t pid=fork();
if (pid == 0) {
close(pin[1]);
dup2(pin[0],STDIN_FILENO);
close(pout[0]);
dup2(pout[1],STDOUT_FILENO);
execlp("cat","cat",(char *)0);
} else if (pid > 0) {
close(pin[0]);
close(pout[1]);
/* I think dead lock occurs here, but i can't figure out a way to avoid it */
while ( (nread=read(fd,buff,CHUNK)) > 0) write(pin[1],buff,nread);
close(pin[1]);
while ( (nread=read(pout[0],buff,CHUNK)) >0) write(STDOUT_FILENO,buff,nread);
waitpid(pid,&rc,0);
exit(rc);
} else {
perror("fork");
exit(errno);
}
}
Any suggestions? I know Python's subprocess class have something like subprocess.communicate() to avoid this kind of dead lock, but i don't know how to deal with it in C.
Many thanks.
The first process pipes into cat and cat pipes back into the first process. Hence, for cat to not block on piping back, the first process must also drain that pipe. E.g.:
fcntl(pout[0], F_SETFL, fcntl(pout[0], F_GETFL) | O_NONBLOCK);
while((nread=read(fd, buff, CHUNK)) > 0) {
write(pin[1], buff, nread); // TODO: check errors and partial writes here.
while((nread=read(pout[0],buff,CHUNK)) > 0) // pout[0] must be set into non-blocking mode.
write(STDOUT_FILENO, buff, nread);
}
A more robust way is to set both pin[1] and pout[0] into non-blocking mode, use select to determine whether pin[1] is ready for write and pout[0] for read and then do write/read correspondingly and handle partial reads and writes.
From your suggestions at least I have 2 ways to solve this problem
1. Setting 'NON-BLOCK' mode by 'fcntl' or 'select/poll/epoll'
Use concurrency such as 'pthread' for stdin pipe
piece of code attached.
struct data {
int from_fd;
int to_fd;
};
and code for pipes should look like
pthread_t t;
struct data d;
d.from_fd=fd;
d.to_fd=pin[1];
pthread_create(&t,NULL,&fd_to_pipe,(void*) &d);
while ( (nread=read(pout[0],buff,CHUNK)) >0) write(STDOUT_FILENO,buff,nread);
waitpid(pid,&rc,0);
pthread_join(t,NULL);
Thank you !
I am trying to create a simple example of named pipes (FIFO). Here, the server will listen for message from the client, which writes on the named pipe, common to both of them. The special thing to be implemented is that the FIFO should be non-blocking (usage of O_NONBLOCK).
By non blocking, I mean that the writer should return immediately after writing, if there is no reader. Similarly, reader should return immediately if there is no message(no writer).
I have created the blocking version though and its working fine. I am then trying to convert it to non-blocking.
Here's the client :
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#define FIFO "/tmp/myFIFO"
/*
This acts as the client, writing to the FIFO
*/
int main(int argc, char *argv[])
{
FILE *fp;
int fifo_fd;
if(argc != 2)
{
printf("Usage : ./fifo_client <message> \n");
exit(1);
}
fifo_fd = open(FIFO, O_WRONLY | O_NONBLOCK);
if(fifo_fd < 0)
{
perror("Error while open call");
exit(1);
}
fp = fdopen(fifo_fd, "w");
if(fp == NULL)
{
perror("Error while opening fd");
exit(1);
}
fputs(argv[1],fp);
/* Close the fp */
fclose(fp);
return 0;
}
Here's the server :
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#define FIFO "/tmp/myFIFO"
/*
This acts as a server waiting for strings to be written by the client, over the FIFO
*/
int main()
{
FILE *fp;
int fifo_fd;
char buf[1024];
/* Create a FIFO */
umask(0);
if(mkfifo(FIFO,0666) < 0) /* 0666 is read and write permission*/
{
perror("Error creating FIFO");
exit(1);
}
while(1) /*endless wait, keep reading strings and print*/
{
fifo_fd = open(FIFO, O_RDONLY | O_NONBLOCK);
if(fifo_fd < 0)
{
perror("Error while open call");
exit(1);
}
fp = fdopen(fifo_fd, "w");
if(fp == NULL)
{
perror("Error while opening fd");
exit(1);
}
if(!fgets(buf,1024,fp))
printf("Nothing to read\n");
else
printf("Message Recieved : %s\n", buf);
fclose(fp);
sleep(1);
}
return 0;
}
I run the server first.
Secondly, on second terminal, when I run the client, I get the error :
Error while open call: No such device or address
What am I missing? I did man, and the parameters seem to be correct.
EDIT
Moving the open and close calls out of the while loop, did the job. But now if the client is started without starting the server, throws the following error :
Error while open call: No such device or address
The file /tmp/myFIFO exists on file-system from previous execution of the server, must be used by the client.
The main problem is opening and closing the file in a loop. This makes no sense. There is a very short time interval between opening and closing, and your client must hit it. It has little chance to do so. The "No such device or address" message happens exactly because the client misses the instant when the file is open. This is the main problem. Try moving open, fopen and fclose out of the server loop.
You also opening for reading but fopening for writing, but I suppose it's just a typo. This combination will not run. You need to change the mode of fopen to "r".
There are other, smaller issues.
You are not checking errors in the client. In your program the client will fail to open most of thee time, but sometimes open will succeed and write will fail.
It makes little sense to use stdio for the pipe in this program. read and write would do just fine.
Last but not least, sleep is an indication of a design issue. Indeed, in this program blocking I/O would make more sense. It's OK to use sleep if you just want to experiment with non-blocking I/O, but in real programs it should be avoided.
When opening the file /dev/urandom in nonblocking mode it is still blocking when reading. Why is the read call still blocking.
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
int main(int argc, char *argv[])
{
int fd = open("/dev/urandom", O_NONBLOCK);
if (fd == -1) {
printf("Unable to open file\n");
return 1;
}
int flags = fcntl(fd, F_GETFL);
if (flags & O_NONBLOCK) {
printf("non block is set\n");
}
int ret;
char* buf = (char*)malloc(10000000);
ret = read(fd, buf, 10000000);
if (ret == -1) {
printf("Error reading: %s\n", strerror(errno));
} else {
printf("bytes read: %d\n", ret);
}
return 0;
}
The output looks like this:
gcc nonblock.c -o nonblock
./nonblock
non block is set
bytes read: 10000000
Opening any (device) file in nonblocking mode does not mean you never need to wait for it.
O_NONBLOCK just says return EAGAIN if there is no data available.
Obviously, the urandom driver always considers to have data available, but isn't necessarily fast to deliver it.
/dev/urandom is non-blocking by design:
When read, the /dev/random device will only return random bytes
within the estimated number of bits of noise in the entropy pool.
/dev/random should be suitable for uses that need very high quality
randomness such as one-time pad or key generation. When the entropy
pool is empty, reads from /dev/random will block until additional
environmental noise is gathered.
A read from the /dev/urandom device will not block waiting for more
entropy. As a result, if there is not sufficient entropy in the
entropy pool, the returned values are theoretically vulnerable to a
cryptographic attack on the algorithms used by the driver.
If you replace it with /dev/random, your program should produce a different result.
In Linux, it is not possible to open regular files in non blocking mode. You have to use the AIO interface to read from /dev/urandom in non blocking mode.
In C pseudo-code:
while (1) {
fifo = open("fifo", O_RDONLY | O_NONBLOCK);
fd_set read;
FD_SET(fifo, &read);
select(nfds, &read, NULL, NULL, NULL);
}
The process sleeps as triggered by select() until another process writes into fifo. Afterwards it will always find fifo as a readable file descriptor.
How to avoid this behavior (that is, after fifo has been read once, how to make it be found as unreadable until it gets another write?)
You opened that FIFO as read only (O_RDONLY), whenever there is no writer to the FIFO, the read end will receive an EOF.
Select system call will return on EOF and for every EOF you handle there will be a new EOF. This is the reason for the observed behavior.
To avoid this open that FIFO for both reading and writing (O_RDWR). This ensures that you have at least one writer on the FIFO thus there wont be an EOF and as a result select won't return unless someone writes to that FIFO.
The simple answer is to read until read() returns EWOULDBLOCK (or EAGAIN), or craps out with an error.
What you are saying simply cannot be happening unless the operating system (or runtime) that you are using is buggy. Otherwise you must be doing something wrong. For example, select() is using level-triggered I/O. I'd think that, most likely, you are not draining the socket completely, and so select() always indicates that you have something left in there (this does not happen with edge-triggered event notifications).
Below is a simple example that shows how one should read until the read() returns EWOULDBLOCK in order to avoid leaving descriptor in readable state (I've compiled and tested this on OS X, and there is also mostly no error checking, but you should get the idea):
/*
* FIFO example using select.
*
* $ mkfifo /tmp/fifo
* $ clang -Wall -o test ./test.c
* $ ./test &
* $ echo 'hello' > /tmp/fifo
* $ echo 'hello world' > /tmp/fifo
* $ killall test
*/
#include <sys/types.h>
#include <sys/select.h>
#include <errno.h>
#include <stdlib.h>
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
int main()
{
int fd;
int n;
fd_set set;
ssize_t bytes;
size_t total_bytes;
char buf[1024];
fd = open("/tmp/fifo", O_RDWR | O_NONBLOCK);
if (fd == -1) {
perror("open");
return EXIT_FAILURE;
}
FD_ZERO(&set);
FD_SET(fd, &set);
for (;;) {
n = select(fd+1, &set, NULL, NULL, NULL);
if (!n)
continue;
if (n == -1) {
perror("select");
return EXIT_FAILURE;
}
if (FD_ISSET(fd, &set)) {
printf("Descriptor %d is ready.\n", fd);
total_bytes = 0;
for (;;) {
bytes = read(fd, buf, sizeof(buf));
if (bytes > 0) {
total_bytes += (size_t)bytes;
} else {
if (errno == EWOULDBLOCK) {
/* Done reading */
printf("done reading (%lu bytes)\n", total_bytes);
break;
} else {
perror("read");
return EXIT_FAILURE;
}
}
}
}
}
return EXIT_SUCCESS;
}
Basically, level-triggered I/O means that you get notified all the time if there is something to read, even though you might have been notified of this before. On a contrary, edge-triggered I/O means that you are getting notified only once every time new data arrives and it doesn't matter whether you read it or not. select() is a level-triggered I/O interface.
Hope it helps. Good Luck!