I've to read program log file and to do that I wanted to use select() and read()
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/select.h>
#include <fcntl.h>
#include <stdio.h>
int main()
{
int fd = -1;
fd_set fds;
struct timeval tv;
int rc;
char buffer[4096];
char * log_path = "/home/mich/a.txt";
if((fd = open(log_path,O_RDONLY) ) == -1 )
{
printf("error\n");
return -1;
}
while(1)
{
FD_ZERO(&fds);
FD_SET(fd,&fds);
tv.tv_sec = 2;
tv.tv_usec = 0;
rc = select(fd+1, &fds, NULL, NULL, &tv);
if (rc < 0) {
printf("failed\n");
continue;
} else if (rc > 0 && FD_ISSET(fd,&fds)) {
printf("read\n");
} else {
printf("timeout\n");
continue;
}
while ((my_getline(buffer,sizeof(buffer),fd)) > 0)
{
printf("%s",buffer);
}
}
close(fd);
}
my_getline is a function which uses read().
Output:
read
aaa
read
read
read
read
read
bbb
read
read
read
read
...
where aaa and bbb are lines from read file.
What is wrong in this program?
select() tells you that a read() won't block.
That includes the case where it will return 0 to indicate end-of-file, which presumably is what you're getting.
select tells you that there is something waiting on fd. This doesn't have to be a line, it could be a single byte, or the information that the end of the file has been reached. Add puts("####"); at the end of the outer while loop to see how your code processes the input file.
To see how to keep reading a file that other processes are appending to, look at the source code for one of the free implementations of the unix tail utility (specifically its -f option). The traditional implementation is to read the whole file, sleep for a small interval (1 second in tail), and repeat from the position where you left off after the last read. A more modern, more reactive, less resource-consuming, but less portable approach uses a file change notification API (inotify under Linux, kqueue under *BSD, ...).
Related
I am trying to write to a file and display the output of the thing i wrote with another process. The code i come up with:
void readLine (int fd, char *str) {
int n;
do {
n = read (fd, str, 1);
} while (*str++ != '\0');
}
int main(int argc,char ** argv){
int fd=open("sharedFile",O_CREAT|O_RDWR|O_TRUNC,0600);
if(fork()==0){
char buf[1000];
while(1) {
readLine(fd,buf);
printf("%s\n",buf);
}
}else{
while(1){
sleep(1);
write(fd,"abcd",strlen("abcd")+1);
}
}
}
the output i want (each result spaced from the other with a period of one second):
abcd
abcd
abcd
....
Unfortunately this code doesn't work, it seems that the child process (the reader of the file "sharedFile") reads junk from the file because somehow it reads values even when the file is empty.
When trying to debug the code, readLine function never reads the written file correctly,it always reads 0 bytes.
Can someone help?
First of all, when a file descriptor becomes shared after forking, both the parent and child are pointing to the same open file description, which means in particular that they share the same file position. This is explained in the fork() man page.
So whenever the parent writes, the position is updated to the end of the file, and thus the child is always attempting to read at the end of the file, where there's no data. That's why read() returns 0, just as normal when you hit the end of a file.
(When this happens, you should not attempt to do anything with the data in the buffer. It's not that you're "reading junk", it's that you're not reading anything but are then pretending that whatever junk was in the buffer is what you just read. In particular your code utterly disregards the return value from read(), which is how you're supposed to tell what you actually read.)
If you want the child to have an independent file position, then the child needs to open() the file separately for itself and get a new fd pointing to a new file description.
But still, when the child has read all the data that's currently in the file, read() will again return 0; it won't wait around for the parent to write some more. The fact that some other process has a file open for writing don't affect the semantics of read() on a regular file.
So what you'll need to do instead is that when read() returns 0, you manually sleep for a while and then try again. When there's more data in the file, read() will return a positive number, and you can then process the data you read. Or, there are more elegant but more complicated approaches using system-specific APIs like Linux's inotify, which can sleep until a file's contents change. You may be familiar with tail -f, which uses some combination of these approaches on different systems.
Another dangerous bug is that if someone else writes text to the file that doesn't contain a null byte where expected, your child will read more data than the buffer can fit, thus overrunning it. This can be an exploitable security vulnerability.
Here is a version of the code that fixes these bugs and works for me:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
void readLine (int fd, char *str, size_t max) {
size_t pos = 0;
while (pos < max) {
ssize_t n = read(fd, str + pos, 1);
if (n == 0) {
sleep(1);
} else if (n == 1) {
if (str[pos] == '\0') {
return;
}
pos++;
} else {
perror("read() failure");
exit(2);
}
}
fprintf(stderr, "Didn't receive null terminator in time\n");
exit(2);
}
int main(int argc, char ** argv){
int fd=open("sharedFile", O_CREAT|O_RDWR|O_TRUNC, 0600);
if (fd < 0) {
perror("parent opening sharedFile");
exit(2);
}
pid_t pid = fork();
if (pid == 0){
int newfd = open("sharedFile", O_RDONLY);
if (newfd < 0) {
perror("child opening sharedFile");
exit(2);
}
char buf[1000];
while (1) {
readLine(newfd, buf, 1000);
printf("%s\n",buf);
}
} else if (pid > 0) {
while (1){
sleep(1);
write(fd,"abcd",strlen("abcd")+1);
}
} else {
perror("fork");
exit(2);
}
return 0;
}
I have a loop. Inside this loop I am trying to detect if a read or write is triggered on a named pipe (FIFO) file by using select().
If a read is triggered I call read() on the FIFO file descriptor.
If a write is triggered I call write() on the FIFO file descriptor.
The issue is that if a write occurs and I write to the FIFO, it will trigger a read. And then when I read from the FIFO it will trigger a write. Causing an infinite loop.
This loop occurs immediately if I use the same file descriptor in mode O_RDWR. This loop occurs after the first write if I create a separate file descriptor for both reading and writing.
#include <errno.h>
#include <sys/select.h>
#include <stdlib.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdbool.h>
#include <unistd.h>
#include <string.h>
int main() {
// Open export fifo
int fd = open("./foo-fifo", O_RDWR | O_CREAT);
if (fd < 0) { // Failed to open
perror("error opening fifo");
}
// Read or write fifo until "quit" is in buffer
while (true) {
fd_set read_fds;
fd_set write_fds;
FD_ZERO(&read_fds);
FD_SET(fd, &read_fds);
FD_ZERO(&write_fds);
FD_SET(fd, &write_fds);
int num_fds = select(fd+1, &read_fds, &write_fds, NULL, NULL);
if (num_fds < 0) { // Failed to select
perror("failed to select fifo fd");
} else if (num_fds == 0) { // Timeout
continue;
}
// If read
if (FD_ISSET(fd, &read_fds)) {
char buf[1000] = "";
if (read(fd, buf, sizeof(buf)) < 0) {
perror("error reading fifo");
}
printf("read: \"%s\"\n", buf);
if (strcmp(buf, "quit\n") == 0) {
break;
}
}
// If write
if (FD_ISSET(fd, &write_fds)) {
char *buf = "foo";
if (write(fd, buf, sizeof(buf)) < 0) {
perror("error writing fifo");
}
printf("write: \"%s\"\n", buf);
}
}
// Close fifo
if (close(fd) < 0) { // Failed to close
perror("failed to close export fifo");
}
return 0;
}
Run the example by downloading the code from here (GitHub Gist). Then run:
gcc -o fifo fifo.c
./fifo
The output will show a loop between reading and writing:
write: "foo"
read: ""
write: "foo"
read: ""
write: "foo"
...
Note: This is prefaced by my top comments.
We need two processes (e.g. a server and a client).
fifos are single direction (a writer and a reader), not like a socket.
So, to do this with fifos, you'll need two of them. (e.g.) Given processes A and B, we need two pipes/fifos: pipeAB and pipeBA.
Process A writes to pipeAB and B reads from pipeAB.
Process B writes to pipeBA and A reads from pipeBA
If you want to use a socket, you could do a PF_UNIX (aka AF_UNIX) socket. See man 7 unix and man 2 socketpair.
Or, you could do a full blown AF_INET socket with the host set to localhost with some fixed port number.
As an exercise [for you], consider doing it in several ways. That is, an argv option like -Tp for dual pipes, -Tu for AF_UNIX, and -Ts for AF_INET, etc. Only the initialization would be different. The protocol would be nearly identical otherwise.
For AF_UNIX sockets, if the client and server are different programs, it may be easier to create a file of type socket in the file system. This can be done by filling in a struct sockaddr_un with the "filename" and then using bind after the socket call. See: https://www.ibm.com/support/knowledgecenter/en/SSB23S_1.1.0.13/gtpc1/unixsock.html for an example
My project is to fork and then use the parent process to read data from a file line by line and then send each line to the child process, which has to use execve to send the line as an argument for bc, and the output has to go back to the parent process. Right now, I'm just trying to send the data to the child and receive it properly, but it doesn't work. I have to use select to figure out if the child has output for the parent to get.
I have a file with 5 lines on it, and I use a while loop to go through the file. For each line I thought I would get the line back from the child, but it only does one line or two and stops. Then I get the same line twice for some reason.
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include <sys/types.h>
#include <errno.h>
#include <wait.h>
int main(int argc, char *argv[])
{
alarm(60);
fd_set rfds;
fd_set r2fds;
struct timeval tv;
int retval;
int retval2;
int i = argc;
int rc;
FILE *fp;
fp = fopen (argv[1],"r");
char *args[3];
int j = 0;
while (j < i)
{
args[j] = argv[j+2];
j++;
}
int stdin_pipe_fds[2], stdout_pipe_fds[2], stderr_pipe_fds[2];
pipe(stdin_pipe_fds);
pipe(stdout_pipe_fds);
pipe(stderr_pipe_fds);
rc = fork();
if (rc == -1)
{
while (rc == -1)
{
rc = fork();
}
}
pid_t child;
pid_t parent;
if (rc == 0)
{
child = getpid();
close(stdin_pipe_fds[1]);
close(stdout_pipe_fds[0]);
close(stdout_pipe_fds[0]);
close(0);
dup(stdin_pipe_fds[0]);//, 0);
close(stdin_pipe_fds[0]);
close(1);
dup(stdout_pipe_fds[1]);//,1);
close(stdout_pipe_fds[1]);
close(2);
dup(stderr_pipe_fds[1]);//,2);
close(stderr_pipe_fds[1]);
}
if (rc > 0)
{
parent = getpid();
close(stdin_pipe_fds[0]);
close(stdout_pipe_fds[1]);
close(stderr_pipe_fds[1]);
}
char str[100];
char buf2[100];
char buf[100];
FD_ZERO(&rfds);
FD_SET(stdout_pipe_fds[0], &rfds);
tv.tv_sec = 1;
tv.tv_usec = 0;
while ((fgets(str,100,fp)) != NULL)
{
if (rc > 0)
{
int wstatus;
int wsta;
int status;
wsta = write(stdin_pipe_fds[1],str,strlen(str));
retval = select(FD_SETSIZE, &rfds, NULL, NULL, &tv);
if (FD_ISSET(stdout_pipe_fds[0], &rfds))
{
wstatus = read(stdout_pipe_fds[0], buf2, 100);
printf("From child: %s\n",buf2);
if (wstatus == -1)
{
printf("read failed\n");
//continue;
}
//wsta = write(stdin_pipe_fds[1],str,strlen(str));
}
}
if (rc == 0)
{
alarm(60);
scanf("%s",buf);
printf("%s", buf);
}
}
fclose(fp);
}
In the child
you close stdout_pipe_fds[0] twice, and not stderr_pipe_fds[0].
The scanf("%s", buf) will strip the newline from the string; you might not want that. Stdout, if you started from a terminal, will be line buffered; that is, it will take a newline to trigger an actual write. Since you snuck in and replaced the underlying FD, it doesn't know that a new buffering strategy may be appropriate.
In the parent:
you treat read and write as if they return a status; they return the number of bytes read or written. If it reads 0 bytes, the buf will contain the values from the previous read. I am not sure of the purpose of the read(), I would assume it just messes things up. I think you need to draw a picture of how the file descriptors are linked.
Suggestions: There is a generally accepted idiom when using fork:
if ((rc = fork()) == 0) {
/* do child stuff */
} else if (rc != -1) {
/* do parent stuff */
} else {
/* do error stuff */
}
Which is mainly followed to avoid warping peoples brains. It is really hard to read when it is interleaved. The "K" in K&R once quipped that "... if you're as clever as you can be when you write it, how will you ever debug it?"
The close(0); dup(); is much better expressed as dup2(fd, 0). The mere fact that you can compile code on your machine assures that dup2 functions correctly.
Running this snippet of code to experiment with piping and signals. I'm trying to learn how to properly utilize the select() function between pipes.
This process will fork. If there is something to be read from stdIn it is then written to the write end of the pipe. It is supposed to execute either a basic command entered via terminal, or it runs hard-coded commands in the code. (It's running hard code right now as "ls.")
When I run this snippet, it should quit and stop running completely when I press the letter "q" followed by ENTER, or it should quit after it runs its assigned process.
Instead, even after I hit "q" or run the process it won't stop the program completely. It is still waiting for input. It will stop running once I have hit ENTER, but it never even executes my process.
For example, if I compile and run this as "./test ls" or even just run "./test" (because ls is hard-coded in so that SHOULD just run I think), it will not run the command ls. And the program will continue to run until I've hit ENTER again.
I'm certain my rudimentary understanding of select() has to do with this issue. I'm pretty sure my select() statement needs to break at some point but I don't know what or how to check for this.
I was told that there is a method WIFEXITED() that might be able to help me but I'm just not sure how it applies in this context.
I also would like to know how to check if your pipes are empty!
I DO know that I want this to be able to both take input from the terminal and record it and also run built in functions.
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdlib.h>
#include <string.h>
#include <sys/select.h>
#include <stdio.h>
int main() {
int in[2]; // parent writes; child reads
pipe(in);
if (fork() == 0) {
// instantiate the values that will be execed
char *a[2];
//a[0] = "./test3";
a[0] = "ls";
a[1] = NULL;
// redirects what is being read from stdin to the pipe
//this redirection is for a separate test that is not included
close(in[1]);
close(0);
dup(in[0]);
close(in[0]); // close read
execvp(a[0], a);
}
else {
close(in[0]); // only want parent to write
// select() params
int nfds = 3;
int check = -1;
int done = 0;
fd_set readfds;
fd_set writefds;
FD_ZERO(&readfds); // set select params
FD_SET(0, &readfds);
FD_SET(in[1], &writefds);
while ((check = select(nfds, &readfds, &writefds, NULL, NULL)) > 0) {
int size = 0;
char buf[1000];
// write to pipe for child
if (FD_ISSET(0, &readfds) && FD_ISSET(in[1], &writefds)) {
while ((size = read(0, buf, sizeof(buf))) != 0) {
write(in[1], buf, size);
}
}
// reset
FD_ZERO(&readfds);
FD_SET(0, &readfds);
FD_SET(in[1], &writefds);
}
printf("%d --------------- %d\n", (FD_ISSET(in[1], &writefds)),
FD_ISSET(0, &readfds));
}
return 0;
}
Here is the potential set of test code that can be run with the above snipped if a[0] = ./test is uncommented and a[0] = ls is commented.
#include <sys/time.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <string.h>
#include <sys/select.h>
int main () {
int fd;
char buf[11];
int ret, sret;
int flag = 0;
fd = 0;
fd_set readfds;
struct timeval timeout;
while(1) {
FD_ZERO(&readfds);
FD_SET(fd, &readfds);
timeout.tv_sec = 5;
timeout.tv_usec = 0;
sret = select(fd + 1, &readfds, NULL, NULL, NULL);
memset((void *) buf, 0, 11);
ret = read(fd, (void *) buf, 10);
flag = strcmp(buf, "q\n") == 0;
if (flag) {
return 0;
}
printf("ret = %d\n", ret);
if(ret != -1) {
printf(" buf = %s\n", buf);
}
}
return 0;
}
I used a FIFO for a simple read/write programme where the input from user is written to standard output by the writer function. The question is however, am I able to run this program without creating a child process (with the fork() operation). From what I see from examples about FIFOs, most read/write programmes with a named pipe/FIFO are done with 2 files - one for reading and one for writing. Could I do these all in a file?
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
/* read from user */
void reader(char *namedpipe) {
char c;
int fd;
while (1) {
/* Read from keyboard */
c = getchar();
fd = open(namedpipe, O_WRONLY);
write(fd, &c, 1);
fflush(stdout);
}
}
/* writes to screen */
void writer(char *namedpipe) {
char c;
int fd;
while (1) {
fd = open(namedpipe, O_RDONLY);
read(fd, &c, 1);
putchar(c);
}
}
int main(int argc, char *argv[]) {
int child,res;
if (access("my_fifo", F_OK) == -1) {
res = mkfifo("my_fifo", 0777);
if (res < 0) {
return errno;
}
}
child = fork();
if (child == -1)
return errno;
if (child == 0) {
reader("my_fifo");
}
else {
writer("my_fifo");
}
return 0;
}
You'll need to put a lock on the file, or else you could attempt to be reading when someone else is writing. You'll also want to flush the write buffer, or your changes to the fifo might actually not be recorded until the kernel write buffer fills and then writes to the file (in linux, write doesn't guarantee a write happens at that exact moment. i see you're flushing stdout, but you should also fsync on the file descriptor. This will cause the file to lock during any write operation so that no one else can write. In order to lock the file for reading, you might have to use a semaphore.