fopen() cannot return a variable [duplicate] - c

I use mkfifo to create a named pipe.
Then I use the following program to open it. However, the program hangs at the line "fopen". Is there something wrong here?
int main(int argc, char** argv) {
char* line = "hello, world!";
FILE* fp = fopen("/tmp/myFIFO", "rw");
fprintf(fp, line);
fclose(fp);
return 0;
}

Try passing "w" as the mode to fopen. "rw" is not a valid mode argument for fopen, and even if it was, you probably don't want to both read and write to the FIFO in the same process (although it is possible, see below).
As an aside, the correct mode argument for opening a file for both reading and writing is either "r+" or "w+" (see the answers to this question for the differences).
This program will correctly write to the FIFO:
#include <stdio.h>
int main(int argc, char** argv) {
FILE* fp = fopen("/tmp/myFIFO", "w");
fprintf(fp, "Hello, world!\n");
fclose(fp);
return 0;
}
Note that fopen in the above program will block until the FIFO is opened for reading. When it blocks, run this in another terminal:
$ cat /tmp/myFIFO
Hello, world!
$
The reason why it blocks is because fopen does not pass O_NONBLOCK to open:
$ strace -P /tmp/myFIFO ./a.out
open("/tmp/myFIFO", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
...
Some background on how FIFOs are opened
Read-only, without O_NONBLOCK: open blocks until another process opens the FIFO for writing. This is the behavior when using fopen with mode argument "r".
Write-only, without O_NONBLOCK: open blocks until another process opens the FIFO for reading. This is the behavior when using fopen with mode argument "w".
Read-only, with O_NONBLOCK: open returns immediately.
Write-only, with O_NONBLOCK: open returns an error with errno set to ENXIO unless another process has the FIFO open for reading.
Info from "Advanced Programming in the UNIX Environment" by W. Richard Stevens.
Opening a FIFO for read and write
Opening a FIFO for reading and writing within the same process is also possible with Linux. The Linux FIFO man page states:
Under Linux, opening a FIFO for read and write will succeed both in
blocking and nonblocking mode. POSIX leaves this behavior undefined.
This can be used to open a FIFO for writing while there are no readers
available. A process that uses both ends of the connection in order
to communicate with itself should be very careful to avoid deadlocks.
Here's a program which writes to and reads from the same FIFO:
#include <stdio.h>
int main(int argc, const char *argv[]) {
char buf[100] = {0};
FILE* fp = fopen("/tmp/myFIFO", "r+");
fprintf(fp, "Hello, world!\n");
fgets(buf, sizeof(buf), fp);
printf("%s", buf);
fclose(fp);
return 0;
}
It does not block, and returns immediately:
$ gcc fifo.c && ./a.out
Hello, world!
Note that this is not portable and may not work on operating systems besides Linux.

The process blocks until the other end of the pipe gets opened.

Related

read() doesn't block on empty FIFOs opened without O_NONBLOCK flag

pipe(7) says:
If a process attempts to read from an empty pipe, then read(2) will block until data is available. If a process attempts to write to a full pipe (see below), then write(2) blocks until sufficient data has been read from the pipe to allow the write to complete. Nonblocking I/O is possible by using the fcntl(2) F_SETFL operation to enable the O_NONBLOCK open file status flag.
Below I have two simple C programs compiled on linux with gcc:
reader.c:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#define STACKBUF_SIZE 128
#define FIFO_PATH "/home/bogdan/.imagedata"
signed int main(int argc, char **argv) {
int fifo_fd = open(FIFO_PATH, O_RDONLY); // blocking... - notice no O_NONBLOCK flag
if (fifo_fd != -1) {
fprintf(stdout, "open() call succeeded\n");
}
while (1) {
char buf[STACKBUF_SIZE] = {0};
ssize_t bread = read(fifo_fd, buf, STACKBUF_SIZE);
fprintf(stdout, "%d - %s\n", bread, buf);
sleep(1);
}
close(fifo_fd);
return EXIT_SUCCESS;
}
writer.c:
#define STACKBUF_SIZE 128
#define FIFO_PATH "/home/bogdan/.imagedata"
#define DATA "data"
int main(void) {
int fifo_fd = open(FIFO_PATH, O_WRONLY); // blocks until reader opens on the reader end, however we always first open the reader so...
if(fifo_fd != -1) {
ssize_t bwritten = write(fifo_fd, DATA, 5);
fprintf(stdout, "writer wrote %ld bytes\n", bwritten);
}
close(fifo_fd);
return EXIT_SUCCESS;
}
The files are compiled into two separate binaries with gcc writer.c -Og -g -o ./writer, same for the reader.
From the shell I first execute the reader binary, and as expected, the initial open() call blocks until I also execute the writer. I then execute the writer, whose open() call immediately succeeds and it writes 5 bytes to the FIFO (which are correctly displayed by the reader), after which it closes the fd, leaving the FIFO empty (?).
However, the following read() calls in the while loop of the reader don't block at all, and instead just return 0.
Unless I am missing something (I probably am) this is in clash with the semantics outlined by the pipe(7) manpage, as the FIFO fd was open without the O_NONBLOCK flag both in the reader and the writer.
The section of the manual that you quoted only applies to pipes with open writers. Two paragraphs down, it says this:
If all file descriptors referring to the write end of a pipe have been closed, then an attempt to read(2) from the pipe will see end-of-file (read(2) will return 0).

Named pipe results in infinite loop/no output [duplicate]

I use mkfifo to create a named pipe.
Then I use the following program to open it. However, the program hangs at the line "fopen". Is there something wrong here?
int main(int argc, char** argv) {
char* line = "hello, world!";
FILE* fp = fopen("/tmp/myFIFO", "rw");
fprintf(fp, line);
fclose(fp);
return 0;
}
Try passing "w" as the mode to fopen. "rw" is not a valid mode argument for fopen, and even if it was, you probably don't want to both read and write to the FIFO in the same process (although it is possible, see below).
As an aside, the correct mode argument for opening a file for both reading and writing is either "r+" or "w+" (see the answers to this question for the differences).
This program will correctly write to the FIFO:
#include <stdio.h>
int main(int argc, char** argv) {
FILE* fp = fopen("/tmp/myFIFO", "w");
fprintf(fp, "Hello, world!\n");
fclose(fp);
return 0;
}
Note that fopen in the above program will block until the FIFO is opened for reading. When it blocks, run this in another terminal:
$ cat /tmp/myFIFO
Hello, world!
$
The reason why it blocks is because fopen does not pass O_NONBLOCK to open:
$ strace -P /tmp/myFIFO ./a.out
open("/tmp/myFIFO", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
...
Some background on how FIFOs are opened
Read-only, without O_NONBLOCK: open blocks until another process opens the FIFO for writing. This is the behavior when using fopen with mode argument "r".
Write-only, without O_NONBLOCK: open blocks until another process opens the FIFO for reading. This is the behavior when using fopen with mode argument "w".
Read-only, with O_NONBLOCK: open returns immediately.
Write-only, with O_NONBLOCK: open returns an error with errno set to ENXIO unless another process has the FIFO open for reading.
Info from "Advanced Programming in the UNIX Environment" by W. Richard Stevens.
Opening a FIFO for read and write
Opening a FIFO for reading and writing within the same process is also possible with Linux. The Linux FIFO man page states:
Under Linux, opening a FIFO for read and write will succeed both in
blocking and nonblocking mode. POSIX leaves this behavior undefined.
This can be used to open a FIFO for writing while there are no readers
available. A process that uses both ends of the connection in order
to communicate with itself should be very careful to avoid deadlocks.
Here's a program which writes to and reads from the same FIFO:
#include <stdio.h>
int main(int argc, const char *argv[]) {
char buf[100] = {0};
FILE* fp = fopen("/tmp/myFIFO", "r+");
fprintf(fp, "Hello, world!\n");
fgets(buf, sizeof(buf), fp);
printf("%s", buf);
fclose(fp);
return 0;
}
It does not block, and returns immediately:
$ gcc fifo.c && ./a.out
Hello, world!
Note that this is not portable and may not work on operating systems besides Linux.
The process blocks until the other end of the pipe gets opened.

Why do the strings output using fprintf end up not being written to the output file if my program is terminated via CTRL-C?

Why does fprintf give different results in the following example programs?
Example 1:
int main(){
FILE *f;
char buf[512];
char name[128] = {"filename"};
f = fopen(name, "w");
fprintf(f, "asdas\n");
fprintf(f, "asdas\n");
while(1){}
return 0;
}
If I terminate this program using CTRL+C, I get an empty file named filename.
However, using
Example 2:
int main(){
FILE *f;
char buf[512];
char name[128] = {"wpa_supplicant.conf"};
f = fopen(name,"w");
while(1){
fprintf(f, "asdas\n");
}
return 0;
}
If I terminate this program using CTRL+C, I get file named filename, and it contains many lines with the string asdas.
Why are the strings not written to the file in the first example, but they are written to the file in the second example?
In the second case, there are enough fprintf calls for the internal buffers to be flushed to disk.
With the first program, if you put a fflush(f) before the while loop, the strings will be written to the file.
#include <stdio.h>
int main(void) {
FILE *f = fopen("filename", "w");
if (!f) {
perror("Failed to open 'filename' for writing");
exit(EXIT_FAILURE);
}
fprintf(f, "asdas\n");
fprintf(f, "asdas\n");
if ( fflush(f) != 0 ) {
perror("Flushing output failed");
exit(EXIT_FAILURE);
}
while(1){}
return 0;
}
Output:
C:\...\Temp> cl file.c
Microsoft (R) C/C++ Optimizing Compiler Version 18.00.31101 for x64
...
/out:file.exe
C:\...\Temp> file
^C
C:\...\Temp> type filename
asdas
asdas
Keep in mind:
Upon successful completion, fflush() shall return 0; otherwise, it shall set the error indicator for the stream, return EOF, and set errno to indicate the error.
As mentioned in the answer by #SinanÜnür this is indeed an issue with the buffering of data in internal buffers. You need to flush manually in the first case to get that data actually written into the file.
However, FWIW, I just want to add here, you see this behavior because of the abnormal termination of the program by a signal (generated by CTRL+C).
If your program would have ended normally, (for example, by calling exit(), after a large-enough but controlled while() loop), then both the cases would have shown the same behavior, as in that scenario, all the open streams would have been flushed automatically.
The exit() function shall then flush all open streams with unwritten buffered data and close all open streams. Finally, the process shall be terminated ...

Why does popen2() hang between write and read calls?

I am trying to integrate use of samtools into a C program. This application reads data in a binary format called BAM, e.g. from stdin:
$ cat foo.bam | samtools view -h -
...
(I realize this is a useless use of cat, but I'm just showing how a BAM file's bytes can be piped to samtools on the command line. These bytes could come from other upstream processes.)
Within a C program, I would like to write chunks of unsigned char bytes to the samtools binary, while simultaneously capturing the standard output from samtools after it processes these bytes.
Because I cannot use popen() to simultaneously write to and read from a process, I looked into using publicly-available implementations of popen2(), which appears to be written to support this.
I wrote the following test code, which attempts to write() 4 kB chunks bytes of a BAM file located in the same directory to a samtools process. It then read()s bytes from the output of samtools into a line buffer, printed to standard error:
#include <sys/types.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#define READ 0
#define WRITE 1
pid_t popen2(const char *command, int *infp, int *outfp)
{
int p_stdin[2], p_stdout[2];
pid_t pid;
if (pipe(p_stdin) != 0 || pipe(p_stdout) != 0)
return -1;
pid = fork();
if (pid < 0)
return pid;
else if (pid == 0)
{
close(p_stdin[WRITE]);
dup2(p_stdin[READ], READ);
close(p_stdout[READ]);
dup2(p_stdout[WRITE], WRITE);
execl("/bin/sh", "sh", "-c", command, NULL);
perror("execl");
exit(1);
}
if (infp == NULL)
close(p_stdin[WRITE]);
else
*infp = p_stdin[WRITE];
if (outfp == NULL)
close(p_stdout[READ]);
else
*outfp = p_stdout[READ];
return pid;
}
int main(int argc, char **argv)
{
int infp, outfp;
/* set up samtools to read from stdin */
if (popen2("samtools view -h -", &infp, &outfp) <= 0) {
printf("Unable to exec samtools\n");
exit(1);
}
const char *fn = "foo.bam";
FILE *fp = NULL;
fp = fopen(fn, "r");
if (!fp)
exit(-1);
unsigned char buf[4096];
char line_buf[65536] = {0};
while(1) {
size_t n_bytes = fread(buf, sizeof(buf[0]), sizeof(buf), fp);
fprintf(stderr, "read\t-> %08zu bytes from fp\n", n_bytes);
write(infp, buf, n_bytes);
fprintf(stderr, "wrote\t-> %08zu bytes to samtools process\n", n_bytes);
read(outfp, line_buf, sizeof(line_buf));
fprintf(stderr, "output\t-> \n%s\n", line_buf);
memset(line_buf, '\0', sizeof(line_buf));
if (feof(fp) || ferror(fp)) {
break;
}
}
return 0;
}
(For a local copy of foo.bam, here is a link to the binary file I am using for testing. But any BAM file would do for testing purposes.)
To compile:
$ cc -Wall test_bam.c -o test_bam
The problem is that the procedure hangs after the write() call:
$ ./test_bam
read -> 00004096 bytes from fp
wrote -> 00004096 bytes to samtools process
[bam_header_read] EOF marker is absent. The input is probably truncated.
If I close() the infp variable immediately after the write() call, then the loop goes through one more iteration before hanging:
...
write(infp, buf, n_bytes);
close(infp); /* <---------- added after the write() call */
fprintf(stderr, "wrote\t-> %08zu bytes to samtools process\n", n_bytes);
...
With the close() statement:
$ ./test_bam
read -> 00004096 bytes from fp
wrote -> 00004096 bytes to samtools process
[bam_header_read] EOF marker is absent. The input is probably truncated.
[main_samview] truncated file.
output ->
#HD VN:1.0 SO:coordinate
#SQ SN:seq1 LN:5000
#SQ SN:seq2 LN:5000
#CO Example of SAM/BAM file format.
read -> 00004096 bytes from fp
wrote -> 00004096 bytes to samtools process
With this change, I get some output that I'd otherwise expect to get if I ran samtools on the command line, but as mentioned, the procedure hangs once again.
How does one use popen2() to write and read data in chunks to internal buffers? If this isn't possible, are there alternatives to popen2() that would work better for this task?
As an alternative to a pipe, why not communicate with samtools through a socket? Checking the samtools source, the file knetfile.c indicates that samtools has socket communications available:
#include "knetfile.h"
/* In winsock.h, the type of a socket is SOCKET, which is: "typedef
* u_int SOCKET". An invalid SOCKET is: "(SOCKET)(~0)", or signed
* integer -1. In knetfile.c, I use "int" for socket type
* throughout. This should be improved to avoid confusion.
*
* In Linux/Mac, recv() and read() do almost the same thing. You can see
* in the header file that netread() is simply an alias of read(). In
* Windows, however, they are different and using recv() is mandatory.
*/
That may provide a better option than using pipe2.
This problem has nothing to do with the particular implementation of popen2. Also note that on OS X, popen lets you open a bidirectional pipe, this may be true on other BSD systems too. If this is to be portable, you'd need a configure check for whether popen allows bidirectional pipes (or something equivalent to a configure check).
You need to switch the pipes to non-blocking mode, and alternate between read and write calls in an endless loop. Such loop, in order not to waste the CPU when the samtools process is busy, needs to use select, poll or a similar mechanism that blocks for a file descriptor to become "available" (more data to read, or ready to accept data for writing).
See this question for some inspiration.

C: Linux command executed by popen() function not showing results

I have the code below that I refer the thread on here to use the popen function
int main(int argc,char *argv[]){
FILE* file = popen("ntpdate", "r");
char buffer[100];
fscanf(file, "%100s", buffer);
pclose(file);
printf("buffer is :%s\n", buffer);
return 0;
}
It outputs:
21 Apr 03:03:03 ntpdate[4393]: no server can be used, exiting
buffer is:
why printf does not output anything? If I use ls as a command, then printf outputs the ls output. what am I doing wrong ntpdate executing?
If I execute the code below (referring the webpage)
#define COMMAND_LEN 8
#define DATA_SIZE 512
int main(int argc,char *argv[]){
FILE *pf;
char command[COMMAND_LEN];
char data[DATA_SIZE];
// Execute a process listing
sprintf(command, "ntpdate");
// Setup our pipe for reading and execute our command.
pf = popen(command,"r");
if(!pf){
fprintf(stderr, "Could not open pipe for output.\n");
return;
}
// Grab data from process execution
fgets(data, DATA_SIZE , pf);
// Print grabbed data to the screen.
fprintf(stdout, "-%s-\n",data);
if (pclose(pf) != 0)
fprintf(stderr," Error: Failed to close command stream \n");
return 0;
}
I get
21 Apr 03:15:45 ntpdate[5334]: no servers can be used, exiting
-�2}�����"|�4#|�-
Error: Failed to close command stream
what are wrongs on the codes above?
Since the output is going to stderr you need to redirect stderr like so:
FILE* file = popen("ntpdate 2>&1", "r");
this will redirect stderr to stdout and so you will see output from both. Second issue fscanf will stop at the first space so you can replace with fgets:
fgets(buffer, 100, file);
As Shafik Yaghmour correctly diagnosed, the output you see from ntpdate is written (correctly) to its standard error, which is the same as your programs standard error.
To get the error messages sent down the pipe, use:
FILE *file = popen("ntpdate 2>&1", "r");
That sends the standard error output from ntpdate to the standard output of the command, which is the pipe you're reading from.
Of course, it looks like using ntpdate isn't going to work well until you've configured something.

Resources