I have tested this program from the shell, and it works correctly when redircting stdin from a file. However, when operating as a CGI program, it times out (TimeForCGI hiawatha webserver setting is set to 30 seconds). This program is contained in just one file. It should be noted that this program was written only to physically verify what I have been reading about C.G.I., and I chose C (or anything else that generates a binary executable) so I can be sure that these things have been untouched by any interpreter, as might be done to facilitate their abstractions.
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <error.h>
int main (void);
int
main
(void)
{
static char buf[BUFSIZ];
size_t size;
if (setvbuf (stdout, NULL, _IOFBF, BUFSIZ) != 0)
error (EXIT_FAILURE, errno, "setvbuf(), stdout");
if (setvbuf (stdin, NULL, _IOFBF, BUFSIZ) != 0)
error (EXIT_FAILURE, errno, "setvbuf(), stdin");
if (setvbuf (stderr, NULL, _IOLBF, BUFSIZ) != 0)
error (EXIT_FAILURE, errno, "setvbuf(), stderr");
printf ("Content-Type: text/plain\n\n");
if (fflush (stdout) == EOF)
error (EXIT_FAILURE, errno, "fflush()");
for (;;)
{
size = fread (buf,1, BUFSIZ, stdin);
if (size == 0)
{
if (feof (stdin) != 0)
goto quit;
else
error (EXIT_FAILURE, errno, "fread(), stdin");
}
size = fwrite (buf, 1, size, stdout);
if (size == 0)
error (EXIT_FAILURE, errno, "write(), stdout");
}
quit:
fflush (stdout);
return EXIT_SUCCESS;
}
Here is the corresponding html form;
<html>
<head>
<title>Form</title>
</head>
<body>
<form action="form-process.cgi" method="post">
input_a: <input name="input_a" type="text"><br>
input_b: <input name="input_b" type="text"><br>
<input type="submit" value="Submit">
</form>
</body>
</html>
Your program attempts to read from its standard input until it reaches its end. That's fine when you redirect the input from a file, but it is inappropriate for a CGI program. The web server in which the CGI runs is not obligated to signal end-of-file on the input when the end of the request body is reached. If it does not, then your program will block indefinitely in fread().
There are several reasons why EOF might not be signaled at the end of the request body. The RFC explicitly postulates the presence of extension data, but it is also plausible that the server connects the CGI's standard input directly to the network socket on which the request is coming in. EOF will not normally be detected there until and unless the client closes its end of the connection, which many clients do not do between requests, and which many of the rest do not do until after they have received the response.
Accordingly, the CGI specifications in RFC 3875 say "the script MUST NOT attempt to read more than CONTENT_LENGTH bytes, even if more data is available" (section 4.2). The CONTENT_LENGTH is conveyed to the script via an environment variable of that name, provided that the request specifies one. Your CGI must not read more bytes than the variable specifies, and it must not read any bytes if the content length is not specified at all. On the other hand, it is at the CGI's not required to read the whole request body, or any of it at all.
In the meantime I have done this;
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <error.h>
#include <dstralg.h>
int main (void);
int
main
(void)
{
int l;
int i;
if (setvbuf (stdin, NULL, _IOFBF, BUFSIZ)!= 0)
error (EXIT_FAILURE, errno, "sevbuf(), stdin");
if (setvbuf (stdout, NULL, _IOFBF, BUFSIZ)!= 0)
error (EXIT_FAILURE, errno, "sevbuf(), stdout");
printf ("Content-Type: text/plain\n\n");
l = atoi (getenv ("CONTENT_LENGTH"));
for (i = 0; i < l; ++i)
putchar (getchar ());
return EXIT_SUCCESS;
}
which exhibits the desired behavior. The full buffering greatly reduces the overhead of processing one character at a time, and is only a function call once getchar and putchar have been unwound (assuming libc has been dynamically linked). As this is only experimental code using data from Hiawatha, which I trust, I didn't bother checking the return values of getchar and putchar being error conditions. Nor did I bother to check if CONTENT_LENGTH was NULL or "". In practice, I would use a domain specific interpretted language, such as PHP, for small projects with light traffic. I would probably use C/C++ for demanding workloads, although FastCGI can improve performance by the lighter operation of opening and closing connection to a unix domain socket in place of the heavier operation of forking a child process with the expense of creating page tables and all the other process management book keeping.
The following code, from your last post, should do also:
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <error.h>
#include <dstralg.h>
int main (void);
int
main
(void)
{
int l;
int i;
printf ("Content-Type: text/plain\n\n");
l = atoi (getenv ("CONTENT_LENGTH"));
for (i = 0; i < l; ++i)
putchar (getchar ());
fflush(stdout);
return EXIT_SUCCESS;
}
and without the final fflush(stdout); also, because you are going to exit(2) inmediately, after last putchar(3);, and that will make stdio to flush all buffers remaining.
Related
I'm trying to understand what is behind this behaviour in my parent process.
Basically, I create a child process and connect its stdout to my pipe. The parent process continuously reads from the pipe and does some stuff.
I noticed that when inserting the while loop in the parent the stdout seems to be lost, nothing appears on the terminal etc I thought that the output of stdout would somehow go to the pipe (maybe an issue with dup2) but that doesn't seem to be the issue. If I don't continuously fflush(stdout) in the parent process, whatever I'm trying to get to the terminal just won't show. Without a while loop in the parent it works fine, but I'm really not sure why it's happening or if the rest of my implementation is problematic somehow.
Nothing past the read system call seems to be going to the stdout in the parent process. Assuming the output of inotifywait in the pipe is small enough ( 30 > bytes ), what exactly is wrong with this program?
What I expect to happen is the stdout of inotifywait to go to the pipe, then for the parent to read the message, run strtok and print the file name (which only appears in stdout when I fflush)
Running the program with inotify installed and creating any file in the current directory of the program should be enough. Removing the while loop does print the created file's name (as expected).
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <errno.h>
int main(void) {
char b[100];
int pipefd;
if (mkfifo("fifo", 0666) == -1) {
if (errno != EEXIST) {
perror("mkfifo");
exit(EXIT_FAILURE);
}
}
pid_t pid = fork();
if (pid < 0) {
perror("fork");
exit(1);
}
if ((pipefd = open("fifo", O_RDWR)) < 0) {
perror("open pipe");
exit(EXIT_FAILURE);
}
if (pid == 0) {
dup2(pipefd, 1);
const char* dir = ".";
const char* args[] = {"inotifywait", dir, "-m", "-e",
"create", "-e", "moved_to", NULL};
execvp("inotifywait", (char**)args);
perror("inotifywait");
} else {
while (1) {
fflush(stdout); // the output only appears in stdout with this here
if (read(pipefd, b, 30) < 0) {
perror("problem # read");
exit(1);
}
char filename[30];
printf("anything");
sscanf(b, "./ CREATE %s", filename);
printf("%s", filename);
}
}
}
The streams used by the C standard library are designed in such a way that they are normally buffered (except for the standard error stream stderr).
The standard output stream is normally line buffered, unless the output device is not an interactive device, in which case it is normally fully buffered. Therefore, in your case, it is probably line buffered.
This means that the buffer will only be flushed
when it is full,
when an \n character is encountered,
when the stream is closed (e.g. during normal program termination),
when reading input from an unbuffered or line-buffered stream (in certain situations), or
when you explicitly call fflush.
This explains why you are not seeing the output, because none of the above are happening in your infinite loop (when you don't call fflush). Although you are reading input, you are not doing this from a C standard library FILE * stream. Instead, you are bypassing the C runtime library (e.g. glibc) by using the read system call directly (i.e. you are using a file descriptor instead of a stream).
The simplest solution to your problem would probably be to replace the line
printf("%s", filename);
with:
printf("%s\n", filename);
If stdout is line-buffered (which should be the case if it is connected to a terminal), then the input should automatically be flushed after every line and an explicit call to fflush should no longer be necessary.
On Linux (Raspbian on a Raspberry Pi) I would like to make it so that anything my C application prints using printf is sent back to me in a callback.
(No, I'm not talking about shell redirection with > some_file.txt. I'm talking about a C program making the decision by itself to send stdout (and therefore printf) to a callback within that same program.)
(Yes, I really do want to do this. I'm making a full-screen program using OpenGL and want to present any printf'd text to the user within that program, using my own rendering code. Replacing all printf calls with something else is not feasible.)
I feel like this should be easy. There are variations of this question on StackOverflow already, but none that I could find are exactly the same.
I can use fopencookie to get a FILE* that ends up calling my callback. So far, so good. The challenge is to get stdout and printf to go there.
I can't use freopen because it takes a string path. The FILE* I want to redirect to is not a file on the filesystem but rather just exists at runtime.
I can't use dup2 because the FILE* from fopencookie does not have a file descriptor (fileno returns -1).
The glibc documentation suggests that I can simply reassign stdout to my new FILE*: "stdin, stdout, and stderr are normal variables which you can set just like any others.". This does almost work. Anything printed with fprintf (stdout, "whatever") does go to my callback, and so does any printf that has any format specifiers. However, any call to printf with a string with no format specifiers at all still goes to the "original" stdout.
How can I achieve what I'm trying to do?
PS: I don't care about portability. This will only ever run on my current environment.
#define _GNU_SOURCE
#include <stdio.h>
#include <unistd.h>
#include <assert.h>
#include <stdarg.h>
#include <alloca.h>
#include <string.h>
static ssize_t my_write_func (void * cookie, const char * buf, size_t size)
{
fprintf (stderr, "my_write_func received %d bytes\n", size);
char * copy = (char*) alloca (size + 1);
assert (copy);
copy[size] = 0;
strncpy (copy, buf, size);
fprintf (stderr, "Text is: \"%s\"\n", copy);
fflush (stderr);
return size;
}
static FILE * create_opencookie ()
{
cookie_io_functions_t funcs;
memset (&funcs, 0, sizeof (funcs));
funcs.write = my_write_func;
FILE * f = fopencookie (NULL, "w", funcs);
assert (f);
return f;
}
int main (int argc, char ** argv)
{
FILE * f = create_opencookie ();
fclose (stdout);
stdout = f;
// These two DO go to my callback:
fprintf (stdout, "This is a long string, fprintf'd to stdout\n");
printf ("Hello world, this is a printf with a digit: %d\n", 123);
// This does not go to my callback.
// If I omit the fclose above then it gets printed to the console.
printf ("Hello world, this is plain printf.\n");
fflush (NULL);
return 0;
}
This appears to be a bug in GLIBC.
The reason that printf("simple string") works differently from printf("foo %d", 123) is that GCC transforms the former into a puts, with the notion that they are equivalent.
As far as I can tell, they should be equivalent. This man page states that puts outputs to stdout, just like printf does.
However, in GLIBC printf outputs to stdout here, but puts outputs to _IO_stdout here, and these are not equivalent. This has already been reported as a glibc bug (upstream bug).
To work around this bug, you could build with -fno-builtin-printf flag. That prevents GCC from transforming printf into puts, and on my system produces:
$ ./a.out
my_write_func received 126 bytes
Text is: "This is a long string, fprintf'd to stdout
Hello world, this is a printf with a digit: 123
Hello world, this is plain printf.
"
This workaround is of course incomplete: if you call puts directly, or link in object files that call printf("simple string") and were not compiled with -fno-builtin-printf (perhaps from 3rd-party library), then you'll still have a problem.
Unfortunately you can't assign to _IO_stdout (which is a macro). The only other thing you could do (that I can think of) is link in your own puts, which just returns printf("%s", arg). That should work if you are linking against libc.so.6, but may cause trouble if you link against libc.a.
You can redirect to a pipe instead and process the written data in a separate thread.
#include <pthread.h>
#include <ctype.h>
#include <unistd.h>
#include <stdio.h>
// this is the original program which you can't change
void print(void) {
printf("Hello, %d\n", 123);
puts("world");
printf("xyz");
}
int p[2];
void *render(void *arg) {
int nread;
char buf[1];
while((nread = read(p[0], buf, sizeof buf)) > 0) {
// process the written data, in this case - make it uppercase and write to stderr
for(int i = 0; i < nread; i++)
buf[i] = toupper(buf[i]);
write(2, buf, nread);
}
return NULL;
}
int main() {
setvbuf(stdout, NULL, _IONBF, 0);
pipe(p);
dup2(p[1], 1);
close(p[1]);
pthread_t t;
pthread_create(&t, NULL, render, NULL);
print();
close(1);
pthread_join(t, NULL);
}
On debian stretch putting:
setvbuf (f, NULL, _IOLBF, 0); // line buffered
after the create_opencookie call worked.
I'm using a library for my project. This library sometimes prints some messages to stdout. This is a problem for me because the messages are mixed up along with the application messages. It will be useful to a stop this behaviour or have them printed to a different window. I'm using C Language and Mingw32 enviroment. How can I do this? Thanks.
You might be able to (nonportably) swap the stdout with another stream:
#include <stdio.h>
FILE *devnull;
#define SWAPSTDOUT() do{ FILE *tmp = stdout; stdout = devnull; devnull = tmp; }while(0)
int main(void)
{
/*program initialization*/
if(0==(devnull= fopen("/dev/null", "r"))) return 1;
fputs("your code 0\n",stdout);
SWAPSTDOUT();
fputs("library code 0\n",stdout); //should be silent
SWAPSTDOUT();
fputs("your code 1\n", stdout);
}
Unfortunately, that's unlikely to work with functions that hardcode stdout (e.g., printf or puts).
If you're on a POSIX platform, you might have freopen but that won't help much if you can't save the original stream. However, on POSIX you could fflush(stdout) and then shuffle the underlying file descriptors, which should be quite reliable:
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
int devnull, tmp;
int swapstdout(void);
int main(void)
{
/*program initialization*/
if(0>(devnull=open("/dev/null", O_RDONLY))) return EXIT_FAILURE;
if(0>(tmp=dup(devnull))) return EXIT_FAILURE; //reserve a fd spot
fputs("your code 0\n",stdout);
if(0>swapstdout()) return EXIT_FAILURE:
fputs("library code 0\n",stdout); //should be silent
if(0>swapstdout()) return EXIT_FAILURE:
fputs("your code 1\n", stdout);
}
int swapstdout(void)
{
if(0>fflush(stdout)) return -1;
if(0>dup2(STDOUT_FILENO,tmp)) return -1; /*really shouldn't happen*/
if(0>dup2(devnull,STDOUT_FILENO)) return -1; /*really shouldn't happen*/
if(0>tmp=dup(devnull)) return -1; /*really shouldn't happen unless we're multithreaded and another thread steals the fd spot*/
}
Either solution depends on your code being single threaded.
In any case, well behaved library functions should leave files they don't own alone, unless you explicitly request them to do something with such files.
I am trying to write 2 programs that will talk to each other using fifo pipe.
I used the example here (section 5.2), but I changed the mknod there to mkfifo and tried to change gets to fgets.
This is the code (of one program which writes into the fifo):
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <sys/types.h> /*mkfifo, open */
#include <sys/wait.h>
#include <sys/stat.h> /* mkfifo, open */
#include <fcntl.h> /*open */
#define FIFO_PATH "/home/hana/Desktop"
#define BUFFER_SIZE 300
int main()
{
char buffer[BUFFER_SIZE];
int fd;
int wStatus;
mkfifo(FIFO_PATH, 666);
printf("waiting for readers\n");
fd = open(FIFO_PATH, O_RDWR);
while (fgets(buffer, BUFFER_SIZE, fd), !feof(stdin))
{
if ((wStatus = write(fd, buffer, strlen(buffer))) == -1)
perror("write");
else
printf("speak: wrote %d bytes\n", wStatus);
}
return 0;
}
I get a compilation error: passing argument 3 of fgets makes pointer from integer.
So fgets is expecting FILE* and not file descriptor.
What should I do? change something so that fgets works? use another function?
I am compiling with gcc (ansi, pedantic).
Thanks
The answer from whjm is the cause of your error diagnostic, but I think you probably meant
fgets(buffer, BUFFER_SIZE, stdin)
// ^^^^^
It doesn't make sense that you would read from a pipe and then immediately write the same thing back to the pipe. Also, if you never read from stdin, feof(stdin) will never be true.
Also, with fgets just check for a null result and then outside the loop, do the check for eof:
while (fgets(...) != NULL)
{
...
}
if (!feof(stdin))
{
// error handling
}
mkfifo() just creates special node in filesystem. And you are free to open it in any way. Actually there are two alternatives - POSIX "non-buffered" I/O: open()/write()/read() or standard buffered I/O: fopen()/fread()/fwrite(). First family operates on file descriptors while second one uses so called file streams: FILE. You can not mix these APIs freely. Just choose one and stick to it.
Standard I/O library offers some useful extra capabilities comparing to low-level non-buffered I/O. Like fgets() that you're trying to use. In this situation would be reasonable to use standard streams and replace open() with:
FILE* stream = fopen(FIFO_PATH, "r+");
Thus program will use FILE* instead of plain file descriptors. Also write() need to be changed to fwrite() immediately followed by fflush() to guarantee that written data are passed to FIFO.
P.S. In case of necessity it is possible to "wrap" low-level descriptors returned by open()(or something other) with standard FILE*. See fdopen(). But it is much like a workaround to use standard I/O API with special file objects that can not be opened with fopen().
My problem is to deal with sparse file reads and understand where the extents of the file are to perform some logic around it.
Since, there is no direct API call to figure these stuff out, I decided to use ioctl api to do this. I got the idea from how cp command deals with problems of copying over sparse files by going through their code and ended up seeing this.
https://github.com/coreutils/coreutils/blob/df88fce71651afb2c3456967a142db0ae4bf9906/src/extent-scan.c#L112
So, I tried to do the same thing in my sample program running in user space and it errors out with "Invalid argument". I am not sure what I am missing or if this is even possible from userspace. I am running on ubuntu 14.04 on an ext4 file system. Could this be a problem with device driver supporting these request modes underneath?
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <sys/fcntl.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <linux/fs.h>
#include "fiemap.h" //This is from https://github.com/coreutils/coreutils/blob/df88fce71651afb2c3456967a142db0ae4bf9906/src/fiemap.h
int main(int argc, char* argv[]) {
int input_fd;
if(argc != 2){
printf ("Usage: ioctl file1");
return 1;
}
/* Create input file descriptor */
input_fd = open (argv [1], O_RDWR);
if (input_fd < 0) {
perror ("open");
return 2;
}
union { struct fiemap f; char c[4096]; } fiemap_buf;
struct fiemap *fiemap = &fiemap_buf.f;
int s = ioctl(input_fd, FS_IOC_FIEMAP, fiemap);
if (s == 0) {
printf("ioctl success\n");
} else {
printf("ioctl failure\n");
char * errmsg = strerror(errno);
printf("error: %d %s\n", errno, errmsg);
}
/* Close file descriptors */
close (input_fd);
return s;
}
As you're not properly setting the fiemap_buf.f parameters before invoking ioctl(), it is likely that the EINVAL is coming from the fiemap invalid contents than from the FS_IOC_FIEMAP request identifier support itself.
For instance, the ioctl_fiemap() (from kernel) will evaluate the fiemap.fm_extent_count in order to determine if it is greater than FIEMAP_MAX_EXTENTS and return -EINVAL in that case. Since no memory reset nor parameterization is being performed on fiemap, this is very likely the root cause of the problem.
Note that from the coreutils code you referenced, it performs the correct parameterization of fiemap before calling ioctl():
fiemap->fm_start = scan->scan_start;
fiemap->fm_flags = scan->fm_flags;
fiemap->fm_extent_count = count;
fiemap->fm_length = FIEMAP_MAX_OFFSET - scan->scan_start;
Note fiemap is not recommended as you have to be sure to pass FIEMAP_FLAG_SYNC which has side effects. The lseek(), SEEK_DATA and SEEK_HOLE interface is the recommended one, though note that will, depending on file system, represent unwritten extents (allocated zeros) as holes.