How to read a command output with pipe and scanf()? - c

Question

I need to be able to send the output of the GET command and store it into a variable inside my program, currently I'm doing it like this:
GET google.com | ./myprogram
And receiving it in my program with the following code:
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
int main(int argc, char *argv[]){
char *a = (char *) malloc (10000000);
scanf("%[^\n]", a);
printf("%s\n",a);
return 0;
}
The problem I have is that the scanf function stops at a new line, and I need to be able to store the whole paragraph output from GET.
Any help will be appreciated. Thanks.

Answer 1

One possibility: Does GET include the size information in the headers? Could you use that to determine how much space to allocate, and how much data to read? That's fiddly though, and requires reading the data in dribs and drabs.
More simply, consider using POSIX (and Linux) getdelim() (a close relative of getline()) and specify the delimiter as the null byte. That's unlikely to appear in the GET output, so the whole content will be a single 'line', and getdelim() will allocate an appropriate amount of space automatically. It also tells you how long the data was.
#include <stdio.h>
int main(void)
{
char *buffer = 0;
size_t buflen = 0;
int length = getdelim(&buffer, &buflen, '\0', stdin);
if (length > 0)
printf("%*.*s\n", length, length, buffer);
free(buffer);
return 0;
}

Answer 2

scanf documentation says
These functions return the number of input items successfully
matched and assigned, which can be fewer than provided for, or even
zero in the event of an early matching failure. The value EOF is
returned if the end of input is reached before either the first
successful conversion or a matching failure occurs. EOF is also
returned if a read error occurs, in which case the error indicator for
the stream (see ferror(3)) is set, and errno is set indicate the
error.
https://www.freebsd.org/cgi/man.cgi?query=scanf&sektion=3
Have you considered writing a loop that calls scanf, monitors its return value and breaks out if EOF

Answer 3

Consider the following readall() function implemented in standard C:
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
char *readall(FILE *source, size_t *length)
{
char *data = NULL;
size_t size = 0;
size_t used = 0;
size_t n;
/* If we have a place to store the length,
we initialize it to zero. */
if (length)
*length = 0;
/* Do not attempt to read the source, if it is
already in end-of-file or error state. */
if (feof(source) || ferror(source))
return NULL;
while (1) {
/* Ensure there is unused chars in data. */
if (used >= size) {
const size_t new_size = (used | 65535) + 65537 - 32;
char *new_data;
new_data = realloc(data, new_size);
if (!new_data) {
/* Although reallocation failed, data is still there. */
free(data);
/* We just fail. */
return NULL;
}
data = new_data;
size = new_size;
}
/* Read more of the source. */
n = fread(data + used, 1, size - used, source);
if (!n)
break;
used += n;
}
/* Read error or other wonkiness? */
if (!feof(source) || ferror(source)) {
free(data);
return NULL;
}
/* Optimize the allocation. For ease of use, we append
at least one nul byte ('\0') at end. */
{
const size_t new_size = (used | 7) + 9;
char *new_data;
new_data = realloc(data, new_size);
if (!new_data) {
if (used >= size) {
/* There is no room for the nul. We fail. */
free(data);
return NULL;
}
/* There is enough room for at least one nul,
so no reason to fail. */
} else {
data = new_data;
size = new_size;
}
}
/* Ensure the buffer is padded with nuls. */
memset(data + used, 0, size - used);
/* Save length, if requested. */
if (length)
*length = used;
return data;
}
It reads everything from the specified file handle (which can be a standard stream like stdin or a pipe opened via popen()) into a dynamically allocated buffer, appends a nul byte (\0), and returns a pointer to the buffer. If not NULL, the actual number of characters read (so, not including the appended nul byte), is stored in the size_t pointed to by the second parameter.
You can use it to read binary data output by programs, say dot -Tpng diagram.dot or image converters, or even wget -O - output (getting data from specific URLs, text or binary).
You can use this for example thus:
int main(void)
{
char *src;
size_t len;
src = readall(stdin, &len);
if (!src) {
fprintf(stderr, "Error reading standard input.\n");
return EXIT_FAILURE;
}
fprintf(stderr, "Read %zu chars.\n", len);
/* As an example, print it to standard output. */
if (len > 0)
fwrite(src, len, 1, stdout);
free(src);
return EXIT_SUCCESS;
}
The readall() function has two quirks: it allocates memory in roughly 131072-byte chunks (but could vary if fread() were to return short reads), and pads the buffer with 7 to 15 nul bytes. (There are reasons why I like doing it this way, but it is all speculative and specific to the C libraries I tend to use, so it is not important.)
Although the ones used above work fine, you can change the size_new calculations if you prefer otherwise. Just make sure that they both are at least used + 1.