I want to determine the stream size retruned by the popen() function call. I tried to use fseek and ftell but it returns the size as -1. Can anyone suggest me how to determine the file size? The following is the code what I am using....
char return_val[256];
FILE *fp = NULL;
char line[256];
memset (return_val, 0, 256);
/* set the defalut value */
strncpy (return_val, "N/A", 4);
char cmd[] = "if [ -f /etc/version ]; then cut -d, -f1 -s /etc/version ; fi";
/* Open the command for reading. */
fp = popen(cmd, "r");
if (fp != NULL)
{
/* read the line from file */
fgets (line, 256, fp);
if( line != NULL)
{
/* copy the data */
strncpy(return_val, line, strnlen (line, 256));
}
/* close the file */
pclose (fp);
}
You can't. popen gives you a pipe, a FIFO First In First Out stream. Characters go in on the one side and they come out at the other. It is in the nature of a pipe that you don't know in advance how many bytes there will be transmitted.
You may either resort to in band signalling, how many bytes there are to come, or you implement buffering and dynamic allocation on your side.
Also seeking is not possible on a pipe, because its merely a kind of "Portal" between those processes. You know, like in the game "Portal"/"Portal2".
Related
I am working on an assignment in socket programming in which I have to send a file between sparc and linux machine. Before sending the file in char stream I have to get the file size and tell the client. Here are some of the ways I tried to get the size but I am not sure which one is the proper one.
For testing purpose, I created a file with content " test" (space + (string)test)
Method 1 - Using fseeko() and ftello()
This is a method I found on https://www.securecoding.cert.org/confluence/display/c/FIO19-C.+Do+not+use+fseek()+and+ftell()+to+compute+the+size+of+a+regular+file
While the fssek() has a problem of "Setting the file position indicator to end-of-file, as with fseek(file, 0, SEEK_END), has undefined behavior for a binary stream", fseeko() is said to have tackled this problem but it only works on POSIX system (which is fine because the environment I am using is sparc and linux)
fd = open(file_path, O_RDONLY);
fp = fopen(file_path, "rb");
/* Ensure that the file is a regular file */
if ((fstat(fd, &st) != 0) || (!S_ISREG(st.st_mode))) {
/* Handle error */
}
if (fseeko(fp, 0 , SEEK_END) != 0) {
/* Handle error */
}
file_size = ftello(fp);
fseeko(fp, 0, SEEK_SET);
printf("file size %zu\n", file_size);
This method works fine and get the size correctly. However, it is limited to regular files only. I tried to google the term "regular file" but I still not quite understand it thoroughly. And I do not know if this function is reliable for my project.
Method 2 - Using strlen()
Since the max. size of a file in my project is 4MB, so I can just calloc a 4MB buffer. After that, the file is read into the buffer, and I tried to use the strlen to get the file size (or more correctly the length of content). Since strlen() is portable, can I use this method instead? The code snippet is like this
fp = fopen(file_path, "rb");
fread(file_buffer, 1024*1024*4, 1, fp);
printf("strlen %zu\n", strlen(file_buffer));
This method works too and returns
strlen 8
However, I couldn't see any similar approach on the Internet using this method. So I am thinking maybe I have missed something or there are some limitations of this approach which I haven't realized.
Regular file means that it is nothing special like device, socket, pipe etc. but "normal" file.
It seems that by your task description before sending you must retrieve size of normal file.
So your way is right:
FILE* fp = fopen(...);
if(fp) {
fseek(fp, 0 , SEEK_END);
long fileSize = ftell(fp);
fseek(fp, 0 , SEEK_SET);// needed for next read from beginning of file
...
fclose(fp);
}
but you can do it without opening file:
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
struct stat buffer;
int status;
status = stat("path to file", &buffer);
if(status == 0) {
// size of file is in member buffer.st_size;
}
OP can do it the easy way as "max. size of a file in my project is 4MB".
Rather than using strlen(), use the return value from fread(). stlen() stops on the first null character, so may report too small a value. #Sami Kuhmonen Also we do not know the data read contains any null character, so it may not be a string. Append a null character (and allocate +1) if code needs to use data as a string. But in that case, I'd expect the file needed to be open in text mode.
Note that many OS's do not even use allocated memory until it is written.
Why is malloc not "using up" the memory on my computer?
fp = fopen(file_path, "rb");
if (fp) {
#define MAX_FILE_SIZE 4194304
char *buf = malloc(MAX_FILE_SIZE);
if (buf) {
size_t numread = fread(buf, sizeof *buf, MAX_FILE_SIZE, fp);
// shrink if desired
char *tmp = realloc(buf, numread);
if (tmp) {
buf = tmp;
// Use buf with numread char
}
free(buf);
}
fclose(fp);
}
Note: Reading the entire file into memory may not be the best idea to begin with.
I was wondering what's the best way to read in a filename in C when the name is echoed on the command line, as in: if I type in
echo test.txt | a1
how would I access the test.txt using
fopen(fname, "r");?
When you pipe data from one program to another it's as if you typed the output of the first into the second. In this case the file name can be read from stdin, and one way to read it is with the fgets function.
Read data from the standard input.
Process the data.
#include <stdio.h>
char buf[1024];
while (fgets(buf, sizeof buf, stdin))
{
printf("Read line: '%s'\n", buf);
FILE * fp = fopen(buf, "r");
if (fp)
{
// process the file
fclose(fp);
}
else
{
printf("No such file: %s\n", buf);
}
}
The obvious caveat is that the file name may be longer than 1023 characters; read the manual for fgets for details on how to detect this.
The way it's written, you'll do one processing attempt per line, and the program stops when the input stream ends.
You're trying to read from stdin, which means you could use fread (man 3 fread on *nix, here if you aren't.) to read from it.
stdin is just a name for a standard file-descriptor, so you can read from it like any other file.
I've been stuck on this for a few days and it's getting really frustrating.
I'm using popen() to call a command line process and get its output and store it in a C string. I was using fgets() but it seems that breaks after a new line, so I'm using fread(). The only problem is that the returned C string is sometimes messed up.
Here's my code:
const char *cmd = "date";//This the shell command
char buf[BUFSIZ];//Output of the command
FILE *ptr;
int c;
if ((ptr = popen(cmd, "r")) != NULL)
while(fread(buf, sizeof(buf),1, ptr))
while ((c = getchar()) != EOF)
printf("output = %s", buf);
(void) pclose(ptr);
The final C string sometimes has weird characters in it that shouldn't be there, or sometimes no string is even available. Can anybody please help? ):
Edit: Here is what I was doing when using fgets() The Shell command can be anything that outputs text though. Not just "date."
if ((ptr = popen(cmd, "r")) != NULL)while (fgets(buf, BUFSIZ, ptr) != NULL)printf("output = %s", buf);(void) pclose(ptr);
fread doesn't insert a NUL terminator after what it reads. You need to check the return value to know how much it read, and only print that much. If you read with fread, you typically want to write the data with fwrite, something on this order:
long bytes;
while ((bytes=fread(buf, sizeof(buf), 1, ptr))>0)
fwrite(buf, bytes, 1, stdout);
Well, fgets is the right way to do it.
FILE *ptr;
if (NULL == (ptr = popen(cmd, "r"))) {
/* ... */
}
while(fgets(buf, sizeof(buf), ptr) != NULL) {
/* There is stuff in 'buf' */
}
I think the reason fgets wasn't working for you is that you were doing something wrong.
Now, here's why I think you are running into trouble with your current code:
You are not checking how much fread actually returned
You are reading with getchar and discarding stuff
You don't have a NUL terminator in the buffer
Get this right and it will all be better: fread might legally read less than you told it to.
The output from date doesn't include the '\0' (NUL) character you need to properly terminate the string. Keep track of the number of characters read and put in the NUL yourself.
Though really, you should be using fgets, getline or similar text-oriented functions to read from a program such as date. getline is especially easy (and safe since it does some memory management for you):
FILE *fp = popen("date", "r");
char *ln = NULL;
size_t len = 0;
while (getline(&ln, &len, fp) != -1)
fputs(ln, stdout);
free(ln);
pclose(fp);
Below is the correct way to use fread for process output with popen:
const char *cmd = "date";
char buf[BUFSIZ];
FILE *ptr;
if ((ptr = popen(cmd, "r")) != NULL) {
/* Read one byte at a time, up to BUFSIZ - 1 bytes, the last byte will be used for null termination. */
size_t byte_count = fread(buf, 1, BUFSIZ - 1, ptr);
/* Apply null termination so that the read bytes can be treated as a string. */
buf[byte_count] = 0;
printf("%s\n", buf);
}
(void) pclose(ptr);
As you can see, the primary problem is to correctly deal with null termination. The two size parameter of fread is also important, you have to let it read character by character. Note that in the case of popen, fread will only return 0 if the process has exited without giving any output. It will not return 0 if it takes a long time for the process to print anything.
If the output is larger than BUFSIZ, you can wrap fread with a while loop.
I'm trying to read from stdin and output the data, things work, EXCEPT that it's not outputting the new incoming data. I'm not quite sure where is the issue. I'm guessing it has something to do when determining the stdin size. Any help would be greatly appreciated! Thanks
tail -f file | my_prog
Updated
#include <stdio.h>
#include <sys/stat.h>
long size(FILE *st_in) {
struct stat st;
if (fstat(fileno(st_in), &st) == 0)
return st.st_size;
return -1;
}
int main (){
FILE *file = stdin;
char line [ 128 ];
while ( fgets ( line, sizeof line, file ) != NULL )
fputs ( line, stdout ); /* write the line */
long s1, s2;
s1 = size(file);
for (;;) {
s2 = size (file);
if (s2 != s1) {
if (!fseek (file, s1, SEEK_SET)) {
while ( fgets ( line, sizeof line, file ) != NULL ) {
fputs ( line, stdout ); /* write the line */
}
}
s1 = s2;
usleep(300000);
}
}
return 0;
}
Edit: Fixed!
After a FILE * has reached EOF, it stays in a state where it will read no more data until you clear the 'EOF' bit either with clearerr() or with fseek(). However, if standard input is connected to a terminal, then that is not a seekable device, so instead of clearing the error, it might not do anything useful:
POSIX says:
The behavior of fseek() on devices which are incapable of seeking is implementation-defined.
Your loop entry condition is suspect; you need to sleep before starting it, and you need to sleep on each iteration. Indeed, normally you write tail -f without worrying about the file size; you sleep, try to read until the next 'EOF', reset the file EOF indicator, and repeat. Note, too, that the size of a pipe or terminal is not defined.
Separately, it is aconventional to call a FILE * argument to a function filename; it has completely the wrong connotations. A filename is a string.
This is not really standard C:
size(file);
Call stat() to get file information - organization type of a file, file size and permissions.
What your code does is to eventually set the file pointer to the end of the file, as it tries to read through it. Consider stat() (or fstat() on a an open file) instead.
rewind() resets the file pointer to the start of the file, fseek() will place it anywhere you need.
tail -f repeatedly tries the file at the EOF point with a short sleep in between tries.... It does not "consider" EOF to be an error. It remembers the current file offset for the EOF, then fseeks() using SEEK_END, then calls ftell(), and compares the offsets. If there is a difference it then fseek()-s back to the last known endpoint and reads the data.
This description is from old unix source. I'm sure it has been tweaked since then.
I am reaing from a file, and when i read, it takes it line by line, and prints it
what i want exactly is i want an array of char holding all the chars in the file and print it once,
this is the code i have
if(strcmp(str[0],"#")==0)
{
FILE *filecomand;
//char fname[40];
char line[100];
int lcount;
///* Read in the filename */
//printf("Enter the name of a ascii file: ");
//fgets(History.txt, sizeof(fname), stdin);
/* Open the file. If NULL is returned there was an error */
if((filecomand = fopen(str[1], "r")) == NULL)
{
printf("Error Opening File.\n");
//exit(1);
}
lcount=0;
int i=0;
while( fgets(line, sizeof(line), filecomand) != NULL ) {
/* Get each line from the infile */
//lcount++;
/* print the line number and data */
//printf("%s", line);
}
fclose(filecomand); /* Close the file */
You need to determine the size of the file. Once you have that, you can allocate an array large enough and read it in a single go.
There are two ways to determine the size of the file.
Using fstat:
struct stat stbuffer;
if (fstat(fileno(filecommand), &stbuffer) != -1)
{
// file size is in stbuffer.st_size;
}
With fseek and ftell:
if (fseek(fp, 0, SEEK_END) == 0)
{
long size = ftell(fp)
if (size != -1)
{
// succesfully got size
}
// Go back to start of file
fseek(fp, 0, SEEK_SET);
}
Another solution would be to map the entire file to the memory and then treat it as a char array.
Under windows MapViewOfFile, and under unix mmap.
Once you mapped the file (plenty of examples), you get a pointer to the file's beginning in the memory. Cast it to char[].
Since you can't assume how big the file is, you need to determine the size and then dynamically allocate a buffer.
I won't post the code, but here's the general scheme. Use fseek() to navigate to the end of file, ftell() to get size of the file, and fseek() again to move the start of the file. Allocate a char buffer with malloc() using the size you found. The use fread() to read the file into the buffer. When you are done with the buffer, free() it.
Use a different open. i.e.
fd = open(str[1], O_RDONLY|O_BINARY) /* O_BINARY for MS */
The read statement would be for a buffer of bytes.
count = read(fd,buf, bytecount)
This will do a binary open and read on the file.