I have been able to write a program that can read any text files... except the ones found in /proc. Any file that I try to read from /proc shows up empty.
But whenever I type
cat /proc/cpuinfo
on terminal, I am presented with my CPU info.
I can also see the file when I open it with a text editor, such as gedit or leafpad.
So it seems that /proc files are indeed text files, but my C program is having a hard time reading them.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
char* readFileString( char* loc ) {
char *fileDat;
FILE * pFile;
long lsize;
pFile = fopen( loc, "r" );
// Grab the file size.
fseek(pFile, 0L, SEEK_END);
lsize = ftell( pFile );
fseek(pFile, 0L, SEEK_SET);
fileDat = calloc( lsize + 1, sizeof(char) );
fread( fileDat, 1, lsize, pFile );
return fileDat;
}
int main( void ) {
char *cpuInfo;
cpuInfo = readFileString( "/proc/cpuinfo" );
printf( "%s\n", cpuInfo );
return 0;
}
Any idea why?
The files from /proc have a size of 0 byte because they are generated on the fly by the kernel.
See here for more information on proc filesystem:
http://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/proc.html
Most /proc/ textual files are intended to be read sequentially by a classical loop like
FILE *f = fopen("/proc/cpuinfo", "r");
size_t sz = 0;
char * lin = 0;
do {
ssize_t lsz = getline (&lin, &sz, f);
if (lsz<0) break;
handle_line_of_size (lin, lsz);
} while (!feof (f));
fclose (f);
seeking don't work on them. A bit like for pipes.
If you want to know the size of a file, stat(2) is the way to go. But for what you're doing, either allocate a very large buffer (RAM is cheap and this is a one-shot program) you fread() into after you fopen() it, or learn about realloc(3) and use that in your file-reading loop. As ouah said, the files in /proc are special.
For general-purpose use, and especially for strings, calloc() is a waste of cpu cycles, as setting the 0th char of the returned allocation area to '\0' is sufficient to make it an empty string, regardless of the data following that first byte.
Related
I am working on an assignment in socket programming in which I have to send a file between sparc and linux machine. Before sending the file in char stream I have to get the file size and tell the client. Here are some of the ways I tried to get the size but I am not sure which one is the proper one.
For testing purpose, I created a file with content " test" (space + (string)test)
Method 1 - Using fseeko() and ftello()
This is a method I found on https://www.securecoding.cert.org/confluence/display/c/FIO19-C.+Do+not+use+fseek()+and+ftell()+to+compute+the+size+of+a+regular+file
While the fssek() has a problem of "Setting the file position indicator to end-of-file, as with fseek(file, 0, SEEK_END), has undefined behavior for a binary stream", fseeko() is said to have tackled this problem but it only works on POSIX system (which is fine because the environment I am using is sparc and linux)
fd = open(file_path, O_RDONLY);
fp = fopen(file_path, "rb");
/* Ensure that the file is a regular file */
if ((fstat(fd, &st) != 0) || (!S_ISREG(st.st_mode))) {
/* Handle error */
}
if (fseeko(fp, 0 , SEEK_END) != 0) {
/* Handle error */
}
file_size = ftello(fp);
fseeko(fp, 0, SEEK_SET);
printf("file size %zu\n", file_size);
This method works fine and get the size correctly. However, it is limited to regular files only. I tried to google the term "regular file" but I still not quite understand it thoroughly. And I do not know if this function is reliable for my project.
Method 2 - Using strlen()
Since the max. size of a file in my project is 4MB, so I can just calloc a 4MB buffer. After that, the file is read into the buffer, and I tried to use the strlen to get the file size (or more correctly the length of content). Since strlen() is portable, can I use this method instead? The code snippet is like this
fp = fopen(file_path, "rb");
fread(file_buffer, 1024*1024*4, 1, fp);
printf("strlen %zu\n", strlen(file_buffer));
This method works too and returns
strlen 8
However, I couldn't see any similar approach on the Internet using this method. So I am thinking maybe I have missed something or there are some limitations of this approach which I haven't realized.
Regular file means that it is nothing special like device, socket, pipe etc. but "normal" file.
It seems that by your task description before sending you must retrieve size of normal file.
So your way is right:
FILE* fp = fopen(...);
if(fp) {
fseek(fp, 0 , SEEK_END);
long fileSize = ftell(fp);
fseek(fp, 0 , SEEK_SET);// needed for next read from beginning of file
...
fclose(fp);
}
but you can do it without opening file:
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
struct stat buffer;
int status;
status = stat("path to file", &buffer);
if(status == 0) {
// size of file is in member buffer.st_size;
}
OP can do it the easy way as "max. size of a file in my project is 4MB".
Rather than using strlen(), use the return value from fread(). stlen() stops on the first null character, so may report too small a value. #Sami Kuhmonen Also we do not know the data read contains any null character, so it may not be a string. Append a null character (and allocate +1) if code needs to use data as a string. But in that case, I'd expect the file needed to be open in text mode.
Note that many OS's do not even use allocated memory until it is written.
Why is malloc not "using up" the memory on my computer?
fp = fopen(file_path, "rb");
if (fp) {
#define MAX_FILE_SIZE 4194304
char *buf = malloc(MAX_FILE_SIZE);
if (buf) {
size_t numread = fread(buf, sizeof *buf, MAX_FILE_SIZE, fp);
// shrink if desired
char *tmp = realloc(buf, numread);
if (tmp) {
buf = tmp;
// Use buf with numread char
}
free(buf);
}
fclose(fp);
}
Note: Reading the entire file into memory may not be the best idea to begin with.
I using some C code that writes binary data to a file. In the process, it seeks around to different positions and then finally seeks to the end with fseeko(fp, 0, SEEK_END);.
However, in some cases, I want to work on a stream in memory instead. I use open_memstream for this, but seeking to the end pads the buffer with zeros and it ends up being twice as big as it should be.
An example just to demonstrate the effect of the fseek to the end of the stream is below. In the actual code, we also fseek to different parts of the stream, patching and editing bits of it, etc., as the stream is processed. Note also that writing the file at the end to the filesystem is just for demonstration to show the contents of the buffer – otherwise I wouldn't need the memory stream.
#include <stdio.h>
#include <stdlib.h>
#if (defined(BSD) || __APPLE__)
#include "open_memstream.h"
#endif
int main(void) {
FILE *stream;
FILE *outfile;
char *buf;
size_t buf_len;
int i;
stream = open_memstream(&buf, &buf_len);
for(i = 0; i < 1000; i++) {
fprintf(stream, "%d\n", i);
}
fseeko(stream, 0, SEEK_END);
fclose(stream);
outfile = fopen("out.txt", "w");
fwrite(buf, buf_len, 1, outfile);
fclose(outfile);
return 0;
}
I was testing this out on Mac OS X with this implementation of open_memstream and it worked as I expected, but when I run this on Linux the file is twice the size with zeros at the end.
What's the best way to deal with this? I'm not sure if it's reliable to divide the buffer length by two and truncate it.
I've just ran into the same problem on Linux.
// It seams that SEEK_END does not work with open_memstream()
fseek(stream, 0, SEEK_END);
I've ended up doing this:
off_t o = ftell(stream);
/* do some things with the stream */
fseek(stream, o, SEEK_SET);
What is the best way to get the contents of a file into a single character array?
I have read this question:
Easiest way to get file's contents in C
But from the comments, I've seen that the solution isn't great for large files. I do have access to the stat function. If the file size is over 4 gb, should I just return an error?
The contents of the file is encrypted and since it's supplied by the user it could be as large as anyone would want it to be. I want it to return an error and not crash if the file is too big. The main purpose of populating the character array with the contents of a file, is to compare it to another character array and also (if needed and configured to do so) to log both of these to a log file (or multiple log files if necessary).
You may use fstat(3) from sys/stat.h. Here is a little function to get size of the file, allocate memory if file is less than 4GB's and return (-1) otherwise. It reads the file to the char array passed to char *buffer a char *, which contains the contents of the whole file.It should be free'd after use.
#include <stdio.h>
#include <sys/stat.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <fcntl.h>
char *loadlfile(const char *path)
{
int file_descr;
FILE *fp;
struct stat buf;
char *p, *buffer;
fstat((file_descr = open(path, O_RDONLY)), &buf);
// This check is done at preprocessing and requires no check at runtime.
// It basically means "If this machine is not of a popular 64bit architecture,
// it's probably not 128bit and possibly has limits in maximum memory size.
// This check is done for the sake of omission of malloc(3)'s unnecessary
// invocation at runtime.
// Amd 64 Arm64 Intel 64 Intel 64 for Microsofts compiler.
#if !defined(__IA_64) || !defined(__aarch64__) || !defined(__ia64__) || !defined(_M_IA64)
#define FILE_MAX_BYTES (4000000000)
// buf.st_size is of off_t, you may need to cast it.
if(buf.st_size >= FILE_MAX_BYTES-1)
return (-1);
#endif
if(NULL == (buffer = malloc(buf.st_size + 1)))
return NULL;
fp = fdopen(file_descr, "rb");
p = buffer;
while((*p++ = fgetc(fp)) != EOF)
;
*p = '\0';
fclose(fp);
close(file_descr);
return buffer;
}
A very broad list of pre-defined macros for various things can be found # http://sourceforge.net/p/predef/wiki/Home/. The reason for the architecture and file size check is, malloc can be expensive at times and it is best to omit/skip it's usage when it is not needed. And querying a memory of max. 4gb for a whole block of 4gb storage is just waste of those precious cycles.
From that guy's code just do, if I understand your question correctly:
char * buffer = 0;
long length;
FILE * f = fopen (filename, "rb");
if (f)
{
fseek (f, 0, SEEK_END);
length = ftell (f);
if(length > MY_MAX_SIZE) {
return -1;
}
fseek (f, 0, SEEK_SET);
buffer = malloc (length);
if (buffer)
{
fread (buffer, 1, length, f);
}
fclose (f);
}
if (buffer)
{
// start to process your data / extract strings here...
}
I'm using fmemopen to create a variable FILE* fid to pass it to a function the reads data from an open file.
Somewhere in that function it uses the following code to find out the size of the file:
fseek(fid, 0, SEEK_END);
file_size = ftell(fid);
this works well in case of regular files, but in case of file ids created by fmemopen I always get file_size = 8192
Any ideas why this happens?
Is there a method to get the correct file size that works for both regular files and files created with fmemopen?
EDIT:
my call to fmemopen:
fid = fmemopen(ptr, memSize, "r");
where memSize != 8192
EDIT2:
I created a minimal example:
#include <cstdlib>
#include <stdio.h>
#include <string.h>
using namespace std;
int main(int argc, char** argv)
{
const long unsigned int memsize = 1000000;
void * ptr = malloc(memsize);
FILE *fid = fmemopen(ptr, memsize, "r");
fseek(fid, 0, SEEK_END);
long int file_size = ftell(fid);
printf("file_size = %ld\n", file_size);
free(ptr);
return 0;
}
btw, I am currently working on another computer, and here I get file_size=0
In case of fmemopen , if you open using the option b then SEEK_END measures the size of the memory buffer. The value you see must be the default buffer size.
OK, I have got this mystery solved by myself. The documentation says:
If the opentype specifies append mode, then the initial file position is set to the first null character in the buffer
and later:
For a stream open for reading, null characters (zero bytes) in the buffer do not count as "end of file". Read operations indicate end of file only when the file position advances past size bytes.
It seems that fseek(fid, 0, SEEK_END) goes to the first zero byte in the buffer, and not to the end of the buffer.
Still looking for a method that will work on both standard and fmemopen files.
I tried to read the /proc/modules using standard c functions:
FILE *pfile;
int sz;
pfile = fopen( "/proc/modules", "r" );
fseek( pfile, 0, SEEK_END );
sz = ftell( pfile );
rewind( ftell );
But my problem is ftell give me 0 value. So I can't read the contents of the file since I have a zero length. Is there another way that I can get the size of the file that I want to read?
Many thanks.
No, it does not have a size. However, you can read parts of it until you reach end-of-file.
/proc files are dynamically created when you read them, so they cannot have a size.
I stand corrected. Some /proc files do indeed have a size, as adobriyan has noted on a comment to Sjoerd's answer. (Is that Alexey Dobriyan of Linux Kernel fame?)
As for how to read the file using fgetc, this works:
int c;
while ( (c = fgetc(pfile)) != EOF) {
printf("%c",c);
}
And your program is segfaulting because you're trying to rewind ftell.