Fwrite write only 4096 instead of 100,000 bytes - c

I use fwrite to write a buffer of 100,000 chars to file, but the return value from fwrite is only 4096.
char buffer [100000];
memset(buffer,0x00,100000);
FILE *f = fopen("<path>","ab+");
if(f ==NULL)
{
return;
}
int ret =fwrite(buffer,1,100000,f);
printf("ret = %d",ret);
ret = 4096
Why this code write only 4096 bytes instead of 100,000 ?
This is Linux embedded system

From man pages:
RETURN VALUE
[...] If an error occurs, or the end of the file is
reached, the return value is a short item count (or zero).
In this case you should use ferror(f) to see if the file handle is in error condition. Also, you can zero the errno before the call, and print the error message with perror:
errno = 0;
int ret = fwrite(buffer, 1, 100000, f);
if (ret != 100000) {
printf("Stream error indication %d", ferror(f));
perror("Short item count");
}

The maximum length for any write call is defined by SSIZE_MAX which can be found in unistd.h.
This holds for every POSIX-compliant system. SSIZE_MAX may differ for different implementations.
Try out the following example to determine the maximum write length on your system:
#include <stdio.h>
#include <limits.h>
int main(void) {
printf("SSIZE_MAX : %ld\n", SSIZE_MAX);
return 0;
}
It prints SSIZE_MAX : 9223372036854775807 on my machine.
EDIT: You can also try to locate your limits.h file, but compiling might be the easier option

Related

what this error description means in fread()

I'm learning the fopen() on Ubuntu, and here is my code. Could you help on what is the specific failure reason?
a.c
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
int main() {
char reg_filename[] = "/home/chuck/Documents/enable";
FILE* f;
char val[2];
f = fopen(reg_filename, "r");
if (f == NULL) {
perror(reg_filename);
printf("error\n");
return 1;
}
setvbuf(f, (char *)NULL, _IONBF, 0);
if (fread(&val, sizeof(val), 1, f) == 0) {
perror(reg_filename);
printf("Read_error\n");
return 1;
}
return 0;
}
Build & run it...
chuck#ubuntu:~/Documents$ gcc a.c
chuck#ubuntu:~/Documents$ ./a.out
/home/chuck/Documents/enable: Success
Read_error
In the code, "enable" is a file in my system.
What I know is that it fails on fread() since "Read_error" pops. But what this "Success" mean? If it failed, why it gives the "Success" word?
And about how to use the fread(), I'm totally new... This fread() will read from f (the file path) with size of val[] length and read to val[], right?
And does it related with the val[] size, in this case, I just put it 2 then sizeof(val) should be 3? Then how will fread gonna to work to read the f (/home/chuck/Documents/enable) into it?
Read the man page for fread
size_t fread(void *ptr, size_t size, size_t nmembFILE *" stream );
And it clearly states,
The function fread() reads nmemb items of data, each size bytes long,
from the stream pointed to by stream, storing them at the location
given by ptr.
Importantly, you might want to change
fread(&val, sizeof(val), 1, f)
to
fread(val, sizeof(val), 1, f)
So it would return 0 if your enable file could be empty!
If you read a short count, (i.e. fewer items than you expected) then fread() has either encountered an error or reached the end of file before reading enough bytes.
In your case, you read 0 items instead of 1 item so there was either an error or end of file was encountered before reading a whole item. Use feof() or ferror() to find out whether you have end of file or an error. i.e. if ferror() returns non zero, there was an error and errno is meaningful.

Buffer size in file I/O

I'm trying to write a small program to find the buffer size of an open file stream. After searching around a little bit, I found the __fbufsize() function. This is the code I wrote:
#include <stdio.h>
#include <stdio_ext.h>
void main() {
FILE *f;
int bufsize;
f = fopen("test.txt","wb");
if (f == NULL) {
perror("fopen failed\n");
return;
}
bufsize = __fbufsize(f);
printf("The buffer size is %d\n",bufsize);
return;
}
I get the buffer size as zero. I'm a bit confused as to why this is happening. Shouldn't the stream be buffered by default? I get a non-zero value if I use setvbuf with _IOFBF before calling fbufsize.
Note that the correct return type for main() is int, not void.
This code compiles on Linux (Ubuntu 14.04 derivative tested):
#include <stdio.h>
#include <stdio_ext.h>
int main(void)
{
FILE *f;
size_t bufsize;
f = fopen("test.txt", "wb");
if (f == NULL)
{
perror("fopen failed\n");
return -1;
}
bufsize = __fbufsize(f);
printf("The buffer size is %zd\n", bufsize);
putc('\n', f);
bufsize = __fbufsize(f);
printf("The buffer size is %zd\n", bufsize);
fclose(f);
return 0;
}
When run, it produces:
The buffer size is 0
The buffer size is 4096
As suggested in the comments, until you use the file stream, the buffer size is not set. Until then, you could change the size with setvbuf(), so the library doesn't set the buffer size until you try to use it.
The macro BUFSIZ defined in <stdio.h> is the default buffer size. There's no standard way to find the buffer size set by setvbuf(). You need to identify the platform you're working on to allow useful commentary on __fbufsize() as a function (though it seems to be a GNU libc extension: __fbufsize()).
There are numerous small improvements that should be made in the program, but they're not immediately germane.
__fbufsize man page says:
The __fbufsize() function returns the size of the buffer currently used by the given stream.
so I think this is buffer size used by the stream.

Getting characters past a certain point in a file in C

I want to take all characters past location 900 from a file called WWW, and put all of these in an array:
//Keep track of all characters past position 900 in WWW.
int Seek900InWWW = lseek(WWW, 900, 0); //goes to position 900 in WWW
printf("%d \n", Seek900InWWW);
if(Seek900InWWW < 0)
printf("Error seeking to position 900 in WWW.txt");
char EverythingPast900[appropriatesize];
int NextRead;
char NextChar[1];
int i = 0;
while((NextRead = read(WWW, NextChar, sizeof(NextChar))) > 0) {
EverythingPast900[i] = NextChar[0];
printf("%c \n", NextChar[0]);
i++;
}
I try to create a char array of length 1, since the read system call requires a pointer, I cannot use a regular char. The above code does not work. In fact, it does not print any characters to the terminal as expected by the loop. I think my logic is correct, but perhaps a misunderstanding of whats going on behind the scenes is what is making this hard for me. Or maybe i missed something simple (hope not).
If you already know how many bytes to read (e.g. in appropriatesize) then just read in that many bytes at once, rather than reading in bytes one at a time.
char everythingPast900[appropriatesize];
ssize_t bytesRead = read(WWW, everythingPast900, sizeof everythingPast900);
if (bytesRead > 0 && bytesRead != appropriatesize)
{
// only everythingPast900[0] to everythingPast900[bytesRead - 1] is valid
}
I made a test version of your code and added bits you left out. Why did you leave them out?
I also made a file named www.txt that has a hundred lines of "This is a test line." in it.
And I found a potential problem, depending on how big your appropriatesize value is and how big the file is. If you write past the end of EverythingPast900 it is possible for you to kill your program and crash it before you ever produce any output to display. That might happen on Windows where stdout may not be line buffered depending on which libraries you used.
See the MSDN setvbuf page, in particular "For some systems, this provides line buffering. However, for Win32, the behavior is the same as _IOFBF - Full Buffering."
This seems to work:
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <stdio.h>
int main()
{
int WWW = open("www.txt", O_RDONLY);
if(WWW < 0)
printf("Error opening www.txt\n");
//Keep track of all characters past position 900 in WWW.
int Seek900InWWW = lseek(WWW, 900, 0); //goes to position 900 in WWW
printf("%d \n", Seek900InWWW);
if(Seek900InWWW < 0)
printf("Error seeking to position 900 in WWW.txt");
int appropriatesize = 1000;
char EverythingPast900[appropriatesize];
int NextRead;
char NextChar[1];
int i = 0;
while(i < appropriatesize && (NextRead = read(WWW, NextChar, sizeof(NextChar))) > 0) {
EverythingPast900[i] = NextChar[0];
printf("%c \n", NextChar[0]);
i++;
}
return 0;
}
As stated in another answer, read more than one byte. The theory behind "buffers" is to reduce the amount of read/write operations due to how slow disk I/O (or network I/O) is compared to memory speed and CPU speed. Look at it as if it is code and consider which is faster: adding 1 to the file size N times and writing N bytes individually, or adding N to the file size once and writing N bytes at once?
Another thing worth mentioning is the fact that read may read fewer than the number of bytes you requested, even if there is more to read. The answer written by #dreamlax illustrates this fact. If you want, you can use a loop to read as many bytes as possible, filling the buffer. Note that I used a function, but you can do the same thing in your main code:
#include <sys/types.h>
/* Read from a file descriptor, filling the buffer with the requested
* number of bytes. If the end-of-file is encountered, the number of
* bytes returned may be less than the requested number of bytes.
* On error, -1 is returned. See read(2) or read(3) for possible
* values of errno.
* Otherwise, the number of bytes read is returned.
*/
ssize_t
read_fill (int fd, char *readbuf, ssize_t nrequested)
{
ssize_t nread, nsum = 0;
while (nrequested > 0
&& (nread = read (fd, readbuf, nrequested)) > 0)
{
nsum += nread;
nrequested -= nread;
readbuf += nread;
}
return nsum;
}
Note that the buffer is not null-terminated as not all data is necessarily text. You can pass buffer_size - 1 as the requested number of bytes and use the return value to add a null terminator where necessary. This is useful primarily when interacting with functions that will expect a null-terminated string:
char readbuf[4096];
ssize_t n;
int fd;
fd = open ("WWW", O_RDONLY);
if (fd == -1)
{
perror ("unable to open WWW");
exit (1);
}
n = lseek (fd, 900, SEEK_SET);
if (n == -1)
{
fprintf (stderr,
"warning: seek operation failed: %s\n"
" reading 900 bytes instead\n",
strerror (errno));
n = read_fill (fd, readbuf, 900);
if (n < 900)
{
fprintf (stderr, "error: fewer than 900 bytes in file\n");
close (fd);
exit (1);
}
}
/* Read a file, printing its contents to the screen.
*
* Caveat:
* Not safe for UTF-8 or other variable-width/multibyte
* encodings since required bytes may get cut off.
*/
while ((n = read_fill (fd, readbuf, (ssize_t) sizeof readbuf - 1)) > 0)
{
readbuf[n] = 0;
printf ("Read\n****\n%s\n****\n", readbuf);
}
if (n == -1)
{
close (fd);
perror ("error reading from WWW");
exit (1);
}
close (fd);
I could also have avoided the null termination operation and filled all 4096 bytes of the buffer, electing to use the precision part of the format specifiers of printf in this case, changing the format specification from %s to %.4096s. However, this may not be feasible with unusually large buffers (perhaps allocated by malloc to avoid stack overflow) because the buffer size may not be representable with the int type.
Also, you can use a regular char just fine:
char c;
nread = read (fd, &c, 1);
Apparently you didn't know that the unary & operator gets the address of whatever variable is its operand, creating a value of type pointer-to-{typeof var}? Either way, it takes up the same amount of memory, but reading 1 byte at a time is something that normally isn't done as I've explained.
Mixing declarations and code is a no no. Also, no, that is not a valid declaration. C should complain about it along the lines of it being variably defined.
What you want is dynamically allocating the memory for your char buffer[]. You'll have to use pointers.
http://www.ontko.com/pub/rayo/cs35/pointers.html
Then read this one.
http://www.cprogramming.com/tutorial/c/lesson6.html
Then research a function called memcpy().
Enjoy.
Read through that guide, then you should be able to solve your problem in an entirely different way.
Psuedo code.
declare a buffer of char(pointer related)
allocate memory for said buffer(dynamic memory related)
Find location of where you want to start at
point to it(pointer related)
Figure out how much you want to store(technically a part of allocating memory^^^)
Use memcpy() to store what you want in the buffer

Reading a large file using C (greater than 4GB) using read function, causing problems

I have to write C code for reading large files. The code is below:
int read_from_file_open(char *filename,long size)
{
long read1=0;
int result=1;
int fd;
int check=0;
long *buffer=(long*) malloc(size * sizeof(int));
fd = open(filename, O_RDONLY|O_LARGEFILE);
if (fd == -1)
{
printf("\nFile Open Unsuccessful\n");
exit (0);;
}
long chunk=0;
lseek(fd,0,SEEK_SET);
printf("\nCurrent Position%d\n",lseek(fd,size,SEEK_SET));
while ( chunk < size )
{
printf ("the size of chunk read is %d\n",chunk);
if ( read(fd,buffer,1048576) == -1 )
{
result=0;
}
if (result == 0)
{
printf("\nRead Unsuccessful\n");
close(fd);
return(result);
}
chunk=chunk+1048576;
lseek(fd,chunk,SEEK_SET);
free(buffer);
}
printf("\nRead Successful\n");
close(fd);
return(result);
}
The issue I am facing here is that as long as the argument passed (size parameter) is less than 264000000 bytes, it seems to be able to read. I am getting the increasing sizes of the chunk variable with each cycle.
When I pass 264000000 bytes or more, the read fails, i.e.: according to the check used read returns -1.
Can anyone point me to why this is happening? I am compiling using cc in normal mode, not using DD64.
In the first place, why do you need lseek() in your cycle? read() will advance the cursor in the file by the number of bytes read.
And, to the topic: long, and, respectively, chunk, have a maximum value of 2147483647, any number greater than that will actually become negative.
You want to use off_t to declare chunk: off_t chunk, and size as size_t.
That's the main reason why lseek() fails.
And, then again, as other people have noticed, you do not want to free() your buffer inside the cycle.
Note also that you will overwrite the data you have already read.
Additionally, read() will not necessarily read as much as you have asked it to, so it is better to advance chunk by the amount of the bytes actually read, rather than amount of bytes you want to read.
Taking everything in regards, the correct code should probably look something like this:
// Edited: note comments after the code
#ifndef O_LARGEFILE
#define O_LARGEFILE 0
#endif
int read_from_file_open(char *filename,size_t size)
{
int fd;
long *buffer=(long*) malloc(size * sizeof(long));
fd = open(filename, O_RDONLY|O_LARGEFILE);
if (fd == -1)
{
printf("\nFile Open Unsuccessful\n");
exit (0);;
}
off_t chunk=0;
lseek(fd,0,SEEK_SET);
printf("\nCurrent Position%d\n",lseek(fd,size,SEEK_SET));
while ( chunk < size )
{
printf ("the size of chunk read is %d\n",chunk);
size_t readnow;
readnow=read(fd,((char *)buffer)+chunk,1048576);
if (readnow < 0 )
{
printf("\nRead Unsuccessful\n");
free (buffer);
close (fd);
return 0;
}
chunk=chunk+readnow;
}
printf("\nRead Successful\n");
free(buffer);
close(fd);
return 1;
}
I also took the liberty of removing result variable and all related logic since, I believe, it can be simplified.
Edit: I have noted that some systems (most notably, BSD) do not have O_LARGEFILE, since it is not needed there. So, I have added an #ifdef in the beginning, which would make the code more portable.
The lseek function may have difficulty in supporting big file sizes. Try using lseek64
Please check the link to see the associated macros which needs to be defined when you use lseek64 function.
If its 32 bit machine, it will cause some problem for reading a file of larger than 4gb. So if you are using gcc compiler try to use the macro -D_LARGEFILE_SOURCE=1 and -D_FILE_OFFSET_BITS=64.
Please check this link also
If you are using any other compiler check for similar types of compiler option.

Why does fread always return 0?

I used this code to read file. But fread function always return 0. What is my mistake?
FILE *file = fopen(pathToSourceFile, "rb");
if(file!=NULL)
{
char aByte[50000];
int ret = fread(aByte, sizeof(aByte), 1, file);
if(ret != 0)
{
not jump into there;
fseek(file, 0, SEEK_SET);
fwrite(aByte, ret, 1, file);
}
}
fclose(file);
are you sure that your file has a size greater than 50000 ? otherwise you could try:
fread(aByte,1, sizeof(aByte), file);
ferror() will tell when something is wrong.
You can print the actual error message using perror().
You can't fwrite to a file open in rb mode.
Your statement that ret is always zero is false. If you had properly instrumented your code, you'd not be making false claims:
#include <stdio.h>
#include <stdlib.h>
int main() {
FILE *file = fopen("junk.dat", "rb");
if(file!=NULL)
{
char aByte[50000];
int ret = fread(aByte, sizeof(aByte), 1, file);
fprintf(stderr, "fread returned %d\n", ret);
if(ret != 0)
{
int fs = fseek(file, 0, SEEK_SET);
if(fs == -1) {
perror("fseek");
exit(1);
}
fs = fwrite(aByte, ret, 1, file);
if(fs != ret) {
perror("fwrite");
exit(1);
}
}
}
fclose(file);
return 0;
}
Yields:
fread returned 1
fwrite: Bad file descriptor
when run.
In my case I wanted to read a file of size 6553600 bytes (an mp3), and it was returning 0 bytes read. It drove me crazy, till, I tried to manually hardcode 30 bytes, and it did read 30 bytes.
I started playing with it and see how much can it read, and it turns out that it can read exactly 262144 (2^18) bytes, if you ask it to read 262145 bytes it reads 0.
Conclusion: at least with this function you can't load the whole file in one go.
In case someone else runs into this. I just ran into a similar issue. It is because the 2nd argument to fread should be the size of each element in the buffer. In OP's code it is the size of the pointer to the buffer.
This should work provided buff has at least 1 element:
int ret = fread(aByte, sizeof(aByte[0]), 1, file);
Please check man fread
man fread(3)
size_t fread(void *restrict ptr, size_t size, size_t nmemb, FILE *restrict stream);
RETURN VALUE
On success, fread() and fwrite() return the number of items read
or written. This number equals the number of bytes transferred
only when size is 1. If an error occurs, or the end of the file
is reached, the return value is a short item count (or zero).
As your file is smaller than 50000Bytes aka. size of a item, the read item count is 0.
In my case,
fseek(rFile, 0, SEEK_END);
iTotalSize = ftell(rFile);
fseek(rFile, 0, SEEK_SET); // <-- I wrote SEEK_END, not SEEK_SET
so it read 0 byte(anything)
Did you:
#include <unistd.h>
If not, and if you compile without -Wall, the C compiler can incorrectly assume that the second argument to fread() is an int rather than an off_t, which can mess up the function call. Your code snippet doesn't show any #include statements, so please make sure you're including everything that you're using.

Resources