What is the most efficient quickest way to write all zeros to a file? including error checking. Would it just be fwrite? or is fseek involved?
I've looked elsewhere and saw code similar to this:
off_t size = fseek(pFile,0,SEEK_END);
fseek(pFile,0,SEEK_SET);
while (size>sizeof zeros)
size -= fwrite(&address, 1, sizeof zeros, pFile);
while (size)
size -= fwrite(&address, 1, size, pFile);
where zeros is an array of file size I suspect. Not sure exactly what off_t was because it wasn't directly intuitive to me anyways
Do you want to replace the contents of the file with a stream of binary zeroes of the same length, or do you want to simply empty the file? (make it have length zero)
Either way, this is best done with the OS file I/O primitives. Option one:
char buf[4096];
struct stat st;
int fd;
off_t pos;
ssize_t written;
memset(buf, 0, 4096);
fd = open(file_to_overwrite, O_WRONLY);
fstat(fd, &st);
for (pos = 0; pos < st.st_size; pos += written)
if ((written = write(fd, buf, min(st.st_size - pos, 4096))) <= 0)
break;
fsync(fd);
close(fd);
Option two:
int fd = open(file_to_truncate, O_WRONLY);
ftruncate(fd, 0);
fsync(fd);
close(fd);
Error handling left as an exercise.
mmap() and memset()
Related
So i got my function here that works to write back any file
int write_file(FILE *f_write) {
// Temp variables
FILE *img = fopen("test.pdf", "wb");
unsigned char buffer[255];
while ( (bytes_read = fread(buffer, 1, sizeof(buffer), f_write) ) > 0) {
fwrite(buffer, 1, bytes_read, img);
}
fclose(img);
return 1;
}
So this works perfecly ive tried with pnj / pdf / jpg etc..
But now i want to stock what ive writen in the memory so i can use it later and not write right away
like an array of uint8_t (maybe) that will contain all the bytes ive writen and that i can send later with sockets to my server and store the file
no idea how to do it
Or maybe i'm making it too complicated and i can just
send(client_socket, FILE, sizeof(FILE), 0); ?
One way to do it would be to create a buffer that exactly fits the size of the file.
In order to do so, you can write a function to get the size of an openned file like so:
size_t get_file_size(FILE *f)
{
size_t pos = ftell(f); // store the cursor position
size_t size;
// go to the end of the file and get the cursor position
fseek(f, 0L, SEEK_END);
size = ftell(f);
// go back to the old position
fseek(f, pos, SEEK_SET);
return size;
}
Then create and fill your buffer:
FILE *f = fopen("your_file", "r");
size_t size = get_file_size(f);
char *buffer = malloc(size);
if (fread(buffer, 1, size, f) != size) { // bytes read != computed file size
// error handling
}
// use your buffer...
// don't forget to free and fclose
free(buffer);
fclose(f);
It is worth mentioning that you should check if the file was opened correctly, and to check if you have enough memory to store the buffer (the one created with malloc).
Edit:
As Andrew Henle said, fseek()/ftell() to get the size of a file is non-portable. Instead, to get the size of your file, you should use one of these techniques depending on your OS (assuming you are trying to open a 'normal' file):
On Linux / MacOS:
#include <sys/stat.h>
struct stat st;
size_t size;
if (stat("your_file", &st) != 0) {
// error handling...
}
size = st.st_size;
On Windows (as answered here) :
__int64 FileSize(const wchar_t* name)
{
HANDLE hFile = CreateFile(name, GENERIC_READ,
FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL, NULL);
if (hFile == INVALID_HANDLE_VALUE)
return -1; // error condition, could call GetLastError to find out more
LARGE_INTEGER size;
if (!GetFileSizeEx(hFile, &size)) {
CloseHandle(hFile);
return -1; // error condition, could call GetLastError to find out more
}
CloseHandle(hFile);
return size.QuadPart;
}
unsigned int file = open(argv[1], O_RDONLY);
printf("%u\n", file);
printf("%u\n", elf.offset);
lseek(file, elf.offset, SEEK_SET);
printf("%u", file);
OutPut:
3
52
3
Shouldn't file be set to 52?
Upon successful completion, the resulting offset, as measured in bytes from the beginning of the file, shall be returned.
try this printf("lseek_offset: %d\n", lseek(file, elf.offset, SEEK_SET));
file is a file descriptor. When you print it, you print the file descriptor, not the offset. When you lseek it to an offset of 52, the file descriptor is still 3, so it still prints 3.
You can read more about file descriptors here.
yon confuse file with file decriptor. The latter is just a non-negative integer that identifies an open file.
maybe this example can help you to understand these two concepts better:
char buf[8];
int main(){
int fd = open("test", O_RDONLY);
off_t offset = lseek(fd, 0, SEEK_CUR);
read(fd, buf, sizeof buf);
printf("first read when offset = %d : %s\n", (int)offset, buf);
offset = lseek(fd, 32, SEEK_SET);
read(fd, buf, sizeof buf);
printf("second read when offset = %d : %s\n", (int)offset, buf);
return 0;
}
and the output is:
first read when offset = 0 : 0000000
second read when offset = 32 : 4444444
here are the contents of test:
0000000\n
1111111\n
2222222\n
3333333\n
4444444\n
I'm trying to read from an input file and ultimately reverse the buffer it reads from and write it to an output file. For now though, I'm testing to see if a buffer I read from would even make it to the output file, and so far it isn't and I'm getting an infinite loop. The buffer should read in PAGESIZE bytes (from a call to sysconf()) and if the file output is larger than the buffer, then the buffer should be written to the output file first then be flushed and reused again to get the rest of the input until the file descriptor returns 0 for no data left. This is what I have so far:
int fdRead = open(inputFile, O_RDONLY);
if (fdRead == -1)
err_sys("Error reading input file '%s', check spelling?\n", inputFile);
int fdWrite = open(outputFile, O_WRONLY | O_CREAT | O_TRUNC, 0644); //overwrites file if it exists
if (fdWrite == -1)
err_sys("Error creating output file '%s'\n", outputFile);
while (1) {
read(fdRead, buf, size);
if (fdRead == 0)
break;
if (fdRead == -1)
err_sys("Error reading from input file '%s'\n", inputFile);
lseek(fdRead, size, SEEK_CUR);
if (fdRead == -1)
err_sys("Error reading from input file '%s'\n", inputFile);
write(fdWrite, buf, size);
if (fdWrite == -1)
err_sys("Error writing to output file '%s'\n", outputFile);
lseek(fdWrite, size, SEEK_CUR);
if (fdWrite == -1)
err_sys("Error writing to output file '%s'\n", outputFile);
memset(buf, '\0', size);
}
close(fdRead);
close(fdWrite);
I suppose that fdRead is never returning 0, and thus not exiting the loop. My question is how do I fix that?
p.s: size is the call from sysconf() that gets the PAGESIZE, e.g
size = sysconf(_SC_PAGESIZE);
And inputFile and outputFile are both char * and I've tested that they return and store good strings.
Transcribing comments into an answer.
You need to capture the return value from read() — you're ignoring it and testing whether the file descriptor is 0 or negative after the read().
So if I did something like int bytesRead then tested for if (bytesRead == 0) instead of if (fdRead == 0), then that should solve my problem?
Yes, you need something like:
int nbytes = read(fdRead, buf, size);
if (nbytes <= 0) break;
You should use the positive nbytes in the write() call; you might not get all size bytes filled by the read().
Testing the file descriptor after the read is wrong (it won't have changed under normal circumstances), and ignoring the value returned by read() is wrong, and not using the value returned by read() in the call to write() is wrong.
OK so it should be write(fdWrite, buf, nbytes)?
Yes, it should be
int obytes;
if ((obytes = write(fdWrite, buf, nbytes)) != nbytes)
{
…oops — short write …
}
You get to decide what's the appropriate response to a short write (a positive value, but not the number of bytes you expected to write). If you're writing to a socket, it might be appropriate to try writing the unwritten section of the data again (that's why obytes is used to capture the number of bytes successfully written). If you're writing to a disk file, it probably means there's no space left, so there's no point (little point) in trying again. If obytes is negative, you've had a write error; there is usually little point in trying to continue.
This all has helped out a lot and it seems to be working okay. I've only run into one other problem. I tested this on a large file (Alice in Wonderland text file) and the output file is almost the whole thing, but cuts off the last two paragraphs or so. …
You need to review why you have the lseek() operations in the code. Neither of them should be necessary, and both are dubious. I think the lseek() on fdRead() means you miss chunks of text of size bytes each; I think the lseek() on fdWrite() means you insert size null bytes into the output file.
/*Here's the code which may help for that.i havemodified in it for your need.*/
int fdRead = open(inputFile, O_RDONLY);
int Ret_Val;
if (fdRead == -1)
err_sys("Error reading input file'%s',check spelling?\n",
inputFile );
int fdWrite = open(outputFile, O_WRONLY | O_CREAT
| O_TRUNC, 0644); //overwrites file if it exists
if (fdWrite == -1)
err_sys("Error creating output file '%s'\n", outputFile);
while (1) {
Ret_Val=read(fdRead, buf, size);
if (Ret_Val == 0)
break;
if (Ret_Val == -1)
err_sys("Error reading from input file'%s'\n",inputFile);
lseek(fdRead, size, SEEK_CUR);
if (Ret_Val == -1)
err_sys("Error reading from input file'%s'\n",
inputFile );
Ret_Val=write(fdWrite, buf, size);
if (Ret_Val == -1)
err_sys("Error writing to output file'%s'\n",outputFile);
lseek(fdWrite, size, SEEK_CUR);
if (Ret_Val == -1)
err_sys("Error writing to output file'%s'\n",outputFile);
memset(buf, '\0', size);
}
close(fdRead);
close(fdWrite);
My code is too long to post all here so i'm going to sum up what's wrong.
In a server part i'm sending on a socket 3 things :
A message
The content of a file
Another message
In a client part i'm receiving these things but :
This first is to print on terminal
The second to write in a new file
The last to print on the terminal too
But my client is stuck on a read and i really don't know why. I'm on the problem for hour so if someone can help me, it will be very great !
edit : Basically, i think my problem is that i don't know what to write on the server to stop the read on the client.. Is it \n, \0.. ?
Here's the 2 part of code :
server
void send_content(t_server *s, FILE *fd, int rfd)
{
int len;
char *buff;
write(s->socket, "150 File status okay;" \
"about to open data connection.\n\0", strlen("150 File status okay;about to open data connection.\n\0"));
fseek(fd, 0, SEEK_END);
len = ftell(fd);
buff = malloc(len * sizeof(char));
read(rfd, buff, len);
write(s->socket, buff, len);
write(s->socket, "\n\0", strlen("\n\0"));
write(s->socket, "226 Closing data connection.\n\0", strlen("226 Closing data connection.\n\0"));
free(buff);
}
client
void getfile(t_client *c, char **tab)
{
int ret;
int fd;
int z;
char buff[4096];
z = 0;
read(c->fd, buff, 4096);
write(1, buff, strlen(buff));
if (strlen(buff) < 25)
return ;
fd = creat(tab[1], S_IRUSR | S_IWUSR);
while (z == 0 && (ret = read(c->fd, buff, 4096)) > 0)
{
if (ret < 4096)
z = -1;
write(fd, buff, strlen(buff));
memset(buff, '\0', 4096);
}
read(c->fd, buff, 4096); // Stuck here
write(1, buff, strlen(buff));
close(fd);
}
Like noted you need a read function like this to make sure you receive
specified number of bytes(this function will loop till it receives number of bytes it was told to). Just use this receivall method instead of read everywhere.
With files you typically first send the file length, and then receive the file.
I did something similar while ago, hope it will help you a bit. This is the client side, which tries to receive first file length from the server, then the file:
/* create file */
FILE * ptrMyFile = fopen(&filenames[i][0],"wb");
if(NULL == ptrMyFile)
{
printf("Unable to open file \n");
return 1;
}
int size = 0;
int t = 4;
/* first receive file size from server */
/* NOTE: error checking is omitted from code, nevertheless users should stil do some error checking when using this code */
readall(sockfd, (unsigned char*) &size, &t);
/* how many 256 byte chunks are there? */
int div = size / 256;
/* loop to receive each chunk. */
for(int k = 0; k < div; k++)
{
int chunk_size = 256;
/* make sure we receive 256 bytes */
readall(sockfd, buffer, &chunk_size);
/* write to file */
fwrite(buffer, chunk_size, 1, ptrMyFile);
}
/* read the final chunk. */
int whatsleft = size - 256 * div;
readall(sockfd, buffer, &whatsleft);
/* write */
fwrite(buffer, whatsleft, 1, ptrMyFile);
/* close file */
fclose(ptrMyFile);
I leave the server part to you.
char buff[4096];
z = 0;
read(c->fd, buff, 4096);
write(1, buff, strlen(buff));
You should be saving the return value of the call to read(), in order to find out how many bytes you just received. You may have to make several calls to read() in order to get the entire message. It's wrong to use strlen() to find out how many bytes were received, because the buffer contents are uninitialized, and the first chunk of the message could be cut off anywhere, so you can't count on it being null-terminated.
I really have a problem in here. It seems that i dont really find the best way to exit a loop when reading characters from a file. I know that every tutorial suggests that i shouldn't use while( !feof() ) but they dont really suggest anything else than putting fgets() in the while and that is not really apropriate because i want to read the whole FILE content in my variable.
while (!feof(newFile))
{
newString[i++] = fgetc(newFile);
}
newString[i] = '\0';
i = 0;
//this is the resoult seen with the debugger
newFile content = ABC
newString[0] = 65 (A)
newString[1] = 66 (B)
newString[2] = 67 (C)
newString[3] = 10 (\n)
newString[4] = -1
newString[5] = 0 (\0)
I am looking for a solution and some advices about how to improve my algorithm.
int c;
while ((c = fgetc(newFile)) != EOF) newString[i++] = c;
newString[i] = '\0';
For reading whole test files into memory, I suggest using mmap. This has the benefit, that all buffering and reading can be handled by your operating system, and you can focus your code on the task at hand. (also, it's usually faster than buffering stuff yourself.)
#include <stdio.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/stat.h>
int
main (void)
{
int fd = open("filename", O_RDONLY);
if (fd == -1)
return 0; // file open failed
struct stat sb;
int res = fstat(fd, &sb);
if (res == -1)
return 0; // stat failed
size_t length = sb.st_size;
char *data = mmap(NULL, sb.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
if (!data)
return 0; // mmap failed
// iterate over characters
size_t i;
for (i = 0; i < length; ++i)
printf("'%c'\n", data[i]);
munmap(data, length);
return 0;
}
they dont really suggest anything else than putting fgets() in the while and that is not really apropriate
That is absolutely, entirely appropriate. fgets() reads the file line by line, and you can append each line onto then end of a dybamically expanding buffer.
However, if you don't want to use fgets(), and you just want to read the file at once: use fread().
FILE *f = fopen("foo.txt", "rb");
if (!f)
abort(); // "handle" error
fseek(f, 0, SEEK_END);
size_t len = ftell(f);
fseek(f, 0, SEEK_SET);
char *buf = malloc(len + 1);
if (!buf)
abort();
if (fread(buf, len, 1, f) != 1) {
// handle reading error
}
buf[len] = 0;
fclose(f);