I'm trying to read from an input file and ultimately reverse the buffer it reads from and write it to an output file. For now though, I'm testing to see if a buffer I read from would even make it to the output file, and so far it isn't and I'm getting an infinite loop. The buffer should read in PAGESIZE bytes (from a call to sysconf()) and if the file output is larger than the buffer, then the buffer should be written to the output file first then be flushed and reused again to get the rest of the input until the file descriptor returns 0 for no data left. This is what I have so far:
int fdRead = open(inputFile, O_RDONLY);
if (fdRead == -1)
err_sys("Error reading input file '%s', check spelling?\n", inputFile);
int fdWrite = open(outputFile, O_WRONLY | O_CREAT | O_TRUNC, 0644); //overwrites file if it exists
if (fdWrite == -1)
err_sys("Error creating output file '%s'\n", outputFile);
while (1) {
read(fdRead, buf, size);
if (fdRead == 0)
break;
if (fdRead == -1)
err_sys("Error reading from input file '%s'\n", inputFile);
lseek(fdRead, size, SEEK_CUR);
if (fdRead == -1)
err_sys("Error reading from input file '%s'\n", inputFile);
write(fdWrite, buf, size);
if (fdWrite == -1)
err_sys("Error writing to output file '%s'\n", outputFile);
lseek(fdWrite, size, SEEK_CUR);
if (fdWrite == -1)
err_sys("Error writing to output file '%s'\n", outputFile);
memset(buf, '\0', size);
}
close(fdRead);
close(fdWrite);
I suppose that fdRead is never returning 0, and thus not exiting the loop. My question is how do I fix that?
p.s: size is the call from sysconf() that gets the PAGESIZE, e.g
size = sysconf(_SC_PAGESIZE);
And inputFile and outputFile are both char * and I've tested that they return and store good strings.
Transcribing comments into an answer.
You need to capture the return value from read() — you're ignoring it and testing whether the file descriptor is 0 or negative after the read().
So if I did something like int bytesRead then tested for if (bytesRead == 0) instead of if (fdRead == 0), then that should solve my problem?
Yes, you need something like:
int nbytes = read(fdRead, buf, size);
if (nbytes <= 0) break;
You should use the positive nbytes in the write() call; you might not get all size bytes filled by the read().
Testing the file descriptor after the read is wrong (it won't have changed under normal circumstances), and ignoring the value returned by read() is wrong, and not using the value returned by read() in the call to write() is wrong.
OK so it should be write(fdWrite, buf, nbytes)?
Yes, it should be
int obytes;
if ((obytes = write(fdWrite, buf, nbytes)) != nbytes)
{
…oops — short write …
}
You get to decide what's the appropriate response to a short write (a positive value, but not the number of bytes you expected to write). If you're writing to a socket, it might be appropriate to try writing the unwritten section of the data again (that's why obytes is used to capture the number of bytes successfully written). If you're writing to a disk file, it probably means there's no space left, so there's no point (little point) in trying again. If obytes is negative, you've had a write error; there is usually little point in trying to continue.
This all has helped out a lot and it seems to be working okay. I've only run into one other problem. I tested this on a large file (Alice in Wonderland text file) and the output file is almost the whole thing, but cuts off the last two paragraphs or so. …
You need to review why you have the lseek() operations in the code. Neither of them should be necessary, and both are dubious. I think the lseek() on fdRead() means you miss chunks of text of size bytes each; I think the lseek() on fdWrite() means you insert size null bytes into the output file.
/*Here's the code which may help for that.i havemodified in it for your need.*/
int fdRead = open(inputFile, O_RDONLY);
int Ret_Val;
if (fdRead == -1)
err_sys("Error reading input file'%s',check spelling?\n",
inputFile );
int fdWrite = open(outputFile, O_WRONLY | O_CREAT
| O_TRUNC, 0644); //overwrites file if it exists
if (fdWrite == -1)
err_sys("Error creating output file '%s'\n", outputFile);
while (1) {
Ret_Val=read(fdRead, buf, size);
if (Ret_Val == 0)
break;
if (Ret_Val == -1)
err_sys("Error reading from input file'%s'\n",inputFile);
lseek(fdRead, size, SEEK_CUR);
if (Ret_Val == -1)
err_sys("Error reading from input file'%s'\n",
inputFile );
Ret_Val=write(fdWrite, buf, size);
if (Ret_Val == -1)
err_sys("Error writing to output file'%s'\n",outputFile);
lseek(fdWrite, size, SEEK_CUR);
if (Ret_Val == -1)
err_sys("Error writing to output file'%s'\n",outputFile);
memset(buf, '\0', size);
}
close(fdRead);
close(fdWrite);
Related
I tried to write basic program in C which copy data from file to another with given source path, destination path and buffer size as input.
my problem is the destination file filled with junk or something because its way larger than the source (get bigger depending on buffer size) and can't be open.
How do i read and write just the bytes in the source?
i'm working in linux, and this is the actually copying part:
char buffer[buffer_size];
int readable=1;
int writeable;
while(readable != 0){
readable = read(sourcef, buffer, buffer_size);
if(readable == -1){
close(sourcef);
close(destf);
exit_with_usage("Could not read.");
}
writeable = write(destf, buffer, buffer_size);
if(writeable == -1){
close(sourcef);
close(destf);
exit_with_usage("Could not write.");
}
}
writeable = write(destf, buffer, buffer_size);
must be
writeable = write(destf, buffer, readable);
Currently you do not write the number of characters you read but all the buffer, so the output file is too large
You also manage wrongly the end of the input file
The return value of read is :
On success, the number of bytes read is returned (zero indicates end of file)
On error, -1 is returned
A proposal :
/* you already check input and output file was open with success */
char buffer[buffer_size];
for(;;){
ssize_t readable = read(sourcef, buffer, buffer_size);
if(readable <= 0){
close(sourcef);
close(destf);
if (readable != 0)
/* not EOF */
exit_with_usage("Could not read.");
/* EOF */
break;
}
if (write(destf, buffer, n) != n) {
close(sourcef);
close(destf);
exit_with_usage("Could not write.");
}
}
I suppose exit_with_usage calls exit() so does not return
Note in theory write may write less than the expected number of characters without being an error, and the write has to be done in a loop, but in that case it is useless to manage that
read function returns how many bytes were read to buffer(which has buffer_size). Its not always the case actual bytes read has same value as buffer size(consider scenario if there are not enough bytes left in source file to fully fill your buffer). So you should write to destination file not buffer_size(third argument of the write function), but how many bytes have you read - that is readable variable in your code
You should exit when readable returns an error.So
while(readable != 0){
should be
while(readable != -1){
So that loop could be terminataed when an readfile is exhausted.
You see currently after the whole readfile has been read, calling read fails but write is being called repeatedly since execution has no exit path for failure on read. Also write should only write the number of bytes read. So the code would look like this:
char buffer[buffer_size];
int readable=1;
int writeable;
while(readable != -1){
readable = read(sourcef, buffer, buffer_size);
if(readable == -1){
close(sourcef);
close(destf);
exit_with_usage("Could not read.");
}
writeable = write(destf, buffer, readable);
if(writeable == -1){
close(sourcef);
close(destf);
exit_with_usage("Could not write.");
}
}
Simple code
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h> // For system calls write, read e close
#include <fcntl.h>
#define BUFFER_SIZE 4096
int main(int argc, char* argv[]) {
if (argc != 3) {
printf("Usage %s Src_file Dest_file\n", argv[0]);
exit(1);
}
unsigned char buffer[BUFFER_SIZE] = {0};
ssize_t ReadByte = 0;
int src_fd, dst_fd;
// open file in read mode
if ((src_fd = open(argv[1], O_RDONLY)) == -1) {
printf("Failed to open input file %s\n", argv[1]);
exit(1);
}
// open file in write mode and already exists to overwrite
if ((dst_fd = open(argv[2], O_WRONLY | O_CREAT | O_TRUNC, 644)) == -1) {
printf("Failed to create output file %s\n", argv[2]);
exit(1);
}
// loop
while (1) {
// read buffer
ReadByte = read(src_fd, buffer, sizeof(buffer));
// error with reading
if (ReadByte == -1) {
printf("Encountered an error\n");
break;
} else if (ReadByte == 0) {
// file end exit loop
printf("File copying successful.\n");
break;
}
// error with writing
if (write(dst_fd, buffer, ReadByte) == -1) {
printf("Failed to copying file\n");
break;
}
}
// Close file
close(src_fd);
close(dst_fd);
exit(0);
}
Run
./program src_file dest_file
Here is my code snippet:
int fd;
bufsize = 30;
char buf[bufsize];
char cmd[100] = "file.txt";
int newfd = 1;
if (fd = open(cmd,O_RDONLY) >=0){
puts("wanna read");
while (read(fd,&bin_buf,bufsize)==1){
puts("reading");
write(newfd,&bin_buf,bufsize);
}
close(fd);
}
So here the program prints "wanna read" but never prints "reading". I have also tried opening using nonblock flag, but no use. Can anybody help me? I must use open() and read() system calls only. Thanks.
Edit: I have made some clarifications in the code. Actually the newfd that I'm writing to is a socket descriptor, but I don't think that is important for this problem because it sticks on the read which is before the write.
The first problem is your if statement. You forgot to use enough parentheses, so if the open() works, the read tries to read from file descriptor 1, aka standard output. If that's your terminal (it probably is) on a Unix box, then that works — surprising though that may be; the program is waiting for you to type something.
Fix: use parentheses!
if ((fd = open(cmd, O_RDONLY)) >= 0)
The assignment is done before, not after, the comparison.
I observe in passing that you don't show how you set cmd, but if you see the 'wanna read' message, it must be OK. You don't show how newfd is initialized; maybe that's 1 too.
You also have the issue with 'what the read() call returns'. You probably need:
int fd;
char buf[bufsize];
int newfd = 1;
if ((fd = open(cmd, O_RDONLY)) >= 0)
{
puts("wanna read");
int nbytes; // ssize_t if you prefer
while ((nbytes = read(fd, buf, sizeof(buf))) > 0)
{
puts("reading");
write(newfd, buf, nbytes);
}
close(fd);
}
You can demonstrate my primary observation by typing something ('Surprise', or 'Terminal file descriptors are often readable and writable' or something) with your original if but my loop body and then writing that somewhere.
Your read() call attempts to read bufsize bytes and returns the number of bytes actually read. Unless bufsize ==, it is quite unlikely read() will return 1, so the block is almost always skipped and nothing get written.
Also note that if (fd = open(cmd, O_RDONLY) >= 0) is incorrect and would set fd to 1, the handle for standard output, if the file exists, causing the read to fail as standard input is most likely not opened for reading.
Note that reading with the read system call is tricky on some environments, because a return value of -1 may be restartable.
Here is an improved version:
int catenate_file(const char *cmd, int newfd, size_t bufsize) {
int fd;
char buf[bufsize];
if ((fd = open(cmd, O_RDONLY)) >= 0) {
puts("wanna read");
ssize_t nc;
while ((nc = read(fd, buf, bufsize)) != 0) {
if (nc < 0) {
if (errno == EINTR)
continue;
else
break;
}
printf("read %zd bytes\n", nc);
write(newfd, buf, nc);
}
close(fd);
return 0;
}
return -1;
}
read returns the number of bytes read from file that can be bufsize or less if the remainder of the file that has to be read is shorter than bufsize.
In your case most probably bufsize is bigger than 1 and the file is bigger than 1 byte so the condition of the while loop is evaluated false, the code is skipped to the point where file is closed.
You should check if there if there are more bytes to be read:
while( read(fd,&bin_buf,bufsize) > 0 ) {
I created an application that send a text file from client to server
So far i'm send it as string like this:
fp = fopen(filename, "r");
if (fp != NULL) {
newLen = fread(source, sizeof(char), 5000, fp);
if (newLen == 0) {
fputs("Error reading file", stderr);
} else {
source[++newLen] = '\0'; /* Just to be safe. */
}
}else{
printf("The file %s does not exist :(");
return 1;
}
fclose(fp);
send(s , source , strlen(source) , 0); //send file
However my professor told me I must send the file in Binary and be ready to accept a file of any size
I'm trying to figure out how to send the file in binary and break it into chunks
You can copy it one byte at a time.
Reading/writing more than a byte at a time theoretically would make it read and write more efficiently to disk. But since the binary is likely short, and disk I/O is already internally buffered it probably doesn't make a noticeable difference.
perror() is a convenient function that displays the text associated with an error code returned from the most recent UNIX system call. The text in the quotes is the title it displays before showing you the system message associated with the code.
exit(EXIT_FAILURE) exits with a -1 value which is what scripts can test to see if your program succeeded or failed, as the exit status can be retrieved for a UNIX program.
size_t is an integer type, but it's named size_t to give a hint as to what you're using it for.
If you wanted to transfer more data at a time you could. But 1-byte xfers is simple and safe and it works.
FILE *exein, *exeout;
exein = fopen("filein.exe", "rb");
if (exein == NULL) {
/* handle error */
perror("file open for reading");
exit(EXIT_FAILURE);
}
exeout = fopen("fileout.exe", "wb");
if (exeout == NULL) {
/* handle error */
perror("file open for writing");
exit(EXIT_FAILURE);
}
size_t n, m;
unsigned char buff[8192];
do {
n = fread(buff, 1, sizeof buff, exein);
if (n)
m = fwrite(buff, 1, n, exeout);
else
m = 0;
} while ((n > 0) && (n == m));
if (m)
perror("copy");
I have a FIFO pipe, which is opened at both ends using open() in O_RDWR mode. At the reading end, read() is not reading all the characters, but lesser than that specified in the call. Is there a way to ensure that all characters are read using open()?
Thanks in advance
if (p != NULL){
printf("Inside p not null!\n");
if((fd = open(p, O_RDWR)) < 0){
perror("File could not be opened!");
exit(EXIT_FAILURE);
}
//FILE *rdptr = fopen(p,"r");
memset(buf,0,file_len);
rc = read(fd, buf, file_len);
printf("Number of bytes read: %d\n", rc);
printf("Data detected on FIFO\n");
buf[rc] = '\0';
char base[20] = "output.txt";
char name[20];
sprintf(name, "%d%s", suffix, base);
FILE *fptr = fopen(name,"ab+");
fd_wr = open(name,O_WRONLY);
charnum = write(fd_wr,buf,rc);
kill(id_A, SIGKILL);
//printf("No. of characters written: %d\n",charnum);
//FD_CLR(fd, &rdfs);
}
First minor comment: you should use O_RDONLY to open the file: don't use more permissions than necessary.
Second issue: if file_len is very large, it's possible that the writer has blocked trying to write the entire chunk of data (since a FIFO can only hold a certain amount of unread data). If that's the case, then read will only read the data that has been stored in the FIFO, and will immediately return with whatever it could read. This will allow the writer to write more bytes, which will then be read in the next read.
You should loop reads, adjusting an offset into the buffer, until the entire file_len bytes are read. Something like this:
size_t offset = 0;
while(offset < file_len) {
rc = read(fd, buf+offset, file_len-offset);
if(rc < 0) {
/* handle I/O error or something... */
} else {
offset += rc;
}
}
I have the following bit of code (it's "example" code, so nothing fancy):
#include <stdio.h>
#include <string.h>
#include <fcntl.h>
#include <sys/types.h>
#include <unistd.h>
int main()
{
char buffer[9];
int fp = open("test.txt", O_RDONLY);
if (fp != -1) // If file opened successfully
{
off_t offset = lseek(fp, 2, SEEK_SET); // Seek from start of file
ssize_t count = read(fp, buffer, strlen(buffer));
if (count > 0) // No errors (-1) and at least one byte (not 0) was read
{
printf("Read test.txt %d characters from start: %s\n", offset, buffer);
}
close(fp);
}
int fp2 = open("test.txt", O_WRONLY);
if (fp2 != -1)
{
off_t offset = lseek(fp2, 2, SEEK_CUR); // Seek fraom current position (0) - same result as above in this case
ssize_t count = write(fp2, buffer, strlen(buffer));
if (count == strlen(buffer)) // We successfully wrote all the bytes
{
printf("Wrote to test.txt %d characters from current (0): %s\n", offset, buffer);
}
close(fp2);
}
}
This code does not return the first printout (reading) as it is, and the second printout reads: "Wrote test.txt 0 characters from current (0): " indicating that it did not seek anywhere in the file and that buffer is empty.
The odd thing is, if I comment out everything from fp2 = open("test.txt", O_WRONLY);, the first printout returns what I expected. As soon as I include the second open statement (even with nothing else) it won't write it. Does it somehow re-order the open statements or something else?
The line
ssize_t count = read(fp, buffer, strlen(buffer));
is wrong, you're taking the strlen of an uninitialized buffer. You likely want the size of the buffer like so:
ssize_t count = read(fp, buffer, sizeof buffer);
You should make sure buffer really contain a nul terminated string as well when you print it as one.
if (fp != -1) // If file opened successfully
{
off_t offset = lseek(fp, 2, SEEK_SET); // Seek from start of file
ssize_t count = read(fp, buffer, sizeof buffer - 1);
if (count > 0) // No errors (-1) and at least one byte (not 0) was read
{
buffer[count] = 0;
Are you perfectly sure you are cleaning out the file every time you run?
As written, the first time you run this, you'll only see the second printout, and the second time you might see the first one.