I have a FIFO pipe, which is opened at both ends using open() in O_RDWR mode. At the reading end, read() is not reading all the characters, but lesser than that specified in the call. Is there a way to ensure that all characters are read using open()?
Thanks in advance
if (p != NULL){
printf("Inside p not null!\n");
if((fd = open(p, O_RDWR)) < 0){
perror("File could not be opened!");
exit(EXIT_FAILURE);
}
//FILE *rdptr = fopen(p,"r");
memset(buf,0,file_len);
rc = read(fd, buf, file_len);
printf("Number of bytes read: %d\n", rc);
printf("Data detected on FIFO\n");
buf[rc] = '\0';
char base[20] = "output.txt";
char name[20];
sprintf(name, "%d%s", suffix, base);
FILE *fptr = fopen(name,"ab+");
fd_wr = open(name,O_WRONLY);
charnum = write(fd_wr,buf,rc);
kill(id_A, SIGKILL);
//printf("No. of characters written: %d\n",charnum);
//FD_CLR(fd, &rdfs);
}
First minor comment: you should use O_RDONLY to open the file: don't use more permissions than necessary.
Second issue: if file_len is very large, it's possible that the writer has blocked trying to write the entire chunk of data (since a FIFO can only hold a certain amount of unread data). If that's the case, then read will only read the data that has been stored in the FIFO, and will immediately return with whatever it could read. This will allow the writer to write more bytes, which will then be read in the next read.
You should loop reads, adjusting an offset into the buffer, until the entire file_len bytes are read. Something like this:
size_t offset = 0;
while(offset < file_len) {
rc = read(fd, buf+offset, file_len-offset);
if(rc < 0) {
/* handle I/O error or something... */
} else {
offset += rc;
}
}
Related
I tried to write basic program in C which copy data from file to another with given source path, destination path and buffer size as input.
my problem is the destination file filled with junk or something because its way larger than the source (get bigger depending on buffer size) and can't be open.
How do i read and write just the bytes in the source?
i'm working in linux, and this is the actually copying part:
char buffer[buffer_size];
int readable=1;
int writeable;
while(readable != 0){
readable = read(sourcef, buffer, buffer_size);
if(readable == -1){
close(sourcef);
close(destf);
exit_with_usage("Could not read.");
}
writeable = write(destf, buffer, buffer_size);
if(writeable == -1){
close(sourcef);
close(destf);
exit_with_usage("Could not write.");
}
}
writeable = write(destf, buffer, buffer_size);
must be
writeable = write(destf, buffer, readable);
Currently you do not write the number of characters you read but all the buffer, so the output file is too large
You also manage wrongly the end of the input file
The return value of read is :
On success, the number of bytes read is returned (zero indicates end of file)
On error, -1 is returned
A proposal :
/* you already check input and output file was open with success */
char buffer[buffer_size];
for(;;){
ssize_t readable = read(sourcef, buffer, buffer_size);
if(readable <= 0){
close(sourcef);
close(destf);
if (readable != 0)
/* not EOF */
exit_with_usage("Could not read.");
/* EOF */
break;
}
if (write(destf, buffer, n) != n) {
close(sourcef);
close(destf);
exit_with_usage("Could not write.");
}
}
I suppose exit_with_usage calls exit() so does not return
Note in theory write may write less than the expected number of characters without being an error, and the write has to be done in a loop, but in that case it is useless to manage that
read function returns how many bytes were read to buffer(which has buffer_size). Its not always the case actual bytes read has same value as buffer size(consider scenario if there are not enough bytes left in source file to fully fill your buffer). So you should write to destination file not buffer_size(third argument of the write function), but how many bytes have you read - that is readable variable in your code
You should exit when readable returns an error.So
while(readable != 0){
should be
while(readable != -1){
So that loop could be terminataed when an readfile is exhausted.
You see currently after the whole readfile has been read, calling read fails but write is being called repeatedly since execution has no exit path for failure on read. Also write should only write the number of bytes read. So the code would look like this:
char buffer[buffer_size];
int readable=1;
int writeable;
while(readable != -1){
readable = read(sourcef, buffer, buffer_size);
if(readable == -1){
close(sourcef);
close(destf);
exit_with_usage("Could not read.");
}
writeable = write(destf, buffer, readable);
if(writeable == -1){
close(sourcef);
close(destf);
exit_with_usage("Could not write.");
}
}
Simple code
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h> // For system calls write, read e close
#include <fcntl.h>
#define BUFFER_SIZE 4096
int main(int argc, char* argv[]) {
if (argc != 3) {
printf("Usage %s Src_file Dest_file\n", argv[0]);
exit(1);
}
unsigned char buffer[BUFFER_SIZE] = {0};
ssize_t ReadByte = 0;
int src_fd, dst_fd;
// open file in read mode
if ((src_fd = open(argv[1], O_RDONLY)) == -1) {
printf("Failed to open input file %s\n", argv[1]);
exit(1);
}
// open file in write mode and already exists to overwrite
if ((dst_fd = open(argv[2], O_WRONLY | O_CREAT | O_TRUNC, 644)) == -1) {
printf("Failed to create output file %s\n", argv[2]);
exit(1);
}
// loop
while (1) {
// read buffer
ReadByte = read(src_fd, buffer, sizeof(buffer));
// error with reading
if (ReadByte == -1) {
printf("Encountered an error\n");
break;
} else if (ReadByte == 0) {
// file end exit loop
printf("File copying successful.\n");
break;
}
// error with writing
if (write(dst_fd, buffer, ReadByte) == -1) {
printf("Failed to copying file\n");
break;
}
}
// Close file
close(src_fd);
close(dst_fd);
exit(0);
}
Run
./program src_file dest_file
I'm trying to read from an input file and ultimately reverse the buffer it reads from and write it to an output file. For now though, I'm testing to see if a buffer I read from would even make it to the output file, and so far it isn't and I'm getting an infinite loop. The buffer should read in PAGESIZE bytes (from a call to sysconf()) and if the file output is larger than the buffer, then the buffer should be written to the output file first then be flushed and reused again to get the rest of the input until the file descriptor returns 0 for no data left. This is what I have so far:
int fdRead = open(inputFile, O_RDONLY);
if (fdRead == -1)
err_sys("Error reading input file '%s', check spelling?\n", inputFile);
int fdWrite = open(outputFile, O_WRONLY | O_CREAT | O_TRUNC, 0644); //overwrites file if it exists
if (fdWrite == -1)
err_sys("Error creating output file '%s'\n", outputFile);
while (1) {
read(fdRead, buf, size);
if (fdRead == 0)
break;
if (fdRead == -1)
err_sys("Error reading from input file '%s'\n", inputFile);
lseek(fdRead, size, SEEK_CUR);
if (fdRead == -1)
err_sys("Error reading from input file '%s'\n", inputFile);
write(fdWrite, buf, size);
if (fdWrite == -1)
err_sys("Error writing to output file '%s'\n", outputFile);
lseek(fdWrite, size, SEEK_CUR);
if (fdWrite == -1)
err_sys("Error writing to output file '%s'\n", outputFile);
memset(buf, '\0', size);
}
close(fdRead);
close(fdWrite);
I suppose that fdRead is never returning 0, and thus not exiting the loop. My question is how do I fix that?
p.s: size is the call from sysconf() that gets the PAGESIZE, e.g
size = sysconf(_SC_PAGESIZE);
And inputFile and outputFile are both char * and I've tested that they return and store good strings.
Transcribing comments into an answer.
You need to capture the return value from read() — you're ignoring it and testing whether the file descriptor is 0 or negative after the read().
So if I did something like int bytesRead then tested for if (bytesRead == 0) instead of if (fdRead == 0), then that should solve my problem?
Yes, you need something like:
int nbytes = read(fdRead, buf, size);
if (nbytes <= 0) break;
You should use the positive nbytes in the write() call; you might not get all size bytes filled by the read().
Testing the file descriptor after the read is wrong (it won't have changed under normal circumstances), and ignoring the value returned by read() is wrong, and not using the value returned by read() in the call to write() is wrong.
OK so it should be write(fdWrite, buf, nbytes)?
Yes, it should be
int obytes;
if ((obytes = write(fdWrite, buf, nbytes)) != nbytes)
{
…oops — short write …
}
You get to decide what's the appropriate response to a short write (a positive value, but not the number of bytes you expected to write). If you're writing to a socket, it might be appropriate to try writing the unwritten section of the data again (that's why obytes is used to capture the number of bytes successfully written). If you're writing to a disk file, it probably means there's no space left, so there's no point (little point) in trying again. If obytes is negative, you've had a write error; there is usually little point in trying to continue.
This all has helped out a lot and it seems to be working okay. I've only run into one other problem. I tested this on a large file (Alice in Wonderland text file) and the output file is almost the whole thing, but cuts off the last two paragraphs or so. …
You need to review why you have the lseek() operations in the code. Neither of them should be necessary, and both are dubious. I think the lseek() on fdRead() means you miss chunks of text of size bytes each; I think the lseek() on fdWrite() means you insert size null bytes into the output file.
/*Here's the code which may help for that.i havemodified in it for your need.*/
int fdRead = open(inputFile, O_RDONLY);
int Ret_Val;
if (fdRead == -1)
err_sys("Error reading input file'%s',check spelling?\n",
inputFile );
int fdWrite = open(outputFile, O_WRONLY | O_CREAT
| O_TRUNC, 0644); //overwrites file if it exists
if (fdWrite == -1)
err_sys("Error creating output file '%s'\n", outputFile);
while (1) {
Ret_Val=read(fdRead, buf, size);
if (Ret_Val == 0)
break;
if (Ret_Val == -1)
err_sys("Error reading from input file'%s'\n",inputFile);
lseek(fdRead, size, SEEK_CUR);
if (Ret_Val == -1)
err_sys("Error reading from input file'%s'\n",
inputFile );
Ret_Val=write(fdWrite, buf, size);
if (Ret_Val == -1)
err_sys("Error writing to output file'%s'\n",outputFile);
lseek(fdWrite, size, SEEK_CUR);
if (Ret_Val == -1)
err_sys("Error writing to output file'%s'\n",outputFile);
memset(buf, '\0', size);
}
close(fdRead);
close(fdWrite);
Here is my code snippet:
int fd;
bufsize = 30;
char buf[bufsize];
char cmd[100] = "file.txt";
int newfd = 1;
if (fd = open(cmd,O_RDONLY) >=0){
puts("wanna read");
while (read(fd,&bin_buf,bufsize)==1){
puts("reading");
write(newfd,&bin_buf,bufsize);
}
close(fd);
}
So here the program prints "wanna read" but never prints "reading". I have also tried opening using nonblock flag, but no use. Can anybody help me? I must use open() and read() system calls only. Thanks.
Edit: I have made some clarifications in the code. Actually the newfd that I'm writing to is a socket descriptor, but I don't think that is important for this problem because it sticks on the read which is before the write.
The first problem is your if statement. You forgot to use enough parentheses, so if the open() works, the read tries to read from file descriptor 1, aka standard output. If that's your terminal (it probably is) on a Unix box, then that works — surprising though that may be; the program is waiting for you to type something.
Fix: use parentheses!
if ((fd = open(cmd, O_RDONLY)) >= 0)
The assignment is done before, not after, the comparison.
I observe in passing that you don't show how you set cmd, but if you see the 'wanna read' message, it must be OK. You don't show how newfd is initialized; maybe that's 1 too.
You also have the issue with 'what the read() call returns'. You probably need:
int fd;
char buf[bufsize];
int newfd = 1;
if ((fd = open(cmd, O_RDONLY)) >= 0)
{
puts("wanna read");
int nbytes; // ssize_t if you prefer
while ((nbytes = read(fd, buf, sizeof(buf))) > 0)
{
puts("reading");
write(newfd, buf, nbytes);
}
close(fd);
}
You can demonstrate my primary observation by typing something ('Surprise', or 'Terminal file descriptors are often readable and writable' or something) with your original if but my loop body and then writing that somewhere.
Your read() call attempts to read bufsize bytes and returns the number of bytes actually read. Unless bufsize ==, it is quite unlikely read() will return 1, so the block is almost always skipped and nothing get written.
Also note that if (fd = open(cmd, O_RDONLY) >= 0) is incorrect and would set fd to 1, the handle for standard output, if the file exists, causing the read to fail as standard input is most likely not opened for reading.
Note that reading with the read system call is tricky on some environments, because a return value of -1 may be restartable.
Here is an improved version:
int catenate_file(const char *cmd, int newfd, size_t bufsize) {
int fd;
char buf[bufsize];
if ((fd = open(cmd, O_RDONLY)) >= 0) {
puts("wanna read");
ssize_t nc;
while ((nc = read(fd, buf, bufsize)) != 0) {
if (nc < 0) {
if (errno == EINTR)
continue;
else
break;
}
printf("read %zd bytes\n", nc);
write(newfd, buf, nc);
}
close(fd);
return 0;
}
return -1;
}
read returns the number of bytes read from file that can be bufsize or less if the remainder of the file that has to be read is shorter than bufsize.
In your case most probably bufsize is bigger than 1 and the file is bigger than 1 byte so the condition of the while loop is evaluated false, the code is skipped to the point where file is closed.
You should check if there if there are more bytes to be read:
while( read(fd,&bin_buf,bufsize) > 0 ) {
I created an application that send a text file from client to server
So far i'm send it as string like this:
fp = fopen(filename, "r");
if (fp != NULL) {
newLen = fread(source, sizeof(char), 5000, fp);
if (newLen == 0) {
fputs("Error reading file", stderr);
} else {
source[++newLen] = '\0'; /* Just to be safe. */
}
}else{
printf("The file %s does not exist :(");
return 1;
}
fclose(fp);
send(s , source , strlen(source) , 0); //send file
However my professor told me I must send the file in Binary and be ready to accept a file of any size
I'm trying to figure out how to send the file in binary and break it into chunks
You can copy it one byte at a time.
Reading/writing more than a byte at a time theoretically would make it read and write more efficiently to disk. But since the binary is likely short, and disk I/O is already internally buffered it probably doesn't make a noticeable difference.
perror() is a convenient function that displays the text associated with an error code returned from the most recent UNIX system call. The text in the quotes is the title it displays before showing you the system message associated with the code.
exit(EXIT_FAILURE) exits with a -1 value which is what scripts can test to see if your program succeeded or failed, as the exit status can be retrieved for a UNIX program.
size_t is an integer type, but it's named size_t to give a hint as to what you're using it for.
If you wanted to transfer more data at a time you could. But 1-byte xfers is simple and safe and it works.
FILE *exein, *exeout;
exein = fopen("filein.exe", "rb");
if (exein == NULL) {
/* handle error */
perror("file open for reading");
exit(EXIT_FAILURE);
}
exeout = fopen("fileout.exe", "wb");
if (exeout == NULL) {
/* handle error */
perror("file open for writing");
exit(EXIT_FAILURE);
}
size_t n, m;
unsigned char buff[8192];
do {
n = fread(buff, 1, sizeof buff, exein);
if (n)
m = fwrite(buff, 1, n, exeout);
else
m = 0;
} while ((n > 0) && (n == m));
if (m)
perror("copy");
I have a "file" as a resource. I can only use read(), write() and fstat() it. This file is a text file which I would like to parse.
Normally I use fgets() to read the text file line by line and parse it. How can I do this in this case?
FILE *fp;
char buffer[128];
fp = fopen( "/home/txtfile", "r" );
if (fp == NULL){
perror("file missing");
}
while (fgets (buffer, sizeof(buffer), fp) != NULL) {
//some code
}
How can I do the same with read() ?
Is this right?
int fd = open("/dev/file",O_RDONLY);
if (fd==-1) {
printf("Failed to open file!!!\n");
}
while (fgets (buffer, sizeof (buffer), fd) != NULL) {
//some code
}
Unless your file is huge, if you're using read(), it would be easier to read in the entire file, then operate on the memory buffer, rather than in discrete chunks. That is, unless each line is of a fixed length.
I'd do something like this:
int rc;
int fd = open("data", O_RDONLY); // open the file for reading
if (fd == -1) {
// error
}
// to be thorough, do a stat() here to find how big to make the buffer
struct stat sb;
rc = fstat(fd, &sb);
if (rc == -1) {
// error
}
char *buffer = calloc(1, sb.st_size);
int bytes_read = 0;
// read in entire file; each read() can be incomplete, hence why it's in a loop,
// and reading from/writing to increasing sections of the memory and file
while ((rc = read(fd, (buffer + bytes_read), (sb.st_size - bytes_read))) > 0) {
if (rc == -1) {
// error
}
bytes_read += rc;
}
close(fd);
// Now, to read it line-by-line...
char line[128]; // 128 is arbitrary
int pos = 0;
while ((rc = sscanf(buffer + pos, "%127[^\n]\n", line)) > 0) {
pos += strlen(line) + 1;
// do stuff with line
}
return 0;
Then you can operate on your memory buffer line-by-line by scanning for newlines, or using sscanf(). Also make sure to free() your buffer!
Edit: I've added some example code for using sscanf() to handle your buffer. If you know the format of the lines (you say you're parsing them) you might be able to make better use of sscanf() by using the format specifiers. All of this is untested, by the way.
Something like this :
int fd = open("/dev/file",O_RDONLY);
ssize_t res = 0;
while((res = read(fd, buffer, sizeof(buffer))) > 0) {
//some code
}
if (res < 0) {
//handle error
} else{
//close fd
}
Is this right?
No.
read() is a system call that operates on a Unix file descriptor, not a stdio FILE*. Other than that, it works by reading data from the file and putting it in the buffer you supply.
int fd = open("/dev/file",O_RDONLY);
if (fd==-1)
{
printf("Failed to open file!!!\n");
}
else
{
char buffer[BUF_SIZE];
ssize_t bytesRead = read(fd, buffer, BUF_SIZE);
while (bytesRead > 0)
{
// do something with the buffer
bytesRead = read(fd, buffer, BUF_SIZE);
}
if (bytesRead == -1)
{
// error
}
// bytesRead == 0 => end of file
}