I want to read a block device file, block by block until last byte even if the block is full of zeroes.
My code is this. size is the no. of bytes I want it to read in one go - could be anything 4K, 8K etc.
for (int i = 1; i <= num; i++){
read_bytes = read(fd, buf, size);
if (read_bytes < 0) {
perror("read failed");
return 1;
}
lseek_offset = lseek(fd, size, SEEK_CUR);
if (lseek_offset < 0){
perror("lseek failed.\n");
return 1;
}
}
When a block is filled with zero bytes (not the length of block but the data in block), lseek fails with EINV.
And I can see from df -h that that disk is half full and rest is zero bytes since it was formatted with ext4 before using it.
As already mentioned by #Mat in the comments read already updates the file offset so you should remove the lseek call. From read(2) man pages:
On files that support seeking, the read operation commences at the current file offset, and the file offset is incremented by the number of bytes read. If the current file offset is at or past the end of file, no bytes are read, and read() returns zero.
Also note that the read call might fail due to an interrupt, so you should check the value of errno for that (I'm guessing you'd still like to continue reading).
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
for (int i = 1; i <= num; i++) {
read_bytes = read(fd, buf, size);
if (read_bytes < 0 && errno != EINTR) {
perror("read failed");
return 1;
}
}
Finally, note that you are not guaranteed that the size number of bytes will be read in one go (see read(2)), most likely because of a signal interrupt. So here's an idea of the top of my head. You could check file size in a while loop within a single iteration of the for loop to determine how much you still need to read. For example:
for (int i = 1; i <= num; i++) {
size_t remain = size;
while(remain) { // keep reading until you've read size bytes
read_bytes = read(fd, buf, remain);
if (read_bytes < 0 && errno != EINTR) {
perror("read failed");
return 1;
}
.... // determine the amount of bytes read in the last iteration
remain = ... // update the size to the bytes still needed to be read
}
}
Related
I am practicing the read and write system call, the below code is working fine with a while loop and also without them. could you please tell me what is the use of while loop here, is it necessary to add it while using read and write system calls. I am a beginner. Thanks.
#include <unistd.h>
#define BUF_SIZE 256
int main(int argc, char *argv[])
{
char buf[BUF_SIZE];
ssize_t rlen;
int i;
char from;
char to;
from = 'e';
to = 'a';
while (1) {
rlen = read(0, buf, sizeof(buf));
if (rlen == 0)
return 0;
for (i = 0; i < rlen; i++) {
if (buf[i] == from)
buf[i] = to;
}
write(1, buf, rlen);
}
return 0;
}
You usually need to use while loops (or some kind of loop in general) with read and write, because, as you should know from the manual page (man 2 read):
RETURN VALUE
On success, the number of bytes read is returned (zero indicates end
of file), and the file position is advanced by this number. It is
not an error if this number is smaller than the number of bytes
requested; this may happen for example because fewer bytes are
actually available right now (maybe because we were close to end-of-
file, or because we are reading from a pipe, or from a terminal), or
because read() was interrupted by a signal. See also NOTES.
Therefore, if you ever want to read more than 1 byte, you need to do this in a loop, because read can always process less than the requested amount.
Similarly, write can also process less than the requested size (see man 2 write):
RETURN VALUE
On success, the number of bytes written is returned (zero indicates nothing was written). It is not an error if this
number is smaller than the number of bytes requested; this may happen for example because the disk device was filled.
See also NOTES.
On error, -1 is returned, and errno is set appropriately.
The only difference here is that when write returns 0 it's not an error or an end of file indicator, you should just retry writing.
Your code is almost correct, in that it uses a loop to keep reading until there are no more bytes left to read (when read returns 0), but there are two problems:
You should check for errors after read (rlen < 0).
When you use write you should also add a loop there too, because as I just said, even write could process less than the requested amount of bytes.
A correct version of your code would be:
#include <stdio.h>
#include <unistd.h>
#define BUF_SIZE 256
int main(int argc, char *argv[])
{
char buf[BUF_SIZE];
ssize_t rlen, wlen, written;
char from, to;
int i;
from = 'e';
to = 'a';
while (1) {
rlen = read(0, buf, sizeof(buf));
if (rlen < 0) {
perror("read failed");
return 1;
} else if (rlen == 0) {
return 0;
}
for (i = 0; i < rlen; i++) {
if (buf[i] == from)
buf[i] = to;
}
for (written = 0; written < rlen; written += wlen) {
wlen = write(1, buf + written, rlen - written);
if (wlen < 0) {
perror("write failed");
return 1;
}
}
}
return 0;
}
Extremely new to socket programming and C in general. I am trying to write a basic program to send and receive data between two machines. I understand that recv will not get all your data at once -- you essentially have to loop it until it has read the whole message.
In lieu of just setting a limit on both machines, I have created a simple Message struct on the client side:
struct Message {
size_t length;
char contents[1024 - sizeof(size_t)];
} message;
message.length = sizeof(struct Message);
message.contents = information_i_want_to_send;
When it arrives at the server, I have recv read into a buffer: received = recv(ioSock, &buffer, 1024, 0) (Which coincidentally is the same size as my Message struct -- but assuming it wasn't...).
I then extract Message.length from the buffer like this:
size_t messagelength;
messagelength = *((size_t *) &buffer);
Then I loop recv into the buffer while received < messagelength.
This works, but I can't help feeling it's really ugly and it feels hacky. (Especially if the first recv call reads less than sizeof(size_t) or the machines are different bit architectures, in which case the size_t cast won't work..). Is there a better way to do this?
You have a fixed-size message, so you can use something like this:
#include <errno.h>
#include <limits.h>
// Returns the number of bytes read.
// EOF was reached if the number of bytes read is less than requested.
// On error, returns -1 and sets errno.
ssize_t recv_fixed_amount(int sockfd, char *buf, size_t size) {
if (size > SSIZE_MAX) {
errno = EINVAL;
return -1;
}
ssize_t bytes_read = 0;
while (size > 0) {
ssize_t rv = recv(sockfd, buf, size, 0);
if (rv < 0)
return -1;
if (rv == 0)
return bytes_read;
size -= rv;
bytes_read += rv;
buf += rv;
}
return bytes_read;
}
It would be used something like this:
typedef struct {
uint32_t length;
char contents[1020];
} Message;
Message message;
ssize_t bytes_read = recv_fixed_amount(sockfd, &(message.length), sizeof(message.length));
if (bytes_read == 0) {
printf("EOF reached\n");
exit(EXIT_SUCCESS);
}
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != sizeof(message.length)) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
bytes_read = recv_fixed_amount(sockfd, &(message.content), sizeof(message.content));
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != msg_size) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
Notes:
size_t is not going to be the same everywhere, so I switched to a uint32_t.
I read the fields independently because the padding within the struct can vary between implementations. They would need to be sent that way as well.
The receiver is populating message.length with the information from the stream, but doesn't actually use it.
A malicious or buggy sender could provide a value for message.length that's too large and crash the receiver (or worse) if it doesn't validate it. Same goes for contents. It might not be NUL-terminated if that's expected.
But what if the length wasn't fixed? Then the sender would need to somehow communicate how much the reader needs to read. A common approach is a length prefix.
typedef struct {
uint32_t length;
char contents[];
} Message;
uint32_t contents_size;
ssize_t bytes_read = recv_fixed_amount(sockfd, &contents_size, sizeof(contents_size));
if (bytes_read == 0) {
printf("EOF reached\n");
exit(EXIT_SUCCESS);
}
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != sizeof(contents_size)) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
Message *message = malloc(sizeof(Message)+contents_size);
if (!message) {
perror("malloc");
exit(EXIT_FAILURE);
}
message->length = contents_size;
bytes_read = recv_fixed_amount(sockfd, &(message->contents), contents_size);
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != contents_size) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
Notes:
message->length contains the size of message->contents instead of the size of the structure. This is far more useful.
Another approach is to use a sentinel value. This is a value that tells the reader the message is over. This is what the NUL that terminates C strings is. This is more complicated because you don't know how much to read in advance. Reading byte-by-byte is too expensive, so one normally uses a buffer.
while (1) {
extend_buffer_if_necessary();
recv_into_buffer();
while (buffer_contains_a_sentinel()) {
// This also shifts the remainder of the buffer's contents.
extract_contents_of_buffer_up_to_sentinel();
process_extracted_message();
}
}
The advantage of using a sentinel value is that one doesn't need to know the length of the message in advance (so the sender can start sending it before it's fully created.)
The disadvantage is the same as for C strings: The message can't contain the sentinel value unless some form of escaping mechanism is used. Between this and the complexity of the reader, you can see why a length prefix is usually preferred over a sentinel value. :)
Finally, there's a better solution than sentinel values for large messages that you want to start sending before they are fully created: A sequence of length-prefixed chunks. One keeps reading chunks until a chunk of size 0 is encountered, signaling the end.
HTTP supports both length-prefixed messages (in the form of Content-Length: <length> header) and this approach (in the form of the Transfer-Encoding: chunked header).
There are Two ways to do that...
1.)
Use Binary Synchronous protocol. (Use of STX - Start of Text and ETX - End of Text ) for identification of the Text start and end.
2.)
Attach the number of bytes of data being sent at the start of Data. The socket will read those number of bytes and will get the number of bytes to be received from the socket. Then read all data and get the amount of data required.
Hmm... Seems tough...?? Let me give you an example.
Actual Data need to be sent: ABCDEFGHIJ
New Data format : 0010ABCDEFGHIJ
Data required in server side: ABCDE
recv function will read the first 4 bytes to get the number of bytes of actual data(In loop untill it gets 4 bytes):
int received1= recv(ioSock, recvbuf, 4, 0);
As per the above case, 'recvbuf' will be 0010 converted to an integer will give value as '10' which can be stored in some integer variable. So we have :
int toReadVal = 10
Now all we need is to read these 10 digits in next recv call :
int received= recv(ioSock, recvbuf1, toReadVal, 0);
Finally, we get the value of recvbuf1 as ABCDEFGHIG. Now you can truncate the value as per your requirement.
I am trying to read all data from a file using open() system call. But I am having difficulties figuring out the end of file. The C EOF flag doesn't work. My program goes into an infinite loop. Here is the code. The file has less than 100 characters in it.
int main()
{
char buf[100] = {""};
i = 0;
int fd = open ("file1.txt", O_RDONLY);
int bytesread = read (fd, &buf[i], 1);
char c = buf[i];
while (c != EOF) {
i++;
int bytesread = read (fd, &buf[i], 1);
c = buf[i];
}
}
read(2) doesn't return EOF. Its return values are: 0 on reaching "end-of-file", -1 on error, positive value when as many bytes are read. Besides you are checking the data for EOF. Your loop condition is wrong.
Typically, you'd also check if read(2) was interrupted and if so, retry.
size_t i = 0;
errno = 0;
while (i < sizeof buf && read (fd, &buf[i], 1) >= 0 && errno != EINTR) {
i++;
errno = 0;
}
I am also not why you are reading only one byte at a time, which is not very efficient. You could always read chunks of data and check the return value to know the number of bytes read.
Note: Typically the macro EOF is also defined with value -1. So it could seem read(2) returns EOF but don't be confused.
The buffer doesn't holds EOF, it just hold the data in the file read.What you can do is
while(bytesread > 0 ){
i++;
bytesread = read (fd, &buf[i], 1);
}
I'm programming in C an IRC chat client. everything it's working well except I can't read the whole answer sent by the server. here's the code:
char buffer[2048];
write_on_screen(current_page(), "LOG COMMAND", command);
write(sockfd, command, strlen(command)); //write to socket
bzero(buffer, sizeof(buffer));
read(sockfd, buffer, sizeof(buffer));
write_on_screen(current_page(), "RESPONSE", buffer);
return buffer;
most of the time buffer will contain just a piece of the response (which is shorter than 2048 bytes) and other times it contains nothing. in both cases if I do another read() after the first one, it returns me the rest of the answer or another small piece (and then I've to do another read() again). if I put a sleep(1) between write() and read() I get the whole answer, but I'm sure this not a good pratice.
Is there some way I can avoid this?
thank you in advance
You're making the usual mistakes. It is impossible to write correct network code without storing the result of read() or recv() into a variable. You have to:
Check it for -1, and if so look at errno to see whether was fatal, which it almost always is except for EAGAIN/EWOULDBLOCK, and if fatal close the socket and abandon the process.
Check it for zero, which means the peer disconnected. Again you must close the socket and abandon the process.
Use it as the count of bytes actually received. These functions are not obliged nor guaranteed to fill the buffer. Their contract in blocking mode is that they block until an error, end of stream, or at least one byte is transferred. If you're expecting more than one byte, you normally have to loop until you get it.
According to RFC-1459, a single line of text in IRC can contain up to 512 characters and is terminated by a CRLF (\r\n) pair. However:
You're not guaranteed to receive exactly 512 bytes each time. For example, you might receive a comparatively short message from someone else one in the channel: Hi!
Related to the above: A group of 512 bytes might represent more than one message. For example, the buffer might contain a whole line, plus part of the next line: PRIVMSG <msgtarget> <message>\r\nPRIVMS
Given that you could have zero-or-more complete lines plus zero-or-one incomplete lines in your buffer[] at any time, you could try doing something along the lines of:
char buffer[2048];
while(keep_going)
{
char **lines;
int i, num_lines;
// Receive data from the internet.
receiveData(buffer);
// Create an array of all COMPLETE lines in the buffer (split on \r\n).
lines = getCompleteLines(buffer, &num_lines);
removeCompleteLinesFromBuffer(buffer);
// Handle each COMPLETE line in the array.
for (i = 0; i < num_lines; ++i) { handle_line(lines[i]); }
freeLines(lines);
}
This would allow you to handle zero or more complete lines in one go, with any incomplete line (i.e anything after the final \r\n pair) being kept around until the next call to receiveData().
You need to loop around read() until a CRLF had been detected.
A possible way to do this would be:
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
ssize_t read_until_crlf(int sd, char * p, size_t s, int break_on_interupt)
{
ssize_t bytes_read = 0;
ssize_t result = 0;
int read_cr = 0;
int read_crlf = 0;
while (bytes_read < s)
{
result = read(sd, p + bytes_read, 1);
if (-1 == result)
{
if ((EAGAIN == errno) || (EWOULDBLOCK == errno))
{
continue;
}
else if (EINTR == errno)
{
if (break_on_interupt)
{
break;
}
continue;
}
else
{
perror("read() failed");
break;
}
}
else if (0 == result)
{
break; /* peer disconnected */
}
if ('\r' == p[bytes_read])
{
read_cr = 1;
}
else if (('\n' == p[bytes_read]) && read_cr)
{
read_crlf = 1;
break; /* CRLF detected */
}
else
{
read_cr = 0;
}
++bytes_read;
}
if (!read_crlf)
{
result = -1; /* Buffer full without having read a CRLF. */
errno = ENOSPC; /* ... or whatever might suite. */
}
return (0 >= result) ?result :bytes_read;
}
Call it like this:
#include <stdio.h>
ssize_t read_until_crlf(int sd, char * p, size_t s, int break_on_interupt);
int main(void)
{
int sd = -1;
/* init sd here */
{
char line[2048] = "";
ssize_t result = read_until_crlf(sd, line, sizeof line, 0);
if (-1 == result)
{
perror("read_until_newline() failed");
}
printf("read '%s'\n", line);
}
return 0;
}
I have a FIFO pipe, which is opened at both ends using open() in O_RDWR mode. At the reading end, read() is not reading all the characters, but lesser than that specified in the call. Is there a way to ensure that all characters are read using open()?
Thanks in advance
if (p != NULL){
printf("Inside p not null!\n");
if((fd = open(p, O_RDWR)) < 0){
perror("File could not be opened!");
exit(EXIT_FAILURE);
}
//FILE *rdptr = fopen(p,"r");
memset(buf,0,file_len);
rc = read(fd, buf, file_len);
printf("Number of bytes read: %d\n", rc);
printf("Data detected on FIFO\n");
buf[rc] = '\0';
char base[20] = "output.txt";
char name[20];
sprintf(name, "%d%s", suffix, base);
FILE *fptr = fopen(name,"ab+");
fd_wr = open(name,O_WRONLY);
charnum = write(fd_wr,buf,rc);
kill(id_A, SIGKILL);
//printf("No. of characters written: %d\n",charnum);
//FD_CLR(fd, &rdfs);
}
First minor comment: you should use O_RDONLY to open the file: don't use more permissions than necessary.
Second issue: if file_len is very large, it's possible that the writer has blocked trying to write the entire chunk of data (since a FIFO can only hold a certain amount of unread data). If that's the case, then read will only read the data that has been stored in the FIFO, and will immediately return with whatever it could read. This will allow the writer to write more bytes, which will then be read in the next read.
You should loop reads, adjusting an offset into the buffer, until the entire file_len bytes are read. Something like this:
size_t offset = 0;
while(offset < file_len) {
rc = read(fd, buf+offset, file_len-offset);
if(rc < 0) {
/* handle I/O error or something... */
} else {
offset += rc;
}
}