I try to fill a named pipe (created by mkfifo /tmp/pipe) by writing to it 3 bytes at a time until the write() function blocks.
On my system, a pipe seems to be limited to 16 pages of 4096 bytes. Thus the pipe can contain 65536 bytes.
I do that with the following C code:
int main ()
{
pid_t child;
child = fork ();
if (child == 0)
{
ssize_t ret;
ssize_t total = 0;
unsigned char *datat = malloc (65536);
assert (datat != NULL);
int fd = open ("/tmp/pipe", O_WRONLY);
assert (fd != -1);
while (1)
{
printf ("Trying writting\n");
ret = write (fd, datat, 3);
assert (ret != -1);
total += ret;
printf ("write : %ld.\n", total);
}
}
else
{
int fd = open ("/tmp/pipe", O_RDONLY);
assert (fd != -1);
while (1); //prevent closing the pipe.
}
return 0;
}
By this way, I succeed to fill the pipe until 65520 bytes. I don't understand why 65520 and not 65536 (or 65535 if we consider that 65536 is not a multiple of 3).
Then I tried to write 65520 bytes and, after, write 3 bytes:
int
main (int argc, char *argv[])
{
pid_t child;
child = fork ();
if (child == 0)
{
ssize_t ret;
ssize_t total = 0;
unsigned char *datat = malloc (65536);
assert (datat != NULL);
int fd = open ("/tmp/pipe", O_WRONLY);
assert (fd != -1);
while(1)
{
printf ("Trying writting\n");
ret = write (fd, datat, 65520);
assert (ret != -1);
total += ret;
printf ("Trying writting\n");
ret = write (fd, datat, 3);
assert (ret != -1);
total += ret;
printf ("write : %ld.\n", total);
}
}
else
{
int fd = open ("/tmp/pipe", O_RDONLY);
assert (fd != -1);
while (1); //prevent closing the pipe.
}
return 0;
}
I expected the second write to block, however it was not the case and I wrote 65523 bytes.
The question is: why can't I write more than 65520 bytes on the first case whereas I can in the second?
EDIT:
More information :
My Operating system is Linux archlinux 4.16.5-1-ARCH
man 7 pipe give information about the size (which is equal to 65536 bytes) of the pipe and is confirmed by fcntl:
int
main (int argc, char *argv[])
{
int fd = open ("/tmp/pipe", O_WRONLY);
printf ("MAX : %d\n", fcntl (fd, F_GETPIPE_SZ));
return 0;
}
It's because of the way 4KB pages are filled with written data in the pipe implementation in the Linux kernel. More specifically, the kernel appends written data to a page only if the data fits entirely in the page, otherwise puts the data into another page with enough free bytes.
If you write 3 bytes at a time, the pipe pages won't be filled at their full capacity, because the page size (4096) is not a multiple of 3: the nearest multiple is 4095, so each page will end up with 1 "wasted" byte. Multiplying 4095 by 16, which is the total number of pages, you get 65520.
In your second use case, when you write 65520 bytes all at once, you are filling 15 pages entirely (61440 bytes), plus you are putting the remaining 4080 bytes in the last page, which will have 16 bytes still available for subsequent writes: that's why your second write() call with 3 bytes succeeds without blocking.
For full details on the Linux pipe implementation, see https://elixir.bootlin.com/linux/latest/source/fs/pipe.c.
Related
I am practicing the read and write system call, the below code is working fine with a while loop and also without them. could you please tell me what is the use of while loop here, is it necessary to add it while using read and write system calls. I am a beginner. Thanks.
#include <unistd.h>
#define BUF_SIZE 256
int main(int argc, char *argv[])
{
char buf[BUF_SIZE];
ssize_t rlen;
int i;
char from;
char to;
from = 'e';
to = 'a';
while (1) {
rlen = read(0, buf, sizeof(buf));
if (rlen == 0)
return 0;
for (i = 0; i < rlen; i++) {
if (buf[i] == from)
buf[i] = to;
}
write(1, buf, rlen);
}
return 0;
}
You usually need to use while loops (or some kind of loop in general) with read and write, because, as you should know from the manual page (man 2 read):
RETURN VALUE
On success, the number of bytes read is returned (zero indicates end
of file), and the file position is advanced by this number. It is
not an error if this number is smaller than the number of bytes
requested; this may happen for example because fewer bytes are
actually available right now (maybe because we were close to end-of-
file, or because we are reading from a pipe, or from a terminal), or
because read() was interrupted by a signal. See also NOTES.
Therefore, if you ever want to read more than 1 byte, you need to do this in a loop, because read can always process less than the requested amount.
Similarly, write can also process less than the requested size (see man 2 write):
RETURN VALUE
On success, the number of bytes written is returned (zero indicates nothing was written). It is not an error if this
number is smaller than the number of bytes requested; this may happen for example because the disk device was filled.
See also NOTES.
On error, -1 is returned, and errno is set appropriately.
The only difference here is that when write returns 0 it's not an error or an end of file indicator, you should just retry writing.
Your code is almost correct, in that it uses a loop to keep reading until there are no more bytes left to read (when read returns 0), but there are two problems:
You should check for errors after read (rlen < 0).
When you use write you should also add a loop there too, because as I just said, even write could process less than the requested amount of bytes.
A correct version of your code would be:
#include <stdio.h>
#include <unistd.h>
#define BUF_SIZE 256
int main(int argc, char *argv[])
{
char buf[BUF_SIZE];
ssize_t rlen, wlen, written;
char from, to;
int i;
from = 'e';
to = 'a';
while (1) {
rlen = read(0, buf, sizeof(buf));
if (rlen < 0) {
perror("read failed");
return 1;
} else if (rlen == 0) {
return 0;
}
for (i = 0; i < rlen; i++) {
if (buf[i] == from)
buf[i] = to;
}
for (written = 0; written < rlen; written += wlen) {
wlen = write(1, buf + written, rlen - written);
if (wlen < 0) {
perror("write failed");
return 1;
}
}
}
return 0;
}
Here is my code snippet:
int fd;
bufsize = 30;
char buf[bufsize];
char cmd[100] = "file.txt";
int newfd = 1;
if (fd = open(cmd,O_RDONLY) >=0){
puts("wanna read");
while (read(fd,&bin_buf,bufsize)==1){
puts("reading");
write(newfd,&bin_buf,bufsize);
}
close(fd);
}
So here the program prints "wanna read" but never prints "reading". I have also tried opening using nonblock flag, but no use. Can anybody help me? I must use open() and read() system calls only. Thanks.
Edit: I have made some clarifications in the code. Actually the newfd that I'm writing to is a socket descriptor, but I don't think that is important for this problem because it sticks on the read which is before the write.
The first problem is your if statement. You forgot to use enough parentheses, so if the open() works, the read tries to read from file descriptor 1, aka standard output. If that's your terminal (it probably is) on a Unix box, then that works — surprising though that may be; the program is waiting for you to type something.
Fix: use parentheses!
if ((fd = open(cmd, O_RDONLY)) >= 0)
The assignment is done before, not after, the comparison.
I observe in passing that you don't show how you set cmd, but if you see the 'wanna read' message, it must be OK. You don't show how newfd is initialized; maybe that's 1 too.
You also have the issue with 'what the read() call returns'. You probably need:
int fd;
char buf[bufsize];
int newfd = 1;
if ((fd = open(cmd, O_RDONLY)) >= 0)
{
puts("wanna read");
int nbytes; // ssize_t if you prefer
while ((nbytes = read(fd, buf, sizeof(buf))) > 0)
{
puts("reading");
write(newfd, buf, nbytes);
}
close(fd);
}
You can demonstrate my primary observation by typing something ('Surprise', or 'Terminal file descriptors are often readable and writable' or something) with your original if but my loop body and then writing that somewhere.
Your read() call attempts to read bufsize bytes and returns the number of bytes actually read. Unless bufsize ==, it is quite unlikely read() will return 1, so the block is almost always skipped and nothing get written.
Also note that if (fd = open(cmd, O_RDONLY) >= 0) is incorrect and would set fd to 1, the handle for standard output, if the file exists, causing the read to fail as standard input is most likely not opened for reading.
Note that reading with the read system call is tricky on some environments, because a return value of -1 may be restartable.
Here is an improved version:
int catenate_file(const char *cmd, int newfd, size_t bufsize) {
int fd;
char buf[bufsize];
if ((fd = open(cmd, O_RDONLY)) >= 0) {
puts("wanna read");
ssize_t nc;
while ((nc = read(fd, buf, bufsize)) != 0) {
if (nc < 0) {
if (errno == EINTR)
continue;
else
break;
}
printf("read %zd bytes\n", nc);
write(newfd, buf, nc);
}
close(fd);
return 0;
}
return -1;
}
read returns the number of bytes read from file that can be bufsize or less if the remainder of the file that has to be read is shorter than bufsize.
In your case most probably bufsize is bigger than 1 and the file is bigger than 1 byte so the condition of the while loop is evaluated false, the code is skipped to the point where file is closed.
You should check if there if there are more bytes to be read:
while( read(fd,&bin_buf,bufsize) > 0 ) {
I am trying to write a program on how to read a file 10 bytes per time using read, however, I do not know how to go about it. How should I modify this code to read 10bytes per time. Thanks!!!!
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/time.h>
int main (int argc, char *argv[])
{
printf("I am here1\n");
int fd, readd = 0;
char* buf[1024];
printf("I am here2\n");
fd =open("text.txt", O_RDWR);
if (fd == -1)
{
perror("open failed");
exit(1);
}
else
{
printf("I am here3\n");
if(("text.txt",buf, 1024)<0)
printf("read error\n");
else
{
printf("I am here3\n");
/*******************************
* I suspect this should be the place I make the modification
*******************************/
if(read("text.txt",buf, 1024)<0)
printf("read error\n");
else
{
printf("I am here4\n");
printf("\nN: %c",buf);
if(write(fd,buf,readd) != readd)
printf("write error\n");
}
}
return 0;
}
The final parameter of read() is the maximum size of the data you wish to read so, to try and read ten bytes at a time, you would need:
read (fd, buf, 10)
You'll notice I've also changed the first parameter to the file descriptor rather than the file name string.
Now, you'll probably want that in a loop since you'll want to do something with the data, and you also need to check the return value since it can give you less than what you asked for.
A good example for doing this would be:
int copyTenAtATime (char *infile, char *outfile) {
// Buffer details (size and data).
int sz;
char buff[10];
// Try open input and output.
int ifd = open (infile, O_RDWR);
int ofd = open (outfile, O_WRONLY|O_CREAT);
// Do nothing unless both opened okay.
if ((ifd >= 0) && (ofd >= 0)) {
// Read chunk, stopping on error or end of file.
while ((sz = read (ifd, buff, sizeof (buff))) > 0) {
// Write chunk, flagging error if not all written.
if (write (ofd, buff, sz) != sz) {
sz = -1;
break;
}
}
}
// Finished or errored here, close files that were opened.
if (ifd >= 0) close (ifd);
if (ofd >= 0) close (ofd);
// Return zero if all okay, otherwise error indicator.
return (sz == 0) ? 0 : -1;
}
change the value in read,
read(fd,buf,10);
From man of read
ssize_t read(int fd, void *buf, size_t count);
read() attempts to read up to count bytes from file descriptor fd into the buffer starting at buf.
if(read("text.txt",buf, 1024)<0)// this will give you the error.
First argument must be an file descriptor.
I have a FIFO pipe, which is opened at both ends using open() in O_RDWR mode. At the reading end, read() is not reading all the characters, but lesser than that specified in the call. Is there a way to ensure that all characters are read using open()?
Thanks in advance
if (p != NULL){
printf("Inside p not null!\n");
if((fd = open(p, O_RDWR)) < 0){
perror("File could not be opened!");
exit(EXIT_FAILURE);
}
//FILE *rdptr = fopen(p,"r");
memset(buf,0,file_len);
rc = read(fd, buf, file_len);
printf("Number of bytes read: %d\n", rc);
printf("Data detected on FIFO\n");
buf[rc] = '\0';
char base[20] = "output.txt";
char name[20];
sprintf(name, "%d%s", suffix, base);
FILE *fptr = fopen(name,"ab+");
fd_wr = open(name,O_WRONLY);
charnum = write(fd_wr,buf,rc);
kill(id_A, SIGKILL);
//printf("No. of characters written: %d\n",charnum);
//FD_CLR(fd, &rdfs);
}
First minor comment: you should use O_RDONLY to open the file: don't use more permissions than necessary.
Second issue: if file_len is very large, it's possible that the writer has blocked trying to write the entire chunk of data (since a FIFO can only hold a certain amount of unread data). If that's the case, then read will only read the data that has been stored in the FIFO, and will immediately return with whatever it could read. This will allow the writer to write more bytes, which will then be read in the next read.
You should loop reads, adjusting an offset into the buffer, until the entire file_len bytes are read. Something like this:
size_t offset = 0;
while(offset < file_len) {
rc = read(fd, buf+offset, file_len-offset);
if(rc < 0) {
/* handle I/O error or something... */
} else {
offset += rc;
}
}
I've got two C files, server.c and client.c. The server has to create a fifo file and constantly read in it, waiting for input. The client gets its PID and writes the PID in the fifo.
This is my server file which I launch first:
int main(){
int fd;
int fd1;
int bytes_read;
char * buffer = malloc(5);
int nbytes = sizeof(buffer);
if((fd = mkfifo("serverfifo",0666)) == -1) printf("create fifo error");
else printf("create fifo ok");
if ((fd1 = open("serverfifo",O_RDWR)) == -1) printf("open fifo error");
else{
printf("open fifo ok");
while(1){
bytes_read = read(fd,buffer,nbytes);
printf("%d",bytes_read);
}
}
return(0);
}
And my client file :
int main(){
int fd;
int pid = 0;
char *fifo;
int bytes;
if ((pid = getpid()) == 0) printf("pid error");
char pid_s[sizeof(pid)];
sprintf(pid_s,"%d",pid);
if ((fd = open ("serverfifo",O_RDWR)) == -1)printf("open fifo error");
else {
printf("open fifo ok");
bytes = write(fd,pid_s, sizeof(pid_s));
printf("bytes = %d",bytes);
}
close(fd);
return(0);
}
The two main problems I'm getting are: When I write the pid to the file it returns the number of bytes I wrote so it looks okay but when I check the properties of the fifo file it says 0 bytes. The second problem is the read doesn't work. If I do a printf before it shows, but after it doesn't and the read isn't returning anything it just freezes.
I realise there are a lot of similar posts on the site but I couldn't find anything that helped.
I'm using Ubuntu and GCC compiler with CodeBlocks.
There are many things wrong here
char pid_s[sizeof(pid)];
sprintf(pid_s,"%d",pid);
sizeof(pid) returns the size of the pid value, not its string representation, i.e. it is sizeof(int) which is either 4 or 8, depending on your architecture. You then proceed to print it. If this works it works only by luck (you are on a 64 bit machine). The correct way to do is, if you choose to do it at all, is to allocate a suitably large buffer, and use snprintf to make sure you don't overflow. PIDs fit in 5 digits, so something like this will do:
char pid_s[8];
snprintf(pid_s, sizeof(pid_s), "%d", pid);
of course, you can skip this step all together and send the raw bytes of the pid instead
write(fd, (void*)&pid, sizeof(pid))
Now in the server you make similar mistakes:
char * buffer = malloc(5);
int nbytes = sizeof(buffer);
sizeof(buffer) returns 4 or 8 again, but you allocated 5 bytes, the correct way to do this, if you want to allocate on the heap (using malloc), is this:
char* buffer = malloc(8);
int nbytes = 8;
alternatively you can allocate on the stack:
char buffer[8];
int nbytes = sizeof(buffer);
sizeof is sort of magical, in that if you pass in an array, it returns the size of the array (8*1) in this case.
When you are reading, you read 5 bytes, which is likely not enough (because you wrote 8 bytes due to the earlier bug), so it would not finish. You should read like this
int pid;
read(fd, (void*)&pid, sizeof(pid));
Also, if you were to actually read and write strings, you'd do something like this:
// client
char pid_s[8];
snprintf(pid_s, sizeof(pid_s), "%d", pid);
write(fd, pid_s, sizeof(pid_s));
// server
char pid_s[8];
read(fd, pid_s, sizeof(pid_s));
Note also that read may return less than what was written, and you need to call it again to keep reading...
Well there are a lot of mistake in this code...
First of all sizeof is not working like that.
Why do you serialize the pid ?
This is wrong :
char pid_s[sizeof(pid)];
123456 is an int and it doesn't fit into this array of size 4, only 3 char can be printed...
And because you are trying to serialize the pid you don't know the expected size to read, unless you take the worst case and write 10 + 1 for the '\0'...