I have to write two separate programs, which one is a Producer and second is a Consumer (both running in separate terminals). I provide an argument to the Producer which can be a text or a single character. Then, producer creates a .txt file, puts single character into it then closes it. Consumer opens that file, reads that character and prints it on a terminal, then closes the file and deletes it.The whole process repeats itself. If provided argument includes *, for example * or text* it finishes the both programs, printing * before ending. I can only use functions: open(), close(), read(), write(), unlink(). The expected result looks like this:
I have written both codes, this is the Producer code:
(I am aware of the fact that i have unnecessarily defined SIZE and used it, please don't mind it)
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/stat.h>
#include <fcntl.h>
#define SIZE 1
int main(int argc, char *argv[]){
char buff;
do{
int fdi=-1;
while(fdi<0){
fdi=open("test.txt",O_WRONLY | O_CREAT | O_EXCL, 0666);
}
read(STDIN_FILENO,&buff,SIZE);
write(fdi,&buff,SIZE);
close(fdi);
}while(buff!='*');
return 0;
}
and this is the Consumer code:
#include <string.h>
#include <unistd.h>
#include <sys/stat.h>
#include <fcntl.h>
#define SIZE 1
int main(int argc, char *argv[]){
char buff;
do{
int fdi=-1;
while(fdi<0){
fdi=open("test.txt",O_RDONLY | O_EXCL);
}
int rdin=read(fdi,&buff,SIZE);
if(rdin>0){
write(STDOUT_FILENO,&buff,SIZE);
close(fdi);
unlink("test.txt");
}
else{
close(fdi);
}
}while(buff!='*');
return 0;
}
My question is: how does the Producer program not insert more than a one character into the file? I mean, if I for example run the Producer program only, and provide argument text, it will insert in the file the letter t only, the rest will be inserted to other files. Shouldn't it loop and add the whole text word to one file? There is no statement that guarantees that file will contain one character, yet it contains only one character and I don't know why.
In the producer, when you use the O_CREAT and O_EXCL flags when opening a file, the call will fail if the file already exists. So on the first iteration of the outer loop (assuming the file doesn't exist) the file is created and the first character is written. On the next iteration the open call fails because the file exists, so it sits in the inner loop for as long as the file exists.
In the consumer, the open call is done in a loop until it succeeds. This will happen when the producer writes and closes the file. The consumer then reads the character from the file and deletes it. When the consumer deletes the file, the open call in the producer will succeed and write the second character to the file.
This process then repeats until a * character is read by the producer and written to the file, after which the producer exits. Then when the consumer reads the * it also exits.
I will limit my answer to your only specific question:
how does the Producer program not insert more than a one character into the file?
You're doing the following in a loop:
do{
int fdi = -1;
while (fdi < 0){
// Open the file only if it does not exist, creating it.
fdi = open("test.txt", O_WRONLY | O_CREAT | O_EXCL, 0666);
}
// Read 1 char from stdin.
read(STDIN_FILENO, &buff, SIZE);
// Write that char to the beginning of the file.
write(fdi, &buff, SIZE);
// Close and truncate the file.
close(fdi);
} while(buff != '*');
The first time you open the file, it is also created. From the second time onwards, the combination of flags O_CREAT | O_EXCL will make open() fail with error EEXIST, since this combination of flags will open the file only if it doesn't already exist. After writing the first character, your program will run in an endless loop (while (fdi < 0)) trying to open the file a second time.
From the manual page for open():
O_EXCL: Ensure that this call creates the file: if this flag is specified in conjunction with O_CREAT, and pathname
already exists, then open() will fail.
So, first of all, you don't need the O_EXCL flag. Other than that, if you want to append data to the file instead of overwriting its content each time, you should add the O_APPEND flag when you open() the file. From the manual page:
O_APPEND: The file is opened in append mode. Before each write(2), the file offset is positioned at the end of the file,
as if with lseek(2). The modification of the file offset and the write operation are performed as a single
atomic step.
Related
I'm trying to trigger some concurrent conflicts by having several processes writing to the same file, but couldn't:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <fcntl.h>
#include <sys/wait.h>
void concurrent_write()
{
int create_fd = open("bar.txt", O_CREAT | O_TRUNC, 0644);
close(create_fd);
int repeat = 20;
int num = 4;
for (int process = 0; process < num; process++)
{
int rc = fork();
if (rc == 0)
{
// child
int write_fd = open("bar.txt", O_WRONLY | O_APPEND, 0644);
for (int idx = 0; idx < repeat; idx++)
{
sleep(1);
write(write_fd, "child writing\n", strlen("child writing\n"));
}
close(write_fd);
exit(0);
}
}
for (int process = 0; process < num; process++)
{
wait(NULL);
// wait for all children to exits
}
printf("write to `bar.txt`\n%d lines written by %d process\n", repeat * num, num);
printf("wc:");
if (fork() == 0)
{
// child
char *args[3];
args[0] = strdup("wc");
args[1] = strdup("bar.txt");
args[2] = NULL;
execvp(args[0], args);
}
}
int main(int argc, char *argv[])
{
concurrent_write();
return 0;
}
This program fork #num children and then have all of them write #repeat lines to a file. But every time (however I change #repeat and #num) I got the same result that the length of bar.txt (output file) matched the number of total written lines. Why is there no concurrent conflicts triggered?
Writing to a file can be divided into a two-step process:
Locate where you want to write.
Write data into the file.
You open a file with flag O_APPEND and it ensures that the two-step process is atomic. So, you can always find the lines of the file as the count you set.
See the open(2) man page:
O_APPEND
The file is opened in append mode. Before each write(2),
the file offset is positioned at the end of the file, as
if with lseek(2). The modification of the file offset and
the write operation are performed as a single atomic step.
In essence, one of the major design features of O_APPEND is precisely to prevent the sort of "concurrent conflicts" you mention. The typical example would be a log file that several processes must write to. Using O_APPEND ensures their messages do not overwrite each other.
Moreover, all data written by a single write call is written atomically, so provided that your write("child writing\n") successfully writes all its bytes (which for a regular file it usually would), they will not be interleaved with the bytes of any other such message.
First, write() calls with the O_APPEND flag should be atomic. Per POSIX write():
If the O_APPEND flag of the file status flags is set, the file offset shall be set to the end of the file prior to each write and no intervening file modification operation shall occur between changing the file offset and the write operation.
But that's not enough when there are multiple threads or processes making parallel write() calls on the same file - that does not guarantee that parallel write() calls are atomic.
POSIX does guarantee that parallel write() calls are also atomic:
All of the following functions shall be atomic with respect to each
other in the effects specified in POSIX.1-2017 when they operate on
regular files or symbolic links:
...
write()
...
See also Is file append atomic in UNIX?
Beware, though. Reading that question and its answers shows that Linux filesystems such as ext3 are not POSIX compliant once you get past a relatively small size operation, or possibly if you cross page and/or file system sector boundaries. I suspect XFS and ZFS will support write() atomicity much better given their origins.
And none of this applies to Windows.
Given the following code, I have: printf("hello\n"); so I am expecting to see hello\n in myfile. but when I run my program I see haha which means \n was ignored, why is that?
Worth Noting: when I replace printf("hello\n"); with printf("hellos"); I don't see the s letter being printed as well. So I think maybe something is writing on top of it but the question is who and why?
#include <stdio.h>
#include <unistd.h>
#include <sys/wait.h>
#include <sys/file.h>
#include <string.h>
int main() {
// creates a new file having full read/write permissions
int fd = open("myfile", O_RDWR | O_CREAT, 0666);
write(fd, "haha\n", 5);
close(fd); // line 6
fd = open("myfile", O_RDWR); // line 7
close(0);
close(1);
dup(fd);
dup(fd);
if (fork() == 0) {
char s[100];
dup(fd);
scanf("%s", s);
printf("hello\n");
write(2, s, strlen(s)); // line 18
return 0; // line 19
}
wait(NULL);
printf("Father finished\n");
close(fd);
return 0;
}
contents of myfile after running:
haha
helloFather finished (new line after this)
First of all, the behavior is undefined. You are starting to use stdout that refers to the same file descriptor as stdin without closing or flushing stdin before doing it. Let's try to take the important stuff from POSIX 2.5.1 Interaction of File Descriptors and Standard I/O Streams:
[...] if two or more handles are used, and any one of them is a stream, the application shall ensure that their actions are coordinated as described below. If this is not done, the result is undefined.
[...]
For a handle to become the active handle, the application shall ensure that the actions below are performed between the last use of the handle (the current active handle) and the first use of the second handle (the future active handle). [...]
[...]
For the first handle, the first applicable condition below applies. [...]
[...]
If the stream is open with a mode that allows reading and the underlying open file description refers to a device that is capable of seeking, the application shall either perform an fflush(), or the stream shall be closed.
Your code does:
scanf("%s", s); // associates stdin with fd
// Ups - no flush(stdin) nor fclose(stdin)
printf("hello\n"); // associates stdout with fd - undefined behavior
The result you are seeing comes from that scanf calls ungetc which increments file position but also "remembers" to un-increment file position once the stream is flushed. Because it is flushed when child terminates, the file position is decremented, and parent overwrites last character of the child.
So I am trying to do some very simple read/writes on a file. Since it's for an assignment I can't use more sophisticated functions using File*.
I can easily create a file and write to it but if I try to read back my content (it's the same content but my problem boils down to this) I don't get what I expect and I can't yet see why.
Here the code snippet that causes me problems:
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
int main(int argc, char** argv){
int fdisk = open("testfile.txt", O_RDWR | O_CREAT | O_APPEND);
char buff [20] = "Just a short text!!!";
write(fdisk, buff, sizeof(buff));
char buff2[20];
read((fdisk), buff2, sizeof(buff2));
printf("Context of deleted file: %c\n",buff[1]);
printf("Context of deleted file: %c\n",buff2[1]);
return 0;
}
When you create a file you need to specify the file access mode:
int fdisk = open("testfile.txt", O_RDWR | O_CREAT | O_APPEND, 0666);
Otherwise the access mode is some indeterminate value.
And before reading it back you need to rewind it:
lseek(fdisk, 0, SEEK_SET); // rewind
The problem is that the write call leaves the file descriptor pointing to just after the data written (so more writes will go after that rather than overwriting the same data), so the following read call tries to read data after that which was written, and probably gets nothing.
I think it is a combination of issues:
You are not rewinding or re-opening the file, so when you read you are always reading from the end of the file.
You are using append mode, so it will add data to the end of the file. This means that after the first run you will be writing data at the end of the file but always reading from the beginning (assuming you address the first problem).
You are not setting the permissions, so you get random file permissions and the file may not be readable after creating it.
Your print statement is only printing the second character from each buffer, rather than a full string.
Here is a minimal working example. This compiles and runs with the expected results on my machine.
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
int main(int argc, char** argv){
int fdisk = open("testfile.txt", O_RDWR | O_CREAT, 0666);
char buff[] = "Just a short text!!!";
write(fdisk, buff, sizeof(buff));
lseek(fdisk, 0, SEEK_SET);
char buff2[sizeof(buff)];
read((fdisk), buff2, sizeof(buff2));
printf("Context of deleted file: %s\n",buff);
printf("Context of deleted file: %s\n",buff2);
return 0;
}
As a final note, the functions you are using (read/write) all return values indicating whether the operation was successful. You should check them. They would have indicated that the read operation in your problem was not actually reading any data (because it was at the end of the file).
I created and written to a named pipe in C under Linux. For how long the text that is written in there is saved in the named pipe?
From what I have done, and the bytes of the pipe file after my program is run I suppose that the text is not preserved in the pipe after the program ends. In the mkfifo manual there is no info about this. I know that ordinary pipes are destroyed after the process that have created them is closed. But what about named pipes, that are still in your file system after the program has finished?
This is the code I use to create a named pipe and to write/read from it.
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <string.h>
#include <sys/stat.h>
#include <fcntl.h>
int main(int argc, char *argv[]) {
int FIFOFileDescriptorID;
FIFOFileDescriptorID = mkfifo(argv[1], 0660);
int ProccesID = fork();
if (ProccesID == 0) {
int TempFileDescriptor = 0;
char buffer[512] = "Some random text goes here...";
TempFileDescriptor = open(argv[1], O_WRONLY);
write(TempFileDescriptor, &buffer, sizeof(buffer));
close(TempFileDescriptor);
} else {
int TempFileDescriptor = 0;
char buffer[512];
TempFileDescriptor = open(argv[1], O_RDONLY);
read(TempFileDescriptor, &buffer, sizeof(buffer));
close(TempFileDescriptor);
printf("Received string: %s\n", buffer);
}
return 0;
}
After I have run this program and created and use the pipe for write/read, I run another one – just to read the text from the given pipe. Indeed, there was no text there.
I will exam this thing better, because there is a good change, after I start the program do delete/create the pipe again.
It'll not save anything. When you read/write something to the named pipe, it the process will be blocked unless some other process writes/reads from the same named pipe.
The file stays in the file-system. But the content goes away when reading/writing finishes.
From linux manual,
Once you have created a FIFO special file in this way, any process
can open it for reading or writing, in the same way as an ordinary file.
However, it has to be open at both ends simultaneously before you can
proceed to do any input or output operations on it. Opening a FIFO for
reading normally blocks until some other process opens the same FIFO for
writing, and vice versa.
Here is some code I wrote up to test named pipes. I made sure to handle all errors:
cleanup in SIGPIPE
Look at Wikipedia: http://en.wikipedia.org/wiki/Named_pipe - named pipes persist beyond the lifetime of the process that created or used them, until they are explicitly deleted.
here's another question about splice(). I'm hoping to use it to copy files, and am trying to use two splice calls joined by a pipe like the example on splice's Wikipedia page. I wrote a simple test case which only tries to read the first 32K bytes from one file and write them to another:
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
int main(int argc, char **argv) {
int pipefd[2];
int result;
FILE *in_file;
FILE *out_file;
result = pipe(pipefd);
in_file = fopen(argv[1], "rb");
out_file = fopen(argv[2], "wb");
result = splice(fileno(in_file), 0, pipefd[1], NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
printf("%d\n", result);
result = splice(pipefd[0], NULL, fileno(out_file), 0, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
printf("%d\n", result);
if (result == -1)
printf("%d - %s\n", errno, strerror(errno));
close(pipefd[0]);
close(pipefd[1]);
fclose(in_file);
fclose(out_file);
return 0;
}
When I run this, the input file seems to be read properly, but the second splice call fails with EINVAL. Anybody know what I'm doing wrong here?
Thanks!
From the splice manpage:
EINVAL Target file system doesn't support splicing; target file is
opened in append mode; neither of the descriptors refers to a
pipe; or offset given for non-seekable device.
We know one of the descriptors is a pipe, and the file's not open in append mode. We also know no offset is given (0 is equivalent to NULL - did you mean to pass in a pointer to a zero offset?), so that's not the problem. Therefore, the filesystem you're using doesn't support splicing to files.
What kind of file system(s) are you copying to/from?
Your example runs on my system when both files are on ext3 but fails when I use an external drive (I forget offhand if it is DOS or NTFS). My guess is that one or both of your files are on a file system that splice does not support.
The splice(2) system call is for copying between files and pipes and not between files, so it can not be used to copy between files, as has been pointed out by the other answers.
As of Linux 4.5 however a new copy_file_range(2) system call is available that can copy between files. In the case of NFS it can even cause server side copying.
The linked man page contains a full example program.