Similar questions has already been asked, but their solutions aren't helping me much
Program that read file and send it to parent process with pipe
Read/writing on a pipe, accomplishing file copying in C
I'm trying to read from a file test.txt ( which contains a single line of text), write it to a pipe, from where a child process will read from the pipe and write the contents to another file.
/* Read the contents of a file and display it using pipe */
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>
void main()
{
char buffer[100];
char childbuff[100];
int fd[2], des, bytes, target;
pipe(fd);
if(fork()) {
/* parent process closes the downstream */
close(fd[0]);
/* reads the file */
des = open("test.txt", O_RDONLY);
bytes = read(des, buffer, sizeof(buffer));
/* puts data in pipe */
write(fd[1], buffer, bytes);
} else {
/* Child process closes the upstream */
close(fd[1]);
/* reads from the pipe */
read(fd[0], childbuff, sizeof(childbuff));
close(fd[0]);
/* output the received string */
printf("\nReceived string is -- %s", childbuff);
target = open("copy.txt", O_CREAT, 00777);
write(target, childbuff, (strlen(childbuff)-1));
}
}
Problem is printf() prints the string on terminal, a file named copy.txt also gets created, but nothing is getting copied to it (it seems there is a problem with the write() function )
however, if I change
write(target, childbuff, (strlen(childbuff)-1));
to
write(1, childbuff, (strlen(childbuff)-1));
string is simply getting written on my terminal.
So what possibly am I doing wrong while writing to file?
You also need O_WRONLY to write to the file:
target = open("copy.txt", O_CREAT |O_WRONLY, 00777);
Note that you can't use strlen() or %s to print it as a C-string. read(2) doesn't return a NUL terminated string.
Instead get the number of bytes read from read() and use it in write():
ssize_t num_bytes = read(fd[0], childbuff, sizeof(childbuff));
write(target, childbuff, num_bytes);
You should check the return of all system calls for failure.
Related
I'm trying to use posix_openpt on Mac. The issue I'm seeing is that I get a file descriptor back from posix_openpt. I use the file descriptor for reading and create a copy using dup for writing. The issue I'm running into is that when I write to the master file descriptor, I read that data back out from the master. So no data ends up at the slave. I confirmed this by using posix_spawnp to run a program with stdin/stdout/stderr set to the slave file. The program hangs indefinitely waiting for input. Here is my code (note, all error handling was removed for legibility):
int master_fd = posix_openpt(O_RDWR);
grantpt(master_fd);
unlockpt(master_fd);
char *slave_filename_orig = ptsname(master_fd);
size_t slave_filename_len = strlen(slave_filename_orig);
char slave_filename[slave_filename_len + 1];
strcpy(slave_filename, slave_filename_orig);
posix_spawn_file_actions_t fd_actions;
posix_spawn_file_actions_init(&fd_actions);
posix_spawn_file_actions_addopen(&fd_actions, STDIN_FILENO, slave_filename, O_RDONLY, 0644);
posix_spawn_file_actions_addopen(&fd_actions, STDOUT_FILENO, slave_filename, O_WRONLY, 0644);
posix_spawn_file_actions_adddup2(&fd_actions, STDOUT_FILENO, STDERR_FILENO);
pid_t pid;
posix_spawnp(&pid, "wc", &fd_actions, NULL, NULL, NULL);
int master_fd_write = dup(master_fd);
char *data = "hello world";
write(master_fd_write, data, strlen(data));
close(master_fd_write);
char buffer[1024];
read(master_fd, buffer, 1024); // <- Issue Here
// buffer now contains hello world. It should contain the output of `wc`
(Note: The above was only tested on Linux; I don't have a Mac to work on, but I have no reason to believe it's any different in the details here.)
There are several problems with your code:
At least on Linux, calling posix_spawn() with a null pointer causes a crash. You need to provide all the arguments. Even if Macs accept it the way you have it, doing this is a Good Idea.
Next, wc reading from standard input will wait until an attempt to read more data gives an End Of File condition before it prints out the statistics it gathers; your code doesn't do this. With a pty, if you write a specific byte (Typically with the value 4, but it can be different, so best to use what the terminal says instead of hardcoding it) to it, the terminal driver will recognize that as signalling EOF without having to close the master like you would when using a pipe (Making it impossible to read the output of wc).
Second, the terminal's default settings include echoing the input; that's what you're reading.
A cleaned up version that addresses these issues and more (Like yours, with most error checking omitted; real code should be checking all these functions for errors):
#define _XOPEN_SOURCE 700
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <fcntl.h>
#include <spawn.h>
#include <termios.h>
#include <unistd.h>
#include <wait.h>
int main(void) {
int master_fd = posix_openpt(O_RDWR);
grantpt(master_fd);
unlockpt(master_fd);
char *slave_filename_orig = ptsname(master_fd);
size_t slave_filename_len = strlen(slave_filename_orig);
char slave_filename[slave_filename_len + 1];
strcpy(slave_filename, slave_filename_orig);
//printf("slave pty filename: %s\n", slave_filename);
// Open the slave pty in this process
int slave_fd = open(slave_filename, O_RDWR);
// Set up slave pty to not echo input
struct termios tty_attrs;
tcgetattr(slave_fd, &tty_attrs);
tty_attrs.c_lflag &= ~ECHO;
tcsetattr(slave_fd, TCSANOW, &tty_attrs);
posix_spawn_file_actions_t fd_actions;
posix_spawn_file_actions_init(&fd_actions);
// Use adddup2 instead of addopen since we already have the pty open.
posix_spawn_file_actions_adddup2(&fd_actions, slave_fd, STDIN_FILENO);
posix_spawn_file_actions_adddup2(&fd_actions, slave_fd, STDOUT_FILENO);
// Also close the master and original slave fd in the child
posix_spawn_file_actions_addclose(&fd_actions, master_fd);
posix_spawn_file_actions_addclose(&fd_actions, slave_fd);
posix_spawnattr_t attrs;
posix_spawnattr_init(&attrs);
pid_t pid;
extern char **environ;
char *const spawn_argv[] = {"wc" , NULL};
posix_spawnp(&pid, "wc", &fd_actions, &attrs, spawn_argv, environ);
close(slave_fd); // No longer needed in the parent process
const char *data = "hello world\n";
ssize_t len = strlen(data);
if (write(master_fd, data, len) != len) {
perror("write");
}
// Send the terminal's end of file interrupt
cc_t tty_eof = tty_attrs.c_cc[VEOF];
if (write(master_fd, &tty_eof, sizeof tty_eof) != sizeof tty_eof) {
perror("write EOF");
}
// Wait for wc to exit
int status;
waitpid(pid, &status, 0);
char buffer[1024];
ssize_t bytes = read(master_fd, buffer, 1024);
if (bytes > 0) {
fwrite(buffer, 1, bytes, stdout);
}
close(master_fd);
return 0;
}
When compiled and run, outputs
1 2 12
There are two problems with this code.
First, you are seeing "hello world" on master_fd because by default terminals echo. You need to set the terminal to raw mode to suppress that.
Second, wc won't output anything until it sees an EOF, and it will not see an EOF until you close the master. Not just master_fd_write mind you, but all copies of master_fd, including master_fd itself. However, once you close the master, you cannot read from it.
Choose some other program that wc to demonstrate the functionality of posix_openpt.
Edit: It is possible to raise the end-of-file condition on the slave without closing the master by writing ^D (EOT, ascii 4).
I'm trying to understand what is behind this behaviour in my parent process.
Basically, I create a child process and connect its stdout to my pipe. The parent process continuously reads from the pipe and does some stuff.
I noticed that when inserting the while loop in the parent the stdout seems to be lost, nothing appears on the terminal etc I thought that the output of stdout would somehow go to the pipe (maybe an issue with dup2) but that doesn't seem to be the issue. If I don't continuously fflush(stdout) in the parent process, whatever I'm trying to get to the terminal just won't show. Without a while loop in the parent it works fine, but I'm really not sure why it's happening or if the rest of my implementation is problematic somehow.
Nothing past the read system call seems to be going to the stdout in the parent process. Assuming the output of inotifywait in the pipe is small enough ( 30 > bytes ), what exactly is wrong with this program?
What I expect to happen is the stdout of inotifywait to go to the pipe, then for the parent to read the message, run strtok and print the file name (which only appears in stdout when I fflush)
Running the program with inotify installed and creating any file in the current directory of the program should be enough. Removing the while loop does print the created file's name (as expected).
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <errno.h>
int main(void) {
char b[100];
int pipefd;
if (mkfifo("fifo", 0666) == -1) {
if (errno != EEXIST) {
perror("mkfifo");
exit(EXIT_FAILURE);
}
}
pid_t pid = fork();
if (pid < 0) {
perror("fork");
exit(1);
}
if ((pipefd = open("fifo", O_RDWR)) < 0) {
perror("open pipe");
exit(EXIT_FAILURE);
}
if (pid == 0) {
dup2(pipefd, 1);
const char* dir = ".";
const char* args[] = {"inotifywait", dir, "-m", "-e",
"create", "-e", "moved_to", NULL};
execvp("inotifywait", (char**)args);
perror("inotifywait");
} else {
while (1) {
fflush(stdout); // the output only appears in stdout with this here
if (read(pipefd, b, 30) < 0) {
perror("problem # read");
exit(1);
}
char filename[30];
printf("anything");
sscanf(b, "./ CREATE %s", filename);
printf("%s", filename);
}
}
}
The streams used by the C standard library are designed in such a way that they are normally buffered (except for the standard error stream stderr).
The standard output stream is normally line buffered, unless the output device is not an interactive device, in which case it is normally fully buffered. Therefore, in your case, it is probably line buffered.
This means that the buffer will only be flushed
when it is full,
when an \n character is encountered,
when the stream is closed (e.g. during normal program termination),
when reading input from an unbuffered or line-buffered stream (in certain situations), or
when you explicitly call fflush.
This explains why you are not seeing the output, because none of the above are happening in your infinite loop (when you don't call fflush). Although you are reading input, you are not doing this from a C standard library FILE * stream. Instead, you are bypassing the C runtime library (e.g. glibc) by using the read system call directly (i.e. you are using a file descriptor instead of a stream).
The simplest solution to your problem would probably be to replace the line
printf("%s", filename);
with:
printf("%s\n", filename);
If stdout is line-buffered (which should be the case if it is connected to a terminal), then the input should automatically be flushed after every line and an explicit call to fflush should no longer be necessary.
I have two file open in two different processes. There's a pipe connecting the two. Is it possible to write directly from one file to another? Especially if the process reading doesn't know the size of the file it's trying to read?
I was hoping to do something like this
#define length 100
int main(){
int frk = fork();
int pip[2];
pipe(pip);
if (frk==0){ //child
FILE* fp fopen("file1", "r");
write(pip[1],fp,length);
}
else {
FILE* fp fopen("file2", "w");
read(pip[0],fp,length);
}
Is it possible to write directly from one file to another?
C does not provide any mechanism for that, and it seems like it would require specialized hardware support. The standard I/O paradigm is that data get read from their source into memory or written from memory to their destination. That pesky "memory" in the middle means copying from one file to another cannot be direct.
Of course, you can write a function or program that performs such a copy, hiding the details from you. This is what the cp command does, after all, but the C standard library does not contain a function for that purpose.
Especially if the process reading doesn't know the size of the file it's trying to read?
That bit isn't very important. One simply reads and then writes (only) what one has read, repeating until there is nothing more to read. "Nothing more to read" means that a read attempt indicates by its return value that the end of the file has been reached.
If you want one process to read one file and the other to write that data to another file, using a pipe to convey data between the two, then you need both processes to implement that pattern. One reads from the source file and writes to the pipe, and the other reads from the pipe and writes to the destination file.
Special note: for the process reading from the pipe to detect EOF on that pipe, the other end has to be closed, in both processes. After the fork, each process can and should close the pipe end that it doesn't intend to use. The one using the write end then closes that end when it has nothing more to write to it.
In other unix systems, like BSD, there's a call to connect directly two file descriptors to do what you want, but don't know if there's a system call to do that in linux. Anywya, this cannot be done with FILE * descriptors, as these are the instance of a buffered file used by <stdio.h> library to represent a file. You can get the file descriptor (as the system knows it) of a FILE * instance by a call to the getfd(3) function call.
The semantics you are trying to get from the system are quite elaborate, as you want something to pass directly the data from one file descriptor to another, without intervention of any process (directly in the kernel), and the kernel needs for that a pool of threads to do the work of copying directly from the read calls to the write ones.
The old way of doing this is to create a thread that makes the work of reading from one file descriptor (not a FILE * pointer) and write to the other.
Another thing to comment is that the pipe(2) system call gives you two connected descriptors, that allow you to read(2) in one (the 0 index) what is write(2)n in the second (the 1 index). If you fork(2) a second process, and you do the pipe(2) call on both, you will have two pipes (with two descriptors each), one in each process, with no relationship between them. You will be able only to communicate each process with itself, but not with the other (which doesn't know anything about the other process' pipe descriptors) so no communication between them will be possible.
Next is a complete example of what you try to do:
#include <errno.h>
#include <stdlib.h>
#include <string.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
#define length 100
#define FMT(fmt) "pid=%d:"__FILE__":%d:%s: " fmt, getpid(), __LINE__, __func__
#define ERR(fmt, ...) do { \
fprintf(stderr, \
FMT(fmt ": %s (errno = %d)\n"), \
##__VA_ARGS__, \
strerror(errno), errno); \
exit(1); \
} while(0)
void copy(int fdi, int fdo)
{
unsigned char buffer[length];
ssize_t res, nread;
while((nread = res = read(fdi, buffer, sizeof buffer)) > 0) {
res = write(fdo, buffer, nread);
if (res < 0) ERR("write");
} /* while */
if (res < 0) ERR("read");
} /* copy */
int main()
{
int pip[2];
int res;
res = pipe(pip);
if (res < 0) ERR("pipe");
char *filename;
switch (res = fork()) {
case -1: /* error */
ERR("fork");
case 0: /* child */
filename = "file1";
res = open(filename, O_RDONLY);
if (res < 0) ERR("open \"%s\"", filename);
close(pip[0]);
copy(res, pip[1]);
break;
default: /* parent, we got the child's pid in res */
filename = "file2";
res = open(filename, O_CREAT | O_TRUNC | O_WRONLY, 0666);
if (res < 0) ERR("open \"%s\"", filename);
close(pip[1]);
copy(pip[0], res);
int status;
res = wait(&status); /* wait for the child to finish */
if (res < 0) ERR("wait");
fprintf(stderr,
FMT("The child %d finished with exit code %d\n"),
res,
status);
break;
} /* switch */
exit(0);
} /* main */
I'm learning about pipe programming in Linux, and am having trouble understanding pipe / FIFO management.
I wrote a small program which opens a FIFO I created (I did mkfifo newfifo in my terminal before executing the program). I then repeatedly read and dump my character buffer. I'm filling the FIFO using echo "message" > newfifo from another terminal's cmd line.
The problem is that when I write to the FIFO, I can read that data in the buffer, but then the read doesn't block anymore. My understanding was that after I read the data from the FIFO, the FIFO should be empty and the read should block. Am I thinking about this wrong, or am I incorrectly managing the FIFO?
Code is below:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#define NEWPIPE "./newfifo"
void main()
{
int great_success = 0;
int fd;
char buffer[20];
fd = open(NEWPIPE, O_RDONLY);
while (1) {
great_success = read(fd, buffer, 20);
if (great_success < 0) {
printf("pipe failed\n");
} else {
printf("buffer : %s\n", buffer);
printf("great_success = %d\n", great_success);
great_success = 0;
}
}
}
Your understanding of how fifos works is incorrect. They are much like pipes: if the write end is closed (the echo command has terminated), the read end will read end-of-file (EOF), i.e. return 0.
Note that when you open the fifo, it isn't read that is blocking. The blocking system call is the open() system call, as explained in http://linux.die.net/man/4/fifo
Because the process(echo "message" > newfifo) is a short program, it terminated quickly. Once the process terminated, there is no write end for the pipe, so the read end in another process gets an EOF.
I am trying to read from a file, write it to a pipe, and in a child process read from the pipe and write it to a new file. The program is passed two parameters: the name of the input file, and the name of the file to be copied to. This is a homework project, but I have spent hours online and have found only ways of making it more confusing. We were given two assignments, this and matrix multiplication with threads. I got the matrix multiplication with no problems, but this one, which should be fairly easy, I am having so much trouble with. I get the first word of the file that I am copying, but then a whole bunch of garble.
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
int main(int argc, char *argv[]) {
if(argc < 3) {
printf("Not enough arguments: FileCopy input.txt copy.txt\n");
exit(0);
}
char buffer[200];
pid_t pid;
int fds[2];
pipe(fds);
pid = fork();
if (pid == 0) { /* The child process */
//wait(NULL);
write(1, "hi i am in child\n", 17);
int copy = open(argv[2], O_WRONLY | O_CREAT, S_IWUSR | S_IRUSR | S_IXUSR | S_IRGRP);
FILE* stream;
close(fds[1]);
stream = fdopen(fds[0], "r");
while (fgets(buffer, sizeof(buffer), stream) != NULL) {
//printf("%s\n", buffer);
write(copy, buffer, 200);
//printf("kjlkjljljlkj\n");
//puts(buffer);
}
close(copy);
close(fds[0]);
exit(0);
}
else {
write(1, "hi i am in parent\n", 18);
FILE* input = fopen(argv[1], "r");
FILE* stream;
close(fds[0]);
stream = fdopen(fds[1], "w");
/*while (fscanf(input, "%s", buffer) != EOF) {
//printf("%s\n", buffer);
fprintf(stream, "%s\n", buffer);
fflush(stream);
//printf("howdy doody\n");
}*/
fgets(buffer, sizeof(buffer), input);
printf("%s", buffer);
fprintf(stream, "%s", buffer);
fflush(stream);
close(fds[1]);
fclose(input);
wait(NULL);
exit(0);
}
return 0;
}
Am I doing the reads and writes wrong?
Am I doing the reads and writes wrong?
Yes.
In the child, you are mixing string-oriented buffered I/O (fgets()) with block-oriented binary I/O. (That is, write().) Either approach will work, but it would be normal practice to pick one or the other.
If you mix them, you have to consider more aspects of the problem. For example, in the child, you are reading just one line from the pipe but then you write the entire buffer to the file. This is the source of the garbage characters you are probably seeing in the file.
In the parent, you are sending only a single line with no loop. And after that, you close the underlying file descriptor before you fclose() the buffered I/O system. This means when fclose tries to flush the buffer, the now-closed descriptor will not work to write any remaining data.
You can either use write()/read()/close(), which are the Posix-specified kernel-level operations, or you can use fdopen/puts/gets/fclose which are the ISO C - specified standard I/O library operations. Now, there is one way of mixing them that will work. If you use stdio in the parent, you could still use read/write in the child, but then you would be making kernel calls for each line, which would not usually be an ideal practice.
You should generally read/write pipes only using the read/write-calls.
You should close the according ends of the pipe for child (read-only) and parent (write-only).
Afterwards, write from the parent into the pipe using write()-systemcall. And in the child read using read()-systemcall.
Look here for a good explanation.