I need to create program with 3 processes:
The first process should repeatedly read /dev/urandom and send 15 chars each cycle to the second process via a pipe.
The second process should convert received data to hex and send the result to the third process via a fifo.
The third process should print the received data.
This is what I wrote so far. Communication using the pipe is working fine, however there is some problem with the fifo - when I change n to a larger number such as 100000 or 1000000, the program doesn't start. When it's smaller, say 500 or 1000, the program works. What could be the reason behind that?
This is how I run it:
cat /dev/urandom | ./a.out
And here is the code:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#define FIFO "my_fifo"
int main(void) {
int pdesk[2];
char buf[15];
char buffer[15];
char hex[30];
char f[30];
int len;
int n;
n = 100;
umask(0);
mkfifo(FIFO, 0666);
pipe(pdesk);
if (fork() == 0) {
for (int i = 0; i < n; i++) {
read(STDIN_FILENO, buffer, 15);
write(pdesk[1], buffer, 15);
}
close(pdesk[1]);
} else {
sleep(1);
int fp;
for(int i = 0; i < n; i++) {
read(pdesk[0], buf, 15);
for(int a = 0, b = 0; b < 30; ++a, b+= 2)
sprintf(hex + b, "%02x", buf[a] & 0xff);
fp = open(FIFO, O_WRONLY);
write(fp, hex, 30);
close(fp);
usleep(10000);
}
close(pdesk[0]);
}
if (fork() == 0) {
sleep(2);
int fp;
for (int i = 0; i < n; i++) {
fp = open(FIFO, O_RDONLY);
read(fp, f, 30);
printf("Odczytano: %s\n", f);
close(fp);
usleep(10000);
}
}
}
If I understand your code correct, it will do the following:
With the first fork you start a child that reads from stdin and writes to the pipe.
Your parent process reads from the pipe and writes to the FIFO.
When your parent process has finished its loop it calls the second fork to create another child which will read from the FIFO and print the data.
When the loop count is too large you will reach the buffer limit of the FIFO and the parent will block because no process is reading from the FIFO. When the process is blocked in writing to the FIFO it will never create the child that is expected to read from the FIFO.
I think the main problem is that you should create the second child before starting the loop that reads the data from the pipe and writes to the FIFO.
Some additional remarks:
With cat /dev/urandom | ./a.out your program does not read /dev/urandom directly. It reads from a pipe which might behave differently.
You should always check the return value of read. It will tell you how many bytes it has read which may be less than you asked it to read. If you want to have exactly 15 characters you might have to read several times if you get less than 15 characters.
The same applies to write.
Thank you very much. When the process that displays data is above other child processes, it finally works.
With cat /dev/urandom | ./a.out your program does not read /dev/urandom directly. It reads from a pipe which might behave differently.
How could I change it?
The programs also needs to read files the same way it reads from /dev/urandom, for example:
cat file.txt | ./a.out
I took your advice and started to check the value of read and now it doesn't go behind the range of file. The problem is I don't know how to check which parameter was called (and hence I can't check the length of file) - if it was file.txt, /dev/urandom, none or anything else. I tried with
int main(char argc, char* argv[])
but argv is always ./a.out, no matter what I call. Is there any way to check that?
Related
I am trying to create a simple program using pipes, even though that are easier options for the same task (fopen(), lseek(), ftell(), etc).
First I use execve() to perform a terminal cat, and send the information through the pipe so I may be able to print the size of the file descriptor and read it to a malloc'd char pointer. My solution is this one:
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/wait.h>
#include <sys/ioctl.h>
#include <fcntl.h>
int main (int argc, char *argv[]) {
int fd[2], active;
int pipe_sz = 0;
char *name;
pipe(fd);
//change /bin/cat according to your system
char *cmd[] = {"/bin/cat", "example.txt", NULL};
if (fork() == 0) {
//CHILD
dup2(fd[1], STDOUT_FILENO);
close(fd[1]);
active = execve(cmd[0], cmd, NULL);
} else {
//PARENT
wait(NULL);
int ioctl_sz = ioctl(fd[0], FIONREAD, &pipe_sz);
name = malloc(pipe_sz + 1);
int result = read(fd[0], name, pipe_sz);
name[pipe_sz] = '\0';
printf("LEN NAME: %ld\n", strlen(name));
close(fd[0]);
close(fd[1]);
}
return 0;
}
Everything works fine as long as I keep myself inside the pipe's limits (my LINUX system is able to hold 65536 bytes). I decided to push the limits, and see what could happen. As expected, the program stays stuck when my file was above 65536 bytes.
I do not want to manually change my pipe's limits through fcntl(pipefd, F_SETPIPE_SZ, size) (since I "don't know" my file's size). Therefore, I researched, and came across the pipe2(pipefd, O_NONBLOCK) in order to avoid my pipe from stopping the reading process. Unfortunately, I received only this message:
/bin/cat: write error: Resource temporarily unavailable
I even tried a while loop read() and realloc() to see if my code could at least give the char pointer result, but I was not successful, and part of file was lost.
Is it possible to produce a code that may provide the same results as mine above with files bigger than 65536 bytes?
Is there a way to interrupt the pipe if it takes too long to finish the process? Or even make a while loop to guess and resize the pipe through
fcntl()?
Thanks everyone for the help!
I wrote this piece of code to show the basic working of how I would like to send some data (Strings) from the parent process to the child process. But I seem to have some problems. (I removed all error checking to make the code more readable)
When I run this piece of code I expect to see the two test strings to be displayed on the terminal, but I only see the first one.
When I uncomment the first “sleep(1)”, then I see both strings displayed.
But when I uncomment only the second “sleep(1)”, then I again only see the first string.
I suspect this problem has something to do with synchronization. That the strings get written to fast and the fifo write end closes before everything is read by the child process. That’s why we see the correct output when we introduce a sleep between the two write() commands.
But what I don’t understand is that we still get a faulty output when we only introduce a sleep after both write commands. Why can’t the child read both strings even if they are both written before it can read one?
How can I solve this problem? Do I need some synchronization code, and if so how should I implement this. Because I won’t write a “sleep(1)” after every write command.
And is the solution also viable for multiple processes that want to write to the same fifo? (but with still only one process that reads from the fifo)
#include <stdio.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>
int main(int argc, char const *argv[]) {
mkfifo("test_fifo", 0666);
pid_t pid = fork();
if (pid == 0) {
int fd = open("test_fifo", O_RDONLY);
char data[400];
int rc;
do {
rc = read(fd, data, 400);
if (rc > 0) printf("Received data: %s\n", data);
} while(rc > 0);
}
else {
int fd = open("test_fifo", O_WRONLY);
char * string1 = "This is the first test string";
write(fd, string1, strlen(string1) + 1);
//sleep(1);
char * string2 = "This is the second test string";
write(fd, string2, strlen(string2) + 1);
//sleep(1);
close(fd);
wait(NULL);
}
return 0;
}
You are receiving both strings at the same time at the first call to read. Because %s prints up until a zero byte, the second string is just not displayed. The poor mans synchronization with sleep(1) allows child to "catch" the messages in two distinct read call.
read returns the count of bytes read. Use that number. Change the parent code to:
ssize_t rc;
do {
rc = read(fd, data, 400);
if (rc > 0) {
printf("Received data: ");
for (size_t i = 0; i < rc; ++i) {
if (data[i] == '\0') {
printf("\\x00");
continue;
}
printf("%c", data[i]);
}
printf("\n");
}
} while(rc >= 0);
and it shows on my pc:
Received data: This is the first test string\x00This is the second test string\x00
Why can’t the child read both strings even if they are both written before it can read one?
Well, the problem is not in reading, it's how you are displaying the data you read. (Still, reading could be improved, one should handle that pesky EAGAIN errno code).
How can I solve this problem?
If you want 1:1 relationship between read/write use a constant size packets or generally you have to know in advance how many bytes you want to read. Bytes written are "concatenated" together and lose structure. Or use pipe(3p) on which messages with size smaller then PIPE_BUF are guaranteed to be atomic. Or you could use POSIX message queue mq_receive/mq_send.
Or write a proper "deserializer", something that will buffer data and keep internal state and notify higher level only when a whole "message" was received, ie. detect when a zero byte was received in the stream of bytes and restore structure the the stream of bytes.
I'm writing a c program in ubuntu that writes and reads in fifo pipe. I already have the program writing working perfectly, I now have a problem reading in a loop.
My reading program :
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#define MAX_CTRLS 80
void main()
{
printf("test1");
char ch2[MAX_CTRLS];
int pipe1;
pipe1 = open("/tmp/ctrl", O_RDONLY);
printf("test2");
read(pipe1, ch2, MAX_CTRLS);
printf("%s",ch2);
close(pipe1);
}
This program is supposed to read in the fifo, store it in ch2 then print in ch2. But when running it normally it works perfectly, while running it in a loop, it doesn't do anything.
Here is writing program 1 :
void send(char ch2[MAX_CTRLS])
{
int create;
create = mkfifo("/tmp/ctrl", 0666);
int pipe1;
pipe1 = open("/tmp/ctrl", O_WRONLY);
write(pipe1, ch2, MAX_CTRLS);
close(pipe1);
}
This one works using another program I have, that has a line like this :
ch2[0] = 'a';
printf("%s",ch2);
send(ch2);
The printf helps me confirm that ch2 is set properly. I need the reading program to loop the reading function, aka print everytime what is set in ch2.
I just ant it to print a in this case, everytime my program launches send(ch2).
Thaks in advance
I figure you want to have a loop that open the pipe and reads if anything is there, with no delay, then goes to sleep for a while before next reading.
Suggest something like this, u might also swap for(;;) to While(boolValue == true)
int fd = -1, naptime = 200; // 200msec
for (;;){
fd = open(pipe, mode|nodelay);
if (fd != -1){
int st = read (fd, buf, sizeof(buf)); /* open OK */
// do your printing of data if any exist - st holds read chars
if(st >0) printf(buf);
}
sleep(naptime); // sleep til next reading
}
I need the reading program to loop the reading function, aka print everytime what is set in ch2.
Simply adding a while(1) to run the reading program on a loop won't do good, because read() from a FIFO that has been closed (which is the case after the first run of the writing program) doesn't block, but rather returns 0, and we surely don't want a busy waiting read() loop. So, we need a means of waiting for something readable from the FIFO. If we have the fortune to run Linux, we can use inotify to get notified of writings to the FIFO. The reading code could be:
#include <sys/inotify.h>
int fd = inotify_init();
inotify_add_watch(fd, "/tmp/ctrl", IN_MODIFY);
pipe1 = open("/tmp/ctrl", O_RDONLY);
struct inotify_event inev;
while (read(fd, &inev, sizeof inev))
if (read(pipe1, ch2, MAX_CTRLS) > 0)
printf("%s", ch2), fflush(stdout);
I'm so confused with this, I need to create named pipes using mkfifo (i know how to do this) the program before would use fork to create child processes that would do something, but now I have to replace fork with poll() to watch multiple streams (that's the part I don't get). In more detailed, when i run my program in terminal, its suppose to make the mkfifo files and then wait till a stream comes in, hence just stay there, not closing. Then I open up a new terminal, and need to input exactly this into terminal "cat file1 > (name of the mkfifo files)" and what that should do is make the program read the data that was in file1, on any of the input pipes made from mkfifo. I've looked everywhere but can never put things together to make it work.
this is what i have so far
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <poll.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <fcntl.h>
#include "battleFieldReader.h" //this is where all the data computation is done
main(int argc, char *argv[])
{
if (sscanf (argv[1], "%i", &N) !=1 )
{
printf ("ERROR: Please input an integer.\n");
_exit(EXIT_SUCCESS);
}
struct pollfd pfd[2];
pid = getpid();
printf("pid=%d\n",pid);
N = atoi(argv[1]);
signal(SIGTERM, removePipes);
int i;
char fileName[32];
char fileName2[32];
snprintf (fileName, sizeof(fileName), "%d_%di", pid, 0);
mkfifo(fileName, 0666);
pfd[0].fd = open(fileName, O_RDONLY | O_NDELAY);
pfd[0].events = POLLIN;
snprintf (fileName2, sizeof(fileName2), "%d_%do", pid, 0);
mkfifo(fileName2, 0666);
pfd[1].fd = open(fileName2, O_WRONLY | O_NDELAY);
pfd[1].events = POLLIN;
while(1)
{
int n;
n = poll(pfd, 2, 3000);
if(n < 1)
{
printf("waiting...\n");
continue;
}
if(pfd[0].revents & POLLIN)
{
printf("test\n");
/*ideally this is where i would put a method to compute data but its just an infinite loop, its suppose to stop, so it can do the same thing whenever another file comes on, but I dont know how to stop it either*/
}
}
}
whats happening is I'm creating a pipe 2N times, one for input and output for whatever process id is running the program. then wait till something comes in on one of the pipes and then spit out what needs to be done with the file data. Can anyone clear things with me, if I'm going in the right direction or something.
poll tells you if you can read from or write to a file descriptor without blocking (or without getting an EAGAIN/EWOULDBLOCK result if the fd in non-blocking).
Your code has a number of obvious problems:
You say you want to create 2N fifos, but then you only create 2. Your pfd array has a fixed size of 2, too. You'll need a bigger array and loop to create more.
You open an O_WRONLY file descriptor to write to a pipe, but then you set the events field to POLLIN, which will test for input available. You want POLLOUT to test for output possible.
In your processing loop, you poll two fds, but you only check pfd[0] for availability, and then you never do anything with it. You should read pfd[0].fd after the pfd[0].revents & POLLIN check succeeds. You should also check pfd[1].revents & POLLOUT and write data to pfd[1].fd if that succeeds.
I'm experimenting with this dup2 command in linux. I've written a code as follows:
#include <stdio.h>
#include <unistd.h>
#include <string.h>
int main()
{
int pipe1_ends[2];
int pipe2_ends[2];
char string[] = "this \n is \n not \n sorted";
char buffer[100];
pid_t pid;
pipe(pipe1_ends);
pipe(pipe2_ends);
pid = fork();
if(pid > 0) { /* parent */
close(pipe1_ends[0]);
close(pipe2_ends[1]);
write(pipe1_ends[1],string,strlen(string));
read(pipe2_ends[0], buffer, 100);
printf("%s",buffer);
return 0;
}
if(pid == 0) { /* child */
close(pipe1_ends[1]);
close(pipe2_ends[0]);
dup2(pipe1_ends[0], 0);
dup2(pipe2_ends[1],1);
char *args[2];
args[0] = "/usr/bin/sort";
args[1] = NULL;
execv("/usr/bin/sort",args);
}
return 0;
}
I expect this program to behave as follows:
It should fork a child and replace its image with sort process. And since the stdin and stdout are replaced with dup2 command, I expect sort to read input from the pipe and write the output into the other pipe which is printed by the parent. But the sort program doesn't seem to be reading any input. If no commandline argument is given, sort reads it from the stdin right? Can someone help me with this problem, please.
Many thanks!
Hm. What's happening is that you aren't finishing your write: after sending data to the child process, you have to tell it you're done writing, either by closing pipe1_ends[1] or calling shutdown(2) on it. You should also call write/read in a loop, since it's quite likely in the general case that read at least won't give you all the results in one go. Obviously the full code checks all return values, doesn't it?
One final thing: Your printf is badly broken. It can only accept null-terminated strings, and the result returned by read will not be null-terminated (it's a buffer-with-length, the other common way of knowing where the end is). You want:
int n = read(pipe2_ends[0], buffer, 99);
if (n < 0) { perror("read"); exit(1); }
buffer[n] = 0;
printf("%s",buffer);