What is the best way to do this theoretically? I need to let the user enter the number of processes to send to a pipe for instance "3" and as it loops through the three [three whats?] on each iteration I need to create a process, send it [what?] to the pipe and print it.
The next time the user enters another number, say "4", it should print the previous 3 + 1.. I am working on this but can't understand how do it. Here is my code. I just need guidance, no need to try to solve it for me (but suggestions would be much appreciated).
Right now I am able to send one through the pipe and return it but then the pipe closes and it does not allow for the other processes to get in there.
Suggestion #1: Use functions
Use functions, even for little jobs such as:
void create_fifo(const char *name)
{
/* Create the first named - pipe */
int ret_val = mkfifo(name, 0666);
if ((ret_val == -1) && (errno != EEXIST))
{
perror("Error creating the named pipe");
exit(1);
}
}
Now you can simply write in your main program:
create_fifo(PIPE1);
create_fifo(PIPE5);
This cuts down on the clutter in your main program. It also adheres to the Agile principle DRY - Don't Repeat Yourself.
Suggestion #2: Error check system calls.
You did that for creating the FIFOs, which is good. You don't for the open() calls, or the read() or write() calls. You probably should. I use a function similar to the following in my programs:
#include <stdarg.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>
static const char *arg0 = "did not call err_setarg0(argv[0])";
void err_setarg0(const char *argv0)
{
arg0 = argv0;
}
void err_exit(const char *fmt, ...)
{
int errnum = errno; /* Capture errno before it is changed */
va_lists args;
fprintf(stderr, "%s: ", arg0);
va_start(args, fmt);
vfprintf(stderr, fmt, args);
va_end(args);
if (errnum != 0)
fprintf(stderr, "%d: %s\n", errnum, strerror(errnum));
exit(1);
}
You can then use:
if ((rdfd1 = open(PIPE1, O_RDONLY)) < 0)
err_exit("Failed to open FIFO %s for reading: ", PIPE1);
if ((wrfd1 = open(PIPE5, O_WRONLY)) < 0)
err_exit("Failed to open FIFO %s for writing: ", PIPE5);
Suggestion #3: Make an iterative server
Your server program currently opens the FIFOs once, then reads from one, write to the other, and terminates. You need a loop around some portion of this code, maybe two nested loops. You have to decide whether you need an inner loop to read until EOF. You also need to know how you will terminate the server.
Suggestion #4: Maybe the server needs pipe names as arguments
Your server currently works on fixed FIFO names. You probably need it to take input and output file names as command line arguments, so that when your client spawns multiple servers, each server can have its own set of FIFOs, rather than all processes sharing the same two FIFOs, which is going to lead to confusion and chaos.
Indeed, the need for generating names calls the whole design into question - are you sure using FIFOs is the best way to do this? It looks to me like a case where anonymous pipes would serve you better; you wouldn't have to invent names, and the server would simply read from its standard input and write the (modified?) data to its standard output, so you could even simply use cat or tr or sed or ... as your server.
Clearly, if you use pipes, you will need to do some careful plumbing, but you also need to do careful plumbing with the pairs of FIFOs per server.
Related
I have two file open in two different processes. There's a pipe connecting the two. Is it possible to write directly from one file to another? Especially if the process reading doesn't know the size of the file it's trying to read?
I was hoping to do something like this
#define length 100
int main(){
int frk = fork();
int pip[2];
pipe(pip);
if (frk==0){ //child
FILE* fp fopen("file1", "r");
write(pip[1],fp,length);
}
else {
FILE* fp fopen("file2", "w");
read(pip[0],fp,length);
}
Is it possible to write directly from one file to another?
C does not provide any mechanism for that, and it seems like it would require specialized hardware support. The standard I/O paradigm is that data get read from their source into memory or written from memory to their destination. That pesky "memory" in the middle means copying from one file to another cannot be direct.
Of course, you can write a function or program that performs such a copy, hiding the details from you. This is what the cp command does, after all, but the C standard library does not contain a function for that purpose.
Especially if the process reading doesn't know the size of the file it's trying to read?
That bit isn't very important. One simply reads and then writes (only) what one has read, repeating until there is nothing more to read. "Nothing more to read" means that a read attempt indicates by its return value that the end of the file has been reached.
If you want one process to read one file and the other to write that data to another file, using a pipe to convey data between the two, then you need both processes to implement that pattern. One reads from the source file and writes to the pipe, and the other reads from the pipe and writes to the destination file.
Special note: for the process reading from the pipe to detect EOF on that pipe, the other end has to be closed, in both processes. After the fork, each process can and should close the pipe end that it doesn't intend to use. The one using the write end then closes that end when it has nothing more to write to it.
In other unix systems, like BSD, there's a call to connect directly two file descriptors to do what you want, but don't know if there's a system call to do that in linux. Anywya, this cannot be done with FILE * descriptors, as these are the instance of a buffered file used by <stdio.h> library to represent a file. You can get the file descriptor (as the system knows it) of a FILE * instance by a call to the getfd(3) function call.
The semantics you are trying to get from the system are quite elaborate, as you want something to pass directly the data from one file descriptor to another, without intervention of any process (directly in the kernel), and the kernel needs for that a pool of threads to do the work of copying directly from the read calls to the write ones.
The old way of doing this is to create a thread that makes the work of reading from one file descriptor (not a FILE * pointer) and write to the other.
Another thing to comment is that the pipe(2) system call gives you two connected descriptors, that allow you to read(2) in one (the 0 index) what is write(2)n in the second (the 1 index). If you fork(2) a second process, and you do the pipe(2) call on both, you will have two pipes (with two descriptors each), one in each process, with no relationship between them. You will be able only to communicate each process with itself, but not with the other (which doesn't know anything about the other process' pipe descriptors) so no communication between them will be possible.
Next is a complete example of what you try to do:
#include <errno.h>
#include <stdlib.h>
#include <string.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
#define length 100
#define FMT(fmt) "pid=%d:"__FILE__":%d:%s: " fmt, getpid(), __LINE__, __func__
#define ERR(fmt, ...) do { \
fprintf(stderr, \
FMT(fmt ": %s (errno = %d)\n"), \
##__VA_ARGS__, \
strerror(errno), errno); \
exit(1); \
} while(0)
void copy(int fdi, int fdo)
{
unsigned char buffer[length];
ssize_t res, nread;
while((nread = res = read(fdi, buffer, sizeof buffer)) > 0) {
res = write(fdo, buffer, nread);
if (res < 0) ERR("write");
} /* while */
if (res < 0) ERR("read");
} /* copy */
int main()
{
int pip[2];
int res;
res = pipe(pip);
if (res < 0) ERR("pipe");
char *filename;
switch (res = fork()) {
case -1: /* error */
ERR("fork");
case 0: /* child */
filename = "file1";
res = open(filename, O_RDONLY);
if (res < 0) ERR("open \"%s\"", filename);
close(pip[0]);
copy(res, pip[1]);
break;
default: /* parent, we got the child's pid in res */
filename = "file2";
res = open(filename, O_CREAT | O_TRUNC | O_WRONLY, 0666);
if (res < 0) ERR("open \"%s\"", filename);
close(pip[1]);
copy(pip[0], res);
int status;
res = wait(&status); /* wait for the child to finish */
if (res < 0) ERR("wait");
fprintf(stderr,
FMT("The child %d finished with exit code %d\n"),
res,
status);
break;
} /* switch */
exit(0);
} /* main */
I'm not an expert in C and I'm looking for some advice to to make my program more robust and reliable. Just to give some context: I've written a program to do some scientific computation that takes quite a long time (about 20h) that I'm executing on a large university HPC linux cluster using a SLRUM scheduling system and NFS mounted file systems. What seems to happen is that some time during the 20h the connection to the file system goes stale (on the entire machine; independent of my program) and the first attempt to open & write a file takes a really long time and that results in a segfault cored dumped error that I have so far not been able to precisely track down. Below is a minimal file that at least conceptually reproduces the error: The program starts, opens a file and everything works. The program does some long computation (simulated by sleep()), tries to open & write to the same file again, and it fails. What are some conventions to make my code more robust and reliably write my results to file without crashing?
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main(int argc, char **argv) {
// Declare variables
FILE *outfile;
char outname[150] = "result.csv";
// Open file for writing
printf("CHECKING if output file '%s' is writable?", outname);
outfile=fopen(outname, "w");
if (outfile == NULL) {
perror("Failed: ");
exit(EXIT_FAILURE);
}
fclose(outfile);
printf(" PASSED.\n");
// Do some computation that takes really long (around 19h)
sleep(3);
// Open file again and Write results
printf("Writing results to %s ...", outname);
outfile=fopen(outname, "w");
if (outfile == NULL) {
perror("Failed writing in tabulate_vector_new: ");
exit(EXIT_FAILURE);
}
fprintf( outfile, "This is the important result.\n");
fclose(outfile);
printf(" DONE.\n");
return 0;
}
It seems odd that your program would segfault due to an NFS issue. I would expect it to hang indefinitely, not crash. That having been said, I would suggest forking a new process to check whether the NFS mount is working. That way, your important code won't be directly involved in testing the problematic file system. Something like the following approach may be useful:
pid_t pid = fork();
if (pid == -1)
{
// error, failed to fork(). should probably give up now. something is really wrong.
}
else if (pid > 0)
{
// if the child exits, it has successfully interacted with the NFS file system
wait(NULL);
// proceed with attempting to write important data
}
else
{
// we are the child; fork df in order to test the NFS file system
execlp("df", "df", "/mnt", (char *)NULL)
// the child has been replaced by df, which will try to statfs(2) /mnt for us
}
The general concept here is that we utilize the df command to check whether the NFS file system (which I assume is at /mnt) is working. If it's temporarily not working, df should hang until it starts working again, and then exit, returning control to your program. If you suspect df might hang forever, you could enhance my example by using alarm(2) to wait a certain period of time, probably at least a few minutes, after which you could retry running df. Note that this could result in zombie df processes sticking around.
In the end, the correct solution is to try to get a more reliable NFS server, but until you can do that, I hope this is helpful.
I want to write a stream into one FILE *fp at the same time the stream should be copied onto another fp too is there a better way to write my debug function by eliminating one fprintf?
const int logflag=1;
#define debug(args ...) if (logflag) { FILE *flog = fopen("test.log", "a+"); fprintf( flog, args); fclose(flog); } fprintf(stderr, args);
int main()
{
debug("test"); // writes test into both stderr and flog
debug("test2");
}
The short answer is no, it's two different file pointers and you can only write to one at a time. Actually, dup still doesn't help you because it closes the duplicated file descriptor:
"dup2() makes newfd be the copy of oldfd, closing newfd first if necessary"
from the dup2 man-pages
However, if your goal is to have both a log to the screen and to a file, you are better served by using the tools Linux already provides you. A generally good practice (I don't remember the source for this) is to have a program print its output and debugging to a stdout/stderr and let the calling user determine how to handle the output.
Following this, if all of your output goes to stderr, you can do the following when executing the program:
$ ./program 2>&1 | tee file.log
I realize this question is asked frequently, mainly by people who want to intercept the password-asking phase of SSH. This is not what I want. I'm after the post-login text.
I want to write a wrapper for ssh, that acts as an intermediary between SSH and the terminal. I want this configuration:
(typing on keyboard / stdin) ----> (wrapper) ----> (ssh client)
and the same for output coming from ssh:
(ssh client) -----> (wrapper) -----> stdout
I seem to be able to attain the effect I want for stdout by doing a standard trick I found online (simplified code):
pipe(fd)
if (!fork()) {
close(fd[READ_SIDE]);
close(STDOUT_FILENO); // close stdout ( fd #1 )
dup(fd[WRITE_SIDE]); // duplicate the writing side of my pipe ( to lowest # free pipe, 1 )
close(STDERR_FILENO);
dup(fd[WRITE_SIDE]);
execv(argv[1], argv + 1); // run ssh
} else {
close(fd[WRITE_SIDE]);
output = fdopen(fd[READ_SIDE], "r");
while ( (c = fgetc(output)) != EOF) {
printf("%c", c);
fflush(stdout);
}
}
Like I said, I think this works. However, I can't seem to do the opposite. I can't close(STDIN_FILENO) and dup the readside of a pipe. It seems that SSH detects this and prevents it. I've read I can use the "-t -t" option to force SSH to ignore the non-stdin nature of its input; but when I try this it still doesn't work.
Any hints?
Thanks very much!
Use popen (instead of execv) to execute the ssh cmd and be able to read and write to the session.
A pipe will not work if you want to allow any interactive use of ssh with the interceptor in place. In this case, you need to create a pseudo-tty. Look up the posix_openpt, ptsname, and grantpt functions. There's also a nonstandard but much-more-intuitive function called openpty, and a wrapper for it called forkpty, which make what you're trying to do extremely easy.
Python's Paramiko does all of this with SSH but it is in Python source code. However, for a C programmer, reading Python is a lot like reading pseudocode so go to the source and learn exactly what works.
Here's a working example that writes to ssh:
#include <unistd.h>
int main(int argc, char **argv)
{
int pid;
int fds[2];
if (pipe(fds))
return -1;
pid = fork();
if (!pid)
{
close(fds[1]);
close(STDERR_FILENO);
dup2(fds[0], STDIN_FILENO);
execvp(argv[1], argv + 1);
}
else
{
char buf[256];
int rc;
close(fds[0]);
while ((rc = read(STDIN_FILENO, buf, 256)) > 0)
{
write(fds[1], buf, rc);
}
}
wait(NULL);
return 0;
}
This line is probably wrong:
execv(argv[1], argv + 1); // run ssh
The array must be terminated by a NULL pointer, if you are using argv[] the parameter from main() I don't think there is any guarantee that this is the case. Edit: just checked the C99 standard and argv is NULL terminated.
execv() does not search the path for the file to execute, so if you are passing ssh as the parameter, it is equivalent to ./ssh which is probably not what you want. You could use execvp() but that is a security risk if a malicious program called ssh appears in $PATH before /bin/ssh. Better to use execv() and force the correct path.
Can I make an anonymous stream in c? I don't want to create a new file on the file system, just have a stream that one function can fwrite to while the other can fread from it. Not c++, c.
Maybe You're looking for pipes.
Forward Your STDOUT to the pipe.
Then the other application would read from the pipe.
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#define RDR 0
#define WTR 1
char ** parseargs(char *string);
int main(void){
char mode = 'r';
char prog[50] = "/bin/ps --version";
char **argv;
int p[2];
pid_t pid;
FILE *readpipe;
int pipein, pipeout;
char buf;
/* create the pipe */
if(pipe(p) != 0){
fprintf(stderr, "error: could not open pipe\n");
}
pipein = p[RDR];
pipeout = p[WTR];
if((pid = fork()) == (pid_t) 0){
close(pipein);
dup2(pipeout, 1);
close(pipeout);
if(execv(argv[0], argv) == -1){
fprintf(stderr, "error: failed to execute %s\n", argv[0]);
}
_exit(1);
}
close(pipeout);
readpipe = fdopen(pipein, &mode);
while(!feof(readpipe)){
if(1 == fread(&buf, sizeof(char), 1, readpipe)){
fprintf(stdout, "%c", buf);
}
}
return 0;
}
Yes, tmpfile() is one way to do it. However, I believe tmpfile() is frowned upon these days due to security concerns.
So, you should use mkstemp in POSIX or tmpfile_s in Windows instead of tmpfile().
These will all still create files in the filesystem, though. They're temporary in that they "go away" when the program exits.
Another option, which doesn't create a physical file is mmap().
Oops, just found it... maybe. tmpfile() returns a tmeporary FILE *
Is that the right way to do it?
If you're on Unix (or a similar OS), you want to read Beej's Guide to Unix Interprocess Communication (it's a good read no matter what your OS is).
Check it out at Beej's Guides.
In a rapid glance there I noticed a few things you could probably use with more or less work (and with the optional creation of a file/resource):
Pipes
FIFOs
Message Queues
Shared Memory Segments
Memory Mapped Files
Unix Sockets