I am learning about ipc in Linux and trying out pipes. I've set up two pipes between the parent and two child processes. While the data goes through the pipes, I get weird newlines. For instance, the output would sometimes have an extra newline or no newline entirely or even appear on the command line itself. Also, I am unsure whether the way I have set up pipes is correct. I may have overlooked some important details and leave dangling file descriptors.
void run_processes(Command_Args *cmd_args, char *file_paths)
{
pipe(pipe_RtoA1);
pipe(pipe_RtoA2);
pipe(pipe_A1toT1);
pipe(pipe_A2toT2);
pipe(pipe_T1toR);
pipe(pipe_T2toR);
if (!(pid_A1 = fork())) {
long read = 0;
size_t size = 0;
char *input_str = NULL;
close(pipe_RtoA1[1]);
dup2(pipe_RtoA1[0], 0);
read = getline(&input_str, &size, stdin);
printf("A1 : %s\n", input_str);
} else if (!(pid_A2 = fork())) {
long read = 0;
size_t size = 0;
char *input_str = NULL;
close(pipe_RtoA2[1]);
dup2(pipe_RtoA2[0], 0);
read = getline(&input_str, &size, stdin);
printf("A2 : %s\n", input_str);
} else {
FILE *fRtoA1 = NULL;
FILE *fRtoA2 = NULL;
fRtoA1 = fdopen(pipe_RtoA1[1], "w");
fRtoA2 = fdopen(pipe_RtoA2[1], "w");
close(pipe_RtoA1[0]);
close(pipe_RtoA2[0]);
fprintf(fRtoA1, "%s", file_paths);
fprintf(fRtoA2, "%s", file_paths);
}
}
I plan on having pipes to other processes, but now I just want to get the pipes from this program R to two other programs A1 and A2 working.
Program R will send file_paths do the pipes and A1 and A2 will print them.
You are launching two distinct processes that are ouputting on th standard output in an unpredictable order. Their output may be interlaced. Just to make sure, open output an file for each subprocess, and check the content.
Related
I am facing some issues when trying to filter specific words from a file, and write them into a new file.
What I am trying to do is writing only words that comes after '&' until the first digit.
For example (This is the content of the file I am reading from):
& some 12 test1 test2
$ thisword 4 no no no no
For the above input , I want to write into a new file the words some and thisword only.
My code is working but , instead of printing only those words , it is printing garbage as well.
int main (argc,argv)
int argc;
char *argv[];
{
int inpfd,outpfd,n;
int i=0;
char tmp[2],buff[BUFFSIZE]; //This is our buffer
//Open the output file of ispell
inpfd = open("outputfile.txt",O_RDONLY);
//Check if open command failed
if(inpfd == -1) {
printf("Failed to open file");
exit(1);
}
//Here we are reading from output file
read(inpfd,buff,999);
buff[999] = '\0';
close(inpfd);
outpfd = open("w.txt",O_WRONLY);
if(outpfd == -1) {
printf("Cannot open file for writing!");
exit(1);
}
//Looping over the Buffer
for (i=0; i <BUFFSIZE; i++) {
printf("This is the char : %c \n",buff[i]);
if(buff[i] == '&') {
i++;
while( !(isdigit(buff[i])) ) { //Write into output file
//As long as we didnt reach
tmp[0] = buff[i]; // To the digit
write(outpfd,tmp,1);
i++;
}
write(outpfd,"\n",1); //Moving to the next line
}
}
close(outpfd);
return 0;
}
This the output of the file after the writing (I am pasting only small part of the garbage):
some
thisword
^#^#^#<FD>^?^#^#<80><B2>-<AD><FD>^?^#^#<B0>
<B0>be^#^#^#^#[^X^?^#^#^#<B4>-<AD><FD>^?^#^#s^X<F0>[^X^?^#^#^#<FF>^#^#^#^#^#^#
^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#/
I have no idea what is this garbage , can someone please assist?
Your problem is in this code
read(inpfd,buff,999);
buff[999] = '\0';
close(inpfd);
where you are ignoring the actual length of what you are reading
You should at least use the length of data actually read -- like this
int len = read(inpfd,buff,999);
buff[len] = '\0';
close(inpfd);
However note that the above have it own issues, as read does not always return everything in one go and can terminate early for interrupts etc etc, but that is beyond the scope of this question. For very simple apps, you may just get away with the simple modification.
Now after null terminating knowing the actual length of the file from the result of read, you also need to fix your loop -- the first step would be to have your outer loop only looking at the data that you read, so
So instead
for (i=0; i <BUFFSIZE; i++) {
use the actual length;
for (i=0; i <len; i++) {
Your code inside the loop contains several issues as well, loop termination for one, that you will have to fix as well.
You are looping over the entire size of your buffer, or 999 items.
for (i=0; i <BUFFSIZE; i++) {
The input file is almost certainly shorter than 999 items. So once you're done processing the supplied input, you're just processing garbage until your counter reaches 999!
I have read previous questions regarding this problem too. fflush(stdin) does not work in this scenario for me.
I want my program to read from a piped stdin and continue from keyboard input in the middle.
int main()
{
int userin = 3;
read_input();
userin = print_menu();
userin = parse_input(userin);
return 0;
}
I want to read data from a file which is passed to the program as a pipied stding like
program < testing_text
int read_input(){
char line[200];
char word[MAX_STRING+1];
int line_number = 0;
while(fgets(line, sizeof(line), stdin) != NULL ){
//do something
printf("%s",line);
line_number++;
}
}
Now read_input must finish reading from the piped input. 'print_menu' must continue reading from the keyboard.
int print_menu()
{
int userinput;
char c;
char num[4];
while((c=getchar()) != '\n' && c != EOF && c != '\r');
printf("\n1. Choice 1 \n");
printf("2. Choice 2\n");
printf("3. Exit\n");
printf("Enter your choice (1-3): ");
/* scanf("%d", &userinput); */
/* fgets(num,80,stdin); */
scanf("%s", num);
userinput = atoi(num);
return userinput;
}
int parse_input(int userinput)
{
char num[4];
while( userinput > 3 || userinput < 1 ){
printf("Sorry, that is not a valid option\n");
printf("Enter your choice (1-3): ");
scanf("%s", num);
userinput = atoi(num);
/* scanf("%d", &userinput); */
/* while( (c = getchar()) == '\n'); */
}
return userinput;
}
My output is a infinite loop of
Enter your choice (1-3): Sorry, that is not a valid option
When I remove read_input method and piped stdin program works fine.
I cannot figure out a get around for this, does someone has a idea..
Pipes come (controlled by the shell) normally on file-descriptor 0, which is the standard input. You can use dup2() to "move" that to a different file descriptor, and use freopen() to continue reading from that. You can also open the /dev/tty device (or whatever the tty program knows about) and make that your standard input.
The dialog program does this for the --gauge (progress bar) option. The reason why it does this is because dialog is a curses application (which by default uses the standard input and output). The actual pipe file-descriptor is as well optional, so quoting from its manual page gives some hints:
--input-fd fd
Read keyboard input from the given file descriptor. Most dialog
scripts read from the standard input, but the gauge widget reads
a pipe (which is always standard input). Some configurations do
not work properly when dialog tries to reopen the terminal. Use
this option (with appropriate juggling of file-descriptors) if
your script must work in that type of environment.
The logic which implements this is in the util.c, from the init_dialog function. Here is the main part of that, to give an idea how the functions are used:
dialog_state.pipe_input = stdin;
if (fileno(input) != fileno(stdin)) {
if ((fd1 = dup(fileno(input))) >= 0
&& (fd2 = dup(fileno(stdin))) >= 0) {
(void) dup2(fileno(input), fileno(stdin));
dialog_state.pipe_input = fdopen(fd2, "r");
if (fileno(stdin) != 0) /* some functions may read fd #0 */
(void) dup2(fileno(stdin), 0);
} else {
dlg_exiterr("cannot open tty-input");
}
close(fd1);
} else if (!isatty(fileno(stdin))) {
if ((fd1 = open_terminal(&device, O_RDONLY)) >= 0) {
if ((fd2 = dup(fileno(stdin))) >= 0) {
dialog_state.pipe_input = fdopen(fd2, "r");
if (freopen(device, "r", stdin) == 0)
dlg_exiterr("cannot open tty-input");
if (fileno(stdin) != 0) /* some functions may read fd #0 */
(void) dup2(fileno(stdin), 0);
}
close(fd1);
}
free(device);
}
I had a nearly-identical situation with a program I wrote to read binary data from stdin (piped in from some other program's stdout), but I wanted to add keyboard commands (via raw terminal support). Obviously reading piped data and the keyboard simultaneously led to some weird results.
Here's my code, working and tested:
FILE * fp;
int fd;
fd = dup(fileno(stdin));
fp = fdopen(fd, "r");
(void) freopen("/dev/tty", "r", stdin);
Now, my program reads from fp (a FILE* object) for its binary data while reading stdin (now associated with "/dev/tty") for keystrokes.
Note that I discard the return value from freopen (a FILE* object) since I can refer to it via stdin if needed. My experience is that there is no need to close() or fclose() a standard text stream. I fclose(fp) at the end of reading the binary data.
If you want your program to accept keyboard input then don't redirect its standard input. If you redirect a program's standard input, don't expect to be able to feed it data via the keyboard.
If your program needs to accept input both from a file and from the keyboard, then give it the file name as an argument, and let it open and read that file.
If you have to deal with some kind of unmodifiable library function that requires input to be provided via stdin, and you want that input to come from a file, and you want elsewhere to accept keyboard input, then you can maybe play games with dup2()ing file descriptors to make it work out. Once you get it working, consider firing the idiot responsible for that library function.
I have the below code (not mine) trying to get it to work out of curiousity.
from here C low-level standard-in to accept filename then printing file contents to stdout
int checkfile(void)
{
char buffer[4096];
char userInput[100];
int input_file1;
int n;
/* Check file existence. */
printf("Enter the filename: \n");
fflush(NULL);
if (!fgets(userInput,sizeof(userInput),stdin))
{
perror("fgets");
exit(EXIT_FAILURE); };
if((input_file1 = open(userInput, O_RDONLY)) < 0)
{
perror(userInput);
exit(1);
}
while((n = read(input_file1, buffer, sizeof(buffer))) > 0)
{
if((write(STDOUT_FILENO, buffer, n)) < 0)
{
perror("failed to display file to output");
close(input_file1);
exit(1);
}
}
}
whenever i run the code and try to open a file called "success.txt" i get a segmentation fault
"RUN FINISHED; Segmentation fault; core dumped; real time: 3s; user: 0ms; system: 0ms"
i'm also calling this from my main code as
checkfile();
if that makes any difference.
Could someone point out to me what i'm missing because i can't see it at the moment. i have a feeling some of my variables are not set properly...but unsure thank you.
if (!fgets(userInput,sizeof(userInput),stdin))
is wrong on couple of accounts.
userInput does not point to any valid memory.
sizeof(userInput) is the same as sizeof(char*), which is not what you want.
Change
char *userInput;
to something like:
char userInput[100];
Next problem
if((input_file1 = open(userInput, O_RDONLY)) < 0)
That's wrong. The return value of open is an int. Type of input_file1 is FILE*. I am surprised you didn't get compiler errors/warnings.
Change
FILE *input_file1;
to
int input_file1;
And the next problem
It's probably caused by fgets() including the newline character in userInput. Add code to trim the newline.
int len = strlen(userInput);
if ( userInput[len-1] == '\n' )
{
userInput[len-1] = '\0';
}
I have to implement an application where user passes multiple words via command line and the application finds count of the word in each line of file. Each word will search the file in its own thread.
So far I have implemented it as single threaded app.
The code looks like:
//Below function reads file line and returns it
char* readLine(FILE* file, char* line)
{
if (file == NULL) {
printf("Error: file pointer is null.");
exit(1);
}
int maximumLineLength = 128;
char *lineBuffer = (char *) malloc(sizeof(char) * maximumLineLength);
if (lineBuffer == NULL) {
printf("Error allocating memory for line buffer.");
exit(1);
}
char ch = getc(file);//Get each character
int count = 0;
//loop for line or EOF
while ((ch != '\n') && (ch != EOF))
{
if (count == maximumLineLength)
{
maximumLineLength += 128;
lineBuffer = realloc(lineBuffer, maximumLineLength);
if (lineBuffer == NULL)
{
printf("Error reallocating space for line buffer.");
exit(1);
}
}
lineBuffer[count] = ch;
count++;
ch = getc(file);
}
lineBuffer[count] = '\0';//Add null character
line = (char *) malloc(sizeof(char) * (count + 1));
strncpy(line, lineBuffer, (count + 1));
free(lineBuffer);
return line;
}
//Below function finds the occurance of
//word in the line
//Need to refine to take into consideration
//scenarios such that {"Am"," am "," am","?Am",".Am"}etc
int findWord(char* line,char* word)
{
int count=0;
int lineLen = strlen(line);
int wordLen = strlen(word);
char* temp= (char *) malloc(sizeof(char) * (lineLen+1));
strcpy(temp,line);
while(true)
{
if( strstr(temp,word) == NULL)
break;
strcpy(temp, strstr(temp,word));
// printf("##%s\n",temp);
strcpy(temp,temp+wordLen+1);
// printf("##%s\n",temp);
count++;
}
//printf("%d\n",count);
free(temp);
return count;
}
//Below function fills the linked list for data structure lineCount
//with word occurance statistics
//line by line and the total
//The number of elements in the list would be number of lines in the
//file
LineCount* findCount(FILE* file, char* word,LineCount** lineCountHead)//Make it multithreaded fn()
{
LineCount* lineHead= NULL;
char* line = NULL;
int lineNumber=1;
int count=0;
if (file == NULL) {
printf("Error: file pointer is null.");
exit(1);
}
while (!feof(file)) {
LineCount* temp=NULL;
line = readLine(file, line);
//printf("%s\n", line);
count=findWord(line,word);
//Critical Section Start
temp=LineCountNode(lineNumber,count);
addToLineCountList(temp,lineCountHead);
//Criticla Section End
lineNumber++;
}
free(line);
return lineHead;
}
So basically I want my calling thread function to be LineCount* findCount(FILE* file, char* word,LineCount** lineCountHead)
My understanding is that, the file will be accessed - only for read purpose by the threads, so no need to take care of synchronization.
Currently I am opening the file as:
pFile = fopen (argv[1],"r");. My question is how do I open in read shared mode ?
I know in C++ there exists a read shared mode. How to achieve this in c?
Also how do I write my function LineCount* findCount(FILE* file, char* word,LineCount** lineCountHead) in the form required by thread call function i.e. the form void* fn(void*)
While in read-only mode there are no issues with the file itself, the IO functions in the standard C library are not designed to be usable from multiple threads in parallel. They are thread-safe (or at least, I think so) but using them correctly from multiple threads is not trivial.
At the lowest level, each FILE structure contains a file position pointer - or the IO functions maintain an OS-provided pointer. Having multiple threads mess with the file cursor position sounds like a good way to make your life more difficult than it should be.
The best approach would be to open your file multiple times - once in each thread. Each thread would then have its own FILE pointer, stream buffer etc. Note that this is not unique to C & POSIX threads - its an inherent issue with using multiple threads.
In any case, I am unsure what you are trying to achieve by using multiple threads. Generally search operations like this are I/O bound - multithreaded accesses to the same file are quite likely to make things worse.
The only case where it might make sense is if you had a huge amount of strings to search for and you had a single I/O thread feeding all other threads through a common buffer. That would distribute the CPU-intensive part, without causing undue I/O...
I need getline() to read the request header sent by my browser to the webserver I'm programming. This is the getMessage function which is supposed to do that task:
char *getMessage(int fd) {
FILE *sstream = fdopen(fd, "r");
// initialise block to 1 char and set it to null
char *block = malloc(sizeof(char));
*block = '\0';
int size = 1;
// Read from the file descriptor fd (using a FILE stream) until a blank line is
// received.
// Read 100 lines (buffersize) from sstream and put into the buffer. If lines have
// been successfully read concatenate them with block.
int buffersize = 100;
char *buffer = malloc (buffersize + 1);
while(getline(&buffer,&buffersize,sstream) != -1){
int length = strlen(buffer);
printf("Buffer length: %d\n",length);
block = realloc(block,strlen(block)+strlen(buffer)+1);
strcat(block,buffer);
if(strcmp(buffer,"\r\n") == 0) break;
}
int len = strlen(block);
printf("Block length: %d\n", len);
printf("%s \n", block);
return block;
}
Basically the input of the getMessage function (fd), is the input from my listening socket declared in my main method. I have verified that the output is correct. Now I need to convert the output from the file descriptor to a string and return that string. But every time I run my server it gets stuck in the while loop. Not executing the statements in the loop.
EDIT: Added a loop-terminating condition: Now it jumps to "Block length" immediatley.
Help is much appreciated!
If you are using the POSIX 2008 getline() function, then you're throwing away useful information (it returns the length of the line it reads, so if you capture that information, you would not need the strlen() in the loop.
If the code blocks on a getline() call, it probably means that the upstream socket is not closed, but there is no data being sent any more. Your sending code needs to close the socket so that this code can detect EOF.
Or, since you discuss 'a blank line', then maybe your code should be checking for a line containing just \r\n (or maybe just \n) and break the loop; your code is not doing that at the moment.
Your loop also exhibits quadratic behaviour because you are repeatedly using strcat(). You would do better to keep tabs on the end of the string and simply strcpy() the new data after the old, then adjust the pointer to the end of the string.
On further review, I note that you use fdopen() to open a file stream based on the file descriptor, but you neither close it nor return the file stream to the caller for closing. This leads to a leakage problem.
Rule of Thumb: if you allocate a resource, you should release it, or pass it back to be released.
I recommend changing the interface to use an already-open FILE *, and doing the fdopen() in the calling code. Alternatively, if you won't need the file descriptor again, you can keep the current interface and use fclose() before returning, but this will close the underlying file descriptor too.
This code works for me (MacOS X 10.7.2; XCode 4.2.1):
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
extern char *getMessage(FILE *);
char *getMessage(FILE *fp)
{
char *block = 0;
size_t size = 0;
size_t buffersize = 0;
char *buffer = 0;
ssize_t newlen;
while ((newlen = getline(&buffer, &buffersize, fp)) > 0)
{
printf("Buffer length: %ld\n", (long)newlen);
block = realloc(block, size + newlen + 1);
strcat(&block[size], buffer);
size += newlen;
if (strcmp(buffer, "\r\n") == 0)
break;
}
printf("Block length: %zd\n", size);
if (size > 0)
printf("<<%s>>\n", block);
return block;
}
int main(void)
{
char *msg;
while ((msg = getMessage(stdin)) != 0)
{
printf("Double check: <<%s>>\n", msg);
free(msg);
}
return 0;
}
I tested it with a file with DOS-style line endings as standard input, with both a blank line as the last line and with a non-blank line. Two blank lines in a row also seemed to be OK.
char buffer = (char *) malloc (buffersize + 1);
should be:
char *buffer = malloc (buffersize + 1);