I can't figure out how to get rid of a delay, it seems like a buffering delay but I havent had any luck with setvbuf or fflush...
I have a c/c++ program that executes a python script which immediately starts printing to stdout (quite a bit), however there seems to be a huge delay when I try to read that input in my program. I have tried to include a basic version of what I am doing below. In the output I see TEST0 immediately and then after quite some time I get a huge dump of prints.... I tried setvbuff but that didnt seem to make a difference. I think either I am doing something wrong or just not understanding what's happening.
Update: I am running in Linux.
Update2: fixed code typo with multiple forks
Update3: Adding stdout flushes in the python script fixed the problem, no more delays! Thanks #DavidGrayson
int pipeFd[2];
pid_t pid;
char buff[PATH_MAX];
std::string path = "/user/bin/python3";
std::string script = "";//use path to python script here
std::string args = ""; //use args for python script here
pid = fork();
if( pid == -1)
{
printf( "[ERROR] cant fork\n" );
}
else if( pid == 0 )
{
close( pipeFd[0] );
dup2( pipeFd[1], STDOUT_FILENO);
close( pipeFd[1] );
execl(path.c_str(), "python3", script.c_str(), args.c_str(), (char*)NULL );
printf( "[ERROR] script execl failed\n" );
exit(1);
}
else
{
//setvbuf(stdout, NULL, _IONBF, 0);
//setvbuf(stdin, NULL, _IONBF, 0);
printf( "TEST0\n" );
fflush(stdout);
//it takes a really long time to see this next print
read( pipeFd[0], buff, 1 );
printf( "TEST1:%c\n", buff[0] );
fflush(stdout);
}
Related
I know this question has been asked a billion times, but all the solutions are not working for me. I need to stream the stdout of an execvp via a socket to a client. The relevant client code is this
static void execute_cmd(int fd, char *cmd, char **params) {
pid_t cid;
size_t i = 0, c = 0;
cid = fork();
if(cid == 0) {
close(STDOUT_FILENO);
dup2(fd, STDOUT_FILENO);
if(execvp(cmd, params) < 0) {
fprintf(stderr, "Could not execute \"%s\": %d\n", cmd, errno);
exit(1);
}
} else {
close(fd);
waitpid(cid, NULL, 0);
exit(1);
}
}
I have tried to copy the answer on this question. However, I get nothing back when I try this code:
int sockfd = serverConnect();
write(sockfd, "echo math", 11);
n = read(sockfd, buffer, 1023);
printf("Got %d bytes\n", n);
printf("%s\n", buffer);
close(sockfd);
I triple checked that the connection is established correctly. When I replace execute_cmd with a simple write, the client correctly prints the answer. But nothing happens when I execute the code above, I get no bytes back. I have removed the dup2 call and got no output from the execvp call either.
Getting quite desperate here, tried pipe and whatever I could find. Where did I go wrong? The command itself is ok, too, works on the shell, and the execvp call does not throw an error.
Turns out the code above is correct. The problem was an incorrect use of an earlier strtok that resulted in a silent crash of strdup. The fork above was simply never executed, and all my tests were above the strtok line. Only after putting printfs into every line of the code I could find the problem.
Frankly, I feel stupid.
I have the following c code which I took from the first answer to this question, I am compiling it with C89 in VS2008 so I made a couple of changes for the code to work it properly, it compiles just fine, but it's not able to create process after creating the namedpipe successfully (CreateProcessA is not working) returning always error 2 and printing the message in the panic function.
The program that I am trying to run in the CreateProcessA can be downloaded here, and I normally run it and use it as follows:
> C:\qhichwa>cmd.exe /c "C:\qhichwa\flookup.exe -bx C:\qhichwa\qhichwa.fst"
wasi <user writes wasi>
wasi <program responds printing wasi>
hola <user writes hola>
+ ? <program responds printing + ?>
<pres ctrl + c to terminate program>
> C:\qhichwa>
The lines between < comments > are just comments.
what are the corrections needed in order to to create the named pipe successfully?
#include <stdio.h>
#include <stdlib.h>
#include <windows.h>
// name of our glorious pipe
#define PIPE_NAME L"\\\\.\\pipe\\whatever" // bloody unicode string
// exit on fatal error
void panic(const char * msg)
{
int err = GetLastError();
fprintf(stderr, "***PANIC*** %s\n", msg);
printf("In the child thread: Last Error is %lu\n", err);
exit(-1);
}
// father process
void father(const char * own_name) // name of our own executable to launch a copy of ourselve
{
printf("Father process starting\n");
// create a monodirectional father->child named pipe
HANDLE pipe = CreateNamedPipe(
PIPE_NAME, // name of the pipe
PIPE_ACCESS_OUTBOUND, // send only
PIPE_TYPE_BYTE, // send data as a byte stream
1, // only one instance
0, 0, 0, NULL); // default junk
if (pipe == INVALID_HANDLE_VALUE) panic("could not create pipe 1");
// spawn child process
{
STARTUPINFOA si;
PROCESS_INFORMATION pi;
ZeroMemory(&si, sizeof(si));
si.cb = sizeof(si);
ZeroMemory(&pi, sizeof(pi));
if (!CreateProcessA( // using ASCII variant to be compatible with argv
"cmd.exe", // executable name (ourself)
"/c \"C:\\qhichwa\\flookup.exe -bx C:\\qhichwa\\qhichwa.fst\"", // command line. This will be seen as argv[0]
NULL, NULL, FALSE, // default junk
CREATE_NEW_CONSOLE, // launch in another console window
NULL, NULL, // more junk
&si, &pi)) // final useless junk
panic("could not create child process 2");
}
// connect to child process
BOOL result = ConnectNamedPipe(pipe, NULL);
if (!result) panic("could not connect to child process");
// talk to child
for (;;)
{
// read an input line
char line[100];
printf("Say something >");
if (fgets(line, sizeof(line), stdin) == NULL)
panic("could not read from standard input");
// exit on an empty line
if (!strcmp(line, "\n")) break;
// send the line to the child
DWORD written = 0;
if (!WriteFile(
pipe,
line, // sent data
strlen(line), // data length
&written, // bytes actually written
NULL))
panic("could not write to pipe");
}
// close the pipe
CloseHandle(pipe);
}
void child(void)
{
printf("Child process starting\n");
// retrieve communication pipe
HANDLE pipe = CreateFile(
PIPE_NAME, // name of the pipe
GENERIC_READ, // read ONLY access (or else the call will fail)
0, NULL, // junk
OPEN_EXISTING, // opens existing pipe
0, NULL); // more junk
if (pipe == INVALID_HANDLE_VALUE) panic("could not connect to the pipe");
// read father's input
for (;;)
{
char buffer[80];
DWORD read = 0;
if (!ReadFile(
pipe,
buffer, // read data
sizeof(buffer)-1, // max length (leave room for terminator)
&read, // bytes actually read
NULL))
break; // exit if the pipe has closed
// display what our father said
buffer[read] = '\0'; // make sure what we just read will be displayable as a string
printf("Father said: %s", buffer);
}
// close pipe
CloseHandle(pipe);
}
int main(int argc, char *argv[])
{
// wait for a <return> keypress on exit
atexit(getchar);
father(argv[0]);
// decide whether we are the father or the child
//if (!strcmp(argv[0], "child")) child();
//else father(argv[0]);
printf("Done\n");
return 0;
}
The problem is located here:
fprintf(stderr, "***PANIC*** %s\n", msg);
printf("In the child thread: Last Error is %lu\n", GetLastError());
This is a standard Windows programming bug, every programmer makes this mistake once. Just once, this is so hard to diagnose that you'll never forget losing the week of your life trying the discover it.
The underlying issue is the way GetLastError() works. It returns the value of an internal global variable, it is stored in the TEB (Thread Environment Block). It has the so common problem with global variables, every call you make to a winapi function is liable to change it, including ones that don't actually fail. Windows itself uses the variable as well. And ERROR_INVALID_NAME is a very popular internal error code.
This is compounded by you not being able to see these winapi calls being made. What is ruining the error code is fprintf(). It is implemented in the CRT with winapi calls. Necessarily so, I/O is an operating system duty.
So what is absolutely essential is that you immediately obtain the error code, before you do anything else. While it is preferable that you pass the value to your panic() function, since it cannot predict what other code runs before it, the quick fix is to rewrite it like this:
int err = GetLastError();
fprintf(stderr, "***PANIC*** %s\n", msg);
printf("In the child thread: Last Error is %lu\n", err);
You'll now get the true error code, the one produced by CreateNamedPipe(). Ought to give you a much better shot at diagnosing the problem. Update your question if you still have a problem interpreting it.
I want to run node.js as a subprocess and feed it input. Using C, here is some sample code of mine that does that.
The issue I have is that although the subprocess's stdout is still directed to the terminal, I see nothing after having fed the subprocess stdin a print 'Hello World' line. Even if I fflush() the pipe, I see nothing on output. However, if I close the pipe's input, then the 'Hello World' appears on the terminal.
The subprocess seems to simply buffer - why is that?
I would like to eventually redirect the subprocess stdout to another pipe and read it
in from main().
int main(int argc, char* argv[]) {
int toNode[2];
pipe(toNode);
pid_t child_pid = fork();
if (child_pid == 0) { // child
// close write end
close(toNode[1]);
// connect read end to stdin
dup2(toNode[0], STDIN_FILENO);
// run node executable
char* arg_list[] = { "/usr/bin/node", NULL};
execvp(arg_list[0], arg_list);
fprintf(stderr, "process failed to start: %s\n", strerror(errno));
abort();
}
else { // parent
FILE* stream;
// close read end
close(toNode[0]);
// convert write fd to FILE object
stream = fdopen(toNode[1], "w");
fprintf(stream, "console.log('Hello World');\n");
fflush(stream);
//close(toNode[1]);
waitpid(child_pid, NULL, 0);
}
return 0; }
There's no problem with the pipe being read. The problem is that /usr/bin/node only invokes the REPL (read-eval-print loop), by default, if it detects that stdin is interactive. If you have a sufficiently recent version of nodejs, then you can provide the -i or --interactive command line flag, but that will do more than just execute each line as it is read; it also really will act as a console, including inserting ANSI colour sequences into the output and printing the value of each expression.
See this forum thread for more information.
I'm writing a short program that's polling the buffer of a named pipe. To test it out, I'll log into 'nobody' and echo into the pipe. while it's hanging, I log in from a different user and run the program that reads the buffer. When it runs, the program returns nothing and the other user is logged out of the system. Here's the read function:
void ReadOut( char * buf )
{
ZERO_MEM( buffer, BUF_SIZE );
int pipe = open( buf, O_RDONLY | O_NONBLOCK );
if( pipe < 0 )
{
printf( "Error %d has occured.\n" , pipe );
return;
}
while( read( pipe, buffer, 2 ) > 0 ) printf( "%s \n" , buffer );
close( pipe );
return;
}
This function also works when I take out O_NONBLOCK
When you mark a file descriptor as non blocking, all the operations that normally can block (for example read(2), and write(2)) return -1 instead and set errno = EAGAIN.
So in your case read immediately returns -1 signaling "I'm not ready right now, try again later".
I need to implement a basic shell in C.
One of things I need is to implement a function that has a command and to execute it.
my code:
pID=fork();
if (pID == 0)
execvp(tmp[0], tmp);
else if (pID > 0)
{
printf("%d", pID);
wait(NULL);
}
else
printf("Failed to create proccess \n");
The problem is that no matter what is the command I put in tmp, the program shows me the prompt again, and do nothing except that.
For example if I write gedit (in order to open the gedit — a ntpad of Ubuntu), it doesn't open it, or if write ls -a it doesn't show me any output as the terminal of Ubuntu does.
execvp should work. As the others mentioned, you really need to show how you populate tmp. That said, I would guess that that's where the error is. tmp needs to be a null terminated array.
#include <stdio.h>
main( int argc, char * argv[] )
{
int pid = fork;
char * tmp[2];
memset( tmp, 0, sizeof(tmp) );
tmp[0] = argv[0];
if( 0 == pid )
{
if( -1 == execvp( tmp[0], tmp ) )
{
char errmsg[64];
snprintf( errmsg, sizeof(errmsg), "exec '%s' failed", tmp[0] );
perror( errmsg );
}
else if( 0 < pid )
{
printf("[%d] %s\n", pid, tmp[0]);
wait(NULL);
}
else
{
perror("fork failed");
}
}
Although you've failed to tell us what you're passing through the tmp variable to execvp, my psychic sense tells me that you forgot to null-terminate your argument list. A NULL argument tells execvp where the last argument is, and if you fail to put in a NULL, it will start reading random garbage off the stack.
If that random garbage points to large strings of non-zero data, it will run out of space to store the supposed arguments to the new process, which is typically a few hundred KB (see this page for some system-specific numbers, as well as various ways of getting your system's maximum arguments size).
When there's too much argument data, the system call execve(2) (called internally by execvp) fails with the error E2BIG.
So to see if this is what's happening to you, check the return value from execvp. If it even returns at all, it failed (if it succeeded, it wouldn't have returned since a new process would be executing!), so check the global value of errno to see why it failed:
if (pID == 0)
{
execvp(tmp[0], tmp);
printf("exec failed: %s\n", strerror(errno));
exit(1);
}
execvp() requires full path . If in tmp[0] isnt the full path of your executable file use execv()
execv(tmp[0], tmp);