Problem
I would like to program an attachable command line interface for my daemon.
I developped a daemon running 24/7 on Linux Openwrt:
#!/bin/sh /etc/rc.common
START=98
USE_PROCD=1
PROCD_DEBUG=1
start_service() {
procd_open_instance
procd_set_param command "/myProgram"
procd_set_param respawn
procd_close_instance
}
I would like to add a debug user interfaces for test. So we could live tune some parameters/actions and print log. Something like the screen package.
Hence i want to create a command line interface for this daemon.
Research
Stdin/Stdout
Ideally i would like to write directly to the stdin of the daemon and read the stdout.
Daemon
Duplicate stdin to a file.
Duplicate stoud to a file.
Client
A launched C program by the tester.
It would relay stdin to stdinfile of daemon and stdoutfile of daemon to stdout.
Critic
That would be maybe the simplest way and I could read stdout.
I couldn't find any exemples, it makes me think i'm overlooking something.
Theres a risk I fill the flash by writing endlessly to the stdoutfile.
Pipes
The creation of 2 named pipe can be possible.
Daemon
The daemon would create a named input pipe and poll the pipe by making non blocking read.
A second output pipe is necessary to write the return of the command received.
Client
A launched C program by the tester.
It would relay stdin to input pipe and output pipe to stdout.
Critic
I don't know if I can properly redirect the stdout of the daemon to output pipe. Which means I wont be able to print the stdout logs but only specific cli coded response.
MessageQ
Same issues as pipe.
Sockets
Seems rather complex for a simple application.
Shared Memory
The paradigm does not seems appropriate.
Pty
Maybe something can be done with pseudo terminals but I don't understand them even after reading explanations: attach a terminal to a process running as a daemon (to run an ncurses UI)
Screen/Tmux
I don't have screen or tmux in my repository.
Question
What is the proper way to create a CLI for a daemon ? Where could I find an exemple ?
I would use a Unix domain stream socket, with the CLI thread in a blocking accept() until a connection is obtained.
These sockets are bidirectional, and you can write a trivial CLI application to read from standard input to the connected socket, and from the connected socket to standard output. (That same trivial CLI program could be used to redirect the output over e.g. SSH to ones local computer with much more storage, running the CLI program remotely using something like ssh -l username openwrt-device.name-or-address cli-program | tee local-logfile. OpenWrt devices often don't have suitable storage for log files, so this can be very useful.)
Use vdprintf() to implement your own printf() that writes to the connected CLI.
Because sockets are bidirectional, if you want to use locking –– for example, to avoid mixing logging output and CLI responses ––, use a mutex for writing; the read side does not need to take the mutex at all.
You cannot really use <stdio.h> FILE * stream handles for this, because its internal buffering can yield unexpected results.
Assuming your service daemon uses sockets or files, it can be very useful to reserve the file descriptor used for the bidirectional CLI connection, by initially opening /dev/null read-write (O_RDWR). Then, when the connection is accept()ed, use dup2() to move the accepted connection descriptor to the reserved one. When the connection is to be closed, use shutdown(fd, SHUT_RDWR) first, then open /dev/null, and dup that descriptor over the connection to be closed. This causes the connection to be closed and the descriptor to be reopened to /dev/null, in an atomic manner: the descriptor is never "unused" in between. (If it is ever close()d in a normal manner, another thread opening a file or socket or accepting a new connection may reuse that descriptor, causing all sorts of odd effects.)
Finally, consider using an internal (cyclic) buffer to store the most recent output messages. You do not need to use a human-readable format, you can use e.g. the first character (codes 1 to 254) to encode the severity or log level, keeping NUL (0) as the end-of-string mark, and 255 as the "CLI response" mark, so that your CLI program can use e.g. ANSI colors to color the output if output is a terminal. (For example, "\033[1;31m" changes output to bright red, and "\033[0m" returns the output back to normal/default. The \033 refers to a single character, code 27, ASCII ESC.) This can be very useful to efficiently indicate the priority/severity of each separate output chunk to the human user. The Linux kernel uses a very similar method in its kernel logging facility.
Related
I've currently written a udp client, which simply listens on specified port, for packets destined to a specific multicast group.
at the end, i'm printing the input to stdout in the following manner:
write(STDOUT_FILENO, buffer, num_of_bytes);
And executing the program in the following manner:
./udp_listen 224.10.10.10 4567 | mpg321 -
That way all output is piped to mpg321 app, which plays the stream of bytes as music.
The above implementation, is of course making me kind of "lose control" over my program, as it opens mpg321 app and plays.
I want to avoid this, and in someway, pipe the write(....) into mpg321, directly within my program.
How can one achieve this? I'm coding in C.
Thank you.
Thanks #kaylum for providing the answer i was looking for.
i've used popen with the command to initiate mpg321 , with redirecting its stderr output to a file.
After that i've used fileno() method to get the file descriptor number,
to be used with write() calls.
Worked great.
Thanks
I'm trying to recode the UNIX command script (as it is on OSX). This is part of an exercise for school to help students learn UNIX APIs. We are only allowed to use system calls, more specifically, only those available on MAN(2) pages on Mac OSX (since that's our OS at school).
I have a 'first version' that kind of works. Running a program such as ls prints the right output to the screen and in an output file.
The problem scenario
I run bash from within the script-clone. First issue is I get the following error:
bash: no job control in this shell
I have tried forcing the bash process into foreground with setpgrp and setpgid but that din't change anything so I concluded that was not the problem.
I also tried to understand why the real script command uses cfmakeraw (at least on Linux), as seen here, but I don't get it. The MAN page is not very helpful.
The real script also dup2s STDIN on the slave, as seen here, but when I do that, it seems like input isn't read anymore.
However, the bash still runs, and I can execute commands inside of it.
But if I run vim inside it, and then hit Ctrl-Z to put vim to the background, the terminal is messed up (which does not happen when I'm in my regular terminal).
So I guess I must have done something wrong. I'd appreciate any advice/help.
Here's the source code:
https://github.com/conradkleinespel/unix-command-script/tree/2587b07e7a36dc74bf6dff0e82c9fdd33cb40411
You can compile by doing: make (it builds on OSX 10.9, hopefully on Linux as well)
And run by doing: ./ft_script
Don't know it it makes more sense to have all the source code in StackOverflow as it would crowd the page with it. If needed, I can replace the Git link with the source.
I don't use OS X, so I can't directly test your code, but I'm currently writing a toy terminal emulator and had similar troubles.
about "bash: no job control in this shell"
In order to perform job control, a shell needs to be a session leader and the controlling process of its terminal. By default, your program inherits the controlling terminal of your own shell which runs your script program and which is also a session leader. Here is how to make your new slave process a session leader after fork:
/* we don't need the inherited master fd */
close(master);
/* discard the previous controlling tty */
ioctl(0, TIOCNOTTY, 0);
/* replace existing stdin/out/err with the slave pts */
dup2(slave, 0);
dup2(slave, 1);
dup2(slave, 2);
/* discard the extra file descriptor for the slave pts */
close(slave);
/* make the pts our controlling terminal */
ioctl(0, TIOCSCTTY, 0);
/* make a new session */
setsid()
At this point, the forked process has stdin/out/err bound to the new pts, the pts became its controlling terminal, and the process is a session leader. The job control should now work.
about raw tty
When you run a program inside a normal terminal, it looks like this:
(term emulator, master side) <=> /dev/pts/42 <=> (program, slave side)
If you press ^Z, the terminal emulator will write the ascii character 0x1A to the pts. It is a control character, so it won't be sent to the program, but instead the kernel will issue SIGSTP to the program and suspend it. The process of transforming characters into something else is called "line cooking" and has various settings that can be adjusted for each tty.
Now let's look at the situation with script:
term emulator <=> /dev/pts/42 <=> script <=> /dev/pts/43 <=> program
With normal line settings, what happens when you press ^Z? It will be transformed into SIGSTP by /dev/pts/42 and script will be suspended. But that's not what we want, instead we'd like the 0x1A character produced by our ^Z to go as-is through /dev/pts/42, then be passed by script to /dev/pts/43 and only then be transformed into SIGSTP to suspend the program.
This is the reason why the pts between your terminal and script must be configured as "raw", so that all control characters reach the pts between script and the program, as if you were directly working with it.
Take the following command:
mysql -u root -p < load_data.sql > output.tab
The -p flag tells the mysql client - a C program - to provide the user with an interactive prompt to enter the password.
AFAIK, input like this is typically handled by writing a prompt to stderr and then blocking on a call like gets, which reads a line from stdin.
But the shell has already opened the load_data.sql file and set the stdin of the mysql client to its file descriptor - so shouldn't calling gets just get the first line from the file?
My initial thought was that the program seeks to the end before reading a line - but you can't seek like that on pipes!
So how does this work? Is there some magic?
Applications that prompt for passwords generally don't actually read them from stdin, on the grounds that this would (a) cause the password to appear on the screen if it was being typed in interactively and (b) encourage plain-text passwords to be bandied around in publicly-visible places when things need to be automated (e.g. in command lines visible to others via ps). PostgreSQL's psql SQL shell opens the terminal device directly, and I suspect mysql will do the same.
Some quick searching found this related question. The top-rated answer mentions the GNU function getpass(), which does indeed open a direct connection to the terminal, bypassing stdin. I suspect that function is what most password-prompting programs use in *nix.
This isn't a pipe that's being opened up, but rather is a redirection of stdin to point to a file. Thus you have both a FILE* (i.e. a stream), as well as a normal file-descriptor you can work with. In the case of the lower-level file-descriptor, there are seeking operations you can do, like lseek(), etc. that can be used along with read() in order to move around the file.
If you are wanting to still read data from the controlling terminal while stdin has been re-directed to a file, you simply need to open the controlling terminal for reading on another file-descriptor. You can use ctermid() in order to determine what the controlling terminal for your process is, and reopen it on another file-descriptor.
in C or bash,
I was wondering how, if possible, do you obtain from inside an ssh session, the file descriptor to the pseudo terminal master responsible for getting input to that's session's slave(pts).
The shell process has no master file descriptor, only slave.
The shell's parent process (be it sshd or xterm or screen or whatever) creates a new master by calling getpt(3) or posix_openpt(3). The function returns the master file descriptor. The parent process then obtains the slave file descriptor by calling a combination of grantpt(3), unlockpt(3), ptsname(3) and open(2). This is for Linux and other POSIXized systems, other *nixes may use other functions, but the net result is the same. The parent process has the master/slave pair of file descriptors.
The slave descriptor, and the slave descriptor only, is then passed to the shell as its standard input, output and error.
From Solaris 5.8 PTS(7D) Man-page - STREAMS pseudo-tty slave driver
Only one open is allowed on a master device.
I guess that answers my question :)
EDIT: actually it does not, because if there is a way to obtain the file descriptor, i won't need to open again, it is a file descriptor lol , no need to open
On unix-based systems, you can open the controlling terminal of the current process by opening /dev/tty. In many cases your program will already have this open as stdin, stdout and stderr, but even if your program is being invoked with stdin, stdout or stderr redirected, /dev/tty will give you the controlling terminal of the process.
I am working on a linux daemon and having some issues with the stdin/stdout. Normally because of the nature of a daemon you do not have any stdin or stdout. However, I do have a function in my daemon that is called when the daemon runs for the first time to specify different parameters that are required for the daemon to run successfully. When this function is called the terminal becomes so sluggish that I have to launch a seperate shell and kill the daemon with top to get a responsive prompt back. Now I suspect that this has something to do with the forking process closing the stdin/stdout but I am not quite sure how I could work around this. If you guys could shed some light on the situation that would be most appreciated. Thanks.
Edit:
int main(argc, char *argv[]) {
/* setup signal handling */
/* check command line arguments */
pid_t pid, sid;
pid = fork();
if (pid < 0) {
exit(EXIT_FAILURE);
}
if(pid > 0){
exit(EXIT_SUCCESS);
}
sid = setsid();
if(sid < 0) {
exit(EXIT_FAILURE);
}
umask(027);
/* set syslogging */
/* do some logic to determine wether we are running the daemon for the first time and if we are call the one time function which uses fgets() to recieve some input */
while(1) {
/* do required work */
}
/* do some clean up procedures and exit */
return 0;
}
You guys mention using a config file. This is is exactly what I do to store the parameters recieved via input. However I still initially need to get these from the user via the stdin. The logic for determining whether we are running for the first time is based off of the existence of the config file.
Normally, the standard input of a daemon should be connected to /dev/null, so that if anything is read from standard input, you get an EOF immediately. Normally, standard output should be connected to a file - either a log file or /dev/null. The latter means all writes will succeed, but no information will be stored. Similarly, standard error should be connected to /dev/null or to a log file.
All programs, including daemons, are entitled to assume that stdin, stdout and stderr are appropriately opened file streams.
It is usually appropriate for a daemon to control where its input comes from and outputs go to. There is seldom occasion for input to come from other than /dev/null. If the code was written to survive without standard output or standard error (for example, it opens a standard log channel, or perhaps uses syslog(3)) then it may be appropriate to close stdout and stderr. Otherwise, it is probably appropriate to redirect them to /dev/null, while still logging messages to a log file. Alternatively, you can redirect both stdout and stderr to a log file - beware continuously growing log files.
Your sluggish-to-impossible response time might be because your program is not paying attention to EOF in a read loop somewhere. It might be prompting for user input on /dev/null, and reading a response from /dev/null, and not getting a 'y' or 'n' back, it tries again, which chews up your system horribly. Of course, the code is flawed in not handling EOF, and counting the number of times it gets an invalid response and stopping being silly after a reasonable number of attempts (16, 32, 64). The program should shut up shop sanely and safely if it expects a meaningful input and continues not to get it.
You guys mention using a config file. This is is exactly what I do to store the parameters recieved via input. However I still initially need to get these from the user via the stdin. The logic for determining whether we are running for the first time is based off of the existence of the config file.
Instead of reading stdin, have the user write the config file themselves; check for its existence before forking, and exit with an error if it doesn't. Include a sample config file with the daemon, and document its format in your daemon's manpage. You do have a manpage, yes? Your config file is textual, yes?
Also, your daemonization logic is missing a key step. After forking, but before calling setsid, you need to close fds 0, 1, and 2 and reopen them to /dev/null (do not attempt to do this with fclose and fopen). That should fix your sluggish terminal problem.
Your design is wrong. Daemon processes should not take input via stdin or deliver output to stdout/stderr. You'll close those descriptors as part of the daemonizing phase. Daemons should take configuration parameters from the command line, a config file, or both. If runtime-input is required you'll have to read a file, open a socket, etc., but the point of a daemon is that it should be able to run and do its thing without a user being present at the console.
If you want to run your program detached, use the shell: (setsid <command> &). Do not fork() inside your program, which will cause sysadmin nightmare.
Don't use syslog() nor redirect stdout or stderr.
Better yet, use a daemon manager such as daemon tools, runit, OpenRC and systemd, to daemonize your program for you.
Use a config file. Do not use STDIN or STDOUT with a daemon. Daemons are meant to run in the background with no user interaction.
If you insist on using stdin/keyboard input to fire up the daemon (e.g. to get some magic passphrase you wouldn't want to store in a file) then handle all I/O before the fork().