What is the equivalent to Posix popen() in the Win32 API? - c

Is there a rough equivalent to the Linux/Unix stdio.h popen() function in the Win32 API? If so, where can I find it?
Edit: I need to know this to patch an omission in the D standard library. Any answer must use only standard Win32 API, no MSVC-specific functions. Also, I'd prefer something that's not horribly low-level, if it exists.

MSDN explains how you do what popen does using the Windows API in Pipe Handle Inheritance . Here it provides a well-documented example. It's way more low-level than the _popen function found in the runtime library linked by Jason, but uses the Win32 API exclusively.

You can call _popen if you're writing a console application. For more information, see the documentation on the MSDN site: http://msdn.microsoft.com/en-us/library/96ayss4b(VS.80).aspx

Sadly it's not particularly easy.
You need to create a pipe using the win32 function (CreatePipe), then typically you need to duplicate the end of the pipe (DuplicateHandle) that you're giving to the subprocess to allow it to be inherited, otherwise it won't be and hence can't be used.
Then you need to create a process using CreateProcess (which takes lots of structure pointers, including a STARTUPINFO), and pass in the STARTUPINFO the handle that you've duplicated as its stdout.
Then you can read from the reading end of the pipe (ReadFile etc) until you reach eof, then you need to clean up by closing all the various win32 handles.

Digging up an old thread ...
Reacting to Jason Coco's reply above, and contrary to what the linked MSDN page claims, it is apparently possible nowadays to call _popen() from non-console apps. I call it from a QuickTime importer component, basically a DLL in disguise. It opens a console window but otherwise shows the expected behaviour. Compiling a standard console tool with mingw32's -mwindows option to make it a GUI app, _popen continues to work correctly (but a console window opens even when running the tool from another console.

This is an old post, but I also had a need several years ago to use a popen() like call in a Windows environment. As several of the comments in the answer here have noted, using the Windows API to implement anything close to the classic POSIX popen() is interesting.
I created an implementation which I submitted to Code Review . This implementation uses pipes to stdin and from stdout, as well as Windows methods for CreateProcess(...).
The linked code is designed to be built into a dll with a very small API.
int __declspec(dllexport) cmd_rsp(const char *command, char **chunk, unsigned int size);
Simple example of usage:
#include "cmd_rsp.h"
int main(void)
{
char *buf = {0};
buf = calloc(100, 1);
if(!buf)return 0;
cmd_rsp("dir /s", &buf, 100);
printf("%s", buf);
free(buf);
//or a custom exe
char *buf2 = {0};
buf2 = calloc(100, 1);
cmd_rsp("some_custom_program.exe arg_1 arg_2 arg_n", &buf2, 100);
printf("%s", buf2);
free(buf2);
return 0;
}
It takes any command that can be for example issued from stdin, creates a separate process to execute the command, then returns all response content (If there is any) to a buffer, and does this without displaying the CMD window popup. The buffer grows as needed to accommodate size of response.

Related

How puts standard library function works in C? [duplicate]

This question already has answers here:
Where can I find the source code for all the C standard libraries?
(4 answers)
Closed 11 months ago.
While going through the standard library functions of C (glibc), I found that printf() actually calls puts() functions (_IO_puts). But I am unable to find out how the puts function actually writes to the stdout ?
Does it uses write() system call defined in unistd.h or something else ? One thing I find out that puts() actually calling _IO_xputn through _IO_putn.
Please help. Thank you.
For Unix based systems for which Linux is part, most functions in stdio library are wrappers that are one layer above the standard I/O system calls. You see, the operating system provides a set of APIs called system calls. Applications cannot directly access hardware resources and hence they usually call these "system calls" whenever they need to do any sort of privileged thing like writing to the screen or reading from the keyboard.
In Unix, everything is abstracted as a file so whenever you need to write characters to a screen, all you need to do is open some file that represents the "screen" and write those characters there. The kernel will take care of the rest. Quite simply, this is how you'd do this in C:
#include <unistd.h>
#include <fcntl.h>
#include <stdlib.h>
#define BUFF_SIZE 2
int main()
{
int terminal;
char buffer[BUFF_SIZE] = "C\n"; // This will store the character to print + new line
terminal = open("/dev/tty", O_WRONLY); // systemcall to open terminal
if (terminal < 0)
exit(1); // some error happened
dup2(terminal, STDOUT_FILENO); // use our newly opened terminal as default Standard output
if (write(terminal, buffer, BUFF_SIZE) != BUFF_SIZE) // systemcall to write to terminal
exit(1); // We couldn't write anything
}
This just goes to show you that everything in stdio is layered on top of the basic I/O system calls. These system calls are read, write, open, etc. If you want to learn more about system calls and some OS internals, read the book "Three Easy Pieces" by Andrea Arpaci-Dusseau

How do C output functions actually work under the hood? [duplicate]

This question already has answers here:
Where can I find the source code for all the C standard libraries?
(4 answers)
Closed 11 months ago.
I am trying to learn some C, but I am finding some of the standard functions a bit opaque.
Take putc or putchar as an example. I am trying to work out what drives this at the most basic level. I have tried to follow their definitions back through the GNU compiler source but it just ends up in this enormous tree of source files.
Is there a primitive "print this character" function that all the others are built from? I had assumed that it was just a write() system call, but an answer to a related question said that this is completely implementation specific. So how else can it actually produce the output if not a system call?
So how else can it actually produce the output if not a system call?
It does use a system call, but the specific system call is implementation-dependent.
Unix implementations use the write() system call. Implementations for other operating systems will use whatever is analogous to this.
There could also be standalone implementations that run directly on hardware without an operating system. These "unhosted" implementations might omit the stdio library, or they could implement its features by accessing the hardware directly. In this case there's no system call, the I/O is done by the stdio library itself.
For Unix based systems for which Linux is part, most functions in stdio library are wrappers that are one layer above the standard I/O system calls. You see, the operating system provides a set of APIs called system calls. Applications cannot directly access hardware resources and hence they usually call these "system calls" whenever they need to do any sort of privileged thing like writing to the screen or reading from the keyboard.
In Unix, everything is abstracted as a file so whenever you need to write characters to a screen, all you need to do is open some file that represents the "screen" and write those characters there. The kernel will take care of the rest. Quite simply, this is how you'd do this in C:
#include <unistd.h>
#include <fcntl.h>
#include <stdlib.h>
#define BUFF_SIZE 2
int main()
{
int terminal;
char buffer[BUFF_SIZE] = "C\n"; // This will store the character to print + new line
terminal = open("/dev/tty", O_WRONLY); // systemcall to open terminal
if (terminal < 0)
exit(1); // some error happened
dup2(terminal, STDOUT_FILENO); // use our newly opened terminal as default Standard output
if (write(terminal, buffer, BUFF_SIZE) != BUFF_SIZE) // systemcall to write to terminal
exit(1); // We couldn't write anything
}
This just goes to show you that everything in stdio is layered on top of the basic I/O system calls. These system calls are read, write, open, etc. If you want to learn more about system calls and some OS internals, read the book "Three Easy Pieces" by Andrea Arpaci-Dusseau

How exactly _fsopen() works?

How exactly _fsopen() works? Does Linux also has similar way of opening files which prepares the file for subsequent shared reading or writing based on shflag?
Referred article here.
How exactly _fsopen() works?
You've linked to the docs. It does what they say it does. If you're asking how it is implemented then we cannot answer because that information is proprietary.
and Does linux also has similar way of opening files which prepares the file for subsequent shared reading or writing based on shflg?
Linux does not have share modes. That's a Windows quirk. Under Linux or other Unix-like operating systems such as macOS, you don't need special flags or modes to share files between processes.
Overall, _fsopen() is an MS-specific variant of the C standard library's fopen() function. In addition to the share-mode flag, which is not relevant to other operating systems, it performs parameter validation in the manner of various other MS extension functions. On Linux, one takes responsibility for validating one's own arguments and simply uses fopen().
On Windows files are opened using the CreateFileW function which uses the NtCreateFile system call.
Argument dwShareMode is used to specify file sharing policy and contains combination of flags FILE_SHARE_DELETE, FILE_SHARE_READ and FILE_SHARE_WRITE which are mapped to shflag argument of _fsopen.
If you want to know how possible implementation of the function can look like, then first you should keep in mind that MSVCRT tries to support to some equivalent of POSIX file descriptor API. Then check the following functions:
_open_osfhandle allows you to convert NT HANDLE to POSIX-like file descriptor
_fdopen allows you to get a FILE * from a file descriptor (equivalent of POSIX fdopen function).
So the possible implementation can look like this (in pseudo code):
FILE *_fsopen(...)
{
HANDLE hFile = CreateFileW(...);
int fd = _open_osfhandle(hFile, ...);
return _fdopen(fd, ...);
}
Linux doesn't provide an equivalent of file sharing policy, so there is no equivalent.
PS: Another related function is _wsopen - combines CreateFileW and _open_osfhandle.

What is the call for the "lp filename" command in linux in a c program?

I want to use the above command in a c program in linux.
I have searched so far that there are system calls and exec calls that one may make in a code. Is there any other way using exec or system commands?
Using the system command isn't an ideal command for a multi-threaded server ,what do you suggest?
First make sure you have lp installed in this path. (Using which lp in the terminal).
You may want to understand the lp command. It's a classic unix command to send data to the "line printer", but it works with e.g. .pdf files too nowadays, depending on your printer system. However, it isn't necessarily installed. Sometimes, lpr may work better, too.
See also: http://en.wikipedia.org/wiki/Lp_%28Unix%29
The second part is about executing unix commands. system is the easiest (also the easiest to introduce a security issue into your program!), using fork and execve is one of a number of alternatives (have a look at man execve).
Yes, this code is ok. It will print the file named filename provided that the lp is found at /usr/bin and the filename file exists. You can add checks for that if you want your program to report if something went wrong, other than that it will do exactly what you expect.
Doing system("lp filename"); would work if you don't mind your program blocking after that system() call and until lp finishes.
You could also use popen(3) (instead of system(3)). But you always need to fork a process (both system and popen are calling fork(2)). BTW, if you have a CUPS server you might use some HTTP client protocol library like libcurl but that is probably inconvenient. Better popen or system an lp (or lpr) command.
BTW, printing is a relatively slow and complex operation, so the overhead of forking a process is negligible (I believe you could do that in a server; after all people usually don't print millions of pages). Some libraries might give you some API (e.g. QPrinter in Qt).
Notice that the lp (or lpr) command is not actually doing the printing, it is simply interacting with some print daemon (cupsd, lpd ...) and its spooling system. See e.g. CUPS. So running the lp or lpr command is reasonably fast (much faster than the printing itself), generally a few milliseconds (certainly compatible with a multi-threaded or server application).
Quite often, the command passed to popen or system is constructed (e.g. with snprintf(3) etc...), e.g.
char cmdbuf[128];
snprintf (cmdbuf, sizeof(cmdbuf), "lp %s", filename);
but beware of code injection (think about filename containing foo; rm -rf $HOME) and of buffer overflow
Of course, notice that library functions like system, popen, fopen are generally built above existing syscalls(2). Read Advanced Linux Programming

how could I intercept linux sys calls?

Besides the LD_PRELOAD trick , and Linux Kernel Modules that replace a certain syscall with one provided by you , is there any possibility to intercept a syscall ( open for example ) , so that it first goes through your function , before it reaches the actual open ?
Why can't you / don't want to use the LD_PRELOAD trick?
Example code here:
/*
* File: soft_atimes.c
* Author: D.J. Capelis
*
* Compile:
* gcc -fPIC -c -o soft_atimes.o soft_atimes.c
* gcc -shared -o soft_atimes.so soft_atimes.o -ldl
*
* Use:
* LD_PRELOAD="./soft_atimes.so" command
*
* Copyright 2007 Regents of the University of California
*/
#define _GNU_SOURCE
#include <dlfcn.h>
#define _FCNTL_H
#include <sys/types.h>
#include <bits/fcntl.h>
#include <stddef.h>
extern int errorno;
int __thread (*_open)(const char * pathname, int flags, ...) = NULL;
int __thread (*_open64)(const char * pathname, int flags, ...) = NULL;
int open(const char * pathname, int flags, mode_t mode)
{
if (NULL == _open) {
_open = (int (*)(const char * pathname, int flags, ...)) dlsym(RTLD_NEXT, "open");
}
if(flags & O_CREAT)
return _open(pathname, flags | O_NOATIME, mode);
else
return _open(pathname, flags | O_NOATIME, 0);
}
int open64(const char * pathname, int flags, mode_t mode)
{
if (NULL == _open64) {
_open64 = (int (*)(const char * pathname, int flags, ...)) dlsym(RTLD_NEXT, "open64");
}
if(flags & O_CREAT)
return _open64(pathname, flags | O_NOATIME, mode);
else
return _open64(pathname, flags | O_NOATIME, 0);
}
From what I understand... it is pretty much the LD_PRELOAD trick or a kernel module. There's not a whole lot of middle ground unless you want to run it under an emulator which can trap out to your function or do code re-writing on the actual binary to trap out to your function.
Assuming you can't modify the program and can't (or don't want to) modify the kernel, the LD_PRELOAD approach is the best one, assuming your application is fairly standard and isn't actually one that's maliciously trying to get past your interception. (In which case you will need one of the other techniques.)
Valgrind can be used to intercept any function call. If you need to intercept a system call in your finished product then this will be no use. However, if you are try to intercept during development then it can be very useful. I have frequently used this technique to intercept hashing functions so that I can control the returned hash for testing purposes.
In case you are not aware, Valgrind is mainly used for finding memory leaks and other memory related errors. But the underlying technology is basically an x86 emulator. It emulates your program and intercepts calls to malloc/free etc. The good thing is, you do not need to recompile to use it.
Valgrind has a feature that they term Function Wrapping, which is used to control the interception of functions. See section 3.2 of the Valgrind manual for details. You can setup function wrapping for any function you like. Once the call is intercepted the alternative function that you provide is then invoked.
First lets eliminate some non-answers that other people have given:
Use LD_PRELOAD. Yeah you said "Besides LD_PRELOAD..." in the question but apparently that isn't enough for some people. This isn't a good option because it only works if the program uses libc which isn't necessarily the case.
Use Systemtap. Yeah you said "Besides ... Linux Kernel Modules" in the question but apparently that isn't enough for some people. This isn't a good option because you have to load a custom kernal module which is a major pain in the arse and also requires root.
Valgrind. This does sort of work but it works be simulating the CPU so it's really slow and really complicated. Fine if you're just doing this for one-off debugging. Not really an option if you're doing something production-worthy.
Various syscall auditing things. I don't think logging syscalls counts as "intercepting" them. We clearly want to modify the syscall parameters / return values or redirect the program through some other code.
However there are other possibilities not mentioned here yet. Note I'm new to all this stuff and haven't tried any of it yet so I may be wrong about some things.
Rewrite the code
In theory you could use some kind of custom loader that rewrites the syscall instructions to jump to a custom handler instead. But I think that would be an absolute nightmare to implement.
kprobes
kprobes are some kind of kernel instrumentation system. They only have read-only access to anything so you can't use them to intercept syscalls, only log them.
ptrace
ptrace is the API that debuggers like GDB use to do their debugging. There is a PTRACE_SYSCALL option which will pause execution just before/after syscalls. From there you can do pretty much whatever you like in the same way that GDB can. Here's an article about how to modify syscall paramters using ptrace. However it apparently has high overhead.
Seccomp
Seccomp is a system that is design to allow you to filter syscalls. You can't modify the arguments, but you can block them or return custom errors. Seccomp filters are BPF programs. If you're not familiar, they are basically arbitrary programs that users can run in a kernel-space VM. This avoids the user/kernel context switch which makes them faster than ptrace.
While you can't modify arguments directly from your BPF program you can return SECCOMP_RET_TRACE which will trigger a ptraceing parent to break. So it's basically the same as PTRACE_SYSCALL except you get to run a program in kernel space to decide whether you want to actually intercept a syscall based on its arguments. So it should be faster if you only want to intercept some syscalls (e.g. open() with specific paths).
I think this is probably the best option. Here's an article about it from the same author as the one above. Note they use classic BPF instead of eBPF but I guess you can use eBPF too.
Edit: Actually you can only use classic BPF, not eBPF. There's a LWN article about it.
Here are some related questions. The first one is definitely worth reading.
Can eBPF modify the return value or parameters of a syscall?
Intercept only syscall with PTRACE_SINGLESTEP
Is this is a good way to intercept system calls?
Minimal overhead way of intercepting system calls without modifying the kernel
There's also a good article about manipulating syscalls via ptrace here.
Some applications can trick strace/ptrace not to run, so the only real option I've had is using systemtap
Systemtap can intercept a bunch of system calls if need be due to its wild card matching. Systemtap is not C, but a separate language. In basic mode, the systemtap should prevent you from doing stupid things, but it also can run in "expert mode" that falls back to allowing a developer to use C if that is required.
It does not require you to patch your kernel (Or at least shouldn't), and once a module has been compiled, you can copy it from a test/development box and insert it (via insmod) on a production system.
I have yet to find a linux application that has found a way to work around/avoid getting caught by systemtap.
I don't have the syntax to do this gracefully with an LKM offhand, but this article provides a good overview of what you'd need to do: http://www.linuxjournal.com/article/4378
You could also just patch the sys_open function. It starts on line 1084 of file/open.c as of linux-2.6.26.
You might also see if you can't use inotify, systemtap or SELinux to do all this logging for you without you having to build a new system.
If you just want to watch what's opened, you want to look at the ptrace() function, or the source code of the commandline strace utility. If you actually want to intercept the call, to maybe make it do something else, I think the options you listed - LD_PRELOAD or a kernel module - are your only options.
If you just want to do it for debugging purposes look into strace, which is built in top of the ptrace(2) system call which allows you to hook up code when a system call is done. See the PTRACE_SYSCALL part of the man page.
if you really need a solution you might be interested in the DR rootkit that accomplishes just this, http://www.immunityinc.com/downloads/linux_rootkit_source.tbz2 the article about it is here http://www.theregister.co.uk/2008/09/04/linux_rootkit_released/
Sounds like you need auditd.
Auditd allows global tracking of all syscalls or accesses to files, with logging. You can set keys for specific events that you are interested in.
Using SystemTap may be an option.
For Ubuntu, install it as indicated in https://wiki.ubuntu.com/Kernel/Systemtap.
Then just execute the following and you will be listening on all openat syscalls:
# stap -e 'probe syscall.openat { printf("%s(%s)\n", name, argstr) }'
openat(AT_FDCWD, "/dev/fb0", O_RDWR)
openat(AT_FDCWD, "/sys/devices/virtual/tty/tty0/active", O_RDONLY)
openat(AT_FDCWD, "/sys/devices/virtual/tty/tty0/active", O_RDONLY)
openat(AT_FDCWD, "/dev/tty1", O_RDONLY)

Resources