Goal
I'd like to write C/C++ applications and provide an API for them to communicate directly with systemd-managed services. Effectively, I'd like my applications to be able to do the equivalent of systemctl start service_name#unit_number.service without using system(), popen(), or exec() calls (hard requirement).
Question
Is there a simple C/C++ API for communicating with systemd, assuming systemd version 219 (i.e. CentOS v7.4, also a hard requirement)?
Work So Far
I've scoured online for examples of controlling systemd-managed services in C/C++ and found the odd discussion on the topic, but haven't found any good examples.
I could always reverse-engineer systemctl version 219 from source, but then I potentially run afoul of GPL licensing.
Since I'm stuck with CentOS v7.4, which means I can't get at version 221 or later of systemd, I can't use the newer "SD Bus" API. Nobody would allow manually upgrading libsystemd just for an application-specific benefit, and I'm apparently not the first person to raise this concern.
I'm hesitant to use the low-level C API for DBUS, since the maintainers themselves state "If you use this low-level API directly, you're signing up for some pain.".
Hard Requirements
systemd v219.
CentOS v7.4
C/C++.
No system()/popen()/exec() calls to the systemctl CLI utility.
Non-propagating license (i.e. LGPLv2 is OK, though BSD/MIT is preferred).
Question (Redux)
Is there a simpler API that I could use on CentOS v7.4 (even if I have to supply the shared library myself) that can be used to communicate with systemd in a simple, reliable manner directly via C/C++ code? If the existing libdbus-1.so API is complicated but still reliable, and I can create a reliable/stable wrapper library around it, I'm open to exploring that option. Also, if it's possible to manually build and deploy the SD-BUS components independent of systemd and use them without having to modify/upgrade the systemd library/daemon already on the system, I could potentially go that route too.
Edits & Comments
I would be open to using a mature C++ library, so long as it doesn't require total release of all source.
Thank you.
As you already figured out, you should only interact with systemd using a dbus library, there is no other supported way to do so.
Even if you lifted the requirement of no execution of binaries, it will remain frowned upon, as the *ctl tools are command line user interfaces not intended or designed to be called from other programs.
Related
I'm working on a project, implementing everything in C language. As a part of the project, we need to be able to control and configure firewalld, firewall of the current system.
firewalld is implemented in Python and an interface is available. However, we don't want to make Python calls from C or vice versa.
There are command line tools to configure firewalld (e.g. firewall-cmd) but we don't want to make such calls from C either.
I recently started working on firewalld, I don't know much about its internals. I've read that it uses D-Bus, I also don't know much about D-Bus.
There is a C library developed by Thomas Woerner: libfirewall.
However, it's been more than a year and a half since the last commit so it's not maintained. Other than libfirewall, I don't know any firewalld interface in C.
I gave libfirewall a shot. It got me some problems when both compiling and running the examples and I still have problems to resolve. Is it worth to continue with libfirewall? Should I use it?
Is there any other interface that I'm not aware of?
Possibly naive question due to lack of understanding of D-Bus: I thought, maybe, with a D-Bus interface, I can issue commands to firewalld. Can it be done? (i.e. Does D-Bus work like that?) Can we write a program that mimics, say firewall-cmd, and interacts with D-Bus in the same way and at the end allows us to control firewalld?
If this is possible, how to do it and what to use? libdbus and GDBus have relatively good documentation although libdbus requires good deal of effort. They even said "If you use this low-level API directly, you're signing up for some pain." in the documentation. In any case I'll be in need of examples or any kind of text demonstrating their usage.
How should I approach this problem?
Yes, you can issue commands to firewalld via D-Bus. I haven't checked but expect that firewall-cmd is itself implemented as a D-Bus client.
The D-Bus API is extensively documented: https://firewalld.org/documentation/man-pages/firewalld.dbus.html. The documentation should give you a rough idea what can be accomplished through the API. You could try the D-Bus debugger d-feet to interact with firewalld without any code.
GDBus is definitely the easiest way use D-Bus from C but it's still not trivial and firewalld is a fairly complex API: Using it may require some expertise (completely depending on what you need to do).
I am not finding documentation for custom protocol support.
From what I understand, Gatling has core engine that does scheduling, thread management etc, and protocol support is designed as an Actor ?
I am trying to develop a custom protocol (thats basically a shell script that will talk to an external service). The latest reference documentation does not seem to have any reference to how to do this ? Any pointers will be greatly appreciated.
If you need to stress test something that is implemented in a shell script, then Gatling probably isn't the best fit. Gatling is designed for stress testing networking protocols. So unless you can duplicate what your shell script is doing in Gatling expressed in networking protocols, you then might want to use something else.
Secondly, if you did implement it, I would check with the core developers of Gatling if it's something that they would consider including (use a github issue to ask). Since the applications of this might not be widespread, they may choose to not include it in their project. If that's the case you would have to either run your own fork with the implementation or add some sort of plugin architecture to Gatling for 3rd part extensibility.
So my suggestions are:
Decompose your shell script into the specific network protocol parts you're interested in stress testing implementing in Gatling.
Use a different tool that's designed to running multiple shell scripts at once for stress testings. Something like GNU Parallel if you're on a Linux box.
Implement it yourself. There's no documentation on how to do this. However a good starting example would be the JMS Protocol Implementation to give you an idea of all that's involved.
My goal is to determine when executing a command, precisely which files it reads and writes. On Linux I can do this using ptrace (with work, akin to what strace does) and on FreeBSD and MacOS I can do this with the ktrace system command. What would you use to obtain this information on Windows?
My research so far suggests that I either use the debugger interface (similar to ptrace in many ways) or perhaps ETW. A third alternative is to interpose a DLL to intercept system calls as they are made. Unfortunately, I don't have the experience to guess as to how challenging each of these approaches will be.
Any suggestions?
Unfortunately it seems there is no easy way to intercept file level operations on Windows.
Here are some hints:
you could try to use FileMon from Sysinternals if it is enough for your needs, or try to look at the source of the tool
you could make use of commercial software like Detours - beware, I never used that myself and I'm not sure it really meets your needs
If you want a better understanding and are not frightened at doing it by hand, the Windows way of intercepting file I/O is using a File System Filter Driver. In fact, there is a FilterManager embedded in Windows system that can forward all file system calls to minifilters.
To build it, the interface with the system is provided by the FilterManager, and you have just (...) to code and install the minifilter that does the actual filtering - beware again never tested that ...
As you suggested, this is a fairly simple task to solve with API hooking with DLL injection.
This is a pretty good article about the application: API hooking revealed
I believe you can find more recent articles about the issue.
However, you probably need to use C++ to implement such a utility. By the way, programs can disable DLL injection. For example, I weren't able to use this approach on the trial version of Photoshop.
So, you may want to check if you can inject DLL files in the process you want with an existing solution before you start writing your own.
Please, take a look to the article CDirectoryChangeWatcher - ReadDirectoryChangesW all wrapped up.
It is a very old, but running, way to watch directory changes.
Microsoft owns a bunch of tools called Sysinternals. There is a program called Process Monitor that will show you all the file accesses for a particular process. This is very likely what you want.
Check this particular Stack Overflow question out for your question... This might help you:
Is there something like the Linux ptrace syscall in Windows?
Also, if you are running lower versions like Windows XP then you should check out Process Monitor.
Also, I would like you to check this out...
Monitoring certain system calls done by a process in Windows
I'm interested in studying the 9P FS, currently been reading the source available from these implementations: http://9p.cat-v.org/implementations
Is 9P obsolete? Are you using it for some application?
(also I've found this, some perfomance test between 9P and NFS: http://graverobbers.blogspot.com/2007/08/v9fs-performance-versus-nfs.html)
No, 9P isn't obsolete; I don't know of a protocol that does what it does and is clean and well defined enough to be implemented correctly in almost any language that exists.
9P is used in a variety of systems. A couple of recent uses in arm-js (an ARM emulator) and 9webdraw (a GSoC project that implements the Plan 9 /dev/draw). Both are HTML5 Javascript implementations.
Just to add a bit, both the Linux client implementation and several servers are under active development, so I'd say that's a pretty clear sign that folks still have use for it. One of the areas its seen heavy use more recently is the virtio-9P (aka virtfs) which is part of qemu/kvm and can be used for direct guest to host file access. It's also been used in several experimental operating systems projects (Libra, PROSE, FusedOS) and incorporated into other operating systems (BSD, MacOSX, Windows, Linux) and hypervisors (in addition to the KVM instance above, its also been incorporated in various ways into Xen). 9P is actually being used in supercomputing deployments (both for Plan 9 and Linux, see the diod project on Sourceforge).
I think the reason is that the protocol is quite simple, so implementations also tend to be quite simple and easy to integrate elsewhere (there are several applications both inside and outside the Plan 9 world which use 9P as an interface to the application, in much the same way that some web developers use RESTful interfaces).
The protocol has a couple of different variations including the 9P.L variant which was developed specifically to match the Linux VFS API better. It adds a bit of complexity to the protocol in the addition of operations, but removes some of the complexity of mapping Linux VFS API -> 9P and vice versa.
It is used in Erlang-on-Xen both as a storage protocol for goofs http://erlangonxen.org/blog/goofs-simple-filesystem
It is the way erlang on xen instances in other ways too, see here:
http://erlangonxen.org/more/9p2000e
Also, it's used by libvirt stuff with QEMU.
http://wiki.qemu.org/Documentation/9psetup
9p, to me, is like the Scheme of network protocols. For the most part, it is very simple, but people see need to extend it to fit their environments. Luckily this is done in ways that are often backwards compatible.
In addition to everything mentioned in the other answers, Microsoft is using 9P as part of their Windows Subsystem for Linux.
They add a 9P server to each Linux distribution that is running as a guest, so that Windows can mount the Linux filesystem over 9P, and Windows processes can transparently access the files on Linux's ext4 partition.
Is there any high performance (ideally evented and open source) web server in C or C++?
I'd like to be able to use it in that it calls a method/function in my application with a filled out HTTP Request class/struct, and then I can return a filled out HTTP Response class/struct to it.
If it isn't open source, I'd need built in support for long-polling connections, keep-alive, etc—otherwise, I think that I can add these things myself.
If you don't know of any such servers available, would you recommend writing my own web server to fit the task? It cannot be file-based, and must be written in high-performance C/C++.
Edit: I'm thinking something like the Ruby Mongrel for C, if that helps.
I had the very same requirements for my job, so I evaluated a number of solutions: mongoose, libmicrohttpd, libevent. And I also was thinking about writing nginx modules. Here is the summary of my findings:
nginx
nginx project page
I love this server and use it a lot. Its performance and resource usage is much better than that of Apache, which I also still use but plan migrating to nginx.
Very good tunable performance. Rich functionality. Portability.
Module API is not documented and seems to be very verbose. See this nginx hello world module as example.
Nginx does not use threads but uses multiple processes. This makes writing modules harder, need to learn nginx API for shared memory, etc.
mongoose
mongoose project page
All server's code is in single mongoose.c file (about 130K), no dependencies. This is good.
One thread per connection, so if you need concurrency you've got to configure lots of threads, ie. high RAM usage. Not too good.
Performance is good, although not exceptional.
API is simple but you have to compose all response HTTP headers yourself, ie. learn HTTP protocol in detail.
libmicrohttpd
libmicrohttpd project page
Official GNU project.
Verbose API, seems awkward to me, although much more simple than writing nginx modules.
Good performance in keep-alive mode (link to my benchmarks below), not so good without keep-alive.
libevent
libevent project page
Libevent library has built-in web server called evhttp.
It is event based, uses libevent for that.
Easy API. Constructs HTTP headers automatically.
Officially single-threaded. This is major disadvantage. I've found a hack, which makes several instances of evhttp run simultaneously accepting connections from the same socket. Not sure if it is all safe and robust.
Performance of single-threaded evhttp is surprisingly poor. Multi-threaded hack works better, but still not good.
G-WAN
G-WAN project is not open source, but I'd like to say a few words about it.
Very good performance, low memory usage, 150 KB executable.
Very convenient 'servlet' deployment: just copy .c file into csp directory, and running server automatically compiles it. Code modifications also compiled on the fly.
Simple API. Although constrained in some ways. Rich functionality (json, key-value store, etc.).
Unstable. I had segfaults on static files. Hangs on some sample scripts. (Experienced on clean install. Never mixed files of different versions).
Only 32-bit binary (not anymore).
So as you can see, none of existing alternatives have fully satisfied me. So I have developed my own server, which is ...
NXWEB
NXWEB project page
Feature highlights:
Very good performance; see benchmarks on project page
Can serve tens of thousands concurrent requests
Small memory footprint
Multi-threaded model designed to scale
Exceptionally light code base
Simple API
Decent HTTP protocol handling
Keep-alive connections
SSL support (via GNUTLS)
HTTP proxy (with keep-alive connection pooling)
Non-blocking sendfile support (with configurable small file memory cache; gzip pre-encoded file serving)
Modular design for developers
Can be run as daemon; relaunches itself on error
Open source
Limitations:
Depends on libev library (not anymore)
Only tested on Linux
I would suggest to write a FastCGI executable that can be used with many high performance web servers (even closed source ones).
mongoose: one file. simple and easy to use. not an asycn io but perfect for embedded and particular purposes.
gwan. excellent. no crashes. ultra well planned configuration. very smart and easy for c/c++ development in other words, very clean sensible api compared to nginx. provides a thread per core. or whatever you specify. a great choice. largest disadvantage (maybe im lacking in this area): cannot step thru code.
libevent: single thread is not a disadvantage on a single core machine. afterall its point is an async i/o. does have multithreads for other cores.
nginx: no personal experience. gaining serious ground on a-patchy server. (terribly confusing api)
boost asio: a c++ library for asynchio (asio). awesome. needs a friendly higher-level api for simpletons like myself. and others who come from php, java, javascript, node.js and other web languages.
python bottle: awesome 1 file lib (framework/system) that makes it easy to build python web apps. has/is a built in httpd server, like libevent and node.js
node.js: javascript asyncio server. an excellent selection. unfortunately, have to program in javascript that does become tedious. while there is something to be said for getting the job done; there is also something to be said for enjoying yourself during the process. hopefully no ones comes up with node.php
I'm going to suggest the same thing as Axel Gneiting - but have provided an answer with my reasons for taking this approach:
1) HTTP is not trivial as a protocol - writing your own server or amending an off-the-shelf solution is a very complex task - a lot more complex than using the available APIs for implementing a separate processing engine
2) Using (an unmodified) mainstream webserver should provide you with more functionality than you require (so you've got growing room).
3) Using (an unmodified) mainstream webserver will usually mean that it has been far more extensively tested and documented than a homebrew system.
4) .. and its more likely to be secure and stable.
5) Using fastCGI you can use all sorts of languages to develop your back-end processing in - including C++ and C. There are standard toolkits available to facilitate this.
6) alternatively many webservers provide support for running interpreter engines in-process (e.g. mod_php, mod_perl). I'd advise against running your own native code as a module though.
It cannot be file-based.
Eh? What does that mean?
I'm an avid nginx user; nginx is written in C; nginx seems like it could work for you. If you want the very best speed out of nginx, I would make a nginx module. Here are 3rd party modules which you can examine to get an idea of what it requires.
As for the long polling requirement, you might want to have a look at the http push modules.