I'm looking for a way to introduce a formal public API into a program (PostgreSQL) that presently lacks any formal boundary between extension-accessible interfaces and those for internal use only.
It has a few headers that say they're for internal use only, and it uses a lot of statics, but there's still a great deal that extensions can just reach into ... but shouldn't. This makes it impractical to offer binary compatibility guarantees across even patch releases. The current approach boils down to "it should work but we don't promise anything," which is not ideal.
It's a large piece of software and it's not going to be practical to introduce a hard boundary all at once since there's such a lively extension community outside the core codebase. But we'd need a way to tell people "this interface is private, don't use it or speak up and ask for it to be declared public if you need it and can justify it".
Portably.
All I've been able to come up with so far is to add macros around gcc's __attribute__ ((deprecated)) and MSVC's __declspec(deprecated) that are empty when building the core server, but defined normally when building extensions. That'll work, but "deprecated" isn't quite right, it's more a case of "use of non-public internal API".
Is there any better way than using deprecated annotations? I'd quite like to actually be able to use them for deprecating use of functionality in-core too, as the server grows and it becomes impractical to always change everything all in one sweep.
Related
First off, I know that any such library would need to have at least some interface shimming to interact with system calls or board support packages or whatever (For example; newlib has a limited and well defined interface to the OS/BSP that doesn't assume intimate access to the OS). I can als see where there will be need for something similar for interacting with the compiler (e.g. details of some things left as "implementation defined" by the standard).
However, most libraries I've looked into end up being way more wedded to the OS and compiler than that. The OS assumptions seems to assume access to whatever parts they want from a specific OS environment and the compiler interactions practically in collusion with the compiler implementation it self.
So, the basic questions end up being:
Would it be possible (and practical) to implement a full C standard library that only has a very limited and well defined set of prerequisites and/or interface for being built by any compiler and run on any OS?
Do any such implementation exist? (I'm not asking for a recommendation of one to use, thought I would be interested in examples that I can make my own evaluations of.)
If either of the above answers is "no"; then why? What fundamentally makes it impossible or impractical? Or why hasn't anyone bothered?
Background:
What I'm working on that has lead me to this rabbit hole is an attempt to make a fully versioned, fully hermetic build chain. The goal being that I should be able to build a project without any dependencies on the local environment (beyond access to the needs source control repo and say "a valid posix shell" or "language-agnostic-build-tool-of-choice is installed and runs"). Given those dependencies, the build should do the exact same thing, with the same compiler and libraries, regardless of which versions of which compilers and libraries are or are not installed. Universal, repeatable, byte identical output is the target I'm wanting to move in the direction of.
Causing the compiler-of-choice to be run from a repo isn't too hard, but I've yet to have found a C standard library that "works right out of the box". They all seem to assume some magic undocumented interface between them and the compiler (e.g. __gnuc_va_list), or at best want to do some sort of config-probing of the hosting environment which would be something between useless and counterproductive for my goals.
This looks to be a bottomless rabbit hole that I'm reluctant to start down without first trying to locate alternatives.
I would like to know which FFI interfaces are supported and functional in Pharo 4, which ones are recommended (if any), and if there are some big changes planned in the future Pharo versions.
Especially I would like to know which stable FFI callback mechanism is available, and what are its restrictions and limitations.
Sorry for taking so long.
Currently, in Pharo you have two possibilities (and three packages –projects– to tackle them).
Out of the box in Pharo 4 you have NativeBoost-FFI who uses ASMJIT as a backend to generate native calls. That means super fast calls, but not so fast callbacks (because it uses some hard trick to make the VM calls back properly).
You can find examples of it all around the source-code, you can take a look specially at NBBasicExamples class.
You also have FFI plugin which implements a more traditional approach. You have to packages to handle this:
FFI, who implements callouts using pragmas (no callbacks). You can install it from the configurations browser and it comes with a set of examples.
AlienFFI, another package to handle same library FFI handles, but implementen a more "alien" approach (each function is an object, not a method). This implements callbacks properly and with good performance.
Installation is a bit more tricky because it has not been (yet) tested in Pharo 4 (so it is not in the configurations browser) but it should load fine.
You can find it here: http://catalog.pharo.org, along with instructions on how to install it (name is OldAlien, because of historical reasons).
All of them are very stable, but we will do some changes in the close future:
We will add a backend for NativeBoost-FFI to use the FFI plugin. The reason is our difficulty to maintain current version and the fact we can find more maintainers willing to work in C than in ASM :)
This change should be backward compatible so you are safe-to-go with NativeBoost.
Hope this info works for you.
I am writing a OS X application in Swift and considering implementing a portion of my model code (MVC pov) in Rust in order to gain portability of usage on a web server (since Swift doesn't extend into the web server space).
If I compile my Rust code as a dynamic library (.dylib) and follow certain guidelines, I understand that it will produce an external interface that will look like a typical "C" interface that Swift can then consume.
What would the performance implications be of Swift communicating with the dylib in this instance? Would they be any different to communicating with something written in C? The model component in the dylib can be assumed to be central to the application and hence the communication is anticipated to be "chatty".
[If I were to code the whole thing in Swift, the model portion would have been contained in a separate framework/library which is very similar to a dylib in any event.]
When using a Foreign Function Interface, I can see two major cost centers (performance-wise):
Conversion: if you need to convert the data back and forth (either on one side or even worse on both sides)
Lost optimization opportunity: no inlining, and thus lots of missed opportunities
The latter can be recovered with a "better" toolchain, especially since in this case both languages compile down to LLVM IR, and I know it is an ambition for at least some Rust developers/users to manage this for Rust/C interaction, but I have never seen it yet.
The former requires a careful design and even then it can be inevitable that some conversions will occur if you wish to use native compound types on either side (String being a prime candidate).
I'm looking for a quick guide to basic dll hooking in windows with C, but all the guides I can find are either not C, or not windows.
(The DLL is not part of windows, but a third party program)
I understand the principle, but I don't know how to go about it.
I have pre-existing source code in C++ that shows what I need to hook into, but I don't have any libraries for C, or know how to hook from scratch.
The detours license terms are quite restrictive.
If you merely want to hook certain functions of a DLL it is often cheaper to use a DLL-placement attack on the application whose DLL you want to hook. In order to do this, provide a DLL with the same set of exports and forward those that you don't care about and intercept the rest. Whether that's C or C++ doesn't really matter. This is often technically feasible even with a large number of exports but has its limitations with exported data and if you don't know or can't discern the calling convention used.
If you must use hooking there are numerous ways including to write a launcher and rewrite the prepopulated (by the loader) IAT to point to your code while the main thread of the launched application is still suspended (see the respective CreateProcess flag). Otherwise you are likely going to need at least a little assembly knowledge to get the jumps correct. There are plenty of liberally licensed disassembler engines out there that will allow you to calculate the proper offsets for patching (because you don't want to patch the middle of a multi-byte opcode, for example).
You may want to edit your question again to include what you wrote in the comments (keyword: "DLL hooking").
loading DLLs by LoadLibrary()
This is well known bad practice.
You might want to look up "witch" or "hctiw", the infamous malware dev. there's a reason he's so infamous - he loaded DLLs with LoadLibrary(). try to refrain from bad practice like that.
I'm finding myself doing more C/C++ code against Win32 lately, and coming from a C# background I've developed an obsession with "clean code" that is completely consistent, so moving away from the beautiful System.* namespace back to the mishmash of #defines that make up the Win32 API header files is a bit of a culture shock.
After reading through MSDN's alphabetical list of core Win32 functions I realised how simple Win32's API design actually is, and it's unfortunate that it's shrouded with all the cruft from the past 25 years, including many references to 16-bit programming that are completely irrelevant in today's 64-bit world.
I'm due to start a new C/C++ project soon, and I was thinking about how I could recreate Win32's headers on an as-needed basis. I could design it to be beautiful, and yet it would maintain 100% binary (and source) compatibility with existing programs (because the #defines ultimately resolve the same thing).
I was wondering if anyone had attempted this in the past (Google turned up nothing), or if anyone wanted to dissuade me from it.
Another thing I thought of, was how with a cleaner C Win32 API, it becomes possible to design a cleaner and easier to use C++ Win32 API wrapper on top, as there wouldn't be any namespace pollution from the old C Win32 items.
EDIT:
Just to clarify, I'm not doing this to improve compilation performance or for any kind of optimisation, I'm fully aware the compiler does away with everything that isn't used. My quest here is to have a Win32 header library that's a pleasure to work with (because I won't need to depress Caps-lock every time I use a function).
Don't do this.
It may be possible, but it will take a long time and will probably lead to subtle bugs.
However, and more importantly, it will make your program utterly impossible for anyone other than you to maintain.
There's no point in doing this. Just because there's additional cruft doesn't mean it's compiled into the binary (anything unused will be optimized out). Furthermore, on the EXTREME off-chance that anything DOES change (I dunno, maybe WM_INPUT's number changes) it's just a lot easier to use the system headers. Furthermore, what's more intuitive? I think #include <windows.h> is a lot easier to understand than #include "a-windows-of-my-own.h".
Also, honestly you never should need to even look at the contents of windows.h. Yeah I've read it, yeah it's ugly as sin, but it does what I need it to and I don't need to maintain it.
Probably the ONLY downside of using the real windows.h is that it MAY slow down compilation by a few milliseconds.
No. What's the point? Just include <windows.h>, and define a few macros like WIN32_LEAN_AND_MEAN, VC_EXTRALEAN, NOGDI, NOMINMAX, etc. to prune out the things you don't want/need to speed up your compile times.
Although the Win32 headers might be considered "messy", you pretty much never have to (or want to) look inside them. All you need to know is documented in the Win32 SDK. The exact contents of the header files are an implementation detail.
There is a ton of stuff in there that would be time-consuming and unnecessarily finicky to replicate, particularly relating to different versions of the Win32 SDK.
I recommend:
#include <windows.h>
In my opinion, this is bad practice. Tidiness and brevity is achieved by keeping to the standard practice as much as possible, and leveraging as much as possible from the platform. You need to assume Microsoft to have the ultimate expertise in their own platform, with some aspects going beyond what you know right now. In simple words, it's their product and they know best.
By rolling your own:
... you branch off from Microsoft's API, so Microsoft could no longer deliver updates to you through their standard channels
... you may introduce bugs due to your own hubris, feeling you've figured something out while you haven't
... you'd be wasting a lot of time for no tangible benefit (as the C headers don't carry any overhead into the compiled binary)
... you'd eventually create a project that's less elegant
The most elegant code is one that carries more LOC of actual program logic and as little as possible LOC for "housekeeping" (i.e. code not directly related to the task at hand). Don't fail to leverage the Platform SDK headers to make your project more elegant.
This has been attempted in the past.
In its include directory, MinGW contains its own version of windows.h. Presumably this exists to make the headers work with gcc. I don't know if it will work with a Microsoft compiler.