We have a product we ship on Solaris amd64 (and x86, SPARC) - we have single pkg that installs on Solaris 10 and 11.
We call some functions from /usr/include/bsm/audit.h, specifically getaudit_addr, and between Solaris 10 and 11 the ABI changed dramatically, for a start reordering the struct fields and changing their lengths:
struct auditinfo_addr {
au_mask_t ai_mask;
au_id_t ai_auid;
au_asid_t ai_asid;
au_tid_addr_t ai_termid;
}; /* Sol 11 version */
struct auditinfo_addr {
au_id_t ai_auid;
au_mask_t ai_mask;
au_tid_addr_t ai_termid;
au_asid_t ai_asid;
}; /* Sol 10 version */
So, our code uses dlopen/dlsym to get a handle to getaudit_addr, which unsurprisingly bails out in a horrible way if you compile on Sol10 and run on Sol11 (because we're using a completely mangled structure). This is not good.
Question
Would we be OK if we linked with -lbsm rather than used dlopen? If so, how, because I can't find any version of the Sol10 ABI symbols inside Sol11's libbsm.so using nm (and both Sol10 and Sol11's version of libbsm have the following version symbols: SUNW_0.7, SUNW_0.8, SUNW_1.1, SUNW_1.2). Update: No, linking with -lbsm on Solaris 10 doesn't make the code run correctly on Solaris 11. It's just a disgusting breaking ABI change they made. Grr.
If Solaris does have symbol versioning that works, can we do it dynamically?
I'm inclined to copy the structure definitions and do a run-time switch between the old and new ones rather than use the headers. Is there another fall-back solution?
Do the headers depend on the architecture? That is, is audit.h identical on SPARC, amd64, and x86? Obviously the size of typedef'd types may change, but will I need to hunt down a Solaris 11 SPARC machine to copy its header and check whether it matches the x86 one?
Really, if the ABI is incompatible you need to treat this in the same way as if it was an incompatible processor type. That is: Build two versions. Install or build the correct version from your installer. Check at runtime you are running the correct version, and quit if you are not.
Alternatively, if this is literally the only structure which changes, you could get away with a typedef'd auditinfo_addr_v10 vs auditinfo_addr_v11, and ship with either two versions of every function which uses the struct, or have a conversion function which is used on V10 to convert everything to the latest structure. I.e. supply your own getaudit_addr_wrapper which takes a v11 structure but will convert to the correct call on v10.
Related
Why does libc_nonshared.a exist? What purpose does it serve? I haven't been able to find a good answer for its existence online.
As far as I can tell it provides certain symbols (stat, lstat, fstat, atexit, etc.). If someone uses one of these functions in their code, it will get linked into the final executable from this archive. These functions are part of the POSIX standard and are pretty common so I don't see why they wouldn't just be put in the shared or static libc.so.6 or libc.a, respectively.
It was a legacy mistake in glibc's implementing extensibility for the definition of struct stat before better mechanisms (symbol redirection or versioning) were thought of. The definitions of the stat-family functions in libc_nonshared.a cause the version of the structure to bind at link-time, and the definitions there call the __xstat-family functions in the real shared libc, which take an extra argument indicating the desired structure version. This implementation is non-conforming to the standard since each shared library ends up gettings its own copy of the stat-family functions with their own addresses, breaking the requirement that pointers to the same function evaluate equal.
Here's the problem. Long ago, members of the struct stat structure had different sizes than they had today. In particular:
uid_t was 2 bytes (though I think this one was fixed in the transition from libc5 to glibc)
gid_t was 2 bytes
off_t was 4 bytes
blkcnt_t was 4 bytes
time_t was 4 bytes
also, timespec wasn't used at all and there was no room for nanosecond precision.
So all of these had to change. The only real solution was to make different versions of the stat() system call and library function and you get the version you compiled against. That is, the .a file matches the header files. These things didn't all change at once, but I think we're done changing them now.
You can't really solve this by a macro because the structure name is the same as the function name; and inline wasn't mandated to exist in the beginning so glibc couldn't demand everybody use it.
I remember there used to be this thing O_LARGEFILE for saying you could handle files bigger than 4GB; otherwise things just wouldn't work. We also used to have to define things like _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE but it's all handled automatically now. Back in the day, if you weren't ready for large file support yet, you didn't define these and you didn't get the 64-bit version of the stat structure; and also worked on older kernel versions lacking the new system calls. I haven't checked; it's possible that 32-bit compilation still doesn't define these automatically, but 64-bit always does.
So you probably think; okay, fine, just don't franken-compile stuff? Just build everything that goes into the final executable with the same glibc version and largefile-choice. Ever use plugins such as browser plugins? Those are pretty much guaranteed to be compiled in different places with different compiler and glibc versions and options; and this didn't require you to upgrade your browser and replace all its plugins at the same time.
I am trying to work with Nyquist (a music programming platform, see: https://www.cs.cmu.edu/~music/nyquist/ or https://www.audacityteam.org/about/nyquist/) as a standalone program and it utilizes libsndfile (a library for reading and writing sound, see: http://www.mega-nerd.com/libsndfile/). I am doing this on an i686 GNU/Linux machine (Gentoo).
After successful set up and launching the program without errors, I tried to generate sound via one of the examples, "(play (osc 60))", and was met with this error:
*** Fatal error : sizeof (off_t) != sizeof (sf_count_t)
*** This means that libsndfile was not configured correctly.
Investigating this further (and emailing the author) has proved somewhat helpful, but the solution is still far from my grasp. The author recommended looking at /usr/include/sndfile.h to see how sf_count_t is defined, and (this portion of) my file is identical to his:
/* The following typedef is system specific and is defined when libsndfile is
** compiled. sf_count_t will be a 64 bit value when the underlying OS allows
** 64 bit file offsets.
** On windows, we need to allow the same header file to be compiler by both GCC
** and the Microsoft compiler.
*/
#if (defined (_MSCVER) || defined (_MSC_VER))
typedef __int64 sf_count_t ;
#define SF_COUNT_MAX 0x7fffffffffffffffi64
#else
typedef int64_t sf_count_t ;
#define SF_COUNT_MAX 0x7FFFFFFFFFFFFFFFLL
#endif
In the above the author notes there is no option for a "32 bit offset". I'm not sure how I would proceed. Here is the particular file the author of Nyquist recommend I investigate: https://github.com/erikd/libsndfile/blob/master/src/sndfile.h.in , and here is the entire source tree: https://github.com/erikd/libsndfile
Here are some relevant snippets from the authors email reply:
"I'm guessing sf_count_t must be showing up as 32-bit and you want
libsndfile to use 64-bit file offsets. I use nyquist/nylsf which is a
local copy of libsndfile sources -- it's more work keeping them up to
date (and so they probably aren't) but it's a lot easier to build and
test when you have a consistent library."
"I use CMake and nyquist/CMakeLists.txt to build nyquist."
"It may be that one 32-bit machines, the default sf_count_t is 32
bits, but I don't think Nyquist supports this option."
And here is the source code for Nyquist: http://svn.code.sf.net/p/nyquist/code/trunk/nyquist/
This problem is difficult for me to solve because it's composed of an niche use case of relatively obscure software. This also makes the support outlook for the problem a bit worrisome. I know a little C++, but I am far from confident in my ability to solve this. Thanks for reading and happy holidays to all. If you have any suggestions, even in terms of formatting or editing, please do not hesitate!
If you look at the sources for the bundled libsndfile in nyquist, i.e. nylsf, then you see that sndfile.h is provided directly. It defines sf_count_t as a 64-bit integer.
The libsndfile sources however do not have this file, rather they have a sndfile.h.in. This is an input file for autoconf, which is a tool that will generate the proper header file from this template. It has currently the following definition for sf_count_t for linux systems (and had it since a while):
typedef #TYPEOF_SF_COUNT_T# sf_count_t ;
The #TYPEOF_SF_COUNT_T# would be replaced by autoconf to generate a header with a working type for sf_count_t for the system that is going to be build for. The header file provided by nyquist is therefore already configured (presumably for the system of the author).
off_t is a type specified by the POSIX standard and defined in the system's libc. Its size on a system using the GNU C library is 32bit if the system is 32bit.
This causes the sanity check in question to fail, because the sizes of sf_count_t and off_t don't match. The error message is also correct, as we are using an unfittingly configured sndfile.h for the build.
As I see it you have the following options:
Ask the nyquist author to provide the unconfigured sndfile.h.in and to use autoconf to configure this file at build time.
Do not use the bundled libsndfile and link against the system's one. (This requires some knowledge and work to change the build scripts and header files, maybe additional unexpected issues)
If you are using the GNU C library (glibc): The preprocessor macro _FILE_OFFSET_BITS can be set to 64 to force the size of off_t and the rest of the file interface to use the 64bit versions even on 32bit systems.
This may or may not work depending on whether your system supports it and it is not a clean solution as there may be additional misconfiguration of libsndfile going unnoticed. This flag could also introduce other interface changes that the code relies on, causing further build or runtime errors/vulnerabilities.
Nonetheless, I think the syntax for cmake would be to add:
add_compile_definitions(_FILE_OFFSET_BITS=64)
or depending on cmake version:
add_definitions(-D_FILE_OFFSET_BITS=64)
in the appropriate CMakeLists.txt.
Actually the README in nyquist/nylsf explains how the files for it were generated. You may try to obtain the source code of the same libsndfile version it is based on and repeat the steps given to produce an nylsf configured to your system. It may cause less further problems than 2. and 3. because there wouldn't be any version/interface changes introduced.
I have managed to build an RV32E cross-compiler on my Intel Ubuntu machine by using the official riscv GitHub toolchain (github.com/riscv/riscv-gnu-toolchain) with the following configuration:-
./configure --prefix=/home/riscv --with-arch=rv32i --with-abi=ilp32e
The ip32e specifies soft float for RV32E. This generates a working compiler that works fine on my simple C source code. If I disassemble the created application then it does indeed stick to the RV32E specification. It only generates assembly for my code that uses the first 16 registers.
I use static linking and it pulls in the expected set of soft float routines such as __divdi3 and __mulsi3. Unfortunately the pulled in routines use all 32 registers and not the restricted lower 16 for RV32E. Hence, not very useful!
I cannot find where this statically linked code is coming from, is it compiled from C source and therefore being compiled without the RV32E restriction? Or maybe it was written as hand coded assembly that has been written only for the full RV32I instead of RV32E? I tried to grep around the source but have had no luck finding anything like the actual code that is statically linked.
Any ideas?
EDIT: Just checked in more details and the compiler is not generating using just the first 16 registers. Turns out with a simple test routine it manages to only use the first 16 but more complex code does use others as well. Maybe RV32E is not implemented yet?
The configure.ac file contains this code:
AS_IF([test "x$with_abi" == xdefault],
[AS_CASE([$with_arch],
[*rv64g* | *rv64*d*], [with_abi=lp64d],
[*rv64*f*], [with_abi=lp64f],
[*rv64*], [with_abi=lp64],
[*rv32g* | *rv32*d*], [with_abi=ilp32d],
[*rv32*f*], [with_abi=ilp32f],
[*rv32*], [with_abi=ilp32],
[AC_MSG_ERROR([Unknown arch])]
)])
Which seems to map your input of rv32i to the ABI ilp32, ignoring the e. So yes, it seems support for the ...e ABIs is not fully implemented yet.
I am trying to build a program that communicates with a 32-bit embedded system, that runs on a Linux based x86_64 machine (host). On the host program I have a structure containing a few pointers that reflects an identical structure on the embedded system.
The problem is that on the host, pointers are natively 64-bits, so the offset of the structure members is not the same as in the embedded system. Thus, when copying the structure (as memcpy), the contents end up at the wrong place in the host copy.
struct {
float a;
float b;
float *p;
float *q;
} mailbox;
// sizeof(mailbox) is 4*4=16 on the embedded, but 2*4+2*8=24 on the host
Luckily, I found out here that gcc has an option -mx32 for generating 32-bit pointers on x86_64 machines. But, when trying to use this, I get an error saying:
$ gcc -mx32 test.c -o test.e
cc1: error: unrecognized command line option "-mx32"
This is for gcc versions 4.4.3 and 4.7.0 20120120 (experimental).
Why doesn't this option work? Is there a way around this?
EDIT: Accrding to the v4.4.7 manual, there was no -mx32 option available, and this is true up to v4.6.3. OTOH, v4.7.0 does show that option, so it may be that the Jan-20 version I am using is not the final one?!
Don't do this. First, x32 is a separate architecture. It's not merely a compiler switch. You need an x32 version of every library you link against to make this work. Linux distros aren't yet producing x32 versions, so that means you'll be either linking statically or rolling your own library environment.
More broadly: that's just asking for trouble. If your structure contains pointers they should be pointers. If it contains "32 bit addresses" they should be a 32 bit integer type.
You might need a newer version of binutils
Though I think gcc 4.8 is recommended
But in general you need a kernel compiled multilib with it: https://unix.stackexchange.com/questions/121424/linux-and-x32-abi-how-to-use
Is there a way to convert a Delphi .dcu file to an .obj file so that it can be linked using a compiler like GCC? I've not used Delphi for a couple of years but would like to use if for a project again if this is possible.
Delphi can output .obj files, but they are in a 32-bit variant of Intel OMF. GCC, on the other hand, works with ELF (Linux, most Unixes), COFF (on Windows) or Mach-O (Mac).
But that alone is not enough. It's hard to write much code without using the runtime library, and the implementation of the runtime library will be dependent on low-level details of the compiler and linker architecture, for things like correct order of initialization.
Moreover, there's more to compatibility than just the object file format; code on Linux, in particular, needs to be position-independent, which means it can't use absolute values to reference global symbols, but rather must index all its global data from a register or relative to the instruction pointer, so that the code can be relocated in memory without rewriting references.
DCU files are a serialization of the Delphi symbol tables and code generated for each proc, and are thus highly dependent on the implementation details of the compiler, which changes from one version to the next.
All this is to say that it's unlikely that you'd be able to get much Delphi (dcc32) code linking into a GNU environment, unless you restricted yourself to the absolute minimum of non-managed data types (no strings, no interfaces) and procedural code (no classes, no initialization section, no data that needs initialization, etc.)
(answer to various FPC remarks, but I need more room)
For a good understanding, you have to know that a delphi .dcu translates to two differernt FPC files, .ppu file with the mentioned symtable stuff, which includes non linkable code like inline functions and generic definitions and a .o which is mingw compatible (COFF) on Windows. Cygwin is mingw compatible too on linking level (but runtime is different and scary). Anyway, mingw32/64 is our reference gcc on Windows.
The PPU has a similar version problem as Delphi's DCU, probably for the same reasons. The ppu format is different nearly every major release. (so 2.0, 2.2, 2.4), and changes typically 2-3 times an year in the trunk
So while FPC on Windows uses own assemblers and linkers, the .o's it generates are still compatible with mingw32 In general FPC's output is very gcc compatible, and it is often possible to link in gcc static libs directly, allowing e.g. mysql and postgres linklibs to be linked into apps with a suitable license. (like e.g. GPL) On 64-bit they should be compatible too, but this is probably less tested than win32.
The textmode IDE even links in the entire GDB debugger in library form. GDB is one of the main reasons for gcc compatibility on Windows.
While Barry's points about the runtime in general hold for FPC too, it might be slightly easier to work around this. It might only require calling certain functions to initialize the FPC rtl from your startup code, and similarly for the finalize. Compile a minimal FPC program with -al and see the resulting assembler (in the .s file, most notably initializeunits and finalizeunits) Moreover the RTL is more flexible and probably more easily cut down to a minimum.
Of course as soon as you also require exceptions to work across gcc<->fpc bounderies you are out of luck. FPC does not use SEH, or any scheme compatible with anything else ATM. (contrary to Delphi, which uses SEH, which at least in theory should give you an advantage there, Barry?) OTOH, gcc might use its own libunwind instead of SEH.
Note that the default calling convention of FPC on x86 is Delphi compatible register, so you might need to insert proper cdecl (which should be gcc compatible) modifiers, or even can set it for entire units at a time using {$calling cdecl}
On *nix this is bog standard (e.g. apache modules), I don't know many people that do this on win32 though.
About compatibility; FPC can compile packages like Indy, Teechart, Zeos, ICS, Synapse, VST
and reams more with little or no mods. The dialect levels of released versions are a mix of D7 and up, with the focus on D7. The dialect level is slowly creeping to D2006 level in trunk versions. (with for in, class abstract etc)
Yes. Have a look at the Project Options dialog box:
(High-Res)
As far as I am aware, Delphi only supports the OMF object file format. You may want to try an object format converter such as Agner Fog's.
Since the DCU format is proprietary and has a tendency of changing from one version of Delphi to the next, there's probably no reliable way to convert a DCU to an OBJ. Your best bet is to build them in OBJ format in the first place, as per Andreas's answer.