Enforce 32-bit enums with GCC linker - c

I'm writing a bare metal application for an ARM device (no OS). I need 32-bit enums, so I compiled the application with the -fno-short-enums compiler flag. Without this flag, I get variable enums (and enforcing the size by adding an additional 0xFFFFFFFF value to each enum is not an option).
Now I get the following linker warning for every object:
c:/gcc-arm-none-eabi/bin/../lib/gcc/arm-none-eabi/6.2.1/../../../../arm-none-eabi/bin/ld.exe: warning: ./src/test.o uses 32-bit enums yet the output is to use variable-size enums; use of enum values across objects may fail
It's just a warning, no error. But what does it mean exactly? How can I specify the "output"?
I tried to recompile the newlib with the above flag to ensure that all objects use the same enum size, but I'm still getting the warning. Is there something I missed?

After a while I got it working.
I rebuilt the whole toolchain including the compiler with this flag. Here is the way I did:
Get the toolchain source from https://developer.arm.com/open-source/gnu-toolchain/gnu-rm/downloads
Add 3 lines to some sections of the buildscript build-toolchain.sh:
saveenv
saveenvvar CFLAGS_FOR_TARGET '-fno-short-enums'
[...build commands...]
restoreenv
Modified sections:
Task [III-1] /$HOST_NATIVE/gcc-first/
Task [III-2] /$HOST_NATIVE/newlib/
Task [III-4] /$HOST_NATIVE/gcc-final/
Task [IV-3] /$HOST_MINGW/gcc-final/
I Skipped building newlib-nano and gcc-size-libstdcxx.
Run the modified Scripts build-prerequisites.sh and build-toolchain.sh to build everything.
After that, the compiler is using the large-enum-mode and the linker is fine with my objects. But now, I get the opposit warning for some objects of the newlib (lib_a-mbtowc_r.o, lib_a-svfiprintf.o, lib_a-svfprintf.o, lib_a-vfprintf.o and lib_a-vfiprintf.o):
c:/gcc-arm-none-eabi/bin/../lib/gcc/arm-none-eabi/6.2.1/../../../../arm-none-eabi/bin/ld.exe: warning: c:/gcc-arm-none-eabi/bin/../lib/gcc/arm-none-eabi/6.2.1/../../../../arm-none-eabi/lib\libc.a(lib_a-mbtowc_r.o) uses variable-size enums yet the output is to use 32-bit enums; use of enum values across objects may fail
I looked into the makefiles of these object and they are explicitly set to variable-size-enums, sadly. The only "solution" for this, was to add a linker flag to mute this warning:
-Xlinker -no-enum-size-warning
That's it.

I ran into the same problem since I have a lot of enums in data structures that map to hardware. In my case, my fix is to do the following:
struct x {
union {
enum my_enum eval;
uint32_t :32;
};
uint32_t some_val;
...
}
This works and is transparent outside of the data structure, though it means that every time an enum is used in a data structure this union wrapper is required. I suppose a macro might be possible as well. I agree it's a pain. Every other 32-bit and 64-bit environment I've worked with treats enums as 32-bits and don't get me started with the decision to make a uint32_t an unsigned long in the ARM embedded ABI (every other one I've worked with it's an unsigned int, making portability with 64-bit ABIs easy).

I have a partial answer as I had the same question. The problem is a warning message from the linker that will appear with the use of -fno-short-enums. The message indicates that the target object is not compatible. So I spent a world of time looking for now to change the target to be compatible.
But that wasn't the problem. The gcc compiler will build 32-bit enums by default!! So this option is no necessary unless you DONT want 32-bit enums. However, if you do specify the -fno-short-enums you will receive the warning message. Unfortunately I don't know why.
So the bottom line is that the no-short-enums flag is not necessary to achieve 32-bit enums. If you do specify it then you will receive the warning message.

Related

Linking additional code for the microcontroller (AVR) against already existing code

The problem definition:
There is a need to have two parts of the code in an AVR microcontroller, a fixed one that is always there and does not change (often), and a transient one, that is (not so) often to be replaced or appended. The challenge is to give the ability for the transient code to call functions and access global variables of the fixed one -- and vice versa.
It is quite obvious that there should be special methods for the fixed code to access transient one -- like having calculated function pointers in RAM and using only them to call transient code procedures.
For the calling in backwards direction, I was thinking about linking transient code against existing .elf file of the fixed code.
I'm using avr-gcc toolchain (as in ubuntu 20.20), gcc version 5.4.0
What's I've already tried:
adding '-shared' as a link argument when building fixed code -- appears to be unsupported for AVR (linker reports an error).
adding instead '-Wl,--export-dynamic' as a link argument -- it seems to be ignored, no .dynsym section appears in the elf.
There is still a .symtab section in the fixed code elf -- could that be somehow used to link against it?
Note: my division of 'fixed' and 'transient' code has nothing to do with boot-area of some AVR microcontroller, boot is just something I do not care here about.
Note2: The question is much alike this one, but gives clear explanation for the need.
You have to forget all big computer knowledge. 8 bits AVRs are timy microcontrollers. Code has to be linked statically. There is no other way.

How is the type sf_count_t in sndfile.h defined in libsndfile?

I am trying to work with Nyquist (a music programming platform, see: https://www.cs.cmu.edu/~music/nyquist/ or https://www.audacityteam.org/about/nyquist/) as a standalone program and it utilizes libsndfile (a library for reading and writing sound, see: http://www.mega-nerd.com/libsndfile/). I am doing this on an i686 GNU/Linux machine (Gentoo).
After successful set up and launching the program without errors, I tried to generate sound via one of the examples, "(play (osc 60))", and was met with this error:
*** Fatal error : sizeof (off_t) != sizeof (sf_count_t)
*** This means that libsndfile was not configured correctly.
Investigating this further (and emailing the author) has proved somewhat helpful, but the solution is still far from my grasp. The author recommended looking at /usr/include/sndfile.h to see how sf_count_t is defined, and (this portion of) my file is identical to his:
/* The following typedef is system specific and is defined when libsndfile is
** compiled. sf_count_t will be a 64 bit value when the underlying OS allows
** 64 bit file offsets.
** On windows, we need to allow the same header file to be compiler by both GCC
** and the Microsoft compiler.
*/
#if (defined (_MSCVER) || defined (_MSC_VER))
typedef __int64 sf_count_t ;
#define SF_COUNT_MAX 0x7fffffffffffffffi64
#else
typedef int64_t sf_count_t ;
#define SF_COUNT_MAX 0x7FFFFFFFFFFFFFFFLL
#endif
In the above the author notes there is no option for a "32 bit offset". I'm not sure how I would proceed. Here is the particular file the author of Nyquist recommend I investigate: https://github.com/erikd/libsndfile/blob/master/src/sndfile.h.in , and here is the entire source tree: https://github.com/erikd/libsndfile
Here are some relevant snippets from the authors email reply:
"I'm guessing sf_count_t must be showing up as 32-bit and you want
libsndfile to use 64-bit file offsets. I use nyquist/nylsf which is a
local copy of libsndfile sources -- it's more work keeping them up to
date (and so they probably aren't) but it's a lot easier to build and
test when you have a consistent library."
"I use CMake and nyquist/CMakeLists.txt to build nyquist."
"It may be that one 32-bit machines, the default sf_count_t is 32
bits, but I don't think Nyquist supports this option."
And here is the source code for Nyquist: http://svn.code.sf.net/p/nyquist/code/trunk/nyquist/
This problem is difficult for me to solve because it's composed of an niche use case of relatively obscure software. This also makes the support outlook for the problem a bit worrisome. I know a little C++, but I am far from confident in my ability to solve this. Thanks for reading and happy holidays to all. If you have any suggestions, even in terms of formatting or editing, please do not hesitate!
If you look at the sources for the bundled libsndfile in nyquist, i.e. nylsf, then you see that sndfile.h is provided directly. It defines sf_count_t as a 64-bit integer.
The libsndfile sources however do not have this file, rather they have a sndfile.h.in. This is an input file for autoconf, which is a tool that will generate the proper header file from this template. It has currently the following definition for sf_count_t for linux systems (and had it since a while):
typedef #TYPEOF_SF_COUNT_T# sf_count_t ;
The #TYPEOF_SF_COUNT_T# would be replaced by autoconf to generate a header with a working type for sf_count_t for the system that is going to be build for. The header file provided by nyquist is therefore already configured (presumably for the system of the author).
off_t is a type specified by the POSIX standard and defined in the system's libc. Its size on a system using the GNU C library is 32bit if the system is 32bit.
This causes the sanity check in question to fail, because the sizes of sf_count_t and off_t don't match. The error message is also correct, as we are using an unfittingly configured sndfile.h for the build.
As I see it you have the following options:
Ask the nyquist author to provide the unconfigured sndfile.h.in and to use autoconf to configure this file at build time.
Do not use the bundled libsndfile and link against the system's one. (This requires some knowledge and work to change the build scripts and header files, maybe additional unexpected issues)
If you are using the GNU C library (glibc): The preprocessor macro _FILE_OFFSET_BITS can be set to 64 to force the size of off_t and the rest of the file interface to use the 64bit versions even on 32bit systems.
This may or may not work depending on whether your system supports it and it is not a clean solution as there may be additional misconfiguration of libsndfile going unnoticed. This flag could also introduce other interface changes that the code relies on, causing further build or runtime errors/vulnerabilities.
Nonetheless, I think the syntax for cmake would be to add:
add_compile_definitions(_FILE_OFFSET_BITS=64)
or depending on cmake version:
add_definitions(-D_FILE_OFFSET_BITS=64)
in the appropriate CMakeLists.txt.
Actually the README in nyquist/nylsf explains how the files for it were generated. You may try to obtain the source code of the same libsndfile version it is based on and repeat the steps given to produce an nylsf configured to your system. It may cause less further problems than 2. and 3. because there wouldn't be any version/interface changes introduced.

C/pre-processor: detect if a __builtin function is available

Is it possible to somehow determine whether an intrinsic function, such as __builtin_bswap16 is provided by the compiler? Preferably, I would like to be able to determine whether this function exists using just preprocessor.
In my particular case, I was using __builtin_bswap16 / 32 / 64 functions in my code which worked fine with GCC 4.x when compiling for 32-bit. Later I switched to a 64-bit Linux and noticed that __builtin_bswap16 suddenly disappeared - I received a linker error:
"undefined reference to `__builtin_bswap16'".
I guess this has something to do with the availability of certain ASM operations in 64-bit mode.
On a later occasion I was trying to compile this code on a different machine where unfortunately only an older version of GCC is installed and does not support these functions at all.
I would like to make this code compilable everywhere, using __builtin_bswap functions if provided, and fall back to hand-coded byteswap routine if not. Is it possible to achieve this somehow with just preprocessor?
My obvious attempt, e.g.:
...
#define MYBSWAP16(v) (v>>8)|(v<<8)
#ifdef __builtin_bswap16
printf("bswap16 is defined : %04x\n", __builtin_bswap16(0x1234));
#else
printf("bswap16 is not defined : %04x\n", MYBSWAP16(0x1234) );
#endif
...
was not successful, as __builtin_bswap16/32/64 are always evaluated to be undefined. Is there any way to make it work automatically within the C source, or is the only way to manually define constants in the Makefile, e.g. HAVE_BSWAP and pass them via -D option?
Please note that my question is not necessarily specific to __builtin_bswap, I'm looking for a general way to detect if the certain functions are available.
Unavailability of __builtin_bswap16 is a gcc bug which was fixed in gcc 4.8.
Sincw it is missing from some versions of gcc you can always add it to your code yourself :
static inline unsigned short __builtin_bswap16(unsigned short a)
{
return (a<<8)|(a>>8);
}

Structure definition in header file for a library and compilation differences

I have a code which is compiled into a library (dll, static library and so). I want the user of this library to use some struct to pass some data as parameters for the library function. I thought about declaring the struct in the API header file.
Is it safe to do so, considering compilation with different compilers, with respect to structure alignment or other things I didn't think about?
Will it require the usage of the same compiler (and flags) for both the library and its user?
Few notes:
I considered giving the user a pointer and set all the struct via functions in the library, but this will make the API really not comfortable to use.
This question is about C, although it would be nice to know if there's a difference in c++.
If it's a regular/static library, the library and application should be compiled using the same compiler. There're a few reasons for this that I can think of:
Different compilers (as in different brands or compilers for different platforms) normally don't understand each other's object and library formats.
You don't want to compile different parts of the same program using different types (e.g. signed vs unsigned char), type sizes (e.g. long = 32 vs 64 bits), alignment and packing and probably some other things, all of which are allowed by the C standard to vary. Mixing and matching those things is usually a bad thing.
You may, however, often use slightly different versions of the same compiler to compile the library and the application using it. Usually, it's OK. Sometimes there're changes that break the code, though.
You may implement some "initialization" function in that header file (declared as static inline) that would ensure that types, type sizes, alignment and packing are the same as expected by the compiled library. The application using this library would have to call this function prior to using any other part of the library. If things aren't the same as expected, the function must fail and cause program termination, possibly with some good textual description of the failure. This won't solve completely the problem of having somewhat incompatible compilers, but it can prevent silent and mysterious malfunctions. Some things can be checked with the preprocessor's #if and #ifdef directives and cause compilation errors with #error.
In addition, structure packing problems can be relieved by inserting explicit padding bytes into structure declarations and forcing tight packing (by e.g. using #pragma pack, which is supported by many compilers). That way if type sizes are the same, it won't matter what the default packing is.
You can apply the same to DLLs as well, but you should really expect that the calling application has been compiled with a different compiler and not depend on the compilers being the same.
All Windows APIs throw structs around like crazy so obviously this is something that is done every day and it works. Of course it doesn't mean that your concerns are not valid :)
I would suggest making your structure's fields have explicit width types (int32_t etc) and maybe specify explicitly that that the packing in a way which would break on any compiler but yours, i.e.
#if defined(_MSC_VER)
#pragma pack(0)
#elif defined ... handle gcc
#else
FAIL // fail compilation on unsupported platform
#endif

__udivdi3 undefined — how to find the code that uses it?

Compiling a kernel module on 32-Bit Linux kernel results in
"__udivdi3" [mymodule.ko] undefined!
"__umoddi3" [mymodule.ko] undefined!
Everything is fine on 64-bit systems. As far as I know, the reason for this is that 64-bit integer division and modulo are not supported inside a 32-bit Linux kernel.
How can I find the code issuing the 64-bit operations. They are hard to find manually because I cannot easily check if an "/" is 32-bit wide or 64-bit wide. If "normal" functions are undefined, I can grep them, but this is not possible here. Is there another good way to search the references? Some kind of "machine code grep"?
The module consists of some thousand lines of code. I can really not check every line manually.
First, you can do 64 bit division by using the do_div macro. (note the prototype is uint32_t do_div(uint64_t dividend, uint32_t divisor) and that "dividend" may be evaluated multiple times.
{
unsigned long long int x = 6;
unsigned long int y = 4;
unsigned long int rem;
rem = do_div(x, y);
/* x now contains the result of x/y */
}
Additionally, you should be able to either find usage of long long int (or uint64_t) types in your code, or alternately, you can build your module with the -g flag and use objdump -S to get a source annotated disassembly.
note: this applies to 2.6 kernels, I have not checked usage for anything lower
Actually, 64-bit integer divison and modulo are supported within a 32-bit Linux kernel; however, you must use the correct macros to do so (which ones depend on your kernel version, since recently new better ones were created IIRC). The macros will do the correct thing in the most efficient way for whichever architecture you are compiling for.
The easiest way to find where they are being used is (as mentioned in #shodanex's answer) to generate the assembly code; IIRC, the way to do so is something like make directory/module.s (together with whatever parameters you already have to pass to make). The next easiest way is to disassemble the .o file (with something like objdump --disassemble). Both ways will give you the functions where the calls are being generated (and, if you know how to read assembly, a general idea of where within the function the division is taking place).
After compilation stage, you should be able to get some documented assembly, and see were those function are called. Try to mess with CFLAGS and add the -S flags.
Compilation should stop at the assembly stage. You can then grep for the offending function call in the assembly file.

Resources