The memory layout of a struct is up to the compiler. So what happens when some code compiled by one compiler uses a struct generated by code compiled by another compiler?
For example, say I have a header file that declares a struct somestruct, and a function that returns the struct. One source file defines that function and is compiled by compiler A. Another source file uses than function and is compiled by compiler B and links against the binary of the other source file.
If the two compilers create two different layouts for somestruct, then what's the layout of the variable returned by the function? Does it defer to one compiler's layout, or will there be a memory bug when the second source file tries to access elements of the struct returned by the first source file? Is it an error at compile time or link time?
The function will return a structure as specified by the ABI of the compiler of the function. The callee compiler, will just treat the function as if it conforms to the ABI of itself.
Assuming the two compilers use a similar ABI, in most cases, no errors will be reported during compile-time or link time or even during runtime. For some compatible compilers like Clang, GCC, and Intel C Compiler on OS X and Linux, no errors should result (if there are errors then it's a bug of the compiler). However in real world it is usually difficult to find fully compatible compilers (in most cases their ABIs are similar but not exactly the same; such ABI errors will be even harder to track down because your app would appear normal and crashes under some really weird circumstances are encountered during runtime).
Just as Basile said, name mangling for C++ poses an additional difference in ABI, but such differences are more easily caught during compile time as the linker literally can't find the symbol of the function, rather than finding a function that is not compatible.
Also, passing structures is another headache in terms of ABI because there are multiple structure-packing ABIs, sometimes even different in "compatible" compilers like GCC/MinGW and MSVC. (See also the -m[no-]ms-bitfields option in GCC, which forces GCC to use the MSVC ABI for structures.) I have also seen some cases where passing structures by pointer is more reliable than passing structures by value.
The layout of data (e.g. structures etc...), and the call protocol (how are call done at the processor level) are defined in a (processor and operating system specific) document called Application Binary Interface. If both compilers are following the same ABI (for the same processor and the same operating system) their generated code should be interoperable.
See e.g. the wikipage for x86 calling conventions and the x86-64 ABI specification.
Name mangling, notably for C++, might also be an issue.
Read also Levine's book on Linkers and Loaders
Related
I am wondering how mixing C and assembly can be possible as compilers generate code in different ways, for example many C compilers will use registers rather than pushing to the stack while making a function call, These functions will then move those registers into the appropiate memory locations because of this what if you write assembly code or link with an object file created by a different compiler that will call the C function but instead push the arguments to the stack rather than set the registers.
My guess is the C compiler assembly output has done it in such a clever way that it doesn't make a difference and it will still work but I can't be sure looking at the assembly code it doesn't appear it would work.
Can anyone answer my question as I am writing a compiler and need to know this so I don't make any mistakes should I want to link with a C module in the future.
The conventions that are used for calling functions are part of what's called the "application binary interface" (ABI). If this interface is specified, then all code that follows the specification can be linked together.
There is no standard ABI for C. However, most popular platforms have one prevailing C compiler that effectively produces a de-facto standard ABI (e.g. there's one for Windows, one for Linux on x86 (32 and 64 bit), one for Linux on ARM, etc.). ABIs may specify a large number of separate "calling conventions", and your C compiler will typically let you specify the desired convention at the point of function declaration using some vendor extension.
Conversely, if there is no documented ABI for your C compiler, or for an existing bit of object code, then you cannot in general link (or otherwise interact) with it successfully.
Say I have two compilers, or even a single compiler with two different option sets. Each compiler compiles some C code into an object and I try to link the two .o files with a common linker. Will this succeed?
My initial thought is: not always. If the compilers are using the same object file format and have compatible options, then it would succeed. But, if the compilers have conflicting options, or (and this is an easy one) are using two different object file formats, it would not work correctly.
Does anyone have more insight on this? What standards would the object files need to comply with to gain confidence that this will work?
Most flavors of *nix OSes have well defined and open ABI and mostly use ELF object file format, so it is not a problem at all for *nix.
Windows is less strictly defined and different compilers may vary in some calling conventions (for example __fastcall may not be supported by some compilers or may have different behavior, see https://en.wikipedia.org/wiki/X86_calling_conventions). But main set of calling conventions (__stdcall, _cdecl, etc) is standard enough to ensure successfull call of function compiled by one compiler from another compiler, otherwise the program won't work at all, since unlike Linux every system call in Windows is wrapped by function from DLL which you need to successfully call.
The other problem is that there is no standard common format for object files. Although most tools (MS, Intel, GCC (MinGW), Clang) use COFF format, some may use OMF (Watcom) or ELF (TinyC).
Another problem is so called "name mangling". Although it was introduced to support overloading C++ functions with the same name, it was adopted by C compilers to prevent linkage of functions defined with different calling conventions. For example, function int _cdecl fun(void); will get compiled name _fun whilst int __stdcall fun(void); will get name _fun#0. More information on name mangling see here: https://en.wikipedia.org/wiki/Name_mangling.
At last, default behavior may differ for some compilers, so yes, options may prevent successful linking of object files produced by different compilers or even by the same compiler. For example, TinyC use default convention _cdecl, whilst CLang use __stdcall. TinyC with default options may not produce code that may be linked with other because it doesn't prepend name by underscore sign. To make it cross-linkable it needs -fleading-underscore option.
But keeping in mind all said above the code may successfully be intermixed. For example, I successfully linked together code produced by Visual Studio, Intel Parallel Studio, GCC (MinGW), Clang, TinyC, NASM.
If you've ever linked a kernel with gcc you may know the parameter -lgcc.
Is this parameter important ? What does it do ?
If you do some driver/kernel dev, you may use the -nostdlib to remove your module from the bloated stdlib. However, you also remove all the internal hacks GCC has in order to have a consistent behaviour on a whole range of hardware.
http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/Link-Options.html
-nostdlib
Do not use the standard system startup files or libraries when linking. No startup files and only the libraries you specify will be
passed to the linker, options specifying linkage of the system
libraries, such as -static-libgcc or -shared-libgcc, will be ignored.
The compiler may generate calls to memcmp, memset, memcpy and memmove.
These entries are usually resolved by entries in libc. These entry
points should be supplied through some other mechanism when this
option is specified.
One of the standard libraries bypassed by -nostdlib and -nodefaultlibs is libgcc.a, a library of internal subroutines that GCC uses to overcome shortcomings of particular machines, or special needs
for some languages. (See Interfacing to GCC Output, for more
discussion of libgcc.a.) In most cases, you need libgcc.a even when
you want to avoid other standard libraries. In other words, when you
specify -nostdlib or -nodefaultlibs you should usually specify -lgcc
as well. This ensures that you have no unresolved references to
internal GCC library subroutines. (For example, `__main', used to
ensure C++ constructors will be called; see collect2.)
https://gcc.gnu.org/onlinedocs/gcc-4.6.1/gccint/Interface.html#Interface
3 Interfacing to GCC Output
GCC is normally configured to use the same function calling convention
normally in use on the target system. This is done with the
machine-description macros described (see Target Macros).
However, returning of structure and union values is done differently
on some target machines. As a result, functions compiled with PCC
returning such types cannot be called from code compiled with GCC, and
vice versa. This does not cause trouble often because few Unix library
routines return structures or unions.
GCC code returns structures and unions that are 1, 2, 4 or 8 bytes
long in the same registers used for int or double return values. (GCC
typically allocates variables of such types in registers also.)
Structures and unions of other sizes are returned by storing them into
an address passed by the caller (usually in a register). The target
hook TARGET_STRUCT_VALUE_RTX tells GCC where to pass this address.
By contrast, PCC on most target machines returns structures and unions
of any size by copying the data into an area of static storage, and
then returning the address of that storage as if it were a pointer
value. The caller must copy the data from that memory area to the
place where the value is wanted. This is slower than the method used
by GCC, and fails to be reentrant.
On some target machines, such as RISC machines and the 80386, the
standard system convention is to pass to the subroutine the address of
where to return the value. On these machines, GCC has been configured
to be compatible with the standard compiler, when this method is used.
It may not be compatible for structures of 1, 2, 4 or 8 bytes.
GCC uses the system's standard convention for passing arguments. On
some machines, the first few arguments are passed in registers; in
others, all are passed on the stack. It would be possible to use
registers for argument passing on any machine, and this would probably
result in a significant speedup. But the result would be complete
incompatibility with code that follows the standard convention. So
this change is practical only if you are switching to GCC as the sole
C compiler for the system. We may implement register argument passing
on certain machines once we have a complete GNU system so that we can
compile the libraries with GCC.
On some machines (particularly the SPARC), certain types of arguments
are passed “by invisible reference”. This means that the value is
stored in memory, and the address of the memory location is passed to
the subroutine.
If you use longjmp, beware of automatic variables. ISO C says that
automatic variables that are not declared volatile have undefined
values after a longjmp. And this is all GCC promises to do, because it
is very difficult to restore register variables correctly, and one of
GCC's features is that it can put variables in registers without your
asking it to.
Assuming the following header file corresponding to, for example, a shared library. The exported function takes a pointer to a custom structure defined in this header:
// lib.h
typedef struct {
char c;
double d;
int i;
} A;
DLL_EXPORT void f(A* p);
If the shared library is built using one compiler and then is used from C code built with another compiler it might not work because of a different memory alignment, as Memory alignment in C-structs suggests. So, is there a way to make my structure definition portable across different compilers on the same platform?
I am interested specifically in Windows platform (apparently it does not have a well-defined ABI), though would be curious to learn about other platforms as well.
TL;DR in practice you should be fine.
The C standard does not define this but a platform ABI generally does. That is, for a given CPU architecture and operating system, there can be a definition for how C maps to assembly that allows different compilers to interoperate.
Struct alignment isn't the only thing that a platform ABI has to define, you also have function calling conventions and stuff like that.
C++ makes it even more complex and the ABI has to specify vtables, exceptions, name mangling, etc.
On Windows I think there are multiple C++ ABIs depending on compiler but C is mostly compatible across compilers. I could be wrong, not a Windows expert.
Some links:
what is an ABI? http://gcc.gnu.org/ml/libstdc++/2001-11/msg00063.html
things an ABI has to define C++ ABI issues list
example C++ ABI spec http://sourcery.mentor.com/public/cxx-abi/abi.html
how the ABI evolved on Solaris http://developers.sun.com/solaris/articles/CC_abi/CC_abi_content.html
Anyway the bottom line is that you're looking for your guarantee in the platform/compiler ABI spec, not the C standard.
The only way to know for sure is to consult the documentation of the compilers in question. However, it is usually the case that C struct layout (except, as you say, for bitfields) is defined by an ABI description for the environment you're using, and C compilers will tend to follow the native ABI.
Not only that it is not guarantied, but even if you use the same compiler there might be differences due to different compiler switches used in the build, or if you use different versions of the same compiler and same switches (happened in an embedded compiler I worked on).
You need to make make sure the structs are represented exactly the same, use switches, #pragmas, whatever the compilers gives you.
My advice - to stay way from this altogether. Pass your arguments in the function, not wrapped within a struct.
And even in this simple form, if you deal with two compilers, it's not trivial. You need to make sure that an int takes the same number of bytes, for example. Also calling conevntion - arguments order - from left to right or from right to left - can differ between compiler.
I have some assembly routines that are called by and take arguments from C functions. Right now, I'm assuming those arguments are passed on the stack in cdecl order. Is that a fair assumption to make?
Would a compiler (GCC) detect this and make sure the arguments are passed correctly, or should I manually go and declare them cdecl? If so, will that attribute still hold if I specify a higher optimisation level?
Calling conventions mean much more than just argument ordering. There is a good pdf explaining all the details, written by Agner Fog: Calling conventions for different C++ compilers and operating systems.
This is a matter of the ABI for the platform you're writing code for. Almost all platforms follow the Unix System V ABI for C calling convention and other ABI issues, which includes both a general ABI (gABI) document detailing the common ABI characteristics across all CPU architectures, and a processor-specific ABI (psABI) document specific to the particular CPU architecture/family. When it comes to x86, this matches what you refer to as "cdecl". So from a practical standpoint, x86 assembly meant to be called from C should be written to assume "cdecl". Basically the only exception to the universality of this calling convention is Windows API functions, which use their own nonstandard "stdcall" calling convention due to legacy Win16 dll thunk compatibility issues; nonetheless, the "default" calling convention on x86 Windows is still "cdecl".
A more important concern when writing asm to be called from C is whether symbol names should be prefixed with an underscore or not. This varies widely between platforms, with the general trend being that ELF-based platforms don't use the prefix, and most other platforms do...
The quick and dirty way to do it is create a dummy C function that matches the asm function you want to implement, do a few things in the dummy C function with the passed in parameters so you can tell them apart, compile then disassemble. Not foolproof but works often.