I have a static library of C files, compiled with g++ on Cygwin. I wish to unit test one function that is defined in the library. That function calls another function defined in that library and I wish to override the dependency to replace it with my own version of that function. I can't modify what's in the static library, so this solution [ Override a function call in C ] doesn't apply.
Usually, I can write a .cpp file and include the .c file containing the function I want to unit test, which essentially extends that file with the code I add. It's a dirty trick I'd never use for production code but it's handy for unit testing C files, because it gives my test code access to static things in that C file. Then, I can write in my fake version of the dependency, and my unit test function that calls the function I'm testing. I compile my.cpp to get my.o, then link it with the static library. In theory, since the linker has found a definition for the dependency already (the one I provide) it won't look in the library and there will be no conflict. Usually this works, but now I'm getting a "multiple definition" error where the linker first finds my fake and then finds the real one. I don't know what might cause this and don't know what to look for. I also can't boil this down to a simple example because my simple examples don't have this problem.
Ideas please?
One possibility (admittedly, and ugly one, but...) is to extract the individual object files from the static library. If the function you're calling and the function it's calling are in separate object files, you can link against the object file containing the function you need to call, but not against the one containing the function it calls.
This only gives you granularity on the level of complete object files though, so if the two functions involved are both in the same object file, it won't work. If you really need to get things to work, and don't mind making a really minor modification to the object file in question, you may be able to use a binary editor to mark the second function as a weak external, which means it'll be used in the absence of any other external with the same name, but if another is provided, that will be used instead.
Whether that latter qualifies as "modifying the library" or not depends a bit on your viewpoint. It's not modifying the code in the library, but is modifying a bit of the object file wrapper around that code. My guess is that you'd rather not do it, but it may still be the cleanest way out of an otherwise untenable situation.
It turns out the reason the linker found both defintions of the function is that the faked function's source file defined a variable which is extern'ed in its header file. That unresolved external in the header file caused the linker to link the faked function's object file (the whole thing) to the tested function's file inside the library. So, it's impossible to extract the definition of the tested function without the definition for the dependency.
What I ended up doing was similar to Override a function call in C where I used a different function name instead of the same one, and a preprocessor directive to swap the two. I put the preprocessor directive and the fake function in a separate file which can be included in a unit test, so the production code in the library does not have to be touched. Plus, if I want to fake that same function for another unit test somewhere else, I can re-use the new file.
Depending on your platform and performance requirements, you might be able to use pin to dynamically modify the application and replace one function with another at runtime.
There's no direct example in the manual, but you could easily modify one of the sample pin tools to do this.
Related
Background
I'm building a C application for an embedded Cortex M4 TI-RTOS SYS/BIOS target, however this question should apply to all embedded targets where a single binary is loaded onto some microprocessor.
What I want
I'd like to do some in situ regression tests on the target where I just replace a single function with some test function instead. E.g. a GetAdcMeasurement() function would return predefined values from a read-only array instead of doing the actual measurement and returning that value.
This could of course be done with a mess of #ifndefs, but I'd rather keep the production code as untouched as possible.
My attempt
I figure one way to achieve this would be to have duplicate symbol definitions at the linker stage, and then have the linker prioritise the definitions from the test suite (given some #define).
I've looked into using LD_PRELOAD, but that doesn't really seem to apply here (since I'm using only static objects).
Details
I'm using TI Code Composer, with TI-RTOS & SYS/BIOS on the Sitara AM57xx platform, compiling for the M4 remote processor (denoted IPU1).
Here's the path to the compiler and linker
/opt/ti/ccsv7/tools/compiler/ti-cgt-arm_16.9.6.LTS/bin/armcl
One solution could be to have multiple .c files for each module, one the production code and one the test code, and compile and link with one of the two. The globals and function signatures in both .c file must be at least the same (at least: there may be more symbols but not less).
Another solution, building on the previous one, is to have two libraries, one with the production code and one with the test code, and link with one of both. You could ieven link with both lubraries, with the test version first, as linkers often resolve symbols in the order they are encountered.
And, as you said, you could work with a bunch of #ifdefs, which would have the advantage of having just one .c file, but making tyhe code less readable.
I would not go for #ifdefs on the function level, i.e. defining just one function of a .c file for test and keeping the others as is; however, if necessary, it could be away. And if necessary, you could have one .c file (two) for each function, but that would negate the module concept.
I think the first approach would be the cleanest.
One additional approach (apart from Paul Ogilvie's) would be to have your mocking header also create a define which will replace the original function symbol at the pre-processing stage.
I.e. if your mocking header looks like this:
// mock.h
#ifdef MOCKING_ENABLED
adcdata_t GetAdcMeasurement_mocked(void);
stuff_t GetSomeStuff_mocked(void);
#define GetAdcMeasurement GetAdcMeasurement_mocked
#define GetSomeStuff GetSomeStuff_mocked
#endif
Then whenever you include the file, the preprocessor will replace the calls before it even hits the compiler:
#include "mock.h"
void SomeOtherFunc(void)
{
// preprocessor will change this symbol into 'GetAdcMeasurement_mocked'
adcdata_t data = GetAdcMeasurement();
}
The approach might confuse the unsuspected reader of your code, because they won't necessarily realize that you are calling a different function altogether. Nevertheless, I find this approach to have the least impact to the production code (apart from adding the include, obviously).
(This is a quick sum up the discussion in the comments, thanks for answers)
A function can be redefined if it has the weak attribute, see
https://en.wikipedia.org/wiki/Weak_symbol
On GCC that would be the weak attribute, e.g.
int __attribute__((weak)) power2(int x);
and on the armcl (as in my question) that would be the pragma directive
#pragma weak power2
int power2(int x);
Letting the production code consist of partly weak functions will allow a test framework to replace single functions.
I am trying to use a callback function. This has worked fine when the caller and the called function were in the same file. I have lately decided to make the called function part of a library, so I have it declared in a header file, defined in its own file. I #include the new header in the calling source file, linking to the new library, and now I get an "Undefined reference" error to the callback function.
Is there something special I have to do to make this work? I notice when I use the same thing in pthread libraries for example, the callback function is defined as a pointer function.
Edit: I am linking to the library, and I can call other functions in the library just fine.
Possible problems:
the definition of the function (in the .c file) does not coincide with its declaration (in the .h file) and the code using this function essentially tells the linker to go and find what's in the header file and not what's actually in the library.
you have forgotten to compile the file implementing the function or put the resulting object file into the library and so the linker can't find the function in the library.
you have some source files open and unsaved and while they look fine and complete in the editor, their on-disk contents is different and something is amiss when you try to compile the code.
you are having some issues with make (bad makefile?) making it think that either the file implementing the function does not need to be compiled or it has already been compiled and needs no recompilation. Fixing the makefile and/or deleting the object and library files will fix the problem.
you have mixed C and C++ code and are having issues because of C++ name mangling. Using extern "C" { } may help here.
you have defined that function as static and so it's invisible in other modules (.c files) at link time. Removing static will help.
Turns out the problem was my header had the definition in there twice, one with static, one without.
You need to link to the actual code that implements the callback, including a header isn't enough.
So, it sounds as if your application needs to link to the library, which it of course already should be doing in order to call functions in the library.
I am new to delphi. I was trying to add C Object files in my Delphi project and link them directly since Delphi Supports C Object Linking. I got it working when i link a single Object file. But when i try to link multiple object files, i am getting error 'Unsatisfied forward or external declaration'. I have tried this in Delphi 2007 as well as XE.So what am i doing wrong here?
Working Code:
function a_function():Integer;cdecl;
implementation
{$Link 'a.obj'}
function a_function():Integer;cdecl;external;
end.
Error Code:
function a_function():Integer;cdecl;
function b_function();Integer;cdecl;
function c_function();Integer;cdecl;
implementation
{$LINK 'a.obj'}
{$LINK 'b.obj'}
{$LINK 'c.obj'}
function a_function():Integer;cdecl;external;
function b_function();Integer;cdecl;external;
function c_function();Integer;cdecl;external;
end.
As an aside, the article linked by #vcldeveloper has a good explanation of some of the common issues. The trick of providing missing C RTL functions in Pascal code is excellent and much quicker than trying to link in the necessary functions as C files, or even as .obj files.
However, I have a suspicion that I know what is going on here. I use this same approach but in fact have over 100 .obj files in the unit. I find that when I add new ones, I get the same linker error as you do. The way I work around this is to try re-ordering my $LINK instructions. I try to add the new obj files one by one and I have always been able, eventually, to get around this problem.
If your C files are totally standalone then you could put each one in a different unit and the linker would handle that. However, I doubt that is the case and indeed I suspect that if they really were standalone then this problem would not occur. Also, it's desirable to have the $LINK instructions in a single unit so that any RTL functions that need to be supplied can be supplied once and once only (they need to appear in the same unit as the $LINK instructions).
This oddity in the linker was present in Delphi 6 and is present in Delphi 2010.
EDIT 1: The realisation has now dawned on me that this issue is probably due to Delphi using a single pass compiler. I suspect that the "missing external reference" error is because the compiler processes the .obj files in the order in which they appear in the unit.
Suppose that a.obj appears before b.obj and yet a.obj calls a function in b() b.obj. The compiler wouldn't know where b() resides at the point where it needs to fixup the function call. When I find the time, I going to try and test if this hypothesis is at the very least plausible!
Finally, another easy way out of the problem would be to combine a.c, b.c and c.c into a single C file which would I believe bypass this issue for the OP.
Edit 2: I found another Stack Overflow question that covers this ground: stackoverflow.com/questions/3228127/why-does-the-order-of-linked-object-file-with-l-directive-matter
Edit 3: I have found another truly wonderful way to work around this problem. Every time the compiler complains
[DCC Error] Unit1.pas(1): E2065 Unsatisfied forward or external declaration: '_a'
you simply add, in the implementation section of the unit, a declaration like so:
procedure _a; external;
If it is a routine that you wish to call from Delphi then you clearly need to get the parameter list, calling conventions etc. correct. Otherwise, if it is a routine internal to the external code, then you can ignore the parameter list, calling conventions etc.
To the best of my knowledge this is the only way to import two objects that refer to each other in a circular manner. I believe that declaring an external procedure in this way is akin to making a forward declaration. The difference is that the implementation is provided by an object rather than Pascal code.
I've now been able to add a couple of more tools to my armory – thank you for asking the question!
If i have two object files both defining a symbol (function) "foobar".
Is it possible to tell the linker to obey the obj file order i give in the command line call and always take the symbol from first file and never the later one?
AFAIK the "weak" pragma works only on shared libraries but not on object files.
Please answer for all the C/C++ compiler/linker/operating system combinations you know cause i'm flexibel and use a lot of compiles (sun studio, intel, msvc, gcc, acc).
I believe that you will need to create a static library from the second object file, and then link the first object file and then the library. If a symbol is resolved by an object file, the linker will not search the libraries for it.
Alternatively place both object files in separate static libraries, and then the link order will be determined by their occurrence in the command line.
Creating a static library from an object file will vary depending on the tool chain. In GCC use the ar utility, and for MSVC lib.exe (or use the static library project wizard).
There is a danger here, the keyword here is called Interpositioning dependant code.
Let me give you an example here:
Supposing you have written a custom routine called malloc. And you link in the standard libraries, what will happen is this, functions that require the usage of malloc (the standard function) will use your custom version and the end result is the code may become unstable as the unintended side effect and something will appear 'broken'.
This is just something to bear in mind.
As in your case, you could 'overwrite' (I use quotes to emphasize) the other function but then how would you know which foobar is being used? This could lead to debugging grief when trying to figure out which foobar is called.
Hope this helps,
Best regards,
Tom.
You can make it as a .a file... Then the compiler gets the symbol and doesn't crib later
I have a fair amount of practice with Java as a programming language, but I am completely new to C. I understand that a header file contains forward declarations for methods and variables. How is this different from an abstract class in Java?
The short answer:
Abstract classes are a concept of object oriented programming. Header files are a necessity due to the way that the C language is constructed. It cannot be compared in any way
The long answer
To understand the header file, and the need for header files, you must understand the concepts of "declaration" and "definition". In C and C++, a declaration means, that you declare that something exists somewhere, for example a function.
void Test(int i);
We have now declared, that somewhere in the program, there exists a function Test, that takes a single int parameter. When you have a definition, you define what it is:
void Test(int i)
{
...
}
Here we have defined what the function void Test(int) actually is.
Global variables are declared using the extern keyword
extern int i;
They are defined without the extern keyword
int i;
When you compile a C program, you compile each source file (.c file) into an .obj file. Definitions will be compiled into the .obj file as actual code. When all these have been compiled, they are linked to the final executable. Therefore, a function should only be defined on one .c file, otherwise, the same function will end up multiple times in the executable. This is not really critical if the function definitions are identical. It is more problematic if a global variable is linked into the same executable twice. That will leave half the code to use the one instance, and the other half of the code to use the other instance.
But functions defined in one .c file cannot see functions defined in another .c files. So if from file1.c file you need to access function Test(int) defined in file2.c, you need to have a declaration of Test(int) present when compiling file1.c. When file1.c is compiled into file1.obj, the resulting .obj file will contain information that it needs Test(int) to be defined somewhere. When the program is linked, the linker will identify that file2.obj contains the function that file1.obj depends on.
If there is no .obj file containing the definition for this function, you will get a linker error, not a compiler error (linker errors are considerably more difficult to find and correct that compiler errors because you get no filename and line number for the resulting file)
So you use the header file to store declarations for the definitions stored in the corresponding source file.
IMO it's mainly because many C programmers seem to think that Java programmers don't know how to program “for real”, e.g. handling pointers, memory and so on.
I would rather compare headers to Java interfaces, in the sense that they generally define how the API must be used.
Headers are basically just a way to avoid copy-pasting: the preprocessor simply includes the content of the header in the source file when encounters an #include directive.
You put in a header every declaration that the user will commonly use.
Here's the answers:
Java has had a bad reputation among some hardcore C programmers mainly because they think:
it's "too easy" (no memory-management, segfaults)
"can't be used for serious work"
"just for the web" or,
"slow".
Java is hardly the easiest language in the world these days, compared to some lanmguages like Python, etc.
It is used in many desktop apps - applets aren't even used that often. Finally, Java will always be slower than C, because it is not compiled directly to machine code. Sometimes, though, extreme speed isn't needed. Anyway, the JVM isn't the slowest language VM ever.
When you're working in C, there aren't abstract classes.
All a header file does is contain code which is pasted into other files. The main reason you put it in a header file is so that it is at the top of the file - this way, you don't need to care where you put your functions in the actual implementation file.
While you can kind-of use OO concepts in C, it doesn't have built-in support for classes and similar fundamentals of OO. It is nigh-impossible to implement inheritance in plain C, therefore there can never actually have OO, or abstract classes for that matter. I would suggest sticking to plain old structs.
If it makes it easier for you to learn, by all means think of them as abstract classes (with the implementation file being the inheriting class) - but IMHO it is a difficult mindset to use when for working in a language without explicit support of said features.
I'm not sure if Java has them, but I think a closer analogue could be partial classes in C#.
If you forward declare something, you have to actually deliver and implement it, else the compiler will complain. The header allows you to display a "module"'s public API and make the declarations available (for type checking and so) to other parts of the program.
Comprehensive reading: Learning C from Java. Recommended reading for developers who are coming from Java to C.
I think that there is much derision (mockery, laughter, contempt, ridicule) for Java simply because it's popular.
Abstract classes and interfaces specify a contract or a set of functions that can be invoked on an object of a certain type. Function prototypes in C only really do compile time type checking of function arguments/return values.
While your first question seems subjective to me, I will answer to the second one:
A header file contains the declarations which are then made available to other files via #inclusion by the preprocessor.
For instance you will declare in a header a function, and you will implement in a .c file. Other files will be able to use the function so long they can see the declaration (by including the header file).
At linking time the linker will look among the object files, or the various libraries linked, for some object which provides the code for the function.
A typical pattern is: you distribute the header files for your library, and a dll (for instance) which contains the object code. Then in your application you include the header, and the compiler will be able to compile because it will find the declaration in the header. No need to provide the actual implementation of the code, which will be available for the linker through the dll.
C programs run directy, while Java programs run inside the JVM, so a common belief is that Java programs are slow. Also in Java you are hidden from some low level constructs (pointer, direct memory access), memory management, etc...
In C the declaration and definition of a function is separated. Declaration "declares" that there exists a function that called by those arguments returns something. Definition "defines" what the function actually does. The former is done in header files, the latter in the actual code. When you are compiling your code, you must use the header files to tell your compiler that there is such a function, and link in a binary that contains the binary code for the function.
In Java, the binary code itself also contains the declaration of the functions, so it is enough for the compiler to look at the class files to get both the definition and declaration of the available functions.