In our environment we're encountering a problem regarding mocking functions for our library unit tests.
The thing is that instead of mocking whole modules (.c files) we would like to mock single functions.
The library is compiled to an archive file and linked statically to the unit test. Without mocking there isn't any issue.
Now when trying to mock single functions of the library we would get multiple definitions obviously.
My approach now is to use the weak function attribute when compiling/linking the library so that the linker takes the mocked (non-weak) function when linking against the unit test. I already tested it and it seems to work as expected.
The downside of this is that we need many attribute declarations in the code.
My final approach would be to pass some compile or link arguments to the compiler, that every function is automatically declared as a weak symbol.
The question now is: Is there anything to do this in a nice way?
btw: We use clang 8 as a compiler.
James Grenning describes several options to solve this problem (http://blog.wingman-sw.com/linker-substitution-in-c-limitations-and-workarounds). The option "function pointer substitution" gives a high degree of freedom. It works as follows: Replace functions by pointers to functions. The function pointers are initialized to point to the original function, but each pointer can be redirected individually to a test double.
This approach allows to have one single test executable where you can still decide for each test case individually for which function you use a test double and for which you use the original function.
It certainly also comes at a price:
One indirection for each call. But, if you use link-time-optimization the optimizer will most likely eliminate that indirection again, so this may not be an issue.
You make it possible to redirect function calls also in production code. This would certainly be a misuse of the concept, however.
I would suggest using VectorCAST
https://www.vector.com/us/en/products/products-a-z/software/vectorcast/
I've used, unity/cmock and others for unit testing C in the past, but after a while its vary tedious to manually create these for a language that isnt really built around that concept and is very much a heres a Hammer and Chissel the world is yours approach.
VectorCAST abstracts majority of the manual work that is required with tools like Unity/Cmock, we can get results across a project/module sooner and quicker than we did in the past with the other tools.
Is vectorCAST expensive and very much an enterprise level tool? yes... but its defiantly worth its weight in gold. And thats coming from someone who is very old school, manual approach to software development... just text editors, terminals and commandline debuggers.
VetorCAST handles function pointers and pointers extremely well, stubbing functions is easy as two clicks away. It saved our team alot of time... allowing us to focus on results and reducing the feedback loop of development.
Related
I create a project for a microcontroller by programming it in C language. Due to its specificity (a microcontroller with a built-in BLE core), I have to use the SDK and a specific project template. How can I test my modules when they have numerous references to other files (modules) in the SDK? (References are needed to use functions to, for example, send data via BLE) Do I have to somehow mock each of the SDK functions? I am using Unity test framework.
Module example:
my_module.c
#include "sdk_module_1.h"
#include "my_module.h"
void init_hardware(void)
{
//function code
}
bool send_data(int data)
{
//prepare data eq.
data++
//send data using SDK function (sdk_module_1.h)
return send_data(data);
}
my_module.h
void init_hardware(void)
void send_data(int data)
my_module_test.c
#include "my_module.h"
#include "//unity files"
TEST_SETUP(Test)
{
}
TEST_TEAR_DOWN(Test)
{
}
TEST(Test, First_test)
{
TEST_ASSERT_EQUAL(send_data(5),true);
}
When I try to test my module, I have a problem with referencing SDK modules and their functions. How can I create tests for such software? Should I change the way my modules are written?
The resource you want is James Grenning's Test Driven Development for Embedded C.
(Note: what follows below is a translation of my ideas into C. If you find a conflict with Grenning's approach, try his first - he has a lot more laps in the embedded space than I do.)
How can I test my modules when they have numerous references to other files (modules) in the SDK?
Sometimes the answer is that you have to change your design. In other words, treating testability as a design constraint, rather than an afterthought.
The way I normally describe it is this: we want to design our code such that (a) all the complicated code is easy to test and (b) anything that's hard to test is "so simple there are obviously no deficiencies".
This often means designing our code so that collaborations between complicated code and hard to test code are configurable, allowing you to provide a substitute implementation (stub/mock/test double) when using the real thing isn't cost effective.
So instead of having A (complicated) directly invoke B (hard for testing), we might instead have A invoke B via a function pointer that, during testing, can be replaced with a pointer to a simpler function.
(In some styles, this gets reversed: the complicated logic points to an inert/stub implementation by default, and you opt in to using the complicated implementation instead.)
In other words, we replace
void A(void) {
B(); // B is the function that makes things hard to test.
}
with
void A(void) {
C(&B);
}
# It's been a long time, please forgive (or correct) the spelling here
void C( void (*fn)()) {
&fn();
}
We test A by looking at it, agreeing that it is "so simple there are obviously no deficiencies", and signing off on it. We test C by passing it pointers to substitute implementations, writing as many substitutes as we need for B to ensure that all of C's edge cases are covered.
Note that "hard to test" can cover a lot of bases - the real function is slow, the real function is unstable, the real function costs money... if it makes testing less pleasant or less effective, then it counts.
Firstly, I would highly recommend using Ceedling to manage and run your unit tests in C. It wraps Unity and CMock very nicely and makes unit testing, particularly for embedded systems, a lot easier than it otherwise would be.
With regards to unit testing when an SDK is involved, you first need to remember that the point of a unit test is to test a unit, so anything outside of that needs to be mocked or stubbed, otherwise it’s not a unit test and cannot be run as one.
So, for something like an I2C hardware abstraction module, any calls to an underlying SDK (that would ordinarily do the actual I2C transaction) need to be mocked so that you can place expectations on what should happen when the call is made instead of the real thing. In this way, the unit test of the I2C HAL module is exercising only its behaviour and how it handles its calls, and nothing else, as it should.
Therefore, all that you generally need to do to unit test with an SDK is to ensure that any module you use has its public functions mocked. You then simply include the mock in the test, rather than the real module.
Now, I find that you don’t really want to mess with the original SDK and its structure for the purpose of unit testing, and moreover you don’t want to change how the real code includes the real modules, so the problem for the test then comes when the SDK function you’re calling in your code sits behind layers of other modules in the SDK, as is often the case. Here, you don’t want to mock everything you’re not using, but you do need to match the include structure. I’ve found that a good way to do this is simply to create a support folder and copy any top-level headers that the code uses from the SDK into it. I then configure the test to include from support before it includes from the SDK.
Once you have this kind of structure in place, unit testing with an SDK is easy. Ceedling will help with all of this.
I recently got into Unit-Testing embedded C Code.
I'd like to test each module fully isolated from all the others. This approach requires me to simulate (or "fake")
all the dependencies and external calls a module makes.
Doing just that, I would end up with multiple definitions for the same function -- all the fakes would have the same identifier.
I believe the most common approach to avoid having multiple definitions is to compile one binary
for each test -- instead of one big program for all tests.
However this introduces new difficulties. I need a main()-Function for every Module under Test. Also each program now prints its own summary instead of one the total
test summary. Oh and its quite tedious to set up the build environment to do this...
There is very likely a smart way to do this. How is it done correctly?
have you ever heard about automatic C code generators?
I have to do a kind of strange API functionality research which includes at least one attempt of every function execution. It may lead to crushes, segmentation faults - no matter. I just need to register every function call.
So i got a long list (several hundreds) of functions from sources using
ctags -x --c-kinds=f *.c
Can i use any tool to generate code calling every of them? Thanks a lot.
UPD: thanks for all your answers.
You could also consider customizing the GCC compiler, e.g. with a MELT extension (which e.g. would generate the testing during some customized compilation). Then you might even define your own #pragma or __attribute__ to parameterize these functions (enabling their auto-testing, giving default arguments for testing, etc etc).
However, I'm not sure it is the right approach for unit testing. There are many unit testing frameworks (but I am not very familiar with them).
Maybe something like autoconf could help you with that: as described here. In particular check for AC_CHECK_FUNCS. Autoconf creates small programs to test the existence of registered functions.
I have a large code base of quite old C code on an embedded system and unfortunately there are no automated test cases/suites. This makes restructuring and refactoring code a dangerous task.
Manually writing test cases is very time consuming, so I thought that it should be possible to automate at least some part of this process for instance by tracing all the function calls and recording of the input and output values. I could then use these values in the test cases (this would not work for all but at least for some functions). It would probably also be possible to create mock functions based on the gathered data.
Having such test cases would make refactoring a less dangerous activity.
Are there any solutions that already can do this? What would be the easiest way to get this to work if I had to code it myself?
I thought about using ctags to find the function definitions, and wrapping them in a function that records the parameter values. Another possibility would probably be a gcc compiler plugin.
There is a gcc option "-finstrument-functions", which mechanism you can use to define your own callbacks for each funtion's entry/exit.
Google it and you can find many good examples.
[Edit] with this gcc option's call back you can only track the function's entry/exit,not the params. but with some tricks you may also track the params. (walk through the current frame pointer to get the param on the stack).
Here is an article talk about the idea of the implementation:
http://linuxgazette.net/151/melinte.html
Furthermore, depends on your embedded system, on linux you can try something like ltrace to show the params(like the strace way). There are many tools do the function trace work either in userspace or kernelspace on linux, ftrace/ust/ltrace/utrace/strace/systemtap/. Anyway, if you do not add any hard debugging code, it's not possible to display the params in the correct way. If you accept the efforts to add entry/exit debugging infomation, then it's much easier.
Also here is a similar thread talk about this problem.
Tool to trace local function calls in Linux
Background:
In a particular project there are about couple of thousand functions in more than hundred files. The functions are divided to reside in two banks of code memory - fast_mem and slow_mem. But now, since the fast_mem area is limited, its running out of space to accommodate any new code changes.
As part of code review, its been found that some functions in fast_mem have no callers. But the list of functions is too huge to check them one by one manually.
Question:
So, coming to the question, is there a tool that can list the callers of all the functions in the project? With this, I can go ahead and remove functions in fast_mem that don't have any callers.
I use cscope for code browsing along with ctags. But this requires one to input the function name manually. Can this be automated some how to get the complete list?
I also tried Doxygen with its caller graph feature. The result is not so comfortable to use though.
I use Scientific Toolworks Understand
If your compiler is a recent GCC (or if you can switch to GCC 4.6, possibly as a cross-compiler) you might develop a GCC plugin or a MELT extension to find out.
Of course, if you are e.g. doing tricks with function pointers (e.g. unportable pointer arithmetic on function pointers) the original question is undecidable.
Actually, if you are using function pointers, often the only reasonable thing to say is that they can reach only functions of the same signature.
And perhaps the project is important enough so that customizing the compiler to make a better (automatic or semi-automatic) trade-off between fast_mem & slow_mem is worthwhile. This is typically an excellent case for GCC plugins or MELT extensions (but that take some work -days or weeks, not hours-, because you need to understand the internal GCC representations to customize GCC), and you are probably the only one who could do it (because your question is very peculiar to some strange systems).
Let's assume there aren't any odd function pointer games going on. Then you can break out the under-used cflow:
http://www.gnu.org/software/cflow/
Generate a "reverse index" with the -r flag. you'll get a list of every function, followed by where it's called. You can feed it multiple files.
You can use static code analysis tool like cppcheck.
If you call it with --enable=unusedFunction parameter it will warn about unused function.