I am trying to implement TDD in C coding. I am building the program structure in a quite modularised way and using as atomic functions as possible. I make one test file (including several suits) for one module (module = header file + source file). I am struggling to make the program files "not know that they are being tested", in other words - I don't want testing parts of code in the proper program. Therefore almost often I need to include the source file in the test file in order to have access to the "private" variables and functions.
That was the intro, now the problem: if in a module I have an aaa() function, which uses inside a bbb() function, which uses some xxx() function from an external module, I can easily test the bbb() function in the atomic way by mocking the x() function: #define bbb mock_bbb and providing a mock xxx module for #include. However, I am unable to find a way of atomic testing of the aaa() function, which uses a function from the same module. Is it possible to do? (note, that apart of mocking bbb() for aaa(), I have to be also able to use the original bbb() to test it)
My closest try was to use -Wl,-wrap,xxx, but the problem is that I haven't found a way to automate this (wildcard or something?) - I will have almost 100 testing files, each containing several functions to test - I cannot allow myself to put manually every function in the makefile.
I never test "private" functions in an atomic way. I usually unit-test a c-module using its public functions and checking its calls to other modules (using mocks through dependency injection) and checking its private data members (by exposing its private data members with a GetDataPtr()-function that is compiled only for the unit test project).
For me that the best tradeof between effort and complexity of the unit test framework, although it not possible to reach 100% statement coverage in some "private" functions.
Related
I create a project for a microcontroller by programming it in C language. Due to its specificity (a microcontroller with a built-in BLE core), I have to use the SDK and a specific project template. How can I test my modules when they have numerous references to other files (modules) in the SDK? (References are needed to use functions to, for example, send data via BLE) Do I have to somehow mock each of the SDK functions? I am using Unity test framework.
Module example:
my_module.c
#include "sdk_module_1.h"
#include "my_module.h"
void init_hardware(void)
{
//function code
}
bool send_data(int data)
{
//prepare data eq.
data++
//send data using SDK function (sdk_module_1.h)
return send_data(data);
}
my_module.h
void init_hardware(void)
void send_data(int data)
my_module_test.c
#include "my_module.h"
#include "//unity files"
TEST_SETUP(Test)
{
}
TEST_TEAR_DOWN(Test)
{
}
TEST(Test, First_test)
{
TEST_ASSERT_EQUAL(send_data(5),true);
}
When I try to test my module, I have a problem with referencing SDK modules and their functions. How can I create tests for such software? Should I change the way my modules are written?
The resource you want is James Grenning's Test Driven Development for Embedded C.
(Note: what follows below is a translation of my ideas into C. If you find a conflict with Grenning's approach, try his first - he has a lot more laps in the embedded space than I do.)
How can I test my modules when they have numerous references to other files (modules) in the SDK?
Sometimes the answer is that you have to change your design. In other words, treating testability as a design constraint, rather than an afterthought.
The way I normally describe it is this: we want to design our code such that (a) all the complicated code is easy to test and (b) anything that's hard to test is "so simple there are obviously no deficiencies".
This often means designing our code so that collaborations between complicated code and hard to test code are configurable, allowing you to provide a substitute implementation (stub/mock/test double) when using the real thing isn't cost effective.
So instead of having A (complicated) directly invoke B (hard for testing), we might instead have A invoke B via a function pointer that, during testing, can be replaced with a pointer to a simpler function.
(In some styles, this gets reversed: the complicated logic points to an inert/stub implementation by default, and you opt in to using the complicated implementation instead.)
In other words, we replace
void A(void) {
B(); // B is the function that makes things hard to test.
}
with
void A(void) {
C(&B);
}
# It's been a long time, please forgive (or correct) the spelling here
void C( void (*fn)()) {
&fn();
}
We test A by looking at it, agreeing that it is "so simple there are obviously no deficiencies", and signing off on it. We test C by passing it pointers to substitute implementations, writing as many substitutes as we need for B to ensure that all of C's edge cases are covered.
Note that "hard to test" can cover a lot of bases - the real function is slow, the real function is unstable, the real function costs money... if it makes testing less pleasant or less effective, then it counts.
Firstly, I would highly recommend using Ceedling to manage and run your unit tests in C. It wraps Unity and CMock very nicely and makes unit testing, particularly for embedded systems, a lot easier than it otherwise would be.
With regards to unit testing when an SDK is involved, you first need to remember that the point of a unit test is to test a unit, so anything outside of that needs to be mocked or stubbed, otherwise it’s not a unit test and cannot be run as one.
So, for something like an I2C hardware abstraction module, any calls to an underlying SDK (that would ordinarily do the actual I2C transaction) need to be mocked so that you can place expectations on what should happen when the call is made instead of the real thing. In this way, the unit test of the I2C HAL module is exercising only its behaviour and how it handles its calls, and nothing else, as it should.
Therefore, all that you generally need to do to unit test with an SDK is to ensure that any module you use has its public functions mocked. You then simply include the mock in the test, rather than the real module.
Now, I find that you don’t really want to mess with the original SDK and its structure for the purpose of unit testing, and moreover you don’t want to change how the real code includes the real modules, so the problem for the test then comes when the SDK function you’re calling in your code sits behind layers of other modules in the SDK, as is often the case. Here, you don’t want to mock everything you’re not using, but you do need to match the include structure. I’ve found that a good way to do this is simply to create a support folder and copy any top-level headers that the code uses from the SDK into it. I then configure the test to include from support before it includes from the SDK.
Once you have this kind of structure in place, unit testing with an SDK is easy. Ceedling will help with all of this.
I'm working in a project targeting a Silicon Labs microcontroller. I want to create a wrapper for the Bluetooth SDK calls. SDK libraries are heavy on dependencies, so by creating a seem in that interface, I can avoid adding them to my tests. Note that this problem is not specific of the Silicon Labs platform, but I will leave the SiLabs SDK file names in case the reader is familiar with the platform.
I'm using CPPUTest as my testing framework. My code is a mix of C and C++, but for the purposes of this question, it's C.
For example, one of my wrapping functions (in a module called Bluetooth_HW) would be
/* On "Bluetooth_HW.c" */
#include "native_gecko.h" // declares gecko_cmd_le_gap_bt5_set_adv_data()
// Set a new advertisement, either as advertisement packet or as a response.
bool BT_setAdvertisementMessage(uint8_t scanResponse, uint8_t adLength, uint8_t* newAdvertisement) {
uint16_t result = gecko_cmd_le_gap_bt5_set_adv_data(0, scanResponse, adLength, newAdvertisement)->result;
if (result != bg_err_success) {
return false;
} // Else, no errors setting the advertisement
return true;
}
BT_setAdvertisementMessage() wraps gecko_cmd_le_gap_bt5_set_adv_data(), which is defined in native_gecko.h, an SDK header file. I declare BT_setAdvertisementMessage() in Bluetooth_HW.h and define it in bluetooth_HW.c. My code that needs to set the advertisement message then includes Bluetooth_HW.h instead of native_gecko.h, and Bluetooth_HW.h is easy to mock.
I would like to test Bluetooth_HW.c. I need to mock native_gecko.h for that, as I am not interested in running its code (it's provided by the manufacturer, I trust it, plus I'm not unit testing native_gecko, I'm unit testing Bluetooth_HW). Normally, I would include the production .h file in my tests, to get declarations of the functions, and then substitute the .c file that contains the production code definitions with my fake implementation.
My problem is that native_gecko.h defines gecko_cmd_le_gap_bt5_set_adv_data() as an inline inside the header, instead of the more common approach of doing it on a .c file.
I could wrap native_gecko.h itself on a header that just declares the functions that I need, and make my code include that header, but then I don't have access to other stuff declared in native_gecko.h, like custom types and enums. I can't move those types and enums to the wrapper, as I can't remove them from native_gecko.h and the compiler would then complain of being defined twice, and I don't want to touch native_gecko.h, because it's an SDK file.
So, my question is how to mock modules that define code in their headers?
I managed to get it working by providing a fake native_gecko.h. The test environment is then pointer to the fake header instead of the production one. In my case, as native_gecko.h is massive, I just surgically added the minimum amount of elements required for it to work. I added just the declarations for the functions needed. Although I could have mimicked the behaviour of the real module and add them as inline functions, this would require including the mocking framework from CPPUTest to be included to native_gecko.h, which is C++. When the compiler tries to compile Bluetooth_HW.c as C code and pulls the header, it will complain about the presence of the C++ code. By adding the mocked function definitions to their own file, I could leave the mocking framework code out of the header, and my original, production code, C file would compile with just C code.
Roughly, the steps would be
Make sure that the test project can't see the real, production code, native_gecko.h.
Provide a fake version of it, substituting any defined functions for declarations (if the file is too big, as is the case of native_gecko.h, just create declarations for the functions used in your Code-Under-Test).
Provide definitions for those functions.
Creating a fake header is not ideal, as the contents can change with an SDK update, for example, and there's room for human error, but it's the cleanest solution I could come up with.
I recently got into Unit-Testing embedded C Code.
I'd like to test each module fully isolated from all the others. This approach requires me to simulate (or "fake")
all the dependencies and external calls a module makes.
Doing just that, I would end up with multiple definitions for the same function -- all the fakes would have the same identifier.
I believe the most common approach to avoid having multiple definitions is to compile one binary
for each test -- instead of one big program for all tests.
However this introduces new difficulties. I need a main()-Function for every Module under Test. Also each program now prints its own summary instead of one the total
test summary. Oh and its quite tedious to set up the build environment to do this...
There is very likely a smart way to do this. How is it done correctly?
I am currently developing an R package where most of the computational work is done in C code. In particular, I have several complex C routines for which I then wrap with an R function. These I know I can test using testthat because the namespace includes the C function called (I am using roxygen comments to generate the namespace).
The difficulty I am having is that some of these C routines have a few complicated subroutines, the output of which is non-trivial to check. These functions are never called from R directly, so there is no code in my R/ directory that has a roxygen comment that results in the namespace making these functions available. What I want to do is write an R routine that performs the same computation and then use testthat to compare against the output of the C function. I have done this and it works great when I load the entire shared library (using dyn.load), but I don't think this is the proper approach for tests in a package that I intend to submit to CRAN.
My current workaround is to create a dummy function in a file in my R/ directory that then has a roxygen comment for each of the functions I wish to load. That way when I run devtools::document(), these functions are included in the namespace.
Is this the best approach, or is there another method that is better/preferred?
I have a static library of C files, compiled with g++ on Cygwin. I wish to unit test one function that is defined in the library. That function calls another function defined in that library and I wish to override the dependency to replace it with my own version of that function. I can't modify what's in the static library, so this solution [ Override a function call in C ] doesn't apply.
Usually, I can write a .cpp file and include the .c file containing the function I want to unit test, which essentially extends that file with the code I add. It's a dirty trick I'd never use for production code but it's handy for unit testing C files, because it gives my test code access to static things in that C file. Then, I can write in my fake version of the dependency, and my unit test function that calls the function I'm testing. I compile my.cpp to get my.o, then link it with the static library. In theory, since the linker has found a definition for the dependency already (the one I provide) it won't look in the library and there will be no conflict. Usually this works, but now I'm getting a "multiple definition" error where the linker first finds my fake and then finds the real one. I don't know what might cause this and don't know what to look for. I also can't boil this down to a simple example because my simple examples don't have this problem.
Ideas please?
One possibility (admittedly, and ugly one, but...) is to extract the individual object files from the static library. If the function you're calling and the function it's calling are in separate object files, you can link against the object file containing the function you need to call, but not against the one containing the function it calls.
This only gives you granularity on the level of complete object files though, so if the two functions involved are both in the same object file, it won't work. If you really need to get things to work, and don't mind making a really minor modification to the object file in question, you may be able to use a binary editor to mark the second function as a weak external, which means it'll be used in the absence of any other external with the same name, but if another is provided, that will be used instead.
Whether that latter qualifies as "modifying the library" or not depends a bit on your viewpoint. It's not modifying the code in the library, but is modifying a bit of the object file wrapper around that code. My guess is that you'd rather not do it, but it may still be the cleanest way out of an otherwise untenable situation.
It turns out the reason the linker found both defintions of the function is that the faked function's source file defined a variable which is extern'ed in its header file. That unresolved external in the header file caused the linker to link the faked function's object file (the whole thing) to the tested function's file inside the library. So, it's impossible to extract the definition of the tested function without the definition for the dependency.
What I ended up doing was similar to Override a function call in C where I used a different function name instead of the same one, and a preprocessor directive to swap the two. I put the preprocessor directive and the fake function in a separate file which can be included in a unit test, so the production code in the library does not have to be touched. Plus, if I want to fake that same function for another unit test somewhere else, I can re-use the new file.
Depending on your platform and performance requirements, you might be able to use pin to dynamically modify the application and replace one function with another at runtime.
There's no direct example in the manual, but you could easily modify one of the sample pin tools to do this.