How can I write a test in Ceedling for a function that uses static global variable?
I would like to test for each possible value of the variable to achieve a good test coverage.
//Pseudo code for file_under_test.c
static int global_var;
int func_under_test(){
switch(global_var){
case x:
return some_value;
case y:
return some_other_value;
.
.
.
.
default:
return something;
}
}
This is a super common problem in unit testing C code and the most common solution I know of is to define the static keyword out of existence when testing. This requires some planning and is hard to do in legacy code but any static that I plan on testing against is replaced by some other string. Usually STATIC or better yet TESTABLE_STATIC.
Remember that ceedling and probable most unit test frameworks sets a compile time macro TEST so you code would be
//Pseudo code for file_under_test.c
#ifdef TEST
#define TESTABLE_STATIC
#else
#define TESTABLE_STATIC static
#endif
TESTABLE_STATIC int global_var;
int func_under_test(){
switch(global_var){
case x:
return some_value;
case y:
return some_other_value;
.
.
.
.
default:
return something;
}
}
Then in your test file you just treat the variable as a global
// your ceedling test
#include <your_functions.h>
extern int global_var;
void test_function_under_test(void)
{
// your test code setting global_var as needed
global_var = some_test_value;
TEST_ASSERT_EQUAL(expected_val, func_under_test());
}
I usually hide the TESTABLE_STATIC in a project header file or if you have a datatypes.h file so it is generally available everywhere in my project.
This also works for unit testing your static functions in a translation unit.
This question does not really have anything specifically to do with Ceedling (or Unity, CMock, etc), but I rather think this is an example of interpreting the word "unit" in a very specific way. The short version of my answer is that the example function you have written here does not really constitute a self-contained "unit", so I would claim that this is not really "unit-testable".
Only think of a "function" as a "unit" if it is a pure function or if you can find an appropriate seam (e.g. stubs, spies, or mocks to external interfaces)! Otherwise you would by neccessity need to check for implementation details inside the test, which makes for very brittle tests.
In order to have a "unit" of testable code, you need to both be able to see the effects of the unit under test (e.g. comparing the returned value and/or checking for other side effects) AND to be able to stimulate the unit under test (e.g. by passing arguments into a the function or by first setting up some side effects which the unit under test relies on).
The example function is relying on the side effects of some other function (one which has the side effect of modifying your static "global" variable), so a "proper unit" in this case would need include whatever function you have which triggers these side effects. I suppose your file already has at least one such function, or your example code would never return anything different*.
*This is unless your example actually has the side effect of modifying the static variable itself. In that case there should at least be a function which resets the "global state", otherwise your tests will not be isolated from each other (i.e. it is hard to make them order-independent). A better solution would be to explicitly expose the dependency of your state through the arguments to func_under_test, like func_under_test(struct some_opaque_type *state) and add a struct some_opaque_type *init_for_func_under_test() function.
TL;DR: If your function is not a pure function (e.g. it relies on hidden state and/or has side effects itself) or if you don't have the appropriate "seams" (e.g. stubs or spies), then also include functions which can modify the hidden state or verify the side effects in your definition of unit under test.
I used a wrapper, that includes the orginal C file and adds some helpers for test
So you have an unchanged orginal c source, but access to all the needed internals.
file wrapped_for_test.h
#include <file_under_test.h>
void set_for_test(int value);
file wrapped_for_test.c
#include <file_under_test.c>
void set_for_test(int value)
{
global_var = value;
}
You can create a test helper function in file_under_test.c that will setup the global_var before calling func_under_test(). If needed this test function helper can be compiled only for testing purpose (using specific #ifdef) so that is it not shipped with the rest of the code if your code is build for eg a product.
file_under_test.h:
void set_for_test(int value);
file_under_test.c
#ifdef TESTS
void set_for_test(int value)
{
global_var = value;
}
#endif
test_file.c:
#include <assert.h>
#include "file_under_test.h"
// some other tests
set_for_test(3);
assert (func_under_test() == something);
//...
Related
I am making a low level library that requires initialization to work properly which I implemented with a init function. I am wondering if there is a way to make the init call be called once the user calls a library function ideally without:
Any overhead
No repeated calls
No exposed global variables. (my current solution does this, which I don't quite like)
my current solution as per comment request:
bool isinit = 0;
void init()
{
isinit = 1;
// init code
}
void lib_function()
{
if(!isinit) init();
// function code
}
The compiler seems to be smart enough (using -0fast on gcc) to not make that comparison each time a lib_function is called, but this still exposes a global variable which I don't like.
Best way to abstract away an init function?
Surely your library has some state. Typically, a library exposes functions that work on a specific structure. Do not use global variables - do not write spaghetti code. Expose the structure that holds the state of your library, and make all functions of your library take a pointer to the structure as an argument. Use a namespace - prepend all exported symbols with a prefix. An init function is just like int lib_init(struct lib_the_struct *t); - it will be self-understandable that users need to initialize the structure with that function before use. For example: fopen(), pthread_create.
Write an init function in your library. Write clear documentation stating, that the user of your library has to call the function once before calling any other function. For example: https://curl.se/libcurl/c/curl_global_init.html .
If you're happy with a solution that is a common extension rather than part of the C standard, you can mark your init function with the constructor attribute, which ensures it will be called automatically during program initialization (or during shared library load if you eventually end up using that).
I would fix this with assert so that the if will dissappear in release build and if you forget to call the init_function somewhere you get the error while developing.
Also turn isinit into a static so every library can have its own variable with the same name.
#include <assert.h>
#ifndef NDEBUG
static int isinit = 0;
#endif
void lib_function()
{
assert(isinit && "library: init not called");
}
There will be overhead if you run if(!isinit) init(); each time you call a function. At least an extra branch.
As for removing global variables, do in your example but static bool isinit = 0;. This reduces the scope of the variable to the local translation unit (.c file and all .h files it includes). It's no longer "global". Note that this isn't ideal in multi-threaded scenarios - you will have to protect the variable with a mutex then.
Overall though, what you are trying to do isn't a good idea. It is very common convention for C libraries to have an init function and the user of the library is expected to call it before calling anything else or they are to blame, not your library. Naturally you have to make this clear to them with source code documentation. It is common to have a list of prerequisites in source code comments together with every function declaration placed in the header file of the library.
I would like to test some functions by implementing unit tests with cmocka framework.
For example I have a non-static function under test which calls two static functions. Because of the fact that these static functions interact with hardware I want to mock/wrap them and use a wrap function instead of the real function when testing.
Like described in the cmocka documentation I've used the --wrap=myfunction linker flags when building/linking my tests.
The tests compile but when I run them the real static functions will be called instead of the wraps.
When I declare the static functions non-static it doesn't work either and also the real functions are called. The only solution I've found is to outsource the functions in an extra .c-file...but that's a really bad workaround because it manipulates the code very much.
As #Paul wrote, this is simply how --wrap works, the functions need to be in a different compilation unit if you want gcc to wrap them. Generally, static methods are private implementation details which you don't want to expose for testing.
So to add some more options to the other answer:
Obviously, the simplest way to mock these functions without polluting the original code with conditionals is to extract them into a separate layer (in this case, a HAL).
You can make the static modifier conditional, which will allow wrapping with cmocka. This does less polluting to the original code than a bunch of #ifdefs:
#ifndef UNIT_TESTING
# define mockable_static static
#else
# define mockable_static
#endif
// then, replace 'static' with 'mockable_static'
mockable_static void some_static_method(void) {
...
}
Use objcopy to globalize and weaken selected static functions, as explained in this answer.
If you have a .c file with a public function and two static functions, and the public function calls the static functions, then you cannot prevent the static functions being called. After all, they are one compilation unit.
You can:
test them as a whole
use #ifdef conditional compilation to replace the static functions with simplified, non-hardware calling static functions to test the public function
use #ifdef conditional compilation to replace the public function with a specialised public function to test the static functions.
I'm looking for a good technical solution to doing DI in C.
I have seen some of the DI questions here already, but I haven't seen one with any actual examples or concrete implementation suggestions.
So, lets say we have the following situation:
We have a set of modules in c; we want to refactor those modules so that we can use DI to run unit tests and so on.
Each module effectively consists of a set of c functions:
module_function(...);
Modules depend on each other. Ie. Typically you may have a call such as:
int module1_doit(int x) {
int y = module2_dosomethingelse(x);
y += 2;
return(y);
}
What is the correct approach to DI for this?
Possible solutions seem to be:
(1) Using function pointers for all module functions, and when invoking a function do this (or similar):
int y = modules->module2->dosomethingelse(x);
(2) Compile multiple libraries (mock, std, etc.) of with the same symbols and dynamically link in the correct implementation.
(2) seems to be the correct way of doing it, but is difficult to configure and annoyingly forces you to build multiple binaries for each unit test.
(1) Seems like it might work, but at some point your DI controller is going to get stuck in a situation where you need to dynamically invoke a generic factory function (void ( factory) (...) say) with a number of other modules that need to be injected at runtime?
Is there another, better way of doing this in c?
What's the 'right' way of doing it?
I don't see any problem with using DI in C. See:
http://devmethodologies.blogspot.com/2012/07/dependency-injection.html
I've concluded that there is no 'right' way of doing this in C. It's always going to be more difficult and tedious than in other languages. I think it's important, however, not to obfuscate your code for the sake of unit tests, though. Making everything a function pointer in C may sound good, but I think it just makes the code horrific to debug in the end.
My latest approach has been to keep things simple. I don't change any code in C modules other than a small #ifdef UNIT_TESTING at the top of a file for externing and memory allocation tracking. I then take the module and compile it with all dependencies removed so that it fails link. Once I've reviewed the unresolved symbols to make sure they are what I want, I run a script that parses these dependencies and generates stub prototypes for all the symbols. These all get dumped in the unit test file. YMMV depending on how complex your external dependencies are.
If I need to mock a dependency in one instance, use the real one in another, or stub it in yet another, then I end up with three unit test modules for the one module under test. Having multiple binaries may not be ideal, but it's the only real option with C. They all get run at the same time, though, so it's not really a problem for me.
This is a perfect use-case for Ceedling.
Ceedling is sort umbrella project that brings together (among other things) Unity and CMock, which together can automate a lot of the work you're describing.
In general Ceedling/Unity/CMock are a set of ruby scripts that scan through your code and auto-generate mocks based on your module header files, as well as test runners that find all the tests and makes runners that will run them.
A separate test runner binary is generated for each test suite, linking in the appropriate mock and real implementations as you request in your test suite implementation.
I was initially hesitant to bring in ruby as a dependency to our build system for testing, and it seemed like a lot of complexity and magic, but after trying it out and writing some tests using the auto-generated mocking code I was hooked.
A little late to the party on this but this has been a recent topic where I work.
The two main ways that I've seen it done is using function pointers, or moving all dependencies to a specific C file.
A good example of the later is FATFS.
http://elm-chan.org/fsw/ff/en/appnote.html
The author of fatfs provides the bulk of the library functions and relegates certain specific dependencies for the user of the library to write (e.g. serial peripheral interface functions).
Function pointers are another useful tool, and using typedefs help to keep the code from getting too ugly.
Here's some simplified snippets from my Analog to Digital Converter (ADC) code:
typedef void (*adc_callback_t)(void);
bool ADC_CallBackSet(adc_callback_t callBack)
{
bool err = false;
if (NULL == ADC_callBack)
{
ADC_callBack = callBack;
}
else
{
err = true;
}
return err;
}
// When the ADC data is ready, this interrupt gets called
bool ADC_ISR(void)
{
// Clear the ADC interrupt flag
ADIF = 0;
// Call the callback function if set
if (NULL != ADC_callBack)
{
ADC_callBack();
}
return true; // handled
}
// Elsewhere
void FOO_Initialize(void)
{
ADC_CallBackSet(FOO_AdcCallback);
// Initialize other FOO stuff
}
void FOO_AdcCallback(void)
{
ADC_RESULT_T rawSample = ADC_ResultGet();
FOO_globalVar += rawSample;
}
Foo's interrupt behavior is now injected into the ADC's interrupt service routine.
You can take it a step further and pass a function pointer into FOO_Initialize so all dependency issues are managed by the application.
//dependency_injection.h
typedef void (*DI_Callback)(void)
typedef bool (*DI_CallbackSetter)(DI_Callback)
// foo.c
bool FOO_Initialize(DI_CallbackSetter CallbackSet)
{
bool err = CallbackSet(FOO_AdcCallback);
// Initialize other FOO stuff
return err;
}
There are two approaches that you can use. Whether you really want to or not, as Rafe is pointing out, are up to you.
First: Create the "dynamically" injected method in a static library. Link against the library and simply substitute it during tests. Voila, the method is replaced.
Second: Simply provide compile-time replacements based on preprocessing:
#ifndef YOUR_FLAG
/* normal method versions */
#else
/* changed method versions */
#endif
/* methods that have no substitute */
I'm very new at testing so please let me know if I am just going off in completely the wrong direction at any point. Having said that, assume I want to test the following function, foo.
int foo(int i) {
//Lots of code here
i = bar();
//Some more changes to i happen here, conditional on what bar returned
return i;
}
In this example, both foo and bar are functions written by myself and I have already tested bar.
Since the output of foo is conditional on the output of bar, I assume that in order to test foo, I need to create a mock of bar. In order to do that, and assuming that the definition of bar is kept inside a separate source file from foo, I could create a new source file, include that instead of the one where the actual definition of bar is found, and put a mock of bar in that file.
int bar(void) {
return HARD_CODED_VALUE;
}
However, there are 2 problems with this approach:
1) What happens if bar returns multiple values (such as an error code or an actual value) and I need to ensure that foo reacts correctly for each possibility? I can't create multiple definitions for bar. One thought I did have was to create a static int in bar and then increment it every time bar gets called. Then I just have a conditional on this int, call bar multiple times and thus return multiple values. However, I am unsure whether introducing more complex logic into a mock function is good practice or if there is a better way to achieve this:
int bar(void) {
static int i = 0;
i++;
if(i == 1) {
return HARD_CODED_VALUE_1
}
else if(i == 2) {
return HARD_CODED_VALUE_2
}
else {
fprintf(stderr, "You called bar too many times\n");
exit(1);
}
}
2) What happens if bar is in the same source file as foo? I can't redefine bar nor separate foo and bar without altering my source code which would be a real pain.
Well, there are a few ways around that problem.
You could use preprocessor hooks to swap out bar() when a UNITTEST flag is set:
#ifdef UNITTEST
return mockBar();
#else
return bar();
#endif
You could simulate Dependency Injection, and require a pointer to bar() as a parameter to the function. I'm not saying that's a great idea in practice, but you could do it.
void foo( void (*bar)() ) {
I'm sure there are others, but that's just 2 that came off the top of my head...
What you want to do is substitute the called function with a stub returning known values. The same would apply when using an external dependency, i.e. a database or networking code. With C there are two usable "seams" (to use the terminology from Working Effectively with Legacy Code) that allow you to perform that substitute:
Using preprocessor commands to replace the function body with a macro, e.g.
#ifdef TEST
#define bar(x) { if (x) then y; else z; }
#endif
Move bar(x) into a separate library, and then maintain two versions of the library. The first is your production code and the second is a test library that contains a test stub of bar(x).
A third option is to use dependency injection, by refactoring the bar(x) call out into a function pointer parameter as ircmaxell demonstrated.
void foo( void (*bar)() )
I have tried these approaches with non-OO C++ code and found the first to be by far the most useful. The second introduces a pretty tough maintainability issue (multiple versions of the same libraries and the functions within need to be maintained in conjunction), while the latter obviously negatively impacts upon the readability and understandability of the code.
The preprocessor directives, on the other hand, can be quite localized and the alternate definitions can be separated out into a header file that is only included if tested, i.e.
#ifdef TEST
#include "subsystem_unittest.h"
#endif
There are libraries for mocking. These libraries usually find a way to address those very questions. Sophisticated libraries will allow you to configure in your test what bar() should return at each point in the test.
I'm not sure they'll be handling the case where bar() and foo() are in the same source file very well but they might. In this case I would consider bar() and foo() to be part of the same unit but that is an entirely different argument.
Here is a C++ code fragment (source) from GoogleMock as an example. It creates a Mock turtle object which the Painter should call the PenDown method once and when it does the PenDown method will return 500. If the Painter doesn't call PenDown then the test would fail.
#include "path/to/mock-turtle.h"
#include "gmock/gmock.h"
#include "gtest/gtest.h"
using ::testing::AtLeast; // #1
TEST(PainterTest, CanDrawSomething) {
MockTurtle turtle; // #2
EXPECT_CALL(turtle, PenDown()) // #3
.WillOnce(Return(500));
Painter painter(&turtle); // #4
EXPECT_TRUE(painter.DrawCircle(0, 0, 10));
} // #5
int main(int argc, char** argv) {
// The following line must be executed to initialize Google Mock
// (and Google Test) before running the tests.
::testing::InitGoogleMock(&argc, argv);
return RUN_ALL_TESTS();
}
Of course this particular library is using OOP which you may or may not be doing. I would guess there are other libraries out there for non-OOP too.
Is bar() an awkward dependency ? Is there a problem with the unit test for foo using the actual implementation of bar ?
If not, then I don't see a problem. You do not have to mock everything.
As applying unit-test to some C code, we run into a problem that some static function can not be called at the test file, without modifying the source code. Is there any simple or reasonable way to overcome this problem?
I have a test harness. In dire cases - like trying to test a static function, I use:
#include "code_under_test.c"
...test framework...
That is, I include the whole of the file containing the function under test in the test harness. It is a last resort - but it works.
Can you give more information as to why you can't call the function?
Is it not available because it's private to a .c file? If so, you're best bet is to use conditional compilation that allows for access to the function in order to allow for other compilation units to access it. For example
SomeHeaderSomewher.h
#if UNIT_TEST
#define unit_static
#else
#define unit_static static
#endif
Foo.h
#if UNIT_TEST
void some_method
#endif
Foo.cpp
unit_static void some_method() ...
For unit tests, we actually have the test code within the source file itself and we conditionally compile it in when testing. This gives the unit tests full access to all functions and file-level variables (static or otherwise).
The unit tests themselves are not static - this allows us to call the unit tests from a single super-test program which unit tests all compilation units.
When we ship the code, we conditionally compile out the unit tests but this isn't actually necessary (if you want to be certain you're shipping exactly the same code you tested).
We've always found it invaluable to have the unit tests in the same place as the code you're testing since it makes it more obvious that you need to update the tests if and when the code changes.
No - you cannot directly test a static function without modifying the source at least a little (that is the definition of static in C - that it cannot be called from a function in a different file).
You could create a separate function within the test file that just calls the static function?
For example:
//Your fn to test
static int foo(int bar)
{
int retVal;
//do something
return retVal;
}
//Wrapper fn
int test_foo(int bar)
{
return foo(bar);
}
We usually don't test our static functions directly, but rather ensure that the logic they perform is adequately tested by different tests of the calling function.
static functions are essentially helper functions to the public (i.e. exposed) functions. So IMO, your unit tests should call the public interface with inputs that exercise all the paths in the static function.
The output (return values / side effects) of the public function should be used to test the effect of the static.
This means you need to have appropriate stubs to 'catch' these side effects. (e.g. if a function calls file IO, you need to provide stubs to override these file IO lib functions). The best way to do this by making each test suite a seperate project/executable and avoid linking to any external lib functions. You can mock even C functions, but it's not worth the effort.
Anyway, this is the approach I've used so far and it works for me. Good luck
#define static
This is a very bad idea. If you have a variable declared local to a function, it changes the behavior of the function. Example:
static int func(int data)
{
static int count = 0;
count += data;
return count;
}
You could call the function from the unit test, as func() would be exported, however the basic functionality of the code would be modified.
--kurt
If you are under Unix environment you can include in test file additional header yourheader_static.h with declarations of your static functions and translate obj file code_under_test.o through objdump --globalize-symbols=syms_name_file to globalize local symbols. They will be visible as if they are non-static functions.
Just to add to the accepted answer by Jonathan Leffler, and elaborate on other's mention of a wrapper function:
Create a test source file, as in test_wrapper_foo.c, where foo.c is the original.
In test_wrapper_foo.c:
#include "foo.c"
// Prototype
int test_wrapper_foo();
// wrapper function
int test_wrapper_foo() {
// static function to test
return foo();
}
Assuming that the static function foo in foo.c has prototype: int foo(void);
build test_wrapper_foo.c through your makefile instead of foo.c (note that this will not break any dependencies on functions in foo.c by other external functions)
In your unit test script, call test_wrapper_foo() instead of foo().
This approach leaves the original source intact, while giving you access to the function from your test framework.
You could add a non-static function to call the static function, then call the non-static function.
static int foo ()
{
return 3;
}
#ifdef UNIT_TEST
int test_foo ()
{
if (foo () == 3)
return 0;
return 1;
}
#endif
If you're using Ceedling and trying to use the #include "code_under_test.c" method, the test build will fail because it will automatically try to build "code_under_test.c" once when #included and also because it's the target of the test.
I've been able to get around it by a slight modification to the code_under_test.c code and a couple of other changes. Wrap the whole code_under_test.c file with this check:
#if defined(BUILD)
...
#endif // defined(BUILD)
Add this to your test harness:
#define BUILD
#include "code_under_test.c"
Add the BUILD define to your Makefile or project config file:
# Makefile example
..
CFLAGS += -DBUILD
..
Your file will now build from your environment and when included from your test harness. Ceedling will now not be able to build the file a second time (ensure your project.yml file does NOT define BUILD).
All the above suggested answers (except a few) suggest conditional compilation which requires modification to source. As such that shouldn't be an issue, it just would add clutter (just for testing). Rather you can do something like this.
Say your function to be tested is
static int foo(int);
You make another header file called testing_headers.h which will have the contents -
static int foo(int);
int foo_wrapper(int a) {
return foo(a);
}
Now while compiling your c file for testing you can force include this header from the compiler options.
For clang and gcc the flag is --include. For Microsoft C compiler it is /FI.
This will require absolutely 0 change to your c file and you will be able to write a non static wrapper to your function.
If you don't want a non static wrapper, you can also create a non static global function pointer initialised to foo.
You can then call the function using this global function pointer.
There are 2 ways to do this.
Include the c source file into the unit testing source file, so the static method now is in the scope of unit testing source file and callable.
Do a trick:
#define static
in the head of unit testing source file.
It will convert keyword static to "nothing" or I can say, it removes static from your c source code.
In some unit testing tool, we can use config option "pre-processor" to do this trick, just put in the config:
static=
The tool willl convert all static keyword to "nothing"
But I must say, using these ways make the static method (or variable) loses the specific meaning of it.