I've started to dig into the GLib documentation and discovered that it also offers a unit testing framework.
But how could you do unit tests in a procedural language? Or does it require to program OO in C?
Unit testing only requires "cut-planes" or boundaries at which testing can be done. It is quite straightforward to test C functions which do not call other functions, or which call only other functions that are also tested. Some examples of this are functions which perform calculations or logic operations, and are functional in nature. Functional in the sense that the same input always results in the same output. Testing these functions can have a huge benefit, even though it is a small part of what is normally thought of as unit testing.
More sophisticated testing, such as the use of mocks or stubs is also possible, but it is not nearly as easy as it is in more dynamic languages, or even just object oriented languages such as C++. One way to approach this is to use #defines. One example of this is this article, Unit testing OpenGL applications, which shows how to mock out OpenGL calls. This allows you to test that valid sequences of OpenGL calls are made.
Another option is to take advantage of weak symbols. For example, all MPI API functions are weak symbols, so if you define the same symbol in your own application, your implementation overrides the weak implementation in the library. If the symbols in the library weren't weak, you would get duplicate symbol errors at link time. You can then implement what is effectively a mock of the entire MPI C API, which allows you to ensure that calls are matched up properly and that there aren't any extra calls that could cause deadlocks. It is also possible to load the library's weak symbols using dlopen() and dlsym(), and pass the call on if necessary. MPI actually provides the PMPI symbols, which are strong, so it is not necessary to use dlopen() and friends.
You can realize many of the benefits of unit testing for C. It is slightly harder, and it may not be possible to get the same level of coverage you might expect from something written in Ruby or Java, but it's definitely worth doing.
At the most basic level, unit tests are just bits of code that execute other bits of code and tell you if they worked as expected.
You could simply make a new console app, with a main() function, that executed a series of test functions. Each test would call a function in your app and return a 0 for success or another value for failure.
I'd give you some example code, but I'm really rusty with C. I'm sure there are some frameworks out there that would make this a little easier too.
You can use libtap which provides a number of functions which can provide diagnostics when a test fails. An example of its use:
#include <mystuff.h>
#include <tap.h>
int main () {
plan(3);
ok(foo(), "foo returns 1");
is(bar(), "bar", "bar returns the string bar");
cmp_ok(baz(), ">", foo(), "baz returns a higher number than foo");
done_testing;
}
Its similar to tap libraries in other languages.
Here's an example of how you would implement multiple tests in a single test program for a given function that might call a library function.
Suppose we want to test the following module:
#include <stdlib.h>
int my_div(int x, int y)
{
if (y==0) exit(2);
return x/y;
}
We then create the following test program:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <setjmp.h>
// redefine assert to set a boolean flag
#ifdef assert
#undef assert
#endif
#define assert(x) (rslt = rslt && (x))
// the function to test
int my_div(int x, int y);
// main result return code used by redefined assert
static int rslt;
// variables controling stub functions
static int expected_code;
static int should_exit;
static jmp_buf jump_env;
// test suite main variables
static int done;
static int num_tests;
static int tests_passed;
// utility function
void TestStart(char *name)
{
num_tests++;
rslt = 1;
printf("-- Testing %s ... ",name);
}
// utility function
void TestEnd()
{
if (rslt) tests_passed++;
printf("%s\n", rslt ? "success" : "fail");
}
// stub function
void exit(int code)
{
if (!done)
{
assert(should_exit==1);
assert(expected_code==code);
longjmp(jump_env, 1);
}
else
{
_exit(code);
}
}
// test case
void test_normal()
{
int jmp_rval;
int r;
TestStart("test_normal");
should_exit = 0;
if (!(jmp_rval=setjmp(jump_env)))
{
r = my_div(12,3);
}
assert(jmp_rval==0);
assert(r==4);
TestEnd();
}
// test case
void test_div0()
{
int jmp_rval;
int r;
TestStart("test_div0");
should_exit = 1;
expected_code = 2;
if (!(jmp_rval=setjmp(jump_env)))
{
r = my_div(2,0);
}
assert(jmp_rval==1);
TestEnd();
}
int main()
{
num_tests = 0;
tests_passed = 0;
done = 0;
test_normal();
test_div0();
printf("Total tests passed: %d\n", tests_passed);
done = 1;
return !(tests_passed == num_tests);
}
By redefining assert to update a boolean variable, you can continue on if an assertion fails and run multiple tests, keeping track of how many succeeded and how many failed.
At the start of each test, set rslt (the variables used by the assert macro) to 1, and set any variables that control your stub functions. If one of your stubs gets called more than once, you can set up arrays of control variables so that the stubs can check for different conditions on different calls.
Since many library functions are weak symbols, they can be redefined in your test program so that they get called instead. Prior to calling the function to test, you can set a number of state variables to control the behavior of the stub function and check conditions on the function parameters.
In cases where you can't redefine like that, give the stub function a different name and redefine the symbol in the code to test. For example, if you want to stub fopen but find that it isn't a weak symbol, define your stub as my_fopen and compile the file to test with -Dfopen=my_fopen.
In this particular case, the function to be tested may call exit. This is tricky, since exit can't return to the function being tested. This is one of the rare times when it makes sense to use setjmp and longjmp. You use setjmp before entering the function to test, then in the stubbed exit you call longjmp to return directly back to your test case.
Also note that the redefined exit has a special variable that it checks to see if you actually want to exit the program and calls _exit to do so. If you don't do this, your test program may not quit cleanly.
This test suite also counts the number of attempted and failed tests and returns 0 if all tests passed and 1 otherwise. That way, make can check for test failures and act accordingly.
The above test code will output the following:
-- Testing test_normal ... success
-- Testing test_div0 ... success
Total tests passed: 2
And the return code will be 0.
There is nothing intrinsically object-oriented about testing small pieces of code in isolation. In procedural languages you test functions and collections thereof.
If you are desperate, and you'd have to be desperate, I banged together a little C preprocessor and gmake based framework. It started as a toy, and never really grew up, but I have used it to develop and test a couple of medium sized (10,000+ line) projects.
Dave's Unit Test is minimally intrusive yet it can do some tests I had originally thought would not be possible for a preprocessor based framework (you can demand that a certain stretch of code throw a segmentation fault under certain conditions, and it will test it for you).
It is also an example of why making heavy use of the preprocessor is hard to do safely.
The simplest way of doing a unit test is to build a simple driver code that gets linked with the other code, and call each function in each case...and assert the values of the results of the functions and build up bit by bit...that's how I do it anyway
int main(int argc, char **argv){
// call some function
int x = foo();
assert(x > 1);
// and so on....
}
Hope this helps.
With C it must go further than simply implementing a framework on top of existing code.
One thing I've always done is make a testing module (with a main) that you can run little tests from to test your code. This allows you to do very small increments between code and test cycles.
The bigger concern is writing your code to be testable. Focus on small, independent functions that do not rely on shared variables or state. Try writing in a "Functional" manner (without state), this will be easier to test. If you have a dependency that can't always be there or is slow (like a database), you may have to write an entire "mock" layer that can be substituted for your database during tests.
The principle unit testing goals still apply: ensure the code under test always resets to a given state, test constantly, etc...
When I wrote code in C (back before Windows) I had a batch file that would bring up an editor, then when I was done editing and exited, it would compile, link, execute tests and then bring up the editor with the build results, test results and the code in different windows. After my break (a minute to several hours depending on what was being compiled) I could just review results and go straight back to editing. I'm sure this process could be improved upon these days :)
I use assert. It's not really a framework though.
You can write a simple minimalistic test framework yourself:
// test_framework.h
#define BEGIN_TESTING int main(int argc, char **argv) {
#define END_TESTING return 0;}
#define TEST(TEST_NAME) if (run_test(TEST_NAME, argc, argv))
int run_test(const char* test_name, int argc, char **argv) {
// we run every test by default
if (argc == 1) { return 1; }
// else we run only the test specified as a command line argument
for (int i = 1; i < argc; i++) {
if (!strcmp(test_name, argv[i])) { return 0; }
}
return 0;
}
Now in the actual test file do this:
#include test_framework.h
BEGIN_TESTING
TEST("MyPassingTest") {
assert(1 == 1);
}
TEST("MyFailingTest") {
assert(1 == 2);
}
END_TESTING
If you want to run all tests, execute ./binary without command line arguments, if you want to run just a particular test, execute ./binary MyFailingTest
Related
I want to write a function in C and to put a condition in it. If the condition isn't met the program gives and error and prevents the user (developer) from compiling the code.
For example:
void func(int x)
{
if (x > 0)
{
//do stuff
}
else
{
//give an error and stops the code from compiling
}
}
prevents the user (developer) from compiling the code.
There's a problem there. You can decide on the user's behaviour, but you can't decide on the compilation of the program. If the code is right (right in the language sense, so it makes sense to the compiler), it will compile, else it won't. You can't make up new arbitrary rules for the compiler.
Before you can even run a program written in C, the compilation needs to be fulfilled.
Functions are called at run-time and so are the parameter values determined at run-time, too.
You can't make the compilation of your code dependent upon the variable x in C.
What you're trying to achieve is basically completely impossible.
Let's take an example. Assume that you want to manufacture an elevator, and you set the weight limit to 800 kilograms. You could build in something that makes the elevator stop if the weight exceeds the limit.
So take the scenario where we program the elevator so that it does not move if the weight limit is exceeded. That would typically be done with an assert() or something like that.
You could also in various way try to prevent this from happening, like making the elevator very small so that you cannot fit too many people. But that is not a fail safe option. We have restricted the volume, but nothing prevents a person from bringing a big chunk of solid gold into the elevator.
The point here is that you can measure the weight before moving the elevator, since this is done at runtime. But preventing someone from even trying to exceed the limit is virtually impossible.
In the general case, what you're asking for is completely impossible. What you can do is something like this:
void func(int x)
{
assert(x>0);
/* Do stuff */
}
And a slightly related thing that is possible is to create a test that is a part of the build process. You cannot prevent compilation the way you want, but you can use it to fail the whole build process. An example.
// main.c
int add(int x, int y)
{
return x+y;
}
bool test()
{
if(add(4,5) != 9) return false;
return true;
}
int main(int argc, char **argv)
{
if(strcmp(argv[1], "--test") == 0) {
if(!test()) {
printf("Test failed\n");
exit(EXIT_FAILURE);
}
// More tests
printf("All tests passed\n");
exit(EXIT_SUCCESS);
/* Rest of the main function */
}
Then you create a Makefile that compiles main.c and then calls ./a.out --test as a part of the build process. The above example is a very simple case, and for a more realistic case I would have made it a bit more sophisticated, but it shows how it can be done. Also, there are libraries that can take care of this kind of stuff, but this is a way to do it without having to use that.
I have implemented a facade pattern that uses C functions underneath and I would like to test it properly.
I do not really have control over these C functions. They are implemented in a header. Right now I #ifdef to use the real headers in production and my mock headers in tests. Is there a way in C to exchange the C functions at runtime by overwriting the C function address or something? I would like to get rid of the #ifdef in my code.
To expand on Bart's answer, consider the following trivial example.
#include <stdio.h>
#include <stdlib.h>
int (*functionPtr)(const char *format, ...);
int myPrintf(const char *fmt, ...)
{
char *tmpFmt = strdup(fmt);
int i;
for (i=0; i<strlen(tmpFmt); i++)
tmpFmt[i] = toupper(tmpFmt[i]);
// notice - we only print an upper case version of the format
// we totally disregard all but the first parameter to the function
printf(tmpFmt);
free(tmpFmt);
}
int main()
{
functionPtr = printf;
functionPtr("Hello world! - %d\n", 2013);
functionPtr = myPrintf;
functionPtr("Hello world! - %d\n", 2013);
return 0;
}
Output
Hello World! - 2013
HELLO WORLD! - %D
It is strange that you even need an ifdef-selected header. The code-to-test and your mocks should have the exact same function signatures in order to be a correct mock of the module-to-test. The only thing that then changes between a production-compilation and a test-compilation would be which .o files you give to the linker.
It is possible With Typemock Isolator++ without creating unnecessary new levels of indirection. It can be done inside the test without altering your production code. Consider the following example:
You have the Sum function in your code:
int Sum(int a, int b)
{
return a+b;
}
And you want to replace it with Sigma for your test:
int Sigma(int a, int b)
{
int sum = 0;
for( ; 0<a ; a--)
{
sum += b;
}
return sum;
}
In your test, mock Sum before using it:
WHEN_CALLED: call the method you want to fake.
ANY_VAL: specify the args values for which the mock will apply. in this case any 2 integers.
*DoStaticOrGlobalInstead: The alternative behavior you want for Sum.
In this example we call Sigma instead.
TEST_CLASS(C_Function_Tests)
{
public:
TEST_METHOD(Exchange_a_C_function_implementation_at_run_time_is_Possible)
{
void* context = NULL; //since Sum global it has no context
WHEN_CALLED(Sum (ANY_VAL(int), ANY_VAL(int))).DoStaticOrGlobalInstead(Sigma, context);
Assert::AreEqual(2, Sum(1,2));
}
};
*DoStaticOrGlobalInstead
It is possible to set other types of behaviors instead of calling an alternative method. You can throw an exception, return a value, ignore the method etc...
For instance:
TEST_METHOD(Alter_C_Function_Return_Value)
{
WHEN_CALLED(Sum (ANY_VAL(int), ANY_VAL(int))).Return(10);
Assert::AreEqual(10, Sum(1,2));
}
I don't think it's a good idea to overwrite functions at runtime. For one thing, the executable segment may be set as read-only and even if it wasn't you could end up stepping on another function's code if your assembly is too large.
I think you should create something like a function pointer collection for the one and the other set of implementations you want to use. Every time you want to call a function, you'll be calling from the selected function pointer collection. Having done that, you may also have proxy functions (that simply call from the selected set) to hide the function pointer syntax.
I have to write code that would look do something like that (of course much more complicated):
int stop;
int isStopped;
void workerFunction(){
while(!stop){
//...
}
isStopeed = 1;
}
startThread(){
int newThreadPid = pthread_create(..., NULL, workerFunction, NULL);
}
stopThread(){
stop = 1;
}
My question is: are there any techniques that can make such code testable? I don't think writing tests like :
startThread();
stopThread();
sleep(1);
ASSERT(isStopped);
is the best idea. How to test stopThread() function without calling system function? Is there any way to mock pthread_create?
I only know one way to mock functions in C: Have function pointers to mockable functions and only ever call them through that pointers. For your tests you would adjust the pointers to point to your mock functions instead. But I don't know if you really want to go down that route....
Is there an mechanism or trick to run a function when a program loads?
What I'm trying to achieve...
void foo(void)
{
}
register_function(foo);
but obviously register_function won't run.
so a trick in C++ is to use initialization to make a function run
something like
int throwaway = register_function(foo);
but that doesn't work in C. So I'm looking for a way around this using standard C (nothing platform / compiler specific )
If you are using GCC, you can do this with a constructor function attribute, eg:
#include <stdio.h>
void foo() __attribute__((constructor));
void foo() {
printf("Hello, world!\n");
}
int main() { return 0; }
There is no portable way to do this in C, however.
If you don't mind messing with your build system, though, you have more options. For example, you can:
#define CONSTRUCTOR_METHOD(methodname) /* null definition */
CONSTRUCTOR_METHOD(foo)
Now write a build script to search for instances of CONSTRUCTOR_METHOD, and paste a sequence of calls to them into a function in a generated .c file. Invoke the generated function at the start of main().
Standard C does not support such an operation. If you don't wish to use compiler specific features to do this, then your next best bet might be to create a global static flag that is initialized to false. Then whenever someone invokes one of your operations that require the function pointer to be registered, you check that flag. If it is false you register the function then set the flag to true. Subsequent calls then won't have to perform the registration. This is similar to the lazy instantiation used in the OO Singleton design pattern.
There is no standard way of doing this although gcc provides a constructor attribute for functions.
The usual way of ensuring some pre-setup has been done (other than a simple variable initialization to a compile time value) is to make sure that all functions requiring that pre-setup. In other words, something like:
static int initialized = 0;
static int x;
int returnX (void) {
if (!initialized) {
x = complicatedFunction();
initialized = 1;
}
return x;
}
This is best done in a separate library since it insulates you from the implementation.
I am writing a large C program for embedded use. Every module in this program has an init() function (like a constructor) to set up its static variables.
The problem is that I have to remember to call all of these init functions from main(). I also have to remember to put them back if I have commented them out for some reason.
Is there anything clever I do to make sure that all of these functions are getting called? Something along the lines of putting a macro in each init function that, when you call a check_inited() function later, sends a warning to STDOUT if not all the functions are called.
I could increment a counter, but I'd have to maintain the correct number of init functions somewhere and that is also prone to error.
Thoughts?
The following is the solution I decided on, with input from several people in this thread
My goal is to make sure that all my init functions are actually being called. I want to do
this without maintaining lists or counts of modules across several files. I can't call
them automatically as Nick D suggested because they need to be called in a certain order.
To accomplish this, a macro included in every module uses the gcc constructor attribute to
add the init function name to a global list.
Another macro included in the body of the init function updates the global list to make a
note that the function was actually called.
Finally, a check function is called in main() after all of the inits are done.
Notes:
I chose to copy the strings into an array. This not strictly necessary because the
function names passed will always be static strings in normal usage. If memory was short
you could just store a pointer to the string that was passed in.
My reusable library of utility functions is called "nx_lib". Thus all the 'nxl' designations.
This isn't the most efficient code in the world but it's only called a boot time so that
doesn't matter for me.
There are two lines of code that need to be added to each module. If either is omitted,
the check function will let you know.
you might be able to make the constructor function static, which would avoid the need to give it a name that is unique across the project.
this code is only lightly tested and it's really late so please check carefully before trusting it.
Thank you to:
pierr who introduced me to the constructor attribute.
Nick D for demonstrating the ## preprocessor trick and giving me the framework.
tod frye for a clever linker-based approach that will work with many compilers.
Everyone else for helping out and sharing useful tidbits.
nx_lib_public.h
This is the relevant fragment of my library header file
#define NX_FUNC_RUN_CHECK_NAME_SIZE 20
typedef struct _nxl_function_element{
char func[NX_FUNC_RUN_CHECK_NAME_SIZE];
BOOL called;
} nxl_function_element;
void nxl_func_run_check_add(char *func_name);
BOOL nxl_func_run_check(void);
void nxl_func_run_check_hit(char *func_name);
#define NXL_FUNC_RUN_CHECK_ADD(function_name) \
void cons_ ## function_name() __attribute__((constructor)); \
void cons_ ## function_name() { nxl_func_run_check_add(#function_name); }
nxl_func_run_check.c
This is the libary code that is called to add function names and check them later.
#define MAX_CHECKED_FUNCTIONS 100
static nxl_function_element m_functions[MAX_CHECKED_FUNCTIONS];
static int m_func_cnt = 0;
// call automatically before main runs to register a function name.
void nxl_func_run_check_add(char *func_name)
{
// fail and complain if no more room.
if (m_func_cnt >= MAX_CHECKED_FUNCTIONS) {
print ("nxl_func_run_check_add failed, out of space\r\n");
return;
}
strncpy (m_functions[m_func_cnt].func, func_name,
NX_FUNC_RUN_CHECK_NAME_SIZE);
m_functions[m_func_cnt].func[NX_FUNC_RUN_CHECK_NAME_SIZE-1] = 0;
m_functions[m_func_cnt++].called = FALSE;
}
// call from inside the init function
void nxl_func_run_check_hit(char *func_name)
{
int i;
for (i=0; i< m_func_cnt; i++) {
if (! strncmp(m_functions[i].func, func_name,
NX_FUNC_RUN_CHECK_NAME_SIZE)) {
m_functions[i].called = TRUE;
return;
}
}
print("nxl_func_run_check_hit(): error, unregistered function was hit\r\n");
}
// checks that all registered functions were called
BOOL nxl_func_run_check(void) {
int i;
BOOL success=TRUE;
for (i=0; i< m_func_cnt; i++) {
if (m_functions[i].called == FALSE) {
success = FALSE;
xil_printf("nxl_func_run_check error: %s() not called\r\n",
m_functions[i].func);
}
}
return success;
}
solo.c
This is an example of a module that needs initialization
#include "nx_lib_public.h"
NXL_FUNC_RUN_CHECK_ADD(solo_init)
void solo_init(void)
{
nxl_func_run_check_hit((char *) __func__);
/* do module initialization here */
}
You can use gcc's extension __attribute__((constructor)) if gcc is ok for your project.
#include <stdio.h>
void func1() __attribute__((constructor));
void func2() __attribute__((constructor));
void func1()
{
printf("%s\n",__func__);
}
void func2()
{
printf("%s\n",__func__);
}
int main()
{
printf("main\n");
return 0;
}
//the output
func2
func1
main
I don't know how ugly the following looks but I post it anyway :-)
(The basic idea is to register function pointers, like what atexit function does.
Of course atexit implementation is different)
In the main module we can have something like this:
typedef int (*function_t)(void);
static function_t vfunctions[100]; // we can store max 100 function pointers
static int vcnt = 0; // count the registered function pointers
int add2init(function_t f)
{
// todo: error checks
vfunctions[vcnt++] = f;
return 0;
}
...
int main(void) {
...
// iterate vfunctions[] and call the functions
...
}
... and in some other module:
typedef int (*function_t)(void);
extern int add2init(function_t f);
#define M_add2init(function_name) static int int_ ## function_name = add2init(function_name)
int foo(void)
{
printf("foo\n");
return 0;
}
M_add2init(foo); // <--- register foo function
Why not write a post processing script to do the checking for you. Then run that script as part of your build process... Or better yet, make it one of your tests. You are writing tests, right? :)
For example, if each of your modules has a header file, modX.c. And if the signature of your init() function is "void init()"...
Have your script grep through all your .h files, and create a list of module names that need to be init()ed. Then have the script check that init() is indeed called on each module in main().
If your single module represents "class" entity and has instance constructor, you can use following construction:
static inline void init(void) { ... }
static int initialized = 0;
#define INIT if (__predict_false(!initialized)) { init(); initialized = 1; }
struct Foo *
foo_create(void)
{
INIT;
...
}
where "__predict_false" is your compiler's branch prediction hint. When first object is created, module is auto-initialized (for once).
Splint (and probably other Lint variants) can give a warning about functions that are defined but not called.
It's interesting that most compilers will warn you about unused variables, but not unused functions.
Larger running time is not a problem
You can conceivably implement a kind of "state-machine" for each module, wherein the actions of a function depend on the state the module is in. This state can be set to BEFORE_INIT or INITIALIZED.
For example, let's say we have module A with functions foo and bar.
The actual logic of the functions (i.e., what they actually do) would be declared like so:
void foo_logic();
void bar_logic();
Or whatever the signature is.
Then, the actual functions of the module (i.e., the actual function declared foo()) will perform a run-time check of the condition of the module, and decide what to do:
void foo() {
if (module_state == BEFORE_INIT) {
handle_not_initialized_error();
}
foo_logic();
}
This logic is repeated for all functions.
A few things to note:
This will obviously incur a huge penalty performance-wise, so is
probably not a good idea (I posted
anyway because you said runtime is
not a problem).
This is not a real state-machine, since there are only two states which are checked using a basic if, without some kind of smart general logic.
This kind of "design-pattern" works great when you're using separate threads/tasks, and the functions you're calling are actually called using some kind of IPC.
A state machine can be nicely implemented in C++, might be worth reading up on it. The same kind of idea can conceivably be coded in C with arrays of function pointers, but it's almost certainly not worth your time.
you can do something along these lines with a linker section. whenever you define an init function, place a pointer to it in a linker section just for init function pointers. then you can at least find out how many init functions have been compiled.
and if it does not matter what order the init functions are called, and the all have the same prototype, you can just call them all in a loop from main.
the exact details elude my memory, but it works soemthing like this::
in the module file...
//this is the syntax in GCC..(or would be if the underscores came through in this text editor)
initFuncPtr thisInit __attribute((section(.myinits)))__= &moduleInit;
void moduleInit(void)
{
// so init here
}
this places a pointer to the module init function in the .myinits section, but leaves the code in the .code section. so the .myinits section is nothing but pointers. you can think of this as a variable length array that module files can add to.
then you can access the section start and end address from the main. and go from there.
if the init functions all have the same protoytpe, you can just iterate over this section, calling them all.
this, in effect, is creating your own static constructor system in C.
if you are doing a large project and your linker is not at least this fully featured, you may have a problem...
Can I put up an answer to my question?
My idea was to have each function add it's name to a global list of functions, like Nick D's solution.
Then I would run through the symbol table produced by -gstab, and look for any functions named init_* that had not been called.
This is an embedded app so I have the elf image handy in flash memory.
However I don't like this idea because it means I always have to include debugging info in the binary.