I create a project for a microcontroller by programming it in C language. Due to its specificity (a microcontroller with a built-in BLE core), I have to use the SDK and a specific project template. How can I test my modules when they have numerous references to other files (modules) in the SDK? (References are needed to use functions to, for example, send data via BLE) Do I have to somehow mock each of the SDK functions? I am using Unity test framework.
Module example:
my_module.c
#include "sdk_module_1.h"
#include "my_module.h"
void init_hardware(void)
{
//function code
}
bool send_data(int data)
{
//prepare data eq.
data++
//send data using SDK function (sdk_module_1.h)
return send_data(data);
}
my_module.h
void init_hardware(void)
void send_data(int data)
my_module_test.c
#include "my_module.h"
#include "//unity files"
TEST_SETUP(Test)
{
}
TEST_TEAR_DOWN(Test)
{
}
TEST(Test, First_test)
{
TEST_ASSERT_EQUAL(send_data(5),true);
}
When I try to test my module, I have a problem with referencing SDK modules and their functions. How can I create tests for such software? Should I change the way my modules are written?
The resource you want is James Grenning's Test Driven Development for Embedded C.
(Note: what follows below is a translation of my ideas into C. If you find a conflict with Grenning's approach, try his first - he has a lot more laps in the embedded space than I do.)
How can I test my modules when they have numerous references to other files (modules) in the SDK?
Sometimes the answer is that you have to change your design. In other words, treating testability as a design constraint, rather than an afterthought.
The way I normally describe it is this: we want to design our code such that (a) all the complicated code is easy to test and (b) anything that's hard to test is "so simple there are obviously no deficiencies".
This often means designing our code so that collaborations between complicated code and hard to test code are configurable, allowing you to provide a substitute implementation (stub/mock/test double) when using the real thing isn't cost effective.
So instead of having A (complicated) directly invoke B (hard for testing), we might instead have A invoke B via a function pointer that, during testing, can be replaced with a pointer to a simpler function.
(In some styles, this gets reversed: the complicated logic points to an inert/stub implementation by default, and you opt in to using the complicated implementation instead.)
In other words, we replace
void A(void) {
B(); // B is the function that makes things hard to test.
}
with
void A(void) {
C(&B);
}
# It's been a long time, please forgive (or correct) the spelling here
void C( void (*fn)()) {
&fn();
}
We test A by looking at it, agreeing that it is "so simple there are obviously no deficiencies", and signing off on it. We test C by passing it pointers to substitute implementations, writing as many substitutes as we need for B to ensure that all of C's edge cases are covered.
Note that "hard to test" can cover a lot of bases - the real function is slow, the real function is unstable, the real function costs money... if it makes testing less pleasant or less effective, then it counts.
Firstly, I would highly recommend using Ceedling to manage and run your unit tests in C. It wraps Unity and CMock very nicely and makes unit testing, particularly for embedded systems, a lot easier than it otherwise would be.
With regards to unit testing when an SDK is involved, you first need to remember that the point of a unit test is to test a unit, so anything outside of that needs to be mocked or stubbed, otherwise it’s not a unit test and cannot be run as one.
So, for something like an I2C hardware abstraction module, any calls to an underlying SDK (that would ordinarily do the actual I2C transaction) need to be mocked so that you can place expectations on what should happen when the call is made instead of the real thing. In this way, the unit test of the I2C HAL module is exercising only its behaviour and how it handles its calls, and nothing else, as it should.
Therefore, all that you generally need to do to unit test with an SDK is to ensure that any module you use has its public functions mocked. You then simply include the mock in the test, rather than the real module.
Now, I find that you don’t really want to mess with the original SDK and its structure for the purpose of unit testing, and moreover you don’t want to change how the real code includes the real modules, so the problem for the test then comes when the SDK function you’re calling in your code sits behind layers of other modules in the SDK, as is often the case. Here, you don’t want to mock everything you’re not using, but you do need to match the include structure. I’ve found that a good way to do this is simply to create a support folder and copy any top-level headers that the code uses from the SDK into it. I then configure the test to include from support before it includes from the SDK.
Once you have this kind of structure in place, unit testing with an SDK is easy. Ceedling will help with all of this.
In our environment we're encountering a problem regarding mocking functions for our library unit tests.
The thing is that instead of mocking whole modules (.c files) we would like to mock single functions.
The library is compiled to an archive file and linked statically to the unit test. Without mocking there isn't any issue.
Now when trying to mock single functions of the library we would get multiple definitions obviously.
My approach now is to use the weak function attribute when compiling/linking the library so that the linker takes the mocked (non-weak) function when linking against the unit test. I already tested it and it seems to work as expected.
The downside of this is that we need many attribute declarations in the code.
My final approach would be to pass some compile or link arguments to the compiler, that every function is automatically declared as a weak symbol.
The question now is: Is there anything to do this in a nice way?
btw: We use clang 8 as a compiler.
James Grenning describes several options to solve this problem (http://blog.wingman-sw.com/linker-substitution-in-c-limitations-and-workarounds). The option "function pointer substitution" gives a high degree of freedom. It works as follows: Replace functions by pointers to functions. The function pointers are initialized to point to the original function, but each pointer can be redirected individually to a test double.
This approach allows to have one single test executable where you can still decide for each test case individually for which function you use a test double and for which you use the original function.
It certainly also comes at a price:
One indirection for each call. But, if you use link-time-optimization the optimizer will most likely eliminate that indirection again, so this may not be an issue.
You make it possible to redirect function calls also in production code. This would certainly be a misuse of the concept, however.
I would suggest using VectorCAST
https://www.vector.com/us/en/products/products-a-z/software/vectorcast/
I've used, unity/cmock and others for unit testing C in the past, but after a while its vary tedious to manually create these for a language that isnt really built around that concept and is very much a heres a Hammer and Chissel the world is yours approach.
VectorCAST abstracts majority of the manual work that is required with tools like Unity/Cmock, we can get results across a project/module sooner and quicker than we did in the past with the other tools.
Is vectorCAST expensive and very much an enterprise level tool? yes... but its defiantly worth its weight in gold. And thats coming from someone who is very old school, manual approach to software development... just text editors, terminals and commandline debuggers.
VetorCAST handles function pointers and pointers extremely well, stubbing functions is easy as two clicks away. It saved our team alot of time... allowing us to focus on results and reducing the feedback loop of development.
I am trying to implement TDD in C coding. I am building the program structure in a quite modularised way and using as atomic functions as possible. I make one test file (including several suits) for one module (module = header file + source file). I am struggling to make the program files "not know that they are being tested", in other words - I don't want testing parts of code in the proper program. Therefore almost often I need to include the source file in the test file in order to have access to the "private" variables and functions.
That was the intro, now the problem: if in a module I have an aaa() function, which uses inside a bbb() function, which uses some xxx() function from an external module, I can easily test the bbb() function in the atomic way by mocking the x() function: #define bbb mock_bbb and providing a mock xxx module for #include. However, I am unable to find a way of atomic testing of the aaa() function, which uses a function from the same module. Is it possible to do? (note, that apart of mocking bbb() for aaa(), I have to be also able to use the original bbb() to test it)
My closest try was to use -Wl,-wrap,xxx, but the problem is that I haven't found a way to automate this (wildcard or something?) - I will have almost 100 testing files, each containing several functions to test - I cannot allow myself to put manually every function in the makefile.
I never test "private" functions in an atomic way. I usually unit-test a c-module using its public functions and checking its calls to other modules (using mocks through dependency injection) and checking its private data members (by exposing its private data members with a GetDataPtr()-function that is compiled only for the unit test project).
For me that the best tradeof between effort and complexity of the unit test framework, although it not possible to reach 100% statement coverage in some "private" functions.
I'm working on an embedded C project that depends on some external HW. I wish to stub out the code accessing these parts, so I can simulate the system without using any HW. Until now I have used some macros but this forces me to change a little on my production code, which I would like to avoid.
Example:
stub.h
#ifdef _STUB_HW
#define STUB_HW(name) Stub_##name
#else /*_STUB_HW*/
#define STUB_HW(name) name
#endif /*_STUB_HW*/
my_hw.c
WORD STUB_HW(clear_RX_TX)()
{ /* clear my rx/tx buffer on target HW */ }
test_my_hw.c
#ifdef _STUB_HW
WORD clear_RX_TX()
{ /* simulate clear rx/tx buffer on target HW */ }
With this code I can turn on/off the stubbing with the preprocessor tag _STUB_HW
Is there a way to acomplish this without having to change my prod code, and avoiding a lot of ifdefs. And I won't mix prod and test code in the same file if I can avoid it. I don't care how the test code looks as long as I can keep as much as possible out of the production code.
Edit:
Would be nice if it was posible to select/rename functions without replacing the whole file. Like take all functions starting on nRF_## and giving then a new name and then inserting test_nRF_## to nRF_## if it is posible
I just make two files ActualDriver.c and StubDriver.c containing exactly the same function names. By making two builds linking the production code against the different objects there is no naming conflicts. This way the production code contains no testing or conditional code.
As Gerhard said, use a common header file "driver.h" and separate hardware layer implementation files containing the actual and stubbed functions.
In eclipse, I have two targets and I "exclude from build" the driver.c file that is not to be used and make sure the proper one is included in the build. Eclipse then generates the makefile at build time.
Another issue to point out is to ensure you are defining fixed size integers so your code behaves the same from an overflow perspective. (Although from your code sample I can see you are doing that.)
I agree with the above. The standard solution to this is to define an opaque abstracted set of function calls that are the "driver" to the hw, and then call that in the main program. Then provide two different driver implementations, one for hw, one for sw. The sw variant will simulate the IO effect of the hw in some appropriate way.
Note that if the goal is at a lower level, i.e., writing code where each hardware access is to be simulated rather than entire functions, it might be a bit tricker. But here, different "write_to_memory" and "read_from_memory" functions (or macros, if speed on target is essential) could be defined.
There is no need in either case to change the names of functions, just have two different batch files, make files, or IDE build targets (depending on what tools you are using).
Finally, in many cases a better technical solution is to go for a full-blown target system simulator, such as Qemu, Simics, SystemC, CoWare, VaST, or similar. This lets you run the same code all the time, and instead you build a model of the hardware that works like the actual hardware from the perspective of the software. It does take a much larger up-front investment, but for many projects it is well worth the effort. It basically gets rid of the nasty issue of having different builds for target and host, and makes sure you always use your cross-compiler with deployment build options. Note that many embedded compiler suites come with some basic such simulation ability built in.
I've heard that there are some things one cannot do as a computer programmer, but I don't know what they are. One thing that occurred to me recently was: wouldn't it be nice to have a class that could make a copy of the source of the program it runs, modify that program and add a method to the class that it is, and then run the copy of the program and terminate itself. Is it possible for code to write code?
If you want to learn about the limits of computability, read about the halting problem
In computability theory, the halting
problem is a decision problem which
can be stated as follows: given a
description of a program and a finite
input, decide whether the program
finishes running or will run forever,
given that input.
Alan Turing proved in 1936 that a
general algorithm to solve the halting problem for all
possible program-input pairs cannot exist
Start by looking at quines, then at Macro-Assemblers and then lex & yacc, and flex & bison. Then consider self-modifying code.
Here's a quine (formatted, use the output as the new input):
#include<stdio.h>
main()
{
char *a = "main(){char *a = %c%s%c; int b = '%c'; printf(a,b,a,b,b);}";
int b = '"';
printf(a,b,a,b,b);
}
Now if you're just looking for things programmers can't do look for the opposite of np-complete.
Sure it is. That's how a lot of viruses work!
Get your head around this: computability theory.
Yes, that's what most Lisp macros do (for just one example).
Yes it certainly is, though maybe not in the context you are referring to check out this post on t4.
If you look at Functional Programming that has many opportunities to write code that generates further code, the way that a language like Lisp doesn't differentiate between code and data is a significant part of it's power.
Rails generates the various default model and controller classes from the database schema when it's creating a new application. It's quite standard to do this kind of thing with dynamic languages- I have a few bits of PHP around that generate php files, just because it was the simplest solution to the problem I was dealing with at the time.
So it is possible. As for the question you are asking, though- that is perhaps a little vague- what environment and language are you using? What do you expect the code to do and why does it need to be added to? A concrete example may bring more directly relevant responses.
Yes it is possible to create code generators.
Most of the time they take user input and produce valid code. But there are other possibilities.
Self modifying programes are also possible. But they were more common in the dos era.
Of course you can! In fact, if you use a dynamic language, the class can change itself (or another class) while the program is still running. It can even create new classes that didn't exist before. This is called metaprogramming, and it lets your code become very flexible.
You are confusing/conflating two meanings of the word "write". One meaning is the physical writing of bytes to a medium, and the other is designing software. Of course you can have the program do the former, if it was designed to do so.
The only way for a program to do something that the programmer did not explicitly intend it to do, is to behave like a living creature: mutate (incorporate in itself bits of environment), and replicate different mutants at different rates (to avoid complete extinction, if a mutation is terminal).
Sure it is. I wrote an effect for Paint.NET* that gives you an editor and allows you to write a graphical effect "on the fly". When you pause typing it compiles it to a dll, loads it and executes it. Now, in the editor, you only need to write the actual render function, everything else necessary to create a dll is written by the editor and sent to the C# compiler.
You can download it free here: http://www.boltbait.com/pdn/codelab/
In fact, there is even an option to see all the code that was written for you before it is sent to the compiler. The help file (linked above) talks all about it.
The source code is available to download from that page as well.
*Paint.NET is a free image editor that you can download here: http://getpaint.net
In relation to artificial intelligence, take a look at Evolutionary algorithms.
make a copy of the source of the program it runs, modify that program and add a method to the class that it is, and then run the copy of the program and terminate itself
You can also generate code, build it into a library instead of an executable, and then dynamically load the library without even exiting the program that is currently running.
Dynamic languages usually don't work quite as you suggest, in that they don't have a completely separate compilation step. It isn't necessary for a program to modify its own source code, recompile, and start from scratch. Typically the new functionality is compiled and linked in on the fly.
Common Lisp is a very good language to practice this in, but there are others where you can created code and run it then and there. Typically, this will be through a function called "eval" or something similar. Perl has an "eval" function, and it's generally common for scripting languages to have the ability.
There are a lot of programs that write other programs, such as yacc or bison, but they don't have the same dynamic quality you seem to be looking for.
Take a look at Langtom's loop. This is the simplest example of self-reproducing "program".
There is a whole class of such things called "Code Generators". (Although, a compiler also fits the description as you set it). And those describe the two areas of these beasts.
Most code generates, take some form of user input (most take a Database schema) and product source code which is then compiled.
More advanced ones can output executable code. With .NET, there's a whole namespace (System.CodeDom) dedicated to the create of executable code. The these objects, you can take C# (or another language) code, compile it, and link it into your currently running program.
I do this in PHP.
To persist settings for a class, I keep a local variable called $data. $data is just a dictionary/hashtable/assoc-array (depending on where you come from).
When you load the class, it includes a php file which basically defines data. When I save the class, it writes the PHP out for each value of data. It's a slow write process (and there are currently some concurrency issues) but it's faster than light to read. So much faster (and lighter) than using a database.
Something like this wouldn't work for all languages. It works for me in PHP because PHP is very much on-the-fly.
It has always been possible to write code generators. With XML technology, the use of code generators can be an essential tool. Suppose you work for a company that has to deal with XML files from other companies. It is relatively straightforward to write a program that uses the XML parser to parse the new XML file and write another program that has all the callback functions set up to read XML files of that format. You would still have to edit the new program to make it specific to your needs, but the development time when a new XML file (new structure, new names) is cut down a lot by using this type of code generator. In my opinion, this is part of the strength of XML technology.
Lisp lisp lisp lisp :p
Joking, if you want code that generates code to run and you got time to loose learning it and breaking your mind with recursive stuff generating more code, try to learn lisp :)
(eval '(or true false))
wouldn't it be nice to have a class that could make a copy of the source of the program it runs, modify that program and add a method to the class that it is, and then run the copy of the program and terminate itself
There are almost no cases where that would solve a problem that cannot be solved "better" using non-self-modifying code..
That said, there are some very common (useful) cases of code writing other code.. The most obvious being any server-side web-application, which generates HTML/Javascript (well, HTML is markup, but it's identical in theory). Also any script that alters a terminals environment usually outputs a shell script that is eval'd by the parent shell. wxGlade generates code to that creates bare-bone wx-based GUIs.
See our DMS Software Reengineering Toolkit. This is general purpose machinery to read and modify programs, or generate programs by assembling fragments.
This is one of the fundamental questions of Artificial Intelligence. Personally I hope it is not possible - otherwise soon I'll be out of a job!!! :)
It is called meta-programming and is both a nice way of writing useful programs, and an interesting research topic. Jacques Pitrat's Artificial Beings: the conscience of a conscious machine book should interest you a lot. It is mostly related to meta-knowledge based computer programs.
Another related term is multi-staged programming (because there are several stages of programs, each generating the next one).