I'm currently working on an embedded device which is able to run unit tests, thanks to the Unity framework. I send the output of these tests into my computer with a JLINK and SEGGER_RTT. Question is, how can i make a web report from unity output?
The best lead I have found, was to transform Unity content to JUnit, in order to have more libraries to work with. Problem is, i didn't find the best approach to have with this JUnit. The idea is to have almost nothing to install, to be able to run tests on a new computer and to have an ergonomic/modern web UI to fastly treat unit tests. The best library i found was Allure (https://github.com/allure-framework/allure2), but I was wondering if it was the best approach (many things to install, and to do before i have anything).
Thomas, have you looked at Ceedling (from the same people that make Unity)? Check out the plugins for it https://github.com/ThrowTheSwitch/Ceedling/tree/master/plugins some of which allow the format of the test ouptut to be adjusted.
Basically Ceedling provides a Ruby build system for Unity with lots of added features like mock generation and the plugin structure - you only need to use the bits you want though
One of the plugins, gcov, also generates test coverage information, which ceedling can also use to generate a HTML test coverage report similar to below
Related
I've used Ceedling in the past on bare-metal projects that don't have many vendor libraries, so creating unit tests and mocking dependencies was pretty easy.
Now I'm trying to integrate Ceedling on a very large project. On this project I'm implementing an application on top of an existing OS and set of libraries provided by some manufacturer. I have the source code for most of it, but not all of it, so my control over those libraries is limited, especially when it reaches low level modules like the scheduler and communications with the other CPU.
So... I decided to give it a go by implementing a very simple unit test that only has one dependency, which I tried to mock. It turned out to be a nightmare right off the bat. That file ended up including a bunch of unrelated modules, so my idea was to mock all those modules until Ceedling stopped complaining. However, I hit a wall when I saw that some of them are automatically generated at compile time in a very obscure way by the ADK.
My question is then, how does one unit test this kind of projects? I feel that I'm missing something very fundamental when dealing with this type of projects, but I can't figure out what.
I have a fork of openssh with some new features and want to write some unit tests to make sure they work at build. Grafting Check into openssh's autotools configuration is diffacult (because I don't really understand autotools)
Is there an easier to use C unit test framework? One less closely tied to autotools?
How about a better way of going about adding Check to this project with out radically changing its build files?
It might not be a good idea to move away from autotools if the objective is to integrate those new features in the main branch. This SO question is pointing to several tutorials for autotools that could be helpful.
Otherwise, there are several unit framework available for C: the most notorious are Cunit, Check, minunit, CMockery, refer to this question or this list.
I am trying to auto-generate Unit Tests for my C code using API sanity autotest.
But, the problem is that it is somewhat complex to use, and some tutorials / howto / other resources on how to use it would be really helpful.
Have you had any luck with API sanity autotest?
Do you think there's a better tool that can be used to auto-generate unit tests for C code?
It is a better tool (among free solutions for Unix) to fully automatically generate smoke tests if your library contains more than hundred functions. The unique feature is an ability to automatically generate reasonable input arguments for each function.
The most popular use case of this framework is a quick search for memory problems (segfaults) in the library. Historically, this framework was used to create LSB certification test suites for too big libraries like Qt3 and Qt4 that cannot be created manually in reasonable time.
Use the following command to generate, build and execute tests:
api-sanity-checker -l name -d descriptor.xml -gen -build -run
XML descriptor is a simple XML file that specifies version number, paths to headers and shared objects:
<version>
0.3.4
</version>
<headers>
/usr/local/libssh/0.3.4/include/
</headers>
<libs>
/usr/local/libssh/0.3.4/lib/
</libs>
You can improve generated tests using specialized types for input parameters.
See example of generated tests for freetype2 2.4.8.
It's a recipe for disaster in the first place. If you auto-generate unit tests, you're going to get a bunch of tests that don't mean a lot. If you have a library that is not covered in automated tests then, by definition, that library is legacy code. Consider following the conventional wisdom for legacy code...
For each change:
Pin behavior with tests
Refactor to the open-closed principle (harder to do with C but not impossible)
Drive changes for new code with tests
Also consider picking up a copy of Working Effectively with Legacy Code.
EDIT:
As a result of our discussion, it has become clear that you only want to enforce some basic standards, such has how null pointer values are handled, with your generated tests. I would argue that you don't need generated tests. Instead you need a tool that inspects a library and exercises its functions dynamically, ensuring that it meets some coding standards you have defined. I'd recommend that you write this tool, yourself, so that it can take advantage of your knowledge of the rules you want enforced and the libraries that are being tested.
I'm starting a new project and decided that I should give this unit testing thing that everybody keeps talking about a try.
The project is a set of C libraries (so no UI or user interaction testing is necessary) and aimed at being cross platform, with Linux, FreeBSD and Windows being my first priority and OS X planned once the first release is out the door (assuming I can get a hold of a machine running OS X to test on).
Does anybody have any experience or recommendations for a good C unit testing framework that easily works across multiple platforms?
I used to use CUnit which I liked. Also google have open sourced their C++ unit testing framework.
Back when I was writing C code all of my unit tests were ad-hoc, no framework involved, so I can't recommend any of these, but you may want to look at the list of C/C++ unit testing tools at http://www.opensourcetesting.org/unit_c.php.
I started with minunit then evolved it to suit my requirements. Rather than having support for test suites from a testing framework, I just use make to build and run different executables, each with a dozen or so related tests in them. So now I've about 250 lines of macros, mainly for printing out different types or doing string or memory comparison, and that's enough for now.
I've grown quite partial to tap (scroll down near the bottom) because its easy to drop in place. It stands for "test anything protocol". I don't think you'd hit many portability issues with it if using compilers released in the last 5 years, but I've only tried it with gcc / glibc.
I use my own unit testing framework CUnitWin32 for embedded C stuff. My host dev environment is Win32 so the framework that is available on-line is for Windows only. However, I did port it to Linux as well, so if the framework works for you, drop me a comment here and we'll work out a way to get your the Linux version.
D
I'm working on a free software (bsd license) project with others. We're searching for a system that check out our source code (svn) and build it also as test it (unit tests with Check / other tools).
It should have a webbased interface and generate reports.
I hope we don't have to write such a system from null by ourselves...
You surely do not have to code this yourself - there are a lot of continuous integration systems which are able to check out source code from systems such as SVN and they are generally easy to extend with your own tasks, so running custom test scripts/programs should not be a problem.
While these CI systems are probably not written in C, this does not matter, since they just need to be able to access and compile your source code, for which they will use an external compiler anyways.
Just to list some of the well known CI tools:
CruiseControl
Hudson
TeamCity
You might also be interested in other questions on Stack Overflow tagged as continuous-integration. :)
I don't think that there's a buildsystem that is capable of doing all this tasks - but what about combining them?
SCons is a nice buildsystem that runs on every machine that has Python. It can even build directly from SVN. For automatic building you can try Buildbot.
Check out buildbot
My vote would be CruiseControl.NET, it has everything you are asking for. It is open source so the costs are low, and it has a very active user community on google groups to help you with your problems as you grow accustomed to it. Also, although .NET based, using MONO it is very nice on Linux and Mac build servers as well so you have everything covered.