I am trying to get started developing unittests for my embedded system, actually its a firmware that was already developed a while ago, now we want to add CI to it and start this by writing unittests.
I read blogs like
https://dmitryfrank.com/articles/unit_testing_embedded_c_applications
or https://interrupt.memfault.com/blog/unit-testing-basics and they recommend to implement the unittests via a unittest framework as unity, cpputest and so on and so on. I tried to figure out how to make this run on my windows machine that I use at work but it seems tedious and prone to problems etc.
I also tried to start working with the microsoft vs unit testing framework but got discouraged after running into problems here, too. I think this can't be so hard, it shouldn't take so much effort to start writing the first littlest test...
Do you have experience with this and what would you recommend? Is it viable to do this on Windows? Should I push my boss to let me use a VM, WSL or a Machine that runs Ubuntu? In the end I would like to transfer the CI System to a lab pc (edit: this pc would then also be connected to the hardware, flash it, maybe run a debug build on it and compile some answers that it gets from the hardware... but now I want to talk about the unit test), so if I start developing on Ubuntu now, I will have to deal with this lateron, too...
Maybe a lot of thoughts but I feel a bit overwhelmed with the problem and don't know how to start.
Thanks for you replies, people!
Edit: The discussion I am looking for is not about whether unit testing for embedded systems is useful or the right approach (also interesting but I would like to seperate it) . It's about the question whether it can be fully developed under Windows or whether a Linux System is necessary to be efficient.
Related
I'm taking my university's trial by fire C course and I need some help with the environment I'm programming in.
So, all assignments have to be uploaded to a linux remote server. All the tests suites they provide are run on this sever, using that servers specific compiler settings. (Which is why I'm really worried about just using an IDE, finishing the project, and then uploading the files to the server. I'm scared the compiler settings will be different and everything will just stop working when moved into a different environment).
Since coding on a remote at the same time as 1000 other students results in lag and crashes that makes the already unintuitive and painful experience of using VIM in a puTTy window even worse is an unbearable experience, what are my options?
Has anyone got some advice on what would be the best option for me in this situation?
I can't figure out how a desktop environment developer test his code. Usually, a C or C++ programmer compiles his code an then run it (i'm not one of those programmers, i'm a web one).
So, you usually build your gui application over some kind of desktop environment (windows, mac os x, gnome, kde, xfce...), sow how they build and test their gui desktop?
And if this is a silly question, how does a kernel programmer test his code? for example linux kernel? how do you know that what you just wrote works?
Testing is a very broad term there are many types (partial list):
unit tests - test small pieces of code. test that the code behaves as expected.
system tests - test whole application in real world scenarios.
performance tests - test what is the performance of the application or part of it.
GUI testing - test operation of GUI elements (not so common as automated tests)
static analysis - compiler warnings on steroids
dynamic analysis - at a minimum memory checks - check mem allocations and usage
coverage tests - check that all code is executed.
formal verification tests (very advanced) - e.g. check when assertions/assumptions are broken.
Kernel code can be debugged by connecting using a 2nd computer (host). Virtual machines uses the same principal and simplify the setup but can't always work as HW might not exist in the guest VM.
The kernel (all OSes) has trace mechanism(s) for printing progress/problems. In Linux the simple trace is shown via the dmesg command (prints a cyclic buffer).
User mode code can easily be stopped and debugged via a debugger.
Desktop Environments
Testing Desktop Environments in real world scenarios can be kind of annoying, so the developer would have to watch out for every small error he makes, if he doesn't, he will have a hard time developing the DE.
As stated by #egur, there are multiple ways of testing his code, the easiest one and most important (but cannot be used in some cases, of course), he can test that code in a simplified program.
A Desktop Environment consists of many parts, however, in your case, I suppose you're talking about the session manager (or window manager) which is responsible for almost everything. So, if he were to test that, he would simply exit his current DE and use the new executable. In case of some error, he can always keep a backup of the old executable or fix the faulty code using some commandline text editor (like vim, or nano).
Kernel
It's quite hard to test, some kernel developers just write some code and make sure it's fine and compiles, then simply let his users test (by ACK'ing the code, etc.), then it can be submitted into the kernel code. Reasoning behind that is, the developer may not have the hardware needed to test the code.
Right now, you can compile and run the kernel in usermode (UML) if you have heard of it, so some developers may go for it. However, some developers may also want to test it themselves (They of course back up the current kernel incase of a screw up).
The way to test a desktop application is related to the way of control the application unassisted or remotely.
The Cross Platform GUI Test Automation tool (I don't know if this project has a web) project helps you to chose the interfaces/libraries required to solve the problem.
In Linux[1] uses the accessibility libraries to control the application, you have Cobra[2] for Windows and PyATOM[3] for MacOS, but I don't know what kind of technology uses in this platforms.
http://ldtp.freedesktop.org/wiki/
https://github.com/ldtp/cobra
https://github.com/pyatom/pyatom
Has anybody experienced using the Invensense IMU3000 with some microcontroller?
I am trying to build the IMU library for a PIC but I am stuck with the dependencies.. any other experience with others microcontrollers will be nice as well!
Basically I don't get whether it is better to take the Visual Studio 2005 project and make the changes there, adding the PIC dependencies (I get stuck..) or compile the whole library in the PIC environment..
Any hint, even with other platforms, would help!
Thank you all!
PC and PIC programming are so very different... Also there are so many pIC variants, they are hugely different from each other, it's hard to answer such an open ended question. However, basically you're writing mathematical algorithms. So write these as ansi c functions, hosted with a load of PC things (dialogs etc) and once they're working, you can move just the math functions over to the PIC - having already got a framework running on the PIC, ready to receive the algorithms. BUT - take care with memory. You have bags of it on the PC, you have to be mean with memory once you work on a PIC. Good luck, enjoy!
While it can be helpful to write code on the PC that will eventually move to the PIC, you will need to make sure that all code that will move has been written with portability in mind. That is, you cannot assume that code that compiles and works perfectly under Visual Studio will work without modification on any other platform.
To run in the PIC, all of the code must be compiled with cross development tools that are designed to target the PIC.
That said, I often develop algorithms and detailed processing code in the PC where a test suite can be easily used to verify its operation, and then recompile it for my target platform.
Incidentally, Google tells me that the IMU-3000 is a MEMS Gyro. It would probably be helpful to include at least the link to its data sheet in the question.
I'm starting a new project and decided that I should give this unit testing thing that everybody keeps talking about a try.
The project is a set of C libraries (so no UI or user interaction testing is necessary) and aimed at being cross platform, with Linux, FreeBSD and Windows being my first priority and OS X planned once the first release is out the door (assuming I can get a hold of a machine running OS X to test on).
Does anybody have any experience or recommendations for a good C unit testing framework that easily works across multiple platforms?
I used to use CUnit which I liked. Also google have open sourced their C++ unit testing framework.
Back when I was writing C code all of my unit tests were ad-hoc, no framework involved, so I can't recommend any of these, but you may want to look at the list of C/C++ unit testing tools at http://www.opensourcetesting.org/unit_c.php.
I started with minunit then evolved it to suit my requirements. Rather than having support for test suites from a testing framework, I just use make to build and run different executables, each with a dozen or so related tests in them. So now I've about 250 lines of macros, mainly for printing out different types or doing string or memory comparison, and that's enough for now.
I've grown quite partial to tap (scroll down near the bottom) because its easy to drop in place. It stands for "test anything protocol". I don't think you'd hit many portability issues with it if using compilers released in the last 5 years, but I've only tried it with gcc / glibc.
I use my own unit testing framework CUnitWin32 for embedded C stuff. My host dev environment is Win32 so the framework that is available on-line is for Windows only. However, I did port it to Linux as well, so if the framework works for you, drop me a comment here and we'll work out a way to get your the Linux version.
D
I'm working on a free software (bsd license) project with others. We're searching for a system that check out our source code (svn) and build it also as test it (unit tests with Check / other tools).
It should have a webbased interface and generate reports.
I hope we don't have to write such a system from null by ourselves...
You surely do not have to code this yourself - there are a lot of continuous integration systems which are able to check out source code from systems such as SVN and they are generally easy to extend with your own tasks, so running custom test scripts/programs should not be a problem.
While these CI systems are probably not written in C, this does not matter, since they just need to be able to access and compile your source code, for which they will use an external compiler anyways.
Just to list some of the well known CI tools:
CruiseControl
Hudson
TeamCity
You might also be interested in other questions on Stack Overflow tagged as continuous-integration. :)
I don't think that there's a buildsystem that is capable of doing all this tasks - but what about combining them?
SCons is a nice buildsystem that runs on every machine that has Python. It can even build directly from SVN. For automatic building you can try Buildbot.
Check out buildbot
My vote would be CruiseControl.NET, it has everything you are asking for. It is open source so the costs are low, and it has a very active user community on google groups to help you with your problems as you grow accustomed to it. Also, although .NET based, using MONO it is very nice on Linux and Mac build servers as well so you have everything covered.