I am new to testing field. I would like to firstly ask in what ways can a C application be tested (a C framework or a C tool), how should I start, what are the steps also which are the best tools I can use for testing C code.
Need some help and some documentation.
Thx
What a unit testing tool or frame work usually does is automate all input sets and check outputs for valid results as well as do negative tests i.e. put invalid values and see appropriate response, such as the system should at least remain stable. For e.g. if a function says it only processes positive numbers, ideally it should be able to say "invalid data" when passed a negative number, instead of giving wrong answers or worst getting crashed)
On an api level say if you have a function which takes a number and returns it's square, you write a script (or have a tool) which calls that function repeatedly passing it all valid inputs (or at least all inputs of different types such that each class is covered).
This would mean testing boundary conditions (min max values), basic use case conditions and negative conditions etc.
Beyond unit tests you can do white box testing. Such as code coverage i.e. ensuring you have executed test cases which cover most if not all code paths.
Automating some/most of above so that they can be repeatedly executed and validated every time a change is made is called regression testing.
Then there are several other areas of testing such as localization, globalization, security testing etc. to name a few.
Related
For a certain time now, I'm looking to build a logging framework in C (not C++!), but for small microcontrollers or devices with a small footprint of some sort. For this, I've had the idea of hashing the strings that are being logged to a certain value and just saving the hashed value with the timestamp instead of the complete ASCII string. The hash can then be correlated with a 'database' file that would be generated from an external process that parses the strings out of the C source files and saves the logged strings along with the hash value.
After doing a little bit of research, this idea is not new, but I do not find an implementation of this idea in C. In other languages, this idea has been worked out, but that is not the goal of my exercise. An example may be this talk where the same concept has been worked out in C++: youtube.com/watch?v=Dt0vx-7e_B0
Some of the requirements that I've set myself for this library are the following:
as portable C code as possible
COMPILE TIME optimization/hashing for the string hash conversion, it should be equivalent to just printf("%d\n", hashed_value) for a single log statement. (Assuming no parameters/arguments for this particular logging statement).
arguments can be passed to the logging statement similar to the printf function.
user can define their own output function (being console, file descriptor, sending the data directly over an UART connection,...)
fast to run!! fast to compile is nice to have, but it should not be terribly slow.
very easy to use, no very complicated API to use the library.
But to achieve this in C, what is a good approach? I've tried several things now, but do not seem to have found a good method of achieving this.
An overview of things I've tried so far, along with the drawbacks are:
Full pre-processor string hashing: did get it working, but the compile time is terribly slow. Also, this code does not feel to be very portable over multiple C compilers.
Semi pre-processor string hashing: The idea was to generate a hash for each string and make an external header file with the defines in of each string with their hash value. The problem here is that I cannot figure out a way of converting the string to the correct define preprocessor value.
Letting go of the default logging macro with a string pointer: Instead of working with the most used method of LOG_DEBUG("Some logging statement"), converting it with an external parser to /*LOG_DEBUG("Some logging statement") */ LOG_RAW(45). This solves the problem of hashing the string since the hash will be replaced by the external parser with the correct hash, but is not the cleanest to read since the original statement will be a comment.
Also expanding this idea to take care of arguments proved to be tricky. How to take care of multiple types of variables as efficiently as possible?
I've tried some other methods but all without success. Especially when I want to add arguments to log the value of a variable, for example, it gets very complicated, and I do not get the required result...
I have found that in CLion it is possible to change functions signature, e.g. parameters' number and/or order alongside with the all places where it is called from. However it allows to do it only from a dialog window and in one-by-one manner, as far as I understood. Is there a way to write a script which will do this for all signatures and calls taken from the script or kind of input data? In other words I want to find a way to quickly refactor all code base having slightly changed something in the "task" script. I mean to modify the refactoring from time to time in one place/file and run it against all code base. Otherwise it is a lot of manual work even for one refactoring concept. But what if it is changed everyday. Here is how it is done in Clion in a one-by-one manner.
https://www.jetbrains.com/help/rider/Refactorings__Change_Signature.html
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I writing an application in C. I am new to writing unit tests. I will be using the glib testing framework.
I read this article in wikipedia. I am unsure of what my unit tests should cover.
I know that my unit test should check whether for a valid input, the expected result is obtained. Is this all that needs to be done while writing a unit test for a function?
Should I also check the value of each variable every time it is modified? Because if the functionality is extended, then more variables might be added and current variables might be modified at various other places, so then I will have to change the test itself.
Please give me your input.
I am unsure of what my unit tests should cover.
Maybe it helps to look at unit tests as a specification (runnable documentation!) for your code.
What is considered valid input?
What happens when the code is given valid input? Does it produce the expected output? (Don't just test one valid input value; focus on maybe 1 typical case, then all edge cases and extremes that should still just work.)
What happens when the code is given invalid input? Are the expected errors / error codes produced?
What environment is the code dependent on?
What happens when that environment isn't properly set up?
The last two points are actually special cases of the first three. That's because unit tests don't necessarily test single, isolated functions:
"Input" doesn't just have to be a function argument. It can be any kind of program state that your code will read (i.e. depends on).
In the same way, "output" is not just a return value of a function. It can be any variable or program state that your code modifies.
Your unit tests might not test just one single isolated function, but the interplay between several functions that must called in sequence to get something done. Read as documentation/specification, such a unit test would suggest that calling the functions in that order is an appropriate or even suggested way to get some task done.
Should I also check the value of each variable every time it is modified?
Unit tests are completely separate from what they are testing (often called the System Under Test, abbreviated to SUT). That is, your unit tests should be separate functions wherein you set up the SUT, exercise it, and then check the outcome against the expected result.
Therefore your unit test functions will be very simple:
set up the input for your SUT.
call/exercise the SUT.
read the output of the SUT and compare against expected output.
As you can see, there's not much room in such a simple procedure for variables that will change their value all the time. If you have such a unit test, chances are that it's too complex and you might want to change it, e.g. split it up.
Changing variables are more likely seen in the tested code (i.e. in the SUT) itself. But that's not where you put your test logic. That goes into a completely separate function, which makes up your unit test.
(Note that I'm speaking very generally, since you haven't said what framework you are using for your unit tests, so I might be slightly off on some issues.)
I´m searching information about how to compare two codes and decide if the code submitted by someone is correct or not (based on a solution code defined before).
I could compare the output but many codes may have the same output. Then I think I must compare someway the codes and give a percentage of similitude.
Anybody can help me?
(the language code is C but I think this isn´t important)
Some of my teachers used online automated program grading systems like http://web-cat.org/
In the assignment they would specify a public api you must provide, and then they would just write tests against your functions, much like unit tests. They would intentionally pick tests that would exploit boundary conditions and other things students are notorious for not thinking about, and just call your code with many different inputs to try to get your code to fail.
Sometimes they would hardcode the expected values, other times they would allow values within a range, and other times they just did the assignment themselves and made it so your own code has to match the results produced by their code.
Obviously, not all programs can be effectively graded this way. It's also kinda error prone in that sometimes even the teacher made a mistake and overflowed an int or something, then the correct student submissions wouldn't match the teachers incorrect results. But, a system doesn't need to be perfect to be useful. But I think this raises an important point in that manually grading by reading the code won't necessarily reveal all mistakes either.
Another possibility is copy the submitted code, strip out all of the white space and search for substrings that must exist for the code to be correct and/or substrings that cannot exist for the code to be considered correct. The troublesome bit might be setting up to allow for some of the more tricky requirements such as [(a or c),((a or b) and c),((a or b) and c)], where the variables are the result of a boolean check as to if the substring related to the variable exists within the code.
For example, [("printf"),("for"), (not "1,2,3,4,5,6,7,9,10")], would require that "printf" and "for" be substrings in the code, while "1,2,3,4,5,6,7,9,10" i I'm not familiar with C, so I'm I'm assuming here that "printf" is required to be able to print anything without involving output streams, which could be accounted for by something like [("printf" or "out"),("for"), (not "1,2,3,4,5,6,7,9,10")], where "out" is part of C code required to make use of output streams.
It might be possible to automatically find required substrings based on a "correct" code, but as others have mentioned, there are alternative ways to do things. Which is why hard-coding the "solution" is probably required. Even so, it's quite possible that you'll miss a required substring, and it'll be marked as wrong, but it's probably the only way you can do what you ask with some degree of success.
Regular expressions might be useful here.
I am writing a relatively simple C program in Visual C++, and have two global variables which I would like to know the values of as the program runs. The values don't change once they are assigned, but my programming ability is not enough to be able to quickly construct a text box that displays the values (I'm working in Win32) so am looking for a quick routine that can perhaps export the values to a text file so I can look at them and check they are what they ought to be. Values are 'double'.
I was under the impression that this was the purpose of the debugger, but for me the debugger doesn't run as the 'file not found' is always the case.
Any ideas how I can easily check the value of a global variable (double) in a Win32 app?
Get the debugger working. You should maybe post another question with information about why it won't work - with as much info as possible.
Once you have done that, set a breakpoint, and under Visual C++ (I just tried with 2010), hover over the variable name.
You could also use the watch window to enter expressions and track their values.
If your debugger isn't working try using printf statements wherever the program iterates.
Sometimes this can be a useful way of watching a variable without having to step into it.
If however you wish to run through the program in debug mode set a breakpoint as suggested (in VS2010 you can right click on the line you want to set a breakpoint on).
Then you just need to go to Toolbars -> Debug Toolbar.
I usually like to put #ifdef _DEBUG (or write an appropriate macro or even extra code) to do the printing, and send to the output everything that can help me tracking what the program's doing. Since your variables are never changing, I would do so.
However, flooding the console with lots of values is bad imo, and in such cases I would rely on assertions and the debugger - you should really see why it's not working.
I've done enough Python and Ruby to tell you that debugging a complex program when all you have is a printf, although doable, is extremely frustrating and takes way longer than what it should.
Finally, since you mention your data type is double (please make sure you have a good reason for not using floats instead), in case you add some assertion, remember that == is to be avoided unless you know 100% that == is what you really really want (which is unlikely if your data comes from calculations).