I would like to see a small but complete snippet of code that will cause Clang's static analyser to complain. My motivation is mostly that I'm trying to get it to work on my PIC32 code, and I need a way to distinguish between "all the code is fine" and "it's not actually doing anything". It's also partly curiosity, since I can't seem to come up with a simple example myself.
C89/ANSI or C99 is fine, and ideally I'd like to see it pick up a simple memory leak. My usage is
clang --analyze test.c
I found a "bug" in my code (the only one ;-) that triggers by that, and that is not detected by -Wall. I cooked it down to the following
struct elem {
struct elem *prev;
struct elem *next;
};
#define ELEM_INITIALIZER(NAME) { .prev = &(NAME), .next = &(NAME), }
struct head {
struct elem header;
};
#define HEAD_INITIALIZER(NAME) { .header = ELEM_INITIALIZER(NAME.header) }
int main(int argc, char ** argv) {
struct head myhead = HEAD_INITIALIZER(myhead);
}
This is a relatively straight forward implementation of a linked list, but this is not important here. The variable myhead is unused in a common sense application of the term, but for the compiler it is used since inside the initializer the address of a field is taken.
clang correctly analyzes this as
/tmp 11:58 <722>% clang --analyze test-clang.c
test-clang.c:25:15: warning: Value stored to 'myhead' during its initialization is never read
struct head myhead = HEAD_INITIALIZER(myhead);
^ ~~~~~~~~~~~~~~~~~~~~~~~~
1 diagnostic generated.
Edit: I found another one that also detects stack memory proliferation
char const* myBuggyFunction(void) {
return (char[len + 1]){ 0 };
}
This is not detected by gcc, open64 or clang with -Wall, but by clang with --analyze.
Related
I have assigned values to a fields in a structure. This resulted in multiple errors.
typedef struct t_queue {
int head = 0;
int tail = 0;
int maxSize = 0;
int size = 0;
SOCKET* queue = NULL;
}Queue;
typedef struct t_threadData {
int topicID;
bool isEngineActive = FALSE;
bool isServerActive = TRUE;
int numberOfConnectedSubs = 0;
Queue* queue;
HANDLE PublisherReady;
HANDLE ThreadReady;
HANDLE BarrierOK;
CRITICAL_SECTION Critical_Section;
int NumberOfThreadsWaiting = 0;
SOCKET sockets[NUMBER_OF_SUBSCRIBERS];
}ThreadData;
Errors are:
E0065 expected a ';'
E0020 identifier "bool" is undefined
I'm working in C on Windows 10 using Visual Studio Enterprise 2017.
Thank you for your help in advance.
Read Modern C and see this C reference. If you compile with a recent GCC compiler, use gcc -Wall -Wextra -g to compile your code. Read also the documentation of your C compiler, and enable all warnings in it. Read also the documentation of your debugger (e.g. GDB)
You should code:
typedef struct t_queue {
int head;
int tail;
int maxSize;
int size;
SOCKET* queue;
} Queue;
BTW, your field names are confusing. Since head and tail suggests a doubly-linked list. Then they should be pointers!
Consider also using the Clang static analyzer -or Frama-C- on your project.
Take inspiration from the source code of some existing open source software coded in C (like Glib, GNU make, the Linux kernel, GNU guile, etc...)
Background
I am working on a modular architecture for embedded devices where different abstraction layers have to communicate with each others. The current approach is to have plenty of functions, variables and defines mangled with the module name they belong to.
This approach is a bit painful and I would like to define some kind of common interfaces for the API. The key idea is to get a better understanding of what modules share the same HAL interface.
Proposal
I would like to use a OOP inspired architecture where I use structures as interfaces. All these structures are populated statically.
This solution looks nice but I might waste a lot of memory because the compiler doesn't know how to chop off structures and only keep what it really needs to.
The following example can be built with or without -DDIRECT and the behavior should be exactly the same.
Example
Source (test.c)
#include <stdlib.h>
int do_foo(int a) {
return 42 * a;
}
#ifdef DIRECT
int (*foo)(int) = do_foo;
int (*bar)(int);
int storage;
#else
struct foo_ts {
int (*do_foo)(int);
int (*do_bar)(int);
int storage;
} foo = {.do_foo = do_foo};
#endif
int main(char argc) {
#ifdef DIRECT
return foo(argc);
#else
return foo.do_foo(argc);
#endif
}
Makefile
CFLAGS=-O2 -mthumb -mcpu=cortex-m3 -g --specs=nosys.specs
CC=arm-none-eabi-gcc
upper=$(shell echo $(1) | tr a-z A-Z)
EXEC=direct.out with_struct.out
all: $(EXEC)
%.out:test.c
$(CC) $(CFLAGS) -D$(call upper,$(basename $#)) -o $# test.c
size $#
Output
One can notice that the memory footprint used with the struct variant is bigger because the compiler doesn't allow itself to remove the unused members.
arm-none-eabi-gcc -O2 -mthumb -mcpu=cortex-m3 -g \
--specs=nosys.specs -DDIRECT -o direct.out test.c
size direct.out
text data bss dec hex filename
992 1092 36 2120 848 direct.out
arm-none-eabi-gcc -O2 -mthumb -mcpu=cortex-m3 -g \
--specs=nosys.specs -DWITH_STRUCT -o with_struct.out test.c
size with_struct.out
text data bss dec hex filename
992 1100 28 2120 848 with_struct.out
Question
With this example I demonstrate that using a structure is good for readability and modularity, but it could decrease efficiency and it increases the memory usage.
Is there a way to get the advantages of both solutions? Said differently, is there a way to tell the compiler to be smarter?
OOP?
Following the comments on this question, one suggestion is to use C++ instead. Unfortunately, the same issue would occur because a Class with unused members will never be simplified by the compiler. So I am falling in the same trap with both languages.
Another raised point was the reason for unused members in structures. To address this question we can imagine a generic 3-axes accelerometer used in an application where only 1 axis is used. The HAL for this accelerometer could have the methods read_x_acc, read_y_acc and read_z_acc while only read_x_acc is used by the application.
If I declare a class in C++ or a structure in C the function pointers for unused methods/functions will still consume memory for nothing.
Let me first show you a possible approach regarding your edit, where your current interface has three functions, but sometimes you would need only one of them. You could define two interfaces:
typedef struct I1DAccel
{
double (*read)(void);
} I1DAccel;
typedef struct I3DAccel
{
union
{
I1DAccel x;
struct
{
double (*read_x)(void);
};
};
union
{
I1DAccel y;
struct
{
double (*read_y)(void);
};
};
union
{
I1DAccel z;
struct
{
double (*read_z)(void);
};
};
} I3DAccel;
Then you can make the implementation of the Accelerometer do this:
I1DAccel accel_x = { read_x_acc };
I1DAccel accel_y = { read_y_acc };
I1DAccel accel_z = { read_z_acc };
I3DAccel accel =
{
.read_x = read_x_acc,
.read_y = read_y_acc,
.read_z = read_z_acc,
};
With proper link-time optimizations, the compiler could then throw away any of these global structs that isn't used by application code.
Of course, you will consume more memory if one part of your code requires only accel_x while another part requires the whole accel. You would have to hunt down these cases manually.
Of course, using "interface" structs, you will always consume more memory than without them, the pointers have to be stored somewhere. Therefore, the typical approach is indeed to just prepend the module name to the functions and call them directly, like e.g. in case of such an accelerometer:
typedef int acchndl; // if you need more than one accelerometer
double Accel_read_x(acchndl accel);
double Accel_read_y(acchndl accel);
double Accel_read_z(acchndl accel);
This is conceptually similar to what you would do in e.g. C++:
class Accel
{
double read_x();
double read_y();
double read_z();
};
double Accel::read_x()
{
// implementation here
}
With the plain C above, instead of an instance pointer, you can use any other type of "object handle", like demonstrated with the typedef to int, which is often an advantage for embedded code.
This solution looks nice but I might waste a lot of memory because the compiler doesn't know how to chop off structures and only keep what it really needs to.
...
Is there a way to get the advantages of both solutions? Said differently, is there a way to tell the compiler to be smarter?
One problem is 'C' compiles things as a module (compilation unit) and there is often no easy way to know what will and will not be used in a structure. Consider the structure can be passed from module to module as you propose. When compiling one module, there is no context/information to know that some other module will use the structure or not.
Okay, so that leads to some possible solutions. gcc has the option -fwhole-program See: -fipa-struct-reorg as well as gold and LTO. Here you must structure your makefile so that the compiler (code generator) has all of the information available to know if it may remove structure members. This is not to say that these options will do what you want with a current gcc, just that they are a requirement to get things to work.
C++
You can do most anything in 'C' that you can in C++; just not as programmer efficiently. See: Operator overloading in 'C'. So how might a 'C' compiler implement your virtual functions? Your C++ virtuals might look like this underneath,
struct foo_ts_virtual_table {
int my_type /* RTTI value for polymorphism. */
int (*do_foo)(int);
int (*do_bar)(int);
} foo_vtable = {.my_type = ENUM_FOO_TS; .do_foo = foo};
struct foo_ts {
void * vtable; /* a foo_ts_virtual_table pointer */
int storage;
} foo = {.vtable = foo_vtable};
This is a win memory wise if you have multiple foo_ts structures. A down side of this is that a function pointer is difficult to in-line. Often a simple functions body maybe less than the call overhead on an ARM CPU. This results in more code memory and slower execution.
C++ compilers will be geared to eliminate these extraneous functions, just because this is a common issue in C++. A 'C' compilers analysis is confounded by your custom code/notation and the language definition of structure and compilation units.
Other possibilities are to wrap the function calls in macros and emit data to a non-linked section. You can examine the non-linked section to see whether or not to include a function (and pointer) in the final link. What is a win or not depends on lots of different design objectives. It is certainly going to complicate the build process and confound other developers.
As well, the ARM Linux MULTI_CPU maybe of interest. This is only to eliminate function pointers if only one 'type' is needed at run time.
If you insist on function pointers, there will be occasions where this actually generates more code. C++ compilers will do book keeping to see when a virtual might be in-lined as well as possibly not emitting unused member functions in a virtual table (and the implementation of the function). These issues maybe more pronounced than the extra structure overhead.
I want gcc to optimize away unused function pointers. Ie remove the code completely from the final executable. So far I was not able to achieve this.
Here is updated code:
#include <stdio.h>
struct inner {
void (*fun)(void);
void (*fun2)(void);
};
struct inner2 {
void (*fun)(void);
};
struct foo {
struct inner in;
struct inner2 in2;
};
void lessfun(){
printf("lessfun\n");
}
void morefun(){
printf("morefun\n");
}
const struct foo inst = {
{ .fun = lessfun, .fun2 = morefun },
{ .fun = lessfun }
};
void test(struct foo *f){
f->in.fun();
}
int main(int argc, char *argv){
struct inner2 in = inst.in2;
inst.in.fun();
inst.in.fun2();
in.fun();
/////////////// alt1: nm out | grep morefun -> found
test(&inst);
///////////////
/////////////// alt2: nm out | grep morefun -> not found
struct inner in;
struct inner in2 = inst.in;
in = in2;
test(&in);
///////////////
}
compiler flags: -Os -fdata-sections -Wl,--relax,--gc-sections -ffunction-sections
link flags: -flto -Os -fdata-sections -Wl,--relax,--gc-sections
Compiler: arm-none-eabi-gcc
Here the compiler will include both method1 and method2 into the final program even if they are never used. It is the assignment that seems to make this happen. But if they are never called, it would be nice to completely remove the code to method1 and method2. This obviously happens because technically the function is in fact referenced in the assignment, but since the variable in the assignment is never user it should still be possible to determine that the method is never called.
Do I need to declare it const somehow? How?
How can I have gcc remove the unused functions?
EDIT: I was able to sort of make it work as you see above. But it only works if I do not make a copy of any members of the struct. If a direct copy is made and passed to a function, the compiler fails to optimize unused functions. I'm now 60% certain that this is some kind of optimizer bug.
EDIT2: you may not even reproduce the bug. But here is the scenario that creates it.
struct mydev dev;
struct dev_spi spi;
struct dev_spi sp2 = board.spi0;
sp2.writereadbyte(0);
spi = sp2;
//test(&cpu.spi0);
// using only this call results in correct optimization
// many unused methods pointed to by members of "board" var are gone.
mydev_init(&dev, &spi);
// using this version breaks optimization
// all methods referenced by "board" struct are included in final program
mydev_init(&dev, &sp2);
// this one breaks optimization as well
// same as above.
mydev_init(&dev, &board.spi0);
// there is no difference other than one passes variable directly to the init function
// and the other uses a temp variable.
Here is the C code:
struct node{
void *value;
struct node *next;
};
void g(void *p){
/*...*/
}
void f(struct node *head, const int ok){
struct node *p=head;
while (p){
/* ...
code 1
...
*/
if (ok!=0){
g(p->value);
}
p=p->next;
}
}
I used gcc to compile this code. If I compiled with -O, would it optimize function f like this:
void f(struct node *head, const int ok){
struct node *p=head;
if (ok!=0){
while (p){
/* ...
code 1
...
*/
g(p->value);
p=p->next;
}
}
else{
while (p){
/* ...
code 1
...
*/
p=p->next;
}
}
}
That would greatly depend on how big /* code 1 */ is. If it is very small, it might. But if it is anything above a few lines, it most probably won't. Duplicating a large amount of code for every single if would have terrible effects on the performance. In fact, that may happen with very aggressive optimization and certainly not just with -O. From the man page of gcc (emphasis mine):
-O
-O1 ...
With -O, the compiler tries to reduce code size and execution time, without performing any optimizations that take a great deal of compilation time.
So reducing code is also part of optimization.
-O2 Optimize even more. GCC performs nearly all supported optimizations that do not involve a space-speed tradeoff. As compared to -O, this option increases
both compilation time and the performance of the generated code.
So -O2 wouldn't do what you want either.
-O3 Optimize yet more. -O3 turns on all optimizations specified by -O2 and also turns on the -finline-functions, -funswitch-loops, -fpredictive-commoning,
-fgcse-after-reload, -ftree-vectorize and -fipa-cp-clone options.
Now we have to look at these options to see if any of them might do what you want:
-funswitch-loops
Move branches with loop invariant conditions out of the loop, with duplicates of the loop on both branches (modified according to result of the condition).
Voila! With -O3 you get the optimization you want.
Well, that depends on many things.
Since, you are using gcc, you can always check if it did for a particular program by invoking gcc -o -S fileName.c
In these kind of situations I find this website http://gcc.godbolt.org/ quite useful
I've got some C code I'm targeting for an AVR. The code is being compiled with avr-gcc, basically the gnu compiler with the right backend.
What I'm trying to do is create a callback mechanism in one of my event/interrupt driven libraries, but I seem to be having some trouble keeping the value of the function pointer.
To start, I have a static library. It has a header file (twi_master_driver.h) that looks like this:
#ifndef TWI_MASTER_DRIVER_H_
#define TWI_MASTER_DRIVER_H_
#define TWI_INPUT_QUEUE_SIZE 256
// define callback function pointer signature
typedef void (*twi_slave_callback_t)(uint8_t*, uint16_t);
typedef struct {
uint8_t buffer[TWI_INPUT_QUEUE_SIZE];
volatile uint16_t length; // currently used bytes in the buffer
twi_slave_callback_t slave_callback;
} twi_global_slave_t;
typedef struct {
uint8_t slave_address;
volatile twi_global_slave_t slave;
} twi_global_t;
void twi_init(uint8_t slave_address, twi_global_t *twi, twi_slave_callback_t slave_callback);
#endif
Now the C file (twi_driver.c):
#include <stdint.h>
#include "twi_master_driver.h"
void twi_init(uint8_t slave_address, twi_global_t *twi, twi_slave_callback_t slave_callback)
{
twi->slave.length = 0;
twi->slave.slave_callback = slave_callback;
twi->slave_address = slave_address;
// temporary workaround <- why does this work??
twi->slave.slave_callback = twi->slave.slave_callback;
}
void twi_slave_interrupt_handler(twi_global_t *twi)
{
(twi->slave.slave_callback)(twi->slave.buffer, twi->slave.length);
// some other stuff (nothing touches twi->slave.slave_callback)
}
Then I build those two files into a static library (.a) and construct my main program (main.c)
#include
#include
#include
#include
#include "twi_master_driver.h"
// ...define microcontroller safe way for mystdout ...
twi_global_t bus_a;
ISR(TWIC_TWIS_vect, ISR_NOBLOCK)
{
twi_slave_interrupt_handler(&bus_a);
}
void my_callback(uint8_t *buf, uint16_t len)
{
uint8_t i;
fprintf(&mystdout, "C: ");
for(i = 0; i < length; i++)
{
fprintf(&mystdout, "%d,", buf[i]);
}
fprintf(&mystdout, "\n");
}
int main(int argc, char **argv)
{
twi_init(2, &bus_a, &my_callback);
// ...PMIC setup...
// enable interrupts.
sei();
// (code that causes interrupt to fire)
// spin while the rest of the application runs...
while(1){
_delay_ms(1000);
}
return 0;
}
I carefully trigger the events that cause the interrupt to fire and call the appropriate handler. Using some fprintfs I'm able to tell that the location assigned to twi->slave.slave_callback in the twi_init function is different than the one in the twi_slave_interrupt_handler function.
Though the numbers are meaningless, in twi_init the value is 0x13b, and in twi_slave_interrupt_handler when printed the value is 0x100.
By adding the commented workaround line in twi_driver.c:
twi->slave.slave_callback = twi->slave.slave_callback;
The problem goes away, but this is clearly a magic and undesirable solution. What am I doing wrong?
As far as I can tell, I've marked appropriate variables volatile, and I've tried marking other portions volatile and removing the volatile markings. I came up with the workaround when I noticed removing fprintf statements after the assignment in twi_init caused the value to be read differently later on.
The problem seems to be with how I'm passing around the function pointer -- and notably the portion of the program that is accessing the value of the pointer (the function itself?) is technically in a different thread.
Any ideas?
Edits:
resolved typos in code.
links to actual files: http://straymark.com/code/ [test.c|twi_driver.c|twi_driver.h]
fwiw: compiler options: -Wall -Os -fpack-struct -fshort-enums -funsigned-char -funsigned-bitfields -mmcu=atxmega128a1 -DF_CPU=2000000UL
I've tried the same code included directly (rather than via a library) and I've got the same issue.
Edits (round 2):
I removed all the optimizations, without my "workaround" the code works as expected. Adding back -Os causes an error. Why is -Os corrupting my code?
Just a hunch, but what happens if you switch these two lines around:
twi->slave.slave_callback = slave_callback;
twi->slave.length = 0;
Does removing the -fpack-struct gcc flag fix the problem? I wonder if you haven't stumbled upon a bug where writing that length field is overwriting part of the callback value.
It looks to me like with the -Os optimisations on (you could try combinations of the individual optimisations enabled by -Os to see exactly which one is causing it), the compiler isn't emitting the right code to manipulate the uint16_t length field when its not aligned on a 2-byte boundary. This happens when you include a twi_global_slave_t inside a twi_global_t that is packed, because the initial uint8_t member of twi_global_t causes the twi_global_slave_t struct to be placed at an odd address.
If you make that initial field of twi_global_t a uint16_t it will probably fix it (or you could turn off struct packing). Try the latest gcc build and see if it still happens - if it does, you should be able to create a minimal test case that shows the problem, so you can submit a bug report to the gcc project.
This really sounds like a stack/memory corruption issue. If you run avr-size on your elf file, what do you get? Make sure (data + bss) < the RAM you have on the part. These types of issues are very difficult to track down. The fact that removing/moving unrelated code changes the behavior is a big red flag.
Replace "&my_callback" with "my_callback" in function main().
Because different threads access the callback address, try protecting it with a mutex or read-write lock.
If the callback function pointer isn't accessed by a signal handler, then the "volatile" qualifier is unnecessary.