Clang Static Analyzer False Positive with bit-fields (C-code) - c

I am wondering if the following example is a Clang SA false positive, and if so, is there a way to suppress it?
The key here is that I am copying a structure containing bit-fields by casting it as a word instead of a field-by-field copy (or memcpy). Both field-by-field copy and memcpy doesn't trigger warnings, but copying as a word (after casting) raises an "uninitialized access" warning. This is on a embedded system where only word-access is possible and these types of word copies are common place.
Below is the example code:
#include <stdio.h>
#include <string.h>
struct my_fields_t {
unsigned int f0: 16;
unsigned int f1: 8;
unsigned int f2: 8;
};
int main(void) {
struct my_fields_t var1, var2;
// initialize all the fields in var1.
var1.f0 = 1;
var1.f1 = 2;
var1.f2 = 3;
// Method #1: copy var1 -> var2 as a word (sizeof(unsigned int) = 4).
unsigned int *src = (unsigned int *) &var1;
unsigned int *dest = (unsigned int *) &var2;
*dest = *src;
// Method #2: copy var1->var2 field-by-field [NO SA WARNINGS]
// var2.f0 = var1.f0;
// var2.f1 = var1.f1;
// var2.f2 = var1.f2;
// Method #3: use memcpy to copy var1 to var2 [NO SA WARNINGS]
// memcpy(&var2, &var1, sizeof(struct my_fields_t));
printf("%d, %d, %d\n", var1.f0, var1.f1, var1.f2);
printf("%d, %d, %d\n", var2.f0, var2.f1, var2.f2); // <--- Function call argument is an uninitialized value
printf("sizeof(unsigned int) = %ld\n", sizeof(unsigned int));
}
Here's the output:
$ clang --version
clang version 4.0.0 (tags/RELEASE_401/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
$ clang -Wall clang_sa.c
$ ./a.out
1, 2, 3
1, 2, 3
sizeof(unsigned int) = 4
$ scan-build clang clang_sa.c
scan-build: Using '<snipped>/clang-4.0' for static analysis
clang_sa.c:33:3: warning: Function call argument is an uninitialized value
printf("%d, %d, %d\n", var2.f0, var2.f1, var2.f2); // <--- Function call argument is an uninitialized value
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
scan-build: 1 bug found.
In the above example, it is quite clear that all the fields in var2 will be initialized by the word copy. So, clang SA shouldn't complain about un-intialized access.
I appreciate any help/insight.

In terms of suppressing a specific warning, from the documentation:
Q: How can I suppress a specific analyzer warning?
There is currently no solid mechanism for suppressing an analyzer warning, although this is currently being investigated. ...
But on the next question, it shows you that you can mark a block of code to be skipped over during static analysis by surrounding the code with an #ifdef block:
Q: How can I selectively exclude code the analyzer examines?
When the static analyzer is using clang to parse source files, it implicitly defines the preprocessor macro __clang_analyzer__. One can use this macro to selectively exclude code the analyzer examines. ...
So, you could do it like this:
#ifdef __clang_analyzer__
#define COPY_STRUCT(DEST, SRC) (DEST) = (SRC)
#else
#define COPY_STRUCT(DEST, SRC) do { \
const unsigned int *src = (const void *)&(SRC); \
unsigned int *dest = (void *)&(DEST); \
*dest = *src; \
} while(0)
#endif
COPY_STRUCT(var2, var1);

Related

How to change the value of a variable without the compiler knowing?

I want to verify the role of volatile by this method. But my inline assembly code doesn't seem to be able to modify the value of i without the compiler knowing. According to the articles I read, I only need to write assembly code like __asm { mov dword ptr [ebp-4], 20h }, I think I write the same as what he did.
actual output:
before = 10
after = 123
expected output:
before = 10
after = 10
Article link: https://www.runoob.com/w3cnote/c-volatile-keyword.html
#include <stdio.h>
int main() {
int a, b;
// volatile int i = 10;
int i = 10;
a = i;
printf("before = %d\n", a);
// Change the value of i in memory without letting the compiler know.
// I can't run the following statement here, so I wrote one myself
// mov dword ptr [ebp-4], 20h
asm("movl $123, -12(%rbp)");
b = i;
printf("after = %d\n", b);
}
I want to verify the role of volatile ...
You can't.
If a variable is not volatile, the compiler may optimize; it does not need to do this.
A compiler may always treat any variable as volatile.
How to change the value of a variable without the compiler knowing?
Create a second thread writing to the variable.
Example
The following example is for Linux (under Windows, you need a different function than pthread_create()):
#include <stdio.h>
#include <pthread.h>
int testVar;
volatile int waitVar;
void * otherThread(void * dummy)
{
while(waitVar != 2) { /* Wait */ }
testVar = 123;
waitVar = 3;
return NULL;
}
int main()
{
pthread_t pt;
waitVar = 1;
pthread_create(&pt, 0, otherThread, NULL);
testVar = 10;
waitVar = 2;
while(waitVar != 3) { /* Wait */ }
printf("%d\n", testVar - 10);
return 0;
}
If you compile with gcc -O0 -o x x.c -lpthread, the compiler does not optimize and works like all variables are volatile. printf() prints 113.
If you compile with -O3 instead of -O0, printf() prints 0.
If you replace int testVar by volatile int testVar, printf() always prints 113 (independent of -O0/-O3).
(Tested with the GCC 9.4.0 compiler.)

Why does clang produces wrong results for my c code compiled with -O1 but not with -O0?

For input 0xffffffff, the following c code works fine with no optimization, but produces wrong results when compiled with -O1. Other compilation options are -g -m32 -Wall. The code is tested with clang-900.0.39.2 in macOS 10.13.2.
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char *argv[]) {
if (argc < 2) return 1;
char *endp;
int x = (int)strtoll(argv[1], &endp, 0);
int mask1 = 0x55555555;
int mask2 = 0x33333333;
int count = (x & mask1) + ((x >> 1) & mask1);
int v1 = count >> 2;
printf("v1 = %#010x\n", v1);
int v2 = v1 & mask2;
printf("v2 = %#010x\n", v2);
return 0;
}
Input: 0xffffffff
Outputs with -O0: (expected)
v1 = 0xeaaaaaaa
v2 = 0x22222222
Outputs with -O1: (wrong)
v1 = 0x2aaaaaaa
v2 = 0x02222222
Below are disassembled instructions for the line "int v1 = count >> 2;" with -O0 and -O1.
With -O0:
sarl $0x2, %esi
With -O1:
shrl $0x2, %esi
Below are disassembled instructions for the line "int v2 = v1 & mask2;" with -O0 and -O1.
With -O0:
andl -0x24(%ebp), %esi //-0x24(%ebp) stores 0x33333333
With -O1:
andl $0x13333333, %esi //why does the optimization changes 0x33333333 to 0x13333333?
In addition, if x is set to 0xffffffff locally instead of getting its value from arguments, the code will work as expected even with -O1.
P.S: The code is an experimental piece based on my solution to the Data Lab from the CS:APP course # CMU. The lab asks the student to implement a function that counts the number of 1 bit of an int variable without using any type other than int.
As several commenters have pointed out, right-shifting signed values is not well defined.
I changed the declaration and initialization of x to
unsigned int x = (unsigned int)strtoll(argv[1], &endp, 0);
and got consistent results under -O0 and -O1. (But before making that change, I was able to reproduce your result under clang under MacOS.)
As you have discovered, you raise Implementation-defined Behavior in your attempt to store 0xffffffff (4294967295) in int x (where INT_MAX is 7fffffff, or 2147483647). C11 Standard ยง6.3.1.3 (draft n1570) - Signed and unsigned integers Whenever using strtoll (or strtoull) (both versions with 1-l would be fine) and attempting to store the value as an int, you must check the result against INT_MAX before making the assignment with a cast. (or if using exact width types, against INT32_MAX, or UINT32_MAX for unsigned)
Further, in circumstance such as this where bit operations are involved, you can remove uncertainty and insure portability by using the exact width types provided in stdint.h and the associated format specifiers provided in inttypes.h. Here, there is no need for use of a signed int. It would make more sense to handle all values as unsigned (or uint32_t).
For example, the following provides a default value for the input to avoid the Undefined Behavior invoked if your code is executed without argument (you can also simply test argc), replaces the use of strtoll with strtoul, validates the input fits within the associated variable before assignment handling the error if it does not, and then makes use of the unambiguous exact types, e.g.
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <inttypes.h>
int main (int argc, char *argv[]) {
uint64_t tmp = argc > 1 ? strtoul (argv[1], NULL, 0) : 0xffffffff;
if (tmp > UINT32_MAX) {
fprintf (stderr, "input exceeds UINT32_MAX.\n");
return 1;
}
uint32_t x = (uint32_t)tmp,
mask1 = 0x55555555,
mask2 = 0x33333333,
count = (x & mask1) + ((x >> 1) & mask1),
v1 = count >> 2,
v2 = v1 & mask2;
printf("v1 = 0x%" PRIx32 "\n", v1);
printf("v2 = 0x%" PRIx32 "\n", v2);
return 0;
}
Example Use/Output
$ ./bin/masktst
v1 = 0x2aaaaaaa
v2 = 0x22222222
Compiled with
$ gcc -Wall -Wextra -pedantic -std=gnu11 -Ofast -o bin/masktst masktst.c
Look things over and let me know if you have further questions.
this statement:
int x = (int)strtoll(argv[1], &endp, 0);
results in a signed overflow, which is undefined behavior.
(on my system, the result is: -1431655766
The resulting values tend to go downhill from there:
The variable: v1 receives: -357913942
The variable: v2 receives: 572662306
the %x format specifier only works correctly with unsigned variables

use macro to define two local variable but have the same address

Code as below:
#include"stdio.h"
#define MySTRING(ident, str) \
({\
char str_##ident[16]; \
memset((char *)str_##ident, 0x00, sizeof(str_##ident)); \
memcpy(str_##ident, (str), strlen((str))); \
str_##ident; \
})
int main(int argc, char **argv)
{
printf("%u, %u\n", MySTRING(qw, "1.1.1.1"), MySTRING(er, "2.2.2.2"));
}
Tetst result:
[root#cza temp]# gcc -O0 ./fly.c
[root#cza temp]# ./a.out
3959297344, 3959297360
[root#cza temp]# gcc -O2 ./fly.c
[root#cza temp]# ./a.out
2017090240, 2017090240
It seems like gcc optimistic make a difference on it.
The second result is not what I want, but in my app build template, O2 have been set.
I'd like to know the detail on why O2 make it difference, or is it a bug on GCC?
P.S. My colleague told me the prefix "volatile" can work.
Your code is a gcc extension called Statement Expression. The statements in the braces are executed, and the value of the final statement is the value of the expression. Any objects created are destroyed when the statement ends.
Being in a macro makes no difference; your code (update: the original code) is:
printf("%u, %u\n", ({ char ip_qw[16]; ip_qw; }), ({ char ip_er[16]; ip_er; }) );
When the ip_qw block ends, ip_qw is destroyed, so that memory is freed up for ip_er to use. This explains why it is possible to see the same address for both.
Your code is invalid because the printf function will access the contents of the 16-byte arrays after they have been destroyed.
Fortunately, Standard C has a solution. Objects that are returned by value are guaranteed to hang around until the end of the statement in which the function call was made. Arrays can't be returned by value, but structs can, so we can go:
#include <stdio.h>
#include <string.h>
struct MS
{
char str[16];
};
struct MS make_MS(char const *str)
{
struct MS ms;
strcpy(ms.str, str);
return ms;
}
#define MySTRING(s) make_MS(s).str
int main(int argc, char **argv)
{
printf("%s, %s\n", MySTRING("1.1.1.1"), MySTRING("2.2.2.2"));
}
Notes:
Use angle brackets for standard includes
Use %p to print pointers (makes a difference on 64bit system)
Your macro with memset and memcpy was not any safer than strcpy; in robust code you should change this strcpy to a snprintf(ms.str, sizeof ms.str, "%s", str);.

How to get a pointer to a binary section in Mac OS X?

I'm writing some code which stores some data structures in a special named binary section. These are all instances of the same struct which are scattered across many C files and are not within scope of each other. By placing them all in the named section I can iterate over all of them.
This works perfectly with GCC and GNU ld. Fails on Mac OS X due to missing __start___mysection and __stop___mysection symbols. I guess llvm ld is not smart enough to provide them automatically.
In GCC and GNU ld, I use __attribute__((section(...)) plus some specially named extern pointers which are magically filled in by the linker. Here's a trivial example:
#include <stdio.h>
extern int __start___mysection[];
extern int __stop___mysection[];
static int x __attribute__((section("__mysection"))) = 4;
static int y __attribute__((section("__mysection"))) = 10;
static int z __attribute__((section("__mysection"))) = 22;
#define SECTION_SIZE(sect) \
((size_t)((__stop_##sect - __start_##sect)))
int main(void)
{
size_t sz = SECTION_SIZE(__mysection);
int i;
printf("Section size is %u\n", sz);
for (i=0; i < sz; i++) {
printf("%d\n", __start___mysection[i]);
}
return 0;
}
What is the general way to get a pointer to the beginning/end of a section with FreeBSD linker. Anyone have any ideas?
For reference linker is:
#(#)PROGRAM:ld PROJECT:ld64-127.2
llvm version 3.0svn, from Apple Clang 3.0 (build 211.12)
Similar question was asked about MSVC here: How to get a pointer to a binary section in MSVC?
You can get the Darwin linker to do this for you.
#include <stdio.h>
extern int start_mysection __asm("section$start$__DATA$__mysection");
extern int stop_mysection __asm("section$end$__DATA$__mysection");
// If you don't reference x, y and z explicitly, they'll be dead-stripped.
// Prevent that with the "used" attribute.
static int x __attribute__((used,section("__DATA,__mysection"))) = 4;
static int y __attribute__((used,section("__DATA,__mysection"))) = 10;
static int z __attribute__((used,section("__DATA,__mysection"))) = 22;
int main(void)
{
long sz = &stop_mysection - &start_mysection;
long i;
printf("Section size is %ld\n", sz);
for (i=0; i < sz; ++i) {
printf("%d\n", (&start_mysection)[i]);
}
return 0;
}
Using Mach-O information:
#include <mach-o/getsect.h>
char *secstart;
unsigned long secsize;
secstart = getsectdata("__SEGMENT", "__section", &secsize);
The above gives information about section declared as:
int x __attribute__((section("__SEGMENT,__section"))) = 123;
More information: https://developer.apple.com/library/mac/documentation/developertools/conceptual/machoruntime/Reference/reference.html

print a __m128i variable

I'm trying to learn to code using intrinsics and below is a code which does addition
compiler used: icc
#include<stdio.h>
#include<emmintrin.h>
int main()
{
__m128i a = _mm_set_epi32(1,2,3,4);
__m128i b = _mm_set_epi32(1,2,3,4);
__m128i c;
c = _mm_add_epi32(a,b);
printf("%d\n",c[2]);
return 0;
}
I get the below error:
test.c(9): error: expression must have pointer-to-object type
printf("%d\n",c[2]);
How do I print the values in the variable c which is of type __m128i
Use this function to print them:
#include <stdint.h>
#include <string.h>
void print128_num(__m128i var)
{
uint16_t val[8];
memcpy(val, &var, sizeof(val));
printf("Numerical: %i %i %i %i %i %i %i %i \n",
val[0], val[1], val[2], val[3], val[4], val[5],
val[6], val[7]);
}
You split 128bits into 16-bits(or 32-bits) before printing them.
This is a way of 64-bit splitting and printing if you have 64-bit support available:
#include <inttypes.h>
void print128_num(__m128i var)
{
int64_t v64val[2];
memcpy(v64val, &var, sizeof(v64val));
printf("%.16llx %.16llx\n", v64val[1], v64val[0]);
}
Note: casting the &var directly to an int* or uint16_t* would also work MSVC, but this violates strict aliasing and is undefined behaviour. Using memcpy is the standard compliant way to do the same and with minimal optimization the compiler will generate the exact same binary code.
Portable across gcc/clang/ICC/MSVC, C and C++.
fully safe with all optimization levels: no strict-aliasing violation UB
print in hex as u8, u16, u32, or u64 elements (based on #AG1's answer)
Prints in memory order (least-significant element first, like _mm_setr_epiX). Reverse the array indices if you prefer printing in the same order Intel's manuals use, where the most significant element is on the left (like _mm_set_epiX). Related: Convention for displaying vector registers
Using a __m128i* to load from an array of int is safe because the __m128 types are defined to allow aliasing just like ISO C unsigned char*. (e.g. in gcc's headers, the definition includes __attribute__((may_alias)).)
The reverse isn't safe (pointing an int* onto part of a __m128i object). MSVC guarantees that's safe, but GCC/clang don't. (-fstrict-aliasing is on by default). It sometimes works with GCC/clang, but why risk it? It sometimes even interferes with optimization; see this Q&A. See also Is `reinterpret_cast`ing between hardware SIMD vector pointer and the corresponding type an undefined behavior?
See GCC AVX _m256i cast to int array leads to wrong values for a real-world example of GCC breaking code which points an int* at a __m256i.
(uint32_t*) &my_vector violates the C and C++ aliasing rules, and is not guaranteed to work the way you'd expect. Storing to a local array and then accessing it is guaranteed to be safe. It even optimizes away with most compilers, so you get movq / pextrq directly from xmm to integer registers instead of an actual store/reload, for example.
Source + asm output on the Godbolt compiler explorer: proof it compiles with MSVC and so on.
#include <immintrin.h>
#include <stdint.h>
#include <stdio.h>
#ifndef __cplusplus
#include <stdalign.h> // C11 defines _Alignas(). This header defines alignas()
#endif
void p128_hex_u8(__m128i in) {
alignas(16) uint8_t v[16];
_mm_store_si128((__m128i*)v, in);
printf("v16_u8: %x %x %x %x | %x %x %x %x | %x %x %x %x | %x %x %x %x\n",
v[0], v[1], v[2], v[3], v[4], v[5], v[6], v[7],
v[8], v[9], v[10], v[11], v[12], v[13], v[14], v[15]);
}
void p128_hex_u16(__m128i in) {
alignas(16) uint16_t v[8];
_mm_store_si128((__m128i*)v, in);
printf("v8_u16: %x %x %x %x, %x %x %x %x\n", v[0], v[1], v[2], v[3], v[4], v[5], v[6], v[7]);
}
void p128_hex_u32(__m128i in) {
alignas(16) uint32_t v[4];
_mm_store_si128((__m128i*)v, in);
printf("v4_u32: %x %x %x %x\n", v[0], v[1], v[2], v[3]);
}
void p128_hex_u64(__m128i in) {
alignas(16) unsigned long long v[2]; // uint64_t might give format-string warnings with %llx; it's just long in some ABIs
_mm_store_si128((__m128i*)v, in);
printf("v2_u64: %llx %llx\n", v[0], v[1]);
}
If you need portability to C99 or C++03 or earlier (i.e. without C11 / C++11), remove the alignas() and use storeu instead of store. Or use __attribute__((aligned(16))) or __declspec( align(16) ) instead.
(If you're writing code with intrinsics, you should be using a recent compiler version. Newer compilers usually make better asm than older compilers, including for SSE/AVX intrinsics. But maybe you want to use gcc-6.3 with -std=gnu++03 C++03 mode for a codebase that isn't ready for C++11 or something.)
Sample output from calling all 4 functions on
// source used:
__m128i vec = _mm_setr_epi8(1, 2, 3, 4, 5, 6, 7,
8, 9, 10, 11, 12, 13, 14, 15, 16);
// output:
v2_u64: 0x807060504030201 0x100f0e0d0c0b0a09
v4_u32: 0x4030201 0x8070605 0xc0b0a09 0x100f0e0d
v8_u16: 0x201 0x403 0x605 0x807 | 0xa09 0xc0b 0xe0d 0x100f
v16_u8: 0x1 0x2 0x3 0x4 | 0x5 0x6 0x7 0x8 | 0x9 0xa 0xb 0xc | 0xd 0xe 0xf 0x10
Adjust the format strings if you want to pad with leading zeros for consistent output width. See printf(3).
I know this question is tagged C, but it was the best search result also when looking for a C++ solution to the same problem.
So, this could be a C++ implementation:
#include <string>
#include <cstring>
#include <sstream>
#if defined(__SSE2__)
template <typename T>
std::string __m128i_toString(const __m128i var) {
std::stringstream sstr;
T values[16/sizeof(T)];
std::memcpy(values,&var,sizeof(values)); //See discussion below
if (sizeof(T) == 1) {
for (unsigned int i = 0; i < sizeof(__m128i); i++) { //C++11: Range for also possible
sstr << (int) values[i] << " ";
}
} else {
for (unsigned int i = 0; i < sizeof(__m128i) / sizeof(T); i++) { //C++11: Range for also possible
sstr << values[i] << " ";
}
}
return sstr.str();
}
#endif
Usage:
#include <iostream>
[..]
__m128i x
[..]
std::cout << __m128i_toString<uint8_t>(x) << std::endl;
std::cout << __m128i_toString<uint16_t>(x) << std::endl;
std::cout << __m128i_toString<uint32_t>(x) << std::endl;
std::cout << __m128i_toString<uint64_t>(x) << std::endl;
Result:
141 114 0 0 0 0 0 0 151 104 0 0 0 0 0 0
29325 0 0 0 26775 0 0 0
29325 0 26775 0
29325 26775
Note: there exists a simple way to avoid the if (size(T)==1), see https://stackoverflow.com/a/28414758/2436175
#include<stdio.h>
#include<emmintrin.h>
int main()
{
__m128i a = _mm_set_epi32(1,2,3,4);
__m128i b = _mm_set_epi32(1,2,3,4);
__m128i c;
const int32_t* q;
//add a pointer
c = _mm_add_epi32(a,b);
q = (const int32_t*) &c;
printf("%d\n",q[2]);
//printf("%d\n",c[2]);
return 0;
}
Try this code.

Resources