What I have done.
Test1
1 #include <stdio.h>
2
3 int test[16];
4
5 int main()
6 {
7 test[17] = -1;
8 }
/tmp $ gcc ./main.c -o main -fsanitize=address
/tmp $ ./main
/tmp $
Test2
1 #include <stdio.h>
2
3 int test[16] = {1};
4
5 int main()
6 {
7 test[17] = -1;
8 }
/tmp $ gcc ./main.c -o main -fsanitize=address
/tmp $ ./main
=================================================================
==19776==ERROR: AddressSanitizer: global-buffer-overflow on address
...
Looks like global buffer overflow detection is not working for global variables which are placed in bss (is it so?). What are the reasons behind this?
Update:
The code which does store is not optimized out.
System information:
$ gcc --version
gcc (Ubuntu 7.2.0-8ubuntu3.2) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
This is in FAQ:
Q: Why didn't ASan report an obviously invalid memory access in my
code?
A1: If your errors is too obvious, compiler might have already
optimized it out by the time Asan runs.
A2: Another, C-only option is accesses to global common symbols
which are not protected by Asan (you can use -fno-common
to disable generation of common symbols and hopefully detect more bugs).
Your case is probly covered by A2 so adding -fno-common should help.
The issue with common symbols (which are generated for zero-initialized global variables by default) is that, due to their weird legacy semantics, Asan can not insert redzones for them (see GCC #55739 for gory details). By supplying -fno-common you disable generation of commons and instead ask GCC to generate normal global symbols in all cases (this has a small chance of breaking ill-written programs that rely on common symbols behavior but usually it's not an issue).
Other tools, including our (Semantic Designs) CheckPointer tool, do this.
(Output edited to remove some irrelevant text):
For test1.c as OP defined it:
c:\DMS\Domains\C\GCC4\Tools\CheckPointer\Example\Source>DMSCheckPointer C~GCC4 test1.c
C~GCC4 CheckPointer Version 1.2.1001
Copyright (C) 2011-2016 Semantic Designs, Inc; All Rights Reserved; SD Confidential
c:\DMS\Domains\C\GCC4\Tools\CheckPointer\Example\Source>gcc.exe -I"c:\DMS\Domains\C\GCC4\Tools\CheckPointer" -I.\Target -otest1.exe Target\test1.c <snip>
test1
*** Error: CWE-119: Improper Restriction of Operations within the Bounds of a Memory Buffer
Dereference of pointer is out of bounds.
in function: main, line: 7, file c:\DMS\Domains\C\GCC4\Tools\CheckPointer\Example\Source\test1.c
For test2.c:
c:\DMS\Domains\C\GCC4\Tools\CheckPointer\Example\Source>DMSCheckPointer C~GCC4 test2.c
C~GCC4 CheckPointer Version 1.2.1001
Copyright (C) 2011-2016 Semantic Designs, Inc; All Rights Reserved; SD Confidential
c:\DMS\Domains\C\GCC4\Tools\CheckPointer\Example\Source>gcc.exe -I"c:\DMS\Domains\C\GCC4\Tools\CheckPointer" -I.\Target -otest2.exe Target\test2.c <snip>
test2
*** Error: CWE-119: Improper Restriction of Operations within the Bounds of a Memory Buffer
Dereference of pointer is out of bounds.
in function: main, line: 7, file c:\DMS\Domains\C\GCC4\Tools\CheckPointer\Example\Source\test2.c
CheckPointer doesn't need "redzones" around data; it checks accesses against the language semantics. That means it will, for example, detect accesses off the end of a char array anywhere in a struct, whether the struct is a local (stack) variable, in the heap, in thread local storage, or embedded in some other structure.
The "odd phrasing" related to "CWE-119" isn't our choice of words, it is defined by the standard for Common Weakness Enumeration. CheckPointer reports errors using the CWE definitions, classifications and descriptions.
Related
When I use the valgrind bbv tool, I encountered a problem. The instruction count of the same executable file valgrind is quite different from that of pmu.
Such as SPEC2006 omnetpp, the instruction count is about 57190 billion by valgrind, but that is 57290 billion by pmu. The environment in which the program runs is the same.
Then we wrote a simple program to verify this, the result is that valgrind's instruction count is about 800 less.
The test code:
#include <unistd.h>
int main(void)
{
unsigned int i = 0, sum = 0;
sum += i;
return 0;
}
valgrind --tool=exp-bbv ./withmain
Total instructions: 5232
simpleperf stat -e instructions:u ./withmain (just count instructions in usespace)
Performance counter statistics:
6,043 instructions:u # (100%)
2. Then we found the functions like _start and _init is UNKNOWN in valgrind, and the symbol size of that functions is zero. Valgrind seems to ignore them during command analysis.
I found that there are such descriptions of this kind of glibc function in m_main.c:
If linking of the final executables is done with glibc present, then Valgrind starts at main() above as usual, and all of the following code is irrelevant.
However, this is not the intended mode of use. The plan is to avoid linking against glibc, by giving gcc the flags -nodefaultlibs -lgcc -nostartfiles at startup.
Q:
1) Anyone known why can't valgrind analyze these functions?
2) The descriptions above mean that we have to avoid using the glibc function? Is there any other way to resolve this difference of instruction count?
3) Why size of some function symbol(such as _start, _init, etc) is zero in elf file? but when I objdump that file, I can see the whole code of that functions.
3. The program environment:
Operating environment: Andriod10 (aarch64 Linux 4.14)
CPU core: armv8 CotexA55
Cross-Compiler: gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu (both valgrind and test code compile by this cross tool, --static, -g -O3 and other para already tried). I have tried gcc-4.8.2 that have same problem.
char a_1[512];
int some_variable;
char a_2[512];
main()
{
...
}
Here in the above program, I have declared some variables, all in bss section of the code. Considering that I have kept in mind the alignment issues, can I be sure that the memory allocated for those 3 variables will always be contiguous?
Considering that I have kept in mind the alignment issues, can I be sure that the memory allocated for those 3 variables will always be contiguous?
Certainly not. Read the C11 standard n1570, and you won't find any guarantee about that.
Different compilers are likely to order variables differently, in particular when they are optimizing. Some variables might even stay in a register, and not even have any memory location. In practice some compilers are following the order of the source, others are using some different order.
And you practically could customize (perhaps with some pain) your GCC or your Clang compiler to change that order. And this does happen in practice. For example, recent versions of the GCC kernel might be configured with some GCC plugin which could reorder variables. With GCC or Clang you might also add some variable attribute to alter that order.
BTW, if you need some specific order, you could pack the fields in some struct e.g. code:
struct {
char a_1[512];
int some_variable;
char a_2[512];
} my_struct;
#define a_1 my_struct.a_1
#define some_variable my_struct.some_variable
#define a_2 my_struct.a_2
BTW, some old versions of GCC had an optional optimization pass which reordered (in some cases) fields in struct-s (but recent GCC removed that optimization pass).
In a comment (which should go into your question) you mention hunting some bug. Consider using the gdb debugger and its watchpoints (and/or valgrind). Don't forget to enable all warnings and debug info when compiling (so gcc -Wall -Wextra -g with GCC). Maybe you want also instrumentation options like -fsanitize=address etc...
Beware of undefined behavior.
First an abstraction of my program:
int main ()
{
My_Struct1 ms1; // sizeof (My_Struct1) is 88712 B -- L1
My_Struct2 ms2; // sizeof (My_Struct2) is 13208 B -- L2
// 1. Invoke parser to fill in the two struct instances. -- L3
printf ("%ul, %ul\n", &ms1, &ms2) // -- **L3b** doesn't produce seg. fault.
my_fun (&ms1, &ms2); // -- L4, does produce seg. fault.
return 0;
}
If I run my program using makefile, then a segmentation fault occurs at L4 (always).
If I execute my program directly from shell (./executable), then the segmentation does occur sometimes but not always.
The error is: Segmentation fault: Cannot access memory at address at L4 for &ms1 and &ms2 both. The type and location of the error is what was pointed out by gdb.
My guess is that the error is because of the size of the structures.
Please explain in detail what is going.
The error behivour is same even after reducing the size of My_Struct1 to 8112 B and My_Struct2 to 1208 B.
I am working on:
Ubuntu 14.04
Intel® Core™ i5-4200M CPU # 2.50GHz × 4
3.8 GiB memory
gcc - 4.8.4
First, compile with all warnings & debug info. Probably with CFLAGS= -g -Wall -Wextra in your Makefile. Perhaps you might sometimes add some sanitize instrumentation options such as -fsanitize=address or -fsanitize=undefined (then it could be worthwhile to upgrade your GCC compiler to GCC 5 in march 2016).You might also want -Wstack-usage=1000 warning & -fstack-usage developer option.
Be very afraid of undefined behavior.
Then, enable core(5) dumps. Probably some ulimit -c 100000 (or whatever number is realistic) in your ~/.bashrc then start a new terminal; check with cat /proc/self/limits (a Linux specific command, related to proc(5)) that the limits are well set. See setrlimit(2).
Run your faulty test, e.g. with make test. You'll get a core dump. Check with ls -l core and file core.
At last, do a post mortem debugging session. If your binary is someprog, run gdb someprog core. Probably the first gdb command you'll type would be bt
Indeed, you are probably wrong in declaring quite large struct as local variables in main. The rule of thumb is to restrict your call frame to a few kilobytes at most (hence, never have a local variable of more than a kilobyte in the call stack). So I would recommend putting your large struct in the heap (so use malloc and free appropriately, read about C dynamic memory allocation). But a typical call stack on Linux can grow to several megabytes.
Also, run your program with valgrind
BTW, the correct format for (void*) pointers in %p so your added printf should be
printf("ms1#%p, ms2#%p\n", (void*)&ms1, (void*)&ms2);
I'm trying to enable gcc' s canaries' generation but I get an undefined reference to __stack_chk_guard.
From gcc's man about canaries :
-mstack-protector-guard=guard
Generate stack protection code using canary at guard. Supported locations are global for
global canary or tls for per-thread canary in the TLS block (the default). This option
has effect only when -fstack-protector or -fstack-protector-all is specified.
These -m switches are supported in addition to the above on x86-64 processors in 64-bit
environments.
I've done this test program :
#define VALUE 2048
int main()
{
char arr[VALUE];
int i;
for (i = 0; i < VALUE + 15; i++) // "i < VALUE + 15" is to test if canaries works but the code doesn't compile anymore with "i < 10"
arr[i] = '0';
return 0;
}
As said in gcc's man, my compilation line is :
gcc main.c -fstack-protector-all -mstack-protector-guard=global
But I get the following error :
/tmp/ccXxxxVd.o: In function `main':
main.c:(.text+0xe): undefined reference to `__stack_chk_guard'
main.c:(.text+0x51): undefined reference to `__stack_chk_guard'
collect2: error: ld returned 1 exit status
How can I remove this error ?
EDIT:
OS: ubuntu 14.10 utopic
architecture: x86-64
environments: 64-bit
It would appear that the -mstack-protector-guard option is only for backwards compatibility with how the stack protector worked in the past. In the past the canary was in a global variable. Later it was switched to TLS. It would appear that the operating system / libc you use either removed or never had support for the global variable canary, so only TLS works.
Don't touch the -mstack-protector-guard option and everything should work. The default should be fine when you use -fstack-protector-all.
Provide __stack_chk_guard with a random value in c file, avoid using regular values like all zero's or FF's because the stack can easily get these values during any memory operation. Wiki on providing magic number implementation. This __stack_chk_guard will be placed at the top and bottom of the stack, which will be checked during every stack access. Any change in the value implies a corrupted stack and returns with error providing the stack protection.
unsigned long __stack_chk_guard;
void __stack_chk_guard_setup(void)
{
__stack_chk_guard = 0xBAAAAAAD;//provide some magic numbers
}
void __stack_chk_fail(void)
{
/* Error message */
}// will be called when guard variable is corrupted
There are two ways to remove this error: 1. From the compiler option disable(comment out) the "stack guard".
Define __stack_chk_guard in you c file.
When you define __stack_chk_guard make sure you provide random value to it. For providing random value you need to pass as an argument to the random function.
For any further detail you can refer to the compiler manual.
For those that get this error in bare metal software development with a custom linker script, make sure to pass the option -nostdlib option:
gcc -nostdlib
since Ubuntu 16.04 for example enables the stack protection by default on the compiler. man gcc says:
NOTE: In Ubuntu 14.10 and later versions, -fstack-protector-strong is enabled by default for C, C++, ObjC, ObjC++, if none of -fno-stack-protector, -nostdlib, nor -ffreestanding are found.
-fno-stack-protector also solved it for me, but you should likely tell your poor compiler that you are doing baremetal stuff to prevent other such problems.
I'm guessing this is because the feature relies on symbols which are normally defined if a linker script is not given? But TODO I found no mention of those symbols by dumping the default linker script with:
aarch64-linux-gnu-gcc -Wl,-verbose main.c
so I'm not sure.
I grepped GCC 6.4.0 source code and it suggests that the symbol comes from libgcc2.c at gcc/doc/tm.texi:
The default version of this hook creates a variable called
#samp{__stack_chk_guard}, which is normally defined in #file{libgcc2.c}.
On doing some experiments whilst learning C, I've come across something odd. This is my program:
int main(void) {sleep(5);}
When it is compiled, the file size of the executable is 8496 bytes (in comparison to the 26 byte source!) This is understandable as sleep is called and the instructions for calling that are written in the executable. Another point to make is that without the sleep, the executable becomes 4312 bytes.
int main(void) {}
My main question is what happens when the first program is run. I'm using clang to compile and Mac OS X to run it. The result (according to Activity Monitor) is that the program uses 504KB of "real memory". Why is it so big when the program is just 4KB? I am assuming that the executable is loaded into memory but I haven't done anything apart from a sleep call. Why does my program need 500KB to sleep for five seconds?
By the way, the reason I'm using sleep is to be able to catch the amount of memory being used using Activity Monitor in the first place.
I ask simply out of curiosity, cheers!
When you compile a C program it is linked into an executable. Even though your program is very small it will link to the C runtime which will include some additional code. There may be some error handling and this error handling may write to the console and this code may include sprintf which adds some footprint to your application. You can request the linker to produce a map of the code in your executable to see what is actually included.
Also, an executable file contains more than machine code. There will be various tables for data and dynamic linking which will increase the size of the executable and there may also be some wasted space because the various parts are stored in blocks.
The C runtime will initialize before main is called and this will result in both some code being loaded (e.g. by dynamically linking to various operating system features) as well as memory being allocated for a a heap, a stack for each thread and probably also some static data. Not all of this data may show as "real memory" - the default stack size on OS X appears to be 8 MB and your application is still using much less than this.
In this case I suppose the size difference you've observed is significantly caused by dynamic linking.
Linkers usually don't place common code into the executable binaries, instead they reserve the information and the code would be loaded when the binary is loaded. Here those common code is stored in files called shared object(SO) or dynamically linked library(DLL).
[pengyu#GLaDOS temp]$ cat test.c
int main(void) {sleep(5);}
[pengyu#GLaDOS temp]$ gcc test.c
[pengyu#GLaDOS temp]$ du -h --apparent-size a.out
6.6K a.out
[pengyu#GLaDOS temp]$ gcc test.c -static
[pengyu#GLaDOS temp]$ du -h --apparent-size a.out
807K a.out
ALso, here I'm listing what are there in the memory of a process:
There're necessary dynamic libraries to be loaded in:
Here ldd gives the result of dynamic libraries to be loaded when invoking the binary. These libraries locates in the part where it's obtained by calling the mmap system call.
[pengyu#GLaDOS temp]$ cat test.c
int main(void) {sleep(5);}
[pengyu#GLaDOS temp]$ gcc test.c
[pengyu#GLaDOS temp]$ ldd ./a.out
linux-vdso.so.1 (0x00007fff576df000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f547a212000)
/lib64/ld-linux-x86-64.so.2 (0x00007f547a5bd000)
There're sections like .data, .code to be allocated for data from your binary file.
This part exists in the binary executable, so the size is supposed to be no lager than the file itself. Contents copied at the loading stage of a executable binary.
There're sections like .bss and also the stack zone to be allocated for dynamically use during execution of the program.
This part does not exist in the binary executable, so the size could be quite large without being affected by size of the file itself.