I'm learning some basics about linking and encountered the following code.
file: f1.c
#include <stdio.h>
static int foo;
int main() {
int *bar();
printf("%ld\n", bar() - &foo);
return 0;
}
file: f2.c
int foo = 0;
int *bar() {
return &foo;
}
Then a problem asks me whether this statement is correct: No matter how the program is compiled, linked or run, the output of it must be a constant (with respect to multiple runs) and it is non-zero.
I think this is correct. Although there are two definitions of foo, one of them is declared with static, so it shadows the global foo, thus the linker will not pick only one foo. Since the relative position of variables should be fixed when run (although the absolute addresses can vary), the output must be a constant.
I experimented with the code and on gcc 7.5.0 with gcc f1.c f2.c -o test && ./test it would always output 1 (but if I remove the static, it would output 0). But the answer says that the statement above is wrong. I wonder why. Are there any mistakes in my understanding?
A result of objdump follows. Both foos go to .bss.
Context. This is a problem related to the linking chapter of Computer Systems: A Programmer's Perspective by Randal E. Bryant and David R. O'Hallaron. But it does not come from the book.
Update. OK now I've found out the reason. If we swap the order and compile as gcc f2.c f1.c -o test && ./test, it will output -1. Quite a boring problem...
Indeed the static variable foo in the f1.c module is a different object from the global foo in the f2.c module referred to by the bar() function. Hence the output should be non zero.
Note however that subtracting 2 pointers that do not point to the same array or one past the end of the same array is meaningless, hence the difference might be 0 even for different objects. This may happen even as &foo == bar() would be non 0 because the objects are different. This behavior was common place in 16-bit segmented systems using the large model where subtracting pointers only affected the offset portion of the pointers whereas comparing them for equality compared both the segment and the offset parts. Modern systems have a more regular architecture where everything is in the same address space. Just be aware that not every system is a linux PC.
Furthermore, the printf conversion format %ld expects a value of type long whereas you pass a value of type ptrdiff_t which may be a different type (namely 64-bit long long on Windows 64-bit targets for example, which is different from 32-bit long there). Either use the correct format %td or cast the argument as (long)(bar() - &foo).
Finally, nothing in the C language guarantees that the difference between the addresses of global objects be constant across different runs of the same program. Many modern systems perform address space randomisation to lessen the risk of successful attacks, leading to different addresses for stack objects and/or static data in successive runs of the same executable.
Abstracting from the wring printf formats and pointer arithmetic problems static global variable from one compilation unit will be different than static and non-static variables having that same name in other compilation units.
to correctly see the difference in chars you should cast both to char pointer and use %td format which will print ptrdiff_t type. If your platform does not support it, cast the result to long long int
int main() {
int *bar();
printf("%td\n", (char *)bar() - (char *)&foo);
return 0;
}
or
printf("%lld\n", (long long)((char *)bar() - (char *)&foo));
If you want to store this difference in the variable use ptrdiff_t type:
ptrdiff_t diff = (char *)bar() - (char *)&foo;
Related
The program below has different behaviors with different option levels. When I compile it with -O3, it will never terminate. when I compile it with -O0, it will always terminate very soon.
#include <stdio.h>
#include <pthread.h>
void *f(void *v) {
int *i = (int *)v;
*i = 0;
printf("set to 0!\n");
return NULL;
}
int main() {
const int c = 1;
int i = 0;
pthread_t thread;
void *ptr = (void *)&c;
while (c) {
i++;
if (i == 1000) {
pthread_create(&thread, NULL, &f, ptr);
}
}
printf("done\n");
}
This is the result of running it with different optimization flags.
username#hostname:/src$ gcc -O0 main.c -o main
username#hostname:/src$ ./main
done
set to 0!
set to 0!
username#hostname:/src$ gcc -O3 main.c -o main
username#hostname:/src$ ./main
set to 0!
set to 0!
set to 0!
set to 0!
set to 0!
set to 0!
^C
username#hostname:/src$
The answer given by the professor's slide is like this:
Will it always terminate?
Depends of gcc options
With –O3 (all optimisations): no
Why?
The variable c is likely to stay local in a register, hence it will not be shared.
Solution « volatile »
Thank you for your replies. I now realize that volatile is a keyword in C. The description of the volatile keyword:
A volatile specifier is a hint to a compiler that an object may change its values in ways not specified by the language so that aggressive optimizations must be avoided.
According to my understanding, there is a shared register that stores the c value when we use -O3 flag. So the main thread and sub-thread will share it. In this case, if a sub-thread modifies c to 0, the main thread will get 0 when it wants to read c to compare in the while(c) statement. Then, the loop stops.
There is no register storing c that can be shared by the main thread and sub-threads when we use -O0 flag. Though the c is modified by a sub-thread, this change may not be written to memory and just be stored in a register, or it is written to memory while the main thread just uses the old value which is read and saved in a register. As a result, the loop is infinite.
If I declared the c value with const: const volatile int c = 1;, the program will terminate finally even if we compiled it with -O3. I guess all threads will read c from the main memory and write back to the main memory if they change the c value.
I know, according to the specifications or rules about C language, we are not allowed to modify a value that is declared by the const keyword. But I don't understand what is un behavior.
I wrote a test program:
#include "stdio.h"
int main() {
const int c = 1;
int *i = &c;
*i = 2;
printf("c is : %d\n", c);
}
output
username#hostname:/src$ gcc test.c -o test
test.c: In function ‘main’:
test.c:9:14: warning: initialization discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
9 | int *i = &c;
| ^
username#hostname:/src$ ./test
c is : 2
username#hostname:/src$
The result is 2 which means a variable declared with the const can be modified but this behavior is not suggested, right?
I also tried changing the judgment condition. If it is changed to while (1){ from while(c){, the loop will be an infinite one no matter using -O0 or -O3
This program is not a good one as it violates the specifications or rules of C language. Actually it comes from the lecture about software security.
Can I just understand like this? All threads share the same register storing c when we compile the program with -O0.
While the value c is in un-shared registers, so main thread is not informed when sub-threads modify value c when we use -O3. Or, while(c){ is replaced by while(1){ when we use -O3 so the loop is infinite.
I know this question can be solved easily if I check the generated assembly code. But I am not good at it.
This is undefined behavior. Per 6.7.3 Type qualifiers, paragraph 6 of the (draft) C11 standard:
If an attempt is made to modify an object defined with a const-qualified type through use of an lvalue with non-const-qualified type, the behavior is undefined.
There's no requirement for any particular behavior on the program. How it behaves is literally outside the specifications of the C language.
Your professor's observation of how it behaves may be correct. But he goes off the rails. There is no "why" for undefined behavior. What happens can change with changes to compiler options, particulars of the source code, time of day, or phase of the moon. Anything. Any expectation for any particular behavior is unfounded.
And
Solution « volatile »
is flat-out WRONG.
volatile does not provide sufficient guarantees for multithreaded access. See Why is volatile not considered useful in multithreaded C or C++ programming?.
volatile can appear to "work" because of particulars of the system, or just because any race conditions just don't happen to be triggered in an observable manner, but that doesn't make it correct. It doesn't "work" - you just didn't observe any failure. "I didn't see it break" does not mean "it works".
Note that some C implementations do define volatile much more extensively than the C standard requires. Microsoft in particular defines volatile much more expansively, making volatile much more effective and even useful and correct in multithreaded programs.
But that does not apply to all C implementations. And if you read that link, you'll find it doesn't even apply to Microsoft-compiled code running on ARM hardware...
The professor's explanation is not quite right.
The initial value of c is 1, which is truthy. It's declared as a constant, so its value can't change. Thus, the condition in while (c) is guaranteed to always be true, so there's no need to test the variable at all when the program is running. Just generate code for an infinite loop.
This optimization of not reading the variable is not done when optimization is disabled. In practice, declaring the variable volatile also forces it to be read whenever the variable is referenced in code.
Note that optimizations are implementation-dependent. Assigning to a const variable by accessing it through a non-const pointer results in undefined behavior, so any result is possible.
The typical use of a const volatile variable is for variables that reference read-only hardware registers that can be changed asynchronously (e.g. I/O ports on microcontrollers). This allows the application to read the register but code that tries to assign to the variable will not compile.
The explanation of "The variable c is likely to stay local in a register, hence it will not be shared." is not quite right. Or I'm having trouble parsing its precise meaning.
Once you take a pointer to it, the compiler has to put it into memory, unless it can convince itself that the pointer will not be used.
Here https://godbolt.org/z/YavbYxqoE
mov DWORD PTR [rsp+4], 1
and
lea rcx, [rsp+4]
suggest to me that the compiler has put the variable on the stack.
It's just that the while loop is not checking it for changes due to it being advertised as const.
I have read in multiple places that when declaring an extern variable, the memory is not designated until the definition is made. I was trying this code which is giving contradictory output in gcc.
#include <stdio.h>
int main() {
extern int a;
printf("%lu", sizeof(a));
return 0;
}
it should have shown error or zero size. but the output was following. please justify the output. Is it example of another undefined behavior?
aditya#theMonster:~$ ./a
4
You're able to get away with it here because a is never actually used. The expression sizeof(a) is evaluated at compile time. So because a is never referenced, the linker doesn't bother looking for it.
Had you done this instead:
printf("%d\n", a);
Then the program would have failed to link, printing "undefined reference to `a'"
The size of a variable is the size of its data type, whether it is presently only an extern or not. Since sizeof is evaluated at compile time, whereas symbol resolution is done at link time, this is acceptable.
Even with -O0, gcc doesn't care that it's extern; it puts 4 in esi for the argument to printf: https://godbolt.org/z/Zv2VYd
Without declaring a, however, any of the following will fail:
a = 3;
printf("%d\n", a);
int *p = &a;
The a is an integer, so its size is 4.
Its location(address) and value are not currently known.(it is extern somewhere at some other location)
But the size is well defined.
size_t sizeof(expr/var_name/data_type) 1 is a unary operator which when not provided with a variable length array, do not evaluate the expression. It just check the data type of expression.
Similarly, here, in sizeof(a), the complier only checks the data type of a which is int and hence returns the size of int.
Another example to clear your confusion is in sizeof(i++), i do not get incremented. Its data type is checked and returned.
One more example:
void main(){
int p=0;
printf("%zu",sizeof(p=2+3.0));
printf("%d",p);
}
will give u output on gcc as:
4
0
There is indeed a problem in your code, but not where you expect it:
passing a value of type size_t for printf conversion specification %ld has undefined behavior if size_t and unsigned long have different sizes or representations, as is the case on many systems (16-bit systems, Windows 64-bit...).
Here is a corrected version, portable to non C99-conforming systems, whose C library printf might not support %zu:
#include <stdio.h>
int main(void) {
extern int a;
printf("%lu\n", (unsigned long)sizeof(a));
return 0;
}
Regarding why the program compiles and executes without an error:
Variable a is declared inside the body of main with extern linkage: no space is allocated for it and a would be undefined outside the body of main.
sizeof(a) is evaluated at compile time as the constant value sizeof(int), which happens to be 4 on your platform.
No further reference to a is generated by the compiler, so the linker does not complain about a not being defined anywhere.
i am new in programming and i started with c . i am learning in the point How Linkers Resolve Global Symbols Defined at Multiple Places. so i made a small program which says:
//main.c
#include <stdio.h>
void b(void);
int x;
int a=10;
int y =500;
int x1 = 2016;
int main()
{
b();
printf("x = %1f y = %d\n",x , y);
return 0;
}
//second_file.c
double x=100.0;
int g=100;
extern int y;
void b()
{
x = -100.0;
}
i know from the rules that linkers follow to solve this case that, the linker will choose the strong symbol which in this case (double x=100.0;) not the integer one defined in main.c. so i expected the output to be x=-100.0 y=500,
but the output was x=0.000000 y=89 can any one explain why did i find this output or where is my wrong?
i tried to use objdump command in cmd (for windows) to view the symbol table may i find something make my understand but i found another thing weird i found the addresses of the variables as follow:
a------ 0x00000000
y------ 0x00000004
x1----- 0x00000008
x------ 0x00000010
g------ 0x00000018
i thought that x should start from 0x0000000c why not?
As you may or may not know, you need to be careful with multiple definitions. Ideally, every global symbol will be defined precisely once. There are various circumstances under which you can get away with multiple definitions, but you have to be careful.
In this case, you're relying on the "common allocation model", which was (and still is) widespread among C compiler because of the early influence of, believe it or not, FORTRAN. This model says that you can have multiple definitions as long as at most one of gives an initializing value. But -- and this is the biggie -- all the definitions must have the same type. (I'm not sure I've ever seen this rule stated; I'm not sure I'd even thought about it explicitly, because it ends up being pretty obvious that it has to be that way.)
C uses the concept of separate compilation. Each .c file compiles down to a separate, standalone "object file". Later, a separate program called the linker links the object files together. It's the linker that assigns the final address to global variables, and ends up resolving common allocation.
But the linker only resolves addresses. It will arrange that x in your main.c and x in your second_file.c are at the same address. But if main.c thinks there's an int located at location x, that's how it's going to interpret it. And if second_file.c thinks there's a double at location x, and indeed stores a value of type double there, it's going to be gibberish when main.c tries to interpret that bit pattern as an int.
As #user58697 discusses in another answer, you've got additional problems in your printf call, trying to use %f to print (what the compiler thinks are) int values, which is never going to work properly, either.
The declaration in main.c
int x;
has nothing to do with the one in second_file.c
double x=100.0;
because their scope (visibility) is limited to its compilation unit. That is, their are globals but (roughly) only visible along functions in the same file.
In case you wanted both files to share the same global variable you have to specify it by using externkeyword:
extern int x;
in one of the files, and that will force the linker to match a symbol at linking time that implements that declaration.
Don't forget: you can declare a variable (consistently in type) multiple times but only define it once. To avoid taking this answer too long, try to search in stackoverflow for the best approach for using globals in multiple files; it's much better to work via *.h for declarations an of course implement it in one *.c file.
Finally, in this example anyway the linker will not match the int xdeclaration with the double xdefinition, and it will throw an error at linking time.
First, in the line
printf("x = %1f y = %d\n",x , y);
you lied to printf. You told the compiler to push an int (apparently 4 bytes worth), but instructed printf to pull a double (apparently 8 bytes). This is an UB. All bets are off. Nobody knows where printf would pull y from.
And this is what happens with x:
vnp$ nm -n main.o
U _b
U _printf
0000000000000000 T _main
0000000000000004 C _x
0000000000000040 D _a
0000000000000044 D _y
0000000000000048 D _x1
You can see that _x is annotated with C (for Common). The compiler took int x; as a definition, realized that x is an uninitialized global, and placed it in BSS. By the time it compiled the
printf("x = %1f y = %d\n",x , y);
line, it generated a relocation record relative to the .bss.
The linker, however, selected x from the second file, and which is placed in .data, but performed the relocation as directed, that is relative to .bss. This is why x prints as 0.
Disclaimer: since there is an UB, the above is a pure (however educated) speculation.
PS: the address of x is due to the alignment requirement or double.
I have code that looks like this :
Foo* create(args) {
Foo *t = malloc (sizeof (Foo)) ;
// Fill up fields in struct t from args.
return t;
}
The call is
Foo *created = create (args)
Note that the function and the call to the function are two separate modules.
The value of the pointer assigned to t on being malloced is slightly different to what is captured in created. Seemingly the MSB of the address is changed and replaced with fffff. The LSB portion is the same for around 6-7 characters.
I'm at a loss as to what's going on.
I'm using GCC 4.6
The most likely explanation one can come up with from what you provided is that at the point of the call the function create is undeclared. A permissive C compiler assumed that unknown function create returned an int and generated code, which effectively truncated the pointer value or, more precisely, sign-extended the assumed int return value into the MSB of the recipient pointer (assuming a platform where pointers are wider than int in terms of their bit-width, e.g. 64-bit platform).
Most likely the compiler issued a warning about an undeclared function being called, but the warning was ignored by the user.
Make sure declaration (or, better, prototype) of create is visible at the point of the call. If you are compiling in C99 mode (or later), ask your compiler to strictly enforce C99 requirements. And don't ignore compiler diagnostic messages.
I have the following piece of C code which prints the rip register and the address of a function foo. Running the executable multiple times results in the same values of rip and &foo being printed.
#include <stdio.h>
#include <inttypes.h>
void foo(int x) {
printf("foo sees %d\n", x);
}
int main(int argc, char *argv[]) {
uint64_t ip;
asm("leaq (%%rip), %0;": "=r"(ip));
printf("rip is 0x%016" PRIx64 "\n", ip);
void (*fp)(int) = &foo;
printf("foo is at offset %p\n", fp);
(*fp)(10);
return 0;
}
Q1: Why does rip remain the same?
Q2: Will &foo remain the same, provided the binary and machine remain the same?
Q3: When can &foo change?
Background: I am trying to store the execution times of functions in a history table. I am thinking of using the function address to index into the table and calculate deviations from previous executions.
Q1:
Depends on your platform. Some platforms load your program into a virtual address space, so the exact same code will have the exact same virtual address for foo (assuming the program and the OS's loader don't change between runs, and the loader isn't one that randomizes the load address per the comments). On other platforms that do not load your executable into a virtual address space, you may or may not get the same address depending on whether other programs have executed and/or terminated between runs.
Q2:
Don't count on it. If nothing changes at all, you will have deterministic behavior (same address). But there are many, many things that can change (again, dependent on the platform).
Q3:
They can change at any time on a platform that doesn't allocate a virtual address (as other processes start/continue doing work/terminate). On a platform that does allocate a virtual address, they addresses can change if your program or related libraries change at all, if there is an OS patch that changes loader behavior, or probably due to other circumstances I'm not thinking of at the moment.
Bottom Line
Storing the address may work for your very specific case, but it's a fragile solution.
Nothing is guaranteed.
The solution is to index using the function name, not its address (The C99 standard provides the __func__ identifier). That way your index is guaranteed to remain the same across all changes in OS, compiler, options, and phase of the moon. Until you refactor the function name, of course :-)
Since you're using Linux you can use dladdr() to ask about symbols near places in memory. For example:
#define _GNU_SOURCE
#include <dlfcn.h>
#include <stdio.h>
void foo() {
}
int main() {
Dl_info info;
void *test = foo; // Note: not standard C
dladdr(test, &info);
printf("closest symbol: %s in %s\n", info.dli_sname, info.dli_fname);
return 0;
}
when compiled with:
gcc -Wall -Wextra test.c -ldl -rdynamic
Correctly identifies the void* as foo, which will be correct no matter where foo gets loaded.