I'm trying to track the usage of malloc'ed area through variables that point to the are in a profiler. For example, for the following assignment inside function func().
uint64_t * dictionary = (uint64_t *) malloc(sizeof(uint64_t)*128);
I need to figure out the variable name (which is 'dictionary' in the above example) that points to the malloc'ed memory region. I instrumented malloc() to record the start address and size of the allocation. However, still no knowledge of variable 'dictionary', what I'm thinking is to examine the stack frame of function func(), finding out the local pointer variable pointing to a data type that matches that of malloc'ed type. The approach would need to instrument malloc() to go back one frame to func() to find out the possible local variables, and then fuzzy match by type. Wondering whether there are any other neat ways to implement this.
In general, I would expect this to be impossible. :)
You can't, of course, assume that the variable name is available, the best bet in general would be (I guess) a stack offset in the calling function's frame. If debugging symbols are available you might perhaps be able to map that to a name, though.
I guess it's possible that there is no name; that the return address is put in a register and perhaps manipulated there, before (if ever) being written to memory. If this means your code needs to start analyzing the calling code to track what it does with the return value, that sounds difficult.
What do you want to do with the variable reference once you've isolated it? I assume you're instrumenting malloc() for debugging purposes, so probably you're going to store it somewhere.
Related
For variables store in the stack, we can use static to avoid accessing from other files. Is there anyway to avoid pointer from other files accessing certain address?
First, to get things out of the way, static variables are never allocated on the stack because they are essentially global variables, they simply don't pollute the global namespace. It's trivial to get a pointer to a static variable and change it, statics are a compiler enforced construct.
Back to the actual question though, no you cannot try to examine the memory access directly. How would you even know if the memory you're accessing is valid or not? You can do something along the line though. You can for example, wrap malloc and free with your own memory management functions, and keep track of the memory allocated and freed along with metadata. You can then use another wrapper function that takes care of pointer dereferencing, and checks the metadata as you desire. You still can use raw pointers to wreak havoc if you want, so it isn't really much though.
Like we do with macros:
#undef SOMEMACRO
Can we also undeclare or delete the variables in C, so that we can save a lot of memory?
I know about malloc() and free(), but I want to delete the variables completely so that if I use printf("%d", a); I should get error
test.c:4:14: error: ‘a’ undeclared (first use in this function)
No, but you can create small minimum scopes to achieve this since all scope local variables are destroyed when the scope is exit. Something like this:
void foo() {
// some codes
// ...
{ // create an extra minimum scope where a is needed
int a;
}
// a doesn't exist here
}
It's not a direct answer to the question, but it might bring some order and understanding on why this question has no proper answer and why "deleting" variables is impossible in C.
Point #1 What are variables?
Variables are a way for a programmer to assign a name to a memory space. This is important, because this means that a variable doesn't have to occupy any actual space! As long as the compiler has a way to keep track of the memory in question, a defined variable could be translated in many ways to occupy no space at all.
Consider: const int i = 10; A compiler could easily choose to substitute all instances of i into an immediate value. i would occupy 0 data memory in this case (depending on architecture it could increase code size). Alternatively, the compiler could store the value in a register and again, no stack nor heap space will be used. There's no point in "undefining" a label that exists mostly in the code and not necessarily in runtime.
Point #2 Where are variables stored?
After point #1 you already understand that this is not an easy question to answer as the compiler could do anything it wants without breaking your logic, but generally speaking, variables are stored on the stack. How the stack works is quite important for your question.
When a function is being called the machine takes the current location of the CPU's instruction pointer and the current stack pointer and pushes them into the stack, replacing the stack pointer to the next location on stack. It then jumps into the code of the function being called.
That function knows how many variables it has and how much space they need, so it moves the frame pointer to capture a frame that could occupy all the function's variables and then just uses stack. To simplify things, the function captures enough space for all it's variables right from the start and each variable has a well defined offset from the beginning of the function's stack frame*. The variables are also stored one after the other.
While you could manipulate the frame pointer after this action, it'll be too costly and mostly pointless - The running code only uses the last stack frame and could occupy all remaining stack if needed (stack is allocated at thread start) so "releasing" variables gives little benefit. Releasing a variable from the middle of the stack frame would require a defrag operation which would be very CPU costly and pointless to recover few bytes of memory.
Point #3: Let the compiler do its job
The last issue here is the simple fact that a compiler could do a much better job at optimizing your program than you probably could. Given the need, the compiler could detect variable scopes and overlap memory which can't be accessed simultaneously to reduce the programs memory consumption (-O3 compile flag).
There's no need for you to "release" variables since the compiler could do that without your knowledge anyway.
This is to complement all said before me about the variables being too small to matter and the fact that there's no mechanism to achieve what you asked.
* Languages that support dynamic-sized arrays could alter the stack frame to allocate space for that array only after the size of the array was calculated.
There is no way to do that in C nor in the vast majority of programming languages, certainly in all programming languages that I know.
And you would not save "a lot of memory". The amount of memory you would save if you did such a thing would be minuscule. Tiny. Not worth talking about.
The mechanism that would facilitate the purging of variables in such a way would probably occupy more memory than the variables you would purge.
The invocation of the code that would reclaim the code of individual variables would also occupy more space than the variables themselves.
So if there was a magic method purge() that purges variables, not only the implementation of purge() would be larger than any amount of memory you would ever hope to reclaim by purging variables in your program, but also, in int a; purge(a); the call to purge() would occupy more space than a itself.
That's because the variables that you are talking about are very small. The printf("%d", a); example that you provided shows that you are thinking of somehow reclaiming the memory occupied by individual int variables. Even if there was a way to do that, you would be saving something of the order of 4 bytes. The total amount of memory occupied by such variables is extremely small, because it is a direct function of how many variables you, as a programmer, declare by hand-typing their declarations. It would take years of typing on a keyboard doing nothing but mindlessly declaring variables before you would declare a number of int variables occupying an amount of memory worth speaking of.
Well, you can use blocks ({ }) and defining a variable as late as possible to limit the scope where it exists.
But unless the variable's address is taken, doing so has no influence on the generated code at all, as the compiler's determination of the scope where it has to keep the variable's value is not significantly impacted.
If the variable's address is taken, failure of escape-analysis, mostly due to inlining-barriers like separate compilation or allowing semantic interpositioning, can make the compiler assume it has to keep it alive till later in the block than strictly neccessary. That's rarely significant (don't worry about a handful of ints, and most often a few lines of code longer keeping it alive are insignificant), but best to keep it in mind for the rare case where it might matter.
If you are that concerned about the tiny amount of memory that is on the stack, then you're probably going to be interested in understanding the specifics of your compiler as well. You'll need to find out what it does when it compiles. The actual shape of the stack-frame is not specified by the C language. It is left to the compiler to figure out. To take an example from the currently accepted answer:
void foo() {
// some codes
// ...
{ // create an extra minimum scope where a is needed
int a;
}
// a doesn't exist here
}
This may or may not affect the memory usage of the function. If you were to do this in a mainstream compiler like gcc or Visual Studio, you would find that they optimize for speed rather than stack size, so they pre-allocate all of the stack space they need at the start of the function. They will do analysis to figure out the minimum pre-allocation needed, using your scoping and variable-usage analysis, but those algorithms literally wont' be affected by extra scoping. They're already smarter than that.
Other compilers, especially those for embedded platforms, may allocate the stack frame differently. On these platforms, such scoping may be the trick you needed. How do you tell the difference? The only options are:
Read the documentation
Try it, and see what works
Also, make sure you understand the exact nature of your problem. I worked on a particular embedded project which eschewed the stack for everything except return values and a few ints. When I pressed the senior developers about this silliness, they explained that on this particular application, stack space was at more of a premium than space for globally allocated variables. They had a process they had to go through to prove that the system would operate as intended, and this process was much easier for them if they allocated everything up front and avoided recursion. I guarantee you would never arrive at such a convoluted solution unless you first knew the exact nature of what you were solving.
As another solution you could look at, you could always build your own stack frames. Make a union of structs, where each struct contains the variables for one stack frame. Then keep track of them yourself. You could also look at functions like alloca, which can allow for growing the stack frame during the function call, if your compiler supports it.
Would a union of structs work? Try it. The answer is compiler dependent. If all variables are stored in memory on your particular device, then this approach will likely minimize stack usage. However, it could also substantially confuse register coloring algorithms, and result in an increase in stack usage! Try and see how it goes for you!
I'm still wondering why in C you can't simply set something to be another thing using plain variables. A variable itself is a pointer to data, is it not? So why make pointers point to the data in the variable when you can simply use the original variable? Is it to access specific bits (or bytes, I guess) of data within said variable?
I'm sure it's logical, however I have never fully grasped the concept and when reading code seeing *pointers always throws me off.
One common place where pointers are helpful is when you are writing functions. Functions take their arguments 'by value', which means that they get a copy of what is passed in and if a function assigns a new value to one of its arguments that will not affect the caller. This means that you couldn't write a "doubling" function like this:
void doubling(int x)
{
x = x * 2;
}
This makes sense because otherwise what would the program do if you called doubling like this:
doubling(5);
Pointers provide a tool for solving this problem because they let you write functions that take the address of a variable, for example:
void doubling2(int *x)
{
(*x) = (*x) * 2;
}
The function above takes the address of an integer as its argument. The one line in the function body dereferences that address twice: on the left-hand side of the equal sign we are storing into that address and on the right-hand side we are getting the integer value from that address and then multiply it by 2. The end result is that the value found at that address is now doubled.
As an aside, when we want to call this new function we can't pass in a literal value (e.g. doubling2(5)) as it won't compile because we are not properly giving the function an address. One way to give it an address would look like this:
int a = 5;
doubling2(&a);
The end result of this would be that our variable a would contain 10.
A variable itself is a pointer to data
No, it is not. A variable represents an object, an lvalue. The concept of lvalue is fundamentally different from the concept of a pointer. You seem to be mixing the two.
In C it is not possible to "rebind" an lvalue to make it "point" to a different location in memory. The binding between lvalues and their memory locations is determined and fixed at compile time. It is not always 100% specific (e.g. absolute location of a local variable is not known at compile time), but it is sufficiently specific to make it non-user-adjustable at run time.
The whole idea of a pointer is that its value is generally determined at run time and can be made to point to different memory locations at run time.
No, a variable is not a pointer to data. If you declare two integers with int x, y;, then there is no way to make x and y refer to the same thing; they are separate.
Whenever you read or write from a variable, your computer has to somehow determine the exact location of that variable in your computer's memory. Your computer will look at the code you wrote and use that to determine where the variable is. A pointer can represent the situation where the location is not known at the time when you compile your code; the exact address is only computed later when you actually run your code.
If you weren't allowed to use pointers or arrays, every line of code you write would have to access specific variables that are known at compile time. You couldn't write a general piece of code that reads and writes from different places in memory that are specified by the caller.
Note: You can also use arrays with a variable index to access variables whose location is not known at compile time, but arrays are mostly just syntactical sugar for pointers. You can think about all array operations in terms of pointer operations instead. Arrays are not as flexible as pointers.
Another caveat: As AnT points out, the location of local variables is usually on the stack, so they are a type of variable where the location isn't known at compile time. But the only reason that the stack works for storing local variables in a reentrant function is that your compiler implements hidden pointers called the stack pointer and/or frame pointer, and functions use these pointers to find out which part of memory holds their arguments and local variables. Pointers are so useful that the compiler actually uses them behind your back without telling you.
Another reason: C was designed to build operating systems and lots of low level code that deals with hardware. Every piece of hardware exposes its interface by means of registers, and on nearly all architectures, registers are mapped into the CPU memory space, and they have not to be in the same address always (thanks to jumper settings, PnP, autoconfig, and so on)
So the OS writer, while writing a driver for instance, needs a way to deal with what seems memory locations, only that they don't refer RAM cells.
Pointers serve to this purpose by allowing the OS writer to specify what memory location he or she wants to access to.
For example, in Linux, I have a pointer pointing to a task_struct. Later, the task_struct might migrate or deleted. How do I know whether the pointer still points to a task_struct or not?
It's not possible.
Pointers only contain addresses, and generally it's not possible to determine whether or not a given address is "valid".
Sometimes you can ask the entity that gave you the pointer to begin with if it's still valid, but that of course depends on the exact details of the entity. The language itself cannot do this.
You don't know, because:
a pointer just contains the address of the object it points to;
the type information is lost at compile time.
So, C provides no facilities for dealing with this kind of problems, you have to track what happens to stuff you point to on your own.
The most you can ask (and it is alreay OS-specific) is to check if the memory page where the structure would reside is still accessible, but usually it's not a particularly useful information.
Depending on your allocation pattern/luck, you might get a segmentation fault (which of course kills your program)...but that at least would tell you the reference is no longer valid.
However, as previously stated, the best way is to track the validity yourself.
If you need to keep moving a struct around in memory (rather than just blanking it and reinitializing it at its current location), you could consider using a pointer to a pointer to make the tracking easier.
"ie. Everything gets a reference to the pointer to the struct, and then when you move or delete the struct you just set that pointer to NULL or to the new memory location."
Also, in general, if you want to do checks on your program for this kind of weirdness, I would recommend looking into valgrind.
It is your responsibility in C to write your code so that you keep track of it. You can use the special value of NULL (representing not pointing to anything), setting the pointer to NULL when you remove (or haven't yet set) whatever it was pointing to & testing for NULL before using it. You might also design your code in a way that the question never comes up.
There is no way to query a random pointer value to see if it represents something, just like there is no way to query an int variable to check if the value in it is uninitialized, junk, or the correct result of a computation.
It is all a matter of software design and, when necessary, using the value of NULL to designate not set.
Why do the variables take garbage values?
I guess the rationale for this is that your program will be faster.
If compiler automatically reset (ie: initialize to 0 or to NaN for float/doubles etc etc) your variables, it would take some time doing that (it'd have to write to memory).
In many cases initializing variables could be unneeded: maybe you will never access your variable, or will write on it the first time you access it.
Today this optimization is arguable: the overhead due to initializing variables is maybe not worth the problems caused by variables uninitialized by mistake, but when C has been defined things were different.
Unassigned variables has so-called indeterminate state that can be implemented in whatever way, usually by just keeping unchanged whatever data was in memory now occupied by the variable.
It just takes whatever is in memory at the address the variable is pointing to.
When you allocate a variable you are allocating some memory. if you dont overwrite it, memory will contain whatever "random" information was there before and that is called garbage value.
Why would it not? A better question might be "Can you explain how it comes about that a member variable in C# which is not initialised has a known default value?"
When variable is declared in C, it involves only assigning memory to variable and no implicit assignment. Thus when you get value from it, it has what is stored in memory cast to your variable datatype. That value we call as garbage value. It remains so, because C language implementations have memory management which does not handle this issue.
This happens with local variables and memory allocated from the heap with malloc(). Local variables are the more typical mishap. They are stored in the stack frame of the function. Which is created simply by adjusting the stack pointer by the amount of storage required for the local variables.
The values those variables will have upon entry of the function is essentially random, whatever happened to be stored in those memory locations from a previous function call that happened to use the same stack area.
It is a nasty source of hard to diagnose bugs. Not in the least because the values aren't really random. As long as the program has predictable call patterns, it is likely that the initial value repeats well. A compiler often has a debug feature that lets it inject code in the preamble of the function that initializes all local variables. A value that's likely to produce bizarre calculation results or a protected mode access violation.
Notable perhaps as well is that managed environments initialize local variables automatically. That isn't done to help the programmer fall into the pit of success, it's done because not initializing them is a security hazard. It lets code that runs in a sandbox access memory that was written by privileged code.