Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Opinion-based Update the question so it can be answered with facts and citations by editing this post.
Improve this question
Context: while compiling some C code compilers may show high RAM consumption. Preliminary investigation shows that (at least some) C compilers do not immediately free "further unused" memories: despite that such (previously allocated) memories are not used anymore, they are still kept in the RAM. The C compiler continues processing the C code, allocating more memories in the RAM, until it reaches OOM (out of memory).
The core question: should C compilers immediately free "further unused" memories?
Rationale:
Efficient RAM utilization: no need of mem_X anymore => free mem_X to let other processes (itself including) to use mem_X.
Ability to compile the "RAM demanding" C code.
UPD20210825. I've memory-profiled some C compiler and have found that it keeps in RAM "C preprocessor data", in particular:
macro table (memory pool for macros);
scanner token objects (memory pool for tokens and for lists).
At certain point X in the middle-end (after the IR is built) these objects seem not needed anymore and, hence, can be freed. (However, now these objects are kept in RAM until a point X+1.) The benefit is seen on "preprocessor-heavy" C programs. Example: "preprocessor-heavy" C program using "ad hoc polymorphism" implemented via C preprocessor (by using a set of macros it progressively implements all the needed "machinery" to support a common interface for an arbitrary (and supported) set of individually specified types). The number of "polymorphic" entries is ~50k * 12 = ~600k (yes, it does not say anything). Results:
before fix: at point X C compiler keeps in RAM ~1.5GB of unused "C preprocessor data";
after fix: at point X C compiler frees from RAM ~1.5GB of unused "C preprocessor data", hence, letting OS processes (itself including) to use these ~1.5GB.
I don't know where you get your analysis from. Most parts like the abstract syntax tree is kept because it is used in all different passes.
It might be that some, especially simple compilers don't free stuff because it's not considered necessary for a C compiler. It's a one shot compilation unit operation and than the process ends.
Of course if you build a compiler library like tinycc did you need to free everything, but even this might happen within a final custom heap clearance at the end of the compilation run.
I have not seen this ever be a problem in real world. But i don't do embedded stuff where a lack of resources can be something to worry.
allocating more memories in the RAM, until it reaches OOM (out of
memory).
None of the compilers I use ever run out of memory. Please give an example of such behaviour.
If you are an Arduino user and think about the code which will not fit into the memory - it is not the problem of the compiler, only the programmer.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
Linux suggests two types of macros for unaligned access: byteshifts and memmoves.
https://elixir.bootlin.com/linux/v4.4/source/include/linux/unaligned
asm-generic which seems to be the most generic implementation of unaligned accesses always includes only byteshifts:
https://elixir.bootlin.com/linux/v4.4/source/include/asm-generic/unaligned.h,
Although my simple benchmarks show that memmove is usually a little more effective.
I also can't find any actual examples of using unaligned memmoves in linux kernel.
Why is that?
Are there any platforms where byteshifts can be more effective than memmoves?
Why is there no implementation which uses memcpy instead of memmove?
Honestly, I'm not the most authoritative source on Linux kernel code. If I were, I would probably argue your assumptions.
As things go:
"Byteshifts" (i.e., (a[0] << 8) | a[1], depending on byte order) is usually more effective than a function call, since we know the amount of data we wish to copy... which is often the case.
The inline static functions used by the kernel are optimized away into a few simple instructions, leaving us with the best possible footprint when all we needed was to read 16, 32 or 64 bits.
I have no idea how you performed your benchmark, but I really doubt memmove can beat this approach unless the compiler optimized it away.
memcpy assumes the memory regions you're working with don't overlap.
This assumption is often out-right wrong, if not merely unsafe.
For example, we might be moving data within a buffer, which means the data might overlap the writing portion, or we might be writing a function that will be used by many other parts of the kernel and can't really make any assumption about the memory source.
I would also point out that the extra little bit of math in memmove is very cheap.
No memory dereferencing is involved which means we won't experience cache misses and we're only likely to experience instruction prediction errors (when the CPU predicts the wrong if result) if memcpy would have been a mistake anyway.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
My understanding of this has come down to C's origins as a "portable assembler" and the option of less overhead. Is thiat all there is to it?
First of all, lets be clear about what garbage is.
The Java definition of garbage is objects that are no longer reachable. The precise meaning of reachable is a bit abstruse, but a practical definition is that if you can get to an object by following references (pointers) from well known places like thread stacks or static variables, then it may be reachable. (In practice, some imprecision is OK, so long as objects that are reachable don't get deleted.)
You could try to apply the same definition to C and C++. An object is garbage if it cannot be reached.
However, the practical problem with this definition ... and garbage collection ... in C or C++ is whether a "pointer like" value is actually a valid pointer. For instance:
An uninitialized C variable can contain a random value that looks like a pointer to an object.
When a C union type that overlays a pointer with an long, a garbage collector cannot be sure whether the union contains one or the other ... or both.
When C application code "compresses" pointers to word aligned heap nodes by dividing them by 4 or 8, a garbage collector won't detect them as "pointer like". Or if it does, it will misinterpret them.
A similar issues is when C application code represents pointers as offsets relative to something else.
However, it is clear that a C program can call malloc, forget to call free, and then forget the address of the heap node. That node is garbage.
There are two reasons why C / C++ doesn't have garbage collection.
It is "culturally inappropriate". The culture of these languages is to leave storage management to the programmer.
It would be technically difficult (and expensive) to implement a precise garbage collector for C / C++. Indeed, doing this would involve things that made the language implementation slow.
Imprecise (i.e. conservative) garbage collectors are practical, but they have performance and (I have heard) reliability issues. (For instance, a conservative collector cannot move non-garbage objects.)
It would be simpler if the implementer (of a C / C++ garbage collector) could assume that the programmer only wrote code that strictly conformed to the C / C++ specs. But they don't.
But your answer seems to be, why did they design C like that?
Questions like that can only be answered authoritatively by the designers (in this case, the late Dennis Ritchie) or their writings.
As you point out in the question, C was designed to be simple and "close to the hardware".
However, C was designed in the early 1970's. In those days programming languages which required a garbage collector were rare, and GC techniques were not as advanced as they are now.
And even now, it is still a fact that garbage collected languages (like Java) are not suitable for applications that require predictable "real-time" performance.
In short, I suspect that the designers were of the view that garbage collection would make the language impractical for its intended purpose.
There are some garbage collectors built for C or C++:
Please check http://www.hboehm.info/gc/.
As you stated, garbage collection defies the purpose of performance claimed by C and C++, as it requires tracking allocations and/or reference counting.
This question already has answers here:
When you exit a C application, is the malloc-ed memory automatically freed?
(9 answers)
Closed 8 years ago.
I am using Visual C++ 2010 for a C project. I have the following code in main():
ARRAY2D a;
arr2_init(&a, 5, 5); /* 5x5 dynamic, multi-demensional array (of type int) */
arr2_release(&a);
I'm not sure if I need the last line. Can I omit out arr2_release() at the end of the program in modern OS's? I'm using Windows 7.
Yes, you can avoid releasing any resource manually which the runtime or the OS will clean up after you.
Still, do not do so please.
It is a valid optimisation for faster shutdown (and sometimes even for faster execution in exchange for memory consumption), though you must be picky about which resources you leave around:
Memory and file descriptors are efficiently handled by the OS (ancient platforms not doing so have mostly succumbed to disuse. Still, there are a few tiny systems not able to free this).
FILE buffers are efficiently cleaned up by the runtime.
Windows GUI resources are not efficiently cleaned up this way, it needs longer.
Anyway, do the cleanup and develop the right mind-set, it makes searching for leaks much easier and is better transferable to bigger and longer-running tasks.
Premature optimisation is the root of all evil. (The expert only option to optimise after measurement and careful consideration does not apply yet)
Always free your memory. The operating system will release a process' resources when it terminates, which includes its memory. But that doesn't give you garbage collection (you have to use different languages for that). Also note that it only does so after your program has ended (also stated in the comments), so as long as your program is running, the memory will not be freed if you don't do it.
Your code might be used as part of a bigger program someday, even if it's now only a few lines. So always make sure to release all resources you acquire. Also, as a C programmer, thinking about resource management should be a habit anyway.
This question already has answers here:
How to determine CPU and memory consumption from inside a process
(10 answers)
Closed 9 years ago.
Is there any way to calculate memory consumption in C. I have checked other answers on Stackoverflow but they were not satisfactory.
Something similar to the one we have in Java:
// Get the Java runtime
Runtime runtime = Runtime.getRuntime();
// Run the garbage collector
runtime.gc();
// Calculate the used memory
long memory = runtime.totalMemory() - runtime.freeMemory();
System.out.println("Used memory is bytes: " + memory + "bytes");
System.out.println("Used memory is kilobytes: " + bytesTokilobytes(memory) +"kb");
C-language itself does not provide any means.
Although every specific platform gives you some support.
For example on Windows you can look at the TaskManager, Details tab. Right click on the listview column headers to add/remove columns. Some of them give insight on how much memory the process consumes. There are a lot of other tools including commercial ones (use google), that give more detailed picture.
On Windows there is also special API that allows writing your own tool. A while ago I wrote one. I do not want this answer to be an ad.
The real question seems to be, can you get the C heap to report how much space it's currently holding. I don't know of a portable way to do this.
Or you could plug in a "debugging heap" implementation which tracks this number and provides an API to retrieve it; debugging heaps are available as second-source libraries, and one MAY come with your compiler. (Many years ago I implemented a debugging heap as a set of macros which intercepted heap calls and redirected them through wrapper routines to perform several kinds of analysis; I wasn't maintaining a usage counter but I could have done so.) ((CAUTION: Anything allocated from a debugging heap MUST be returned to that heap, not the normal heap, and vice versa, or things get very ugly very quickly.))
Or your compiler may have some other nonstandard way to retrieve this information. Check its documentation.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
I was impressed by http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html and http://shaurz.wordpress.com/2008/06/02/finding-linux-gateso1-in-assembly/ , and now i want to use these to do something practical.
That tutorial already tells you about all the things you need to do in order to not have to link an executable against libc (namely, define appropriate entry and exit points).
From there, you can do whatever you like, it's just that you'll have to re-write even the simplest stuff like printf et cetera, since practically everything that does stuff, apart from manipulating variables, needs system calls.
Let me reiterate that you are not going to save anything by going down this route. Your shell, your init system, your desktop environment all need libc already, so it's loaded anyways. Read about 'shared libraries' to see why this means an extra executable with libc dependence doesn't waste memory. On the contrary, if you reimplement all the system calls you need, all that implementation will have to be loaded into RAM, where it can't be shared with anything else because everything else uses libc.
Well, since you're going to have to do many things that require communicating with the operating system (let's assume Linux), you must do system calls. Since those are generally implemented by the C runtime library, you are going to have to re-implement them all.
At a minimum I would assume you need to do system calls to:
Query the file, to learn its size
Allocate the required memory
Map the memory into your process' address space, making sure its executable
Ask the kernel to load the named file into the newly allocated memory
Then it's probably more or less straight-forward application-level code to:
Inspect the ELF headers of the loaded data, identifying entry points
Applying initalizers for static data
Relocating any position-independent code to properly reference the mapped space
I'm almost certainly missing a bunch of things; while this isn't "magic", it's still pretty heavy lifting. You will have your work cut out for you, to say the least. Enjoy.