Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
In C, we can allocate memory automatically or dynamically:
size_t foo = 128;
char str1[64]; // Automatic
char str2[foo] // Automatic (variable-length, C99)
char *str3 = malloc(64); // Dynamic
char *str4 = malloc(foo); // Dynamic
With automatic allocation, variables have a limited scope and the memory is usually allocated on the stack, whereas dynamic allocation uses the heap.
Let's stick with strings and say a function is supposed to return a string that it somehow creates. There seem to be two common practices, both can be seen with standard library functions:
Take a buffer and a size, then write into the provided buffer
Allocate a buffer with malloc() and return the pointer to it
The first has the advantage that we don't need to remember to free() the memory. On the other hand, we need to hand in two additional arguments and somehow need to determine from the outside what size the buffer should have:
char str1[128]; // Let's hope this is large enough!
fill_my_buffer(str1, 128); // Might need more than 128, who knows
char *str2 = return_a_ptr(); // Just give me a pointer
size_t len = strlen(str2); // Ah, so that's what we got!
What (other) considerations should be taken into account when choosing between automatic or dynamic memory allocation in C?
The question was closed as opinion-based, but I'm convinced it should be possible to compile a comprehensive list of objective pros and cons for either approach, which would then allow making informed decision on a case-by-case basis.
One rule of thumb is that you should avoid "large" allocations on the stack. A page (4096 bytes) or two is fine, but anything larger should be heap-allocated.
Changing the stack pointer too much will kill your cache performance, and cache is king.
Also, you might overflow the stack and miss the guard page, although this is mostly only a problem with threads which necessarily have limited-size stacks, or else on otherwise constrained systems which can't have automatic main-thread stack growth.
Other than that, the main question is semantics: does this (small) object "belong" to this stack frame, or does it need to live for a different period of time (longer or shorter).
The need to call free should not be a concern at all, there are all sorts of functions that always must be called in pairs (which is the main problem that C++ tried to address, and also GNU's __attribute__((cleanup)) extension) and you should develop the discipline of always knowing to do it.
Notably, every function that takes a pointer should have a concept of "who is in charge of this ownership"? Common ownership models include: borrowed (owned by some higher stack frame), uniquely-owned, and shared (often refcounted) ownership, although there can be dozens of subtle variants (though not all in the same program).
It's basically a judgement call — can the calling code know how big a buffer is likely to be?
If so, it is probably better that it provides the buffer and the called code must ensure it doesn't overflow it (which in turn means that the interface must specify the size of the buffer provided).
If not, then the called code should allocate and return the memory.
The complication with the caller-provided buffer is "what to do when the provided space isn't big enough". The complications with the caller-provided buffer are ensuring it is freed appropriately and deciding what to do if memory allocation fails.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I want to move on from writing small one/two/three-filed C programs to slightly larger C projects. For that I really want to get the memory management right. Now I know that similar questions have been asked, but they won't quite answer mine. I know some of the theory already and put it to use. Therefore I'd like to present what I know or think to know and you correct me or add information I miss.
There is the stack, static memory and the heap
static int n; // goes to static memory, thread lifetime
char *array = malloc(256 * sizeof(*array)); // goes to heap, needs free(array)
func(int n) {
float f; // goes to stack, dies with func return
static double d; // thread lifetime again
}
Static memory can't overflow since it's set for all static variables,
however heap and stack can overflow, heap overflows when non allocated memory is accessed in most cases, stack is set to ~1MB Windows or ~8MB Linux, if that is exhausted (I got a message "core dumped" on my Ubuntu for setting up an array of structs for every pixel of an image on the stack)
Does static, stack and heap memory behave like this in every scope?
I know heap memory does. But does a global non static array of structs go on the stack? What if I have a static array in file b where there is no main?
Goes on to static memory right? And what is if a function has a local static variable with initialized value? Is it initialized everytime i call the function? and do functions take from the stack?
How to avoid exhausting the stack in large programs with long lifed stack variables and plenty of them? And most of all, where do string literals go?? They are supposed to be pointers not arrays, but if i change the pointer what happens to the original string? One more thing: It always bothers me how it looks straight up bad practise to write code like
if(!strcmp(a, "comparethis")) do ...
or
fprintf(stderr, "There was a problem .... %d", something);
EDIT
Where is the difference in terms of memory management?
char arr[3] = {'A', 'B', 'C'};
char arr[3] = "ABC";
char *arr = "ABC";
EDIT END
Is it good to include string literals anyway or rather read them from a file or whatnot?
Finally, sorry if my grammar failed me here and there but this was written in a fast manner. Thanks for reading and please no hate. Outline my mistakes, don't judge them, if I can ask that much. I wanna improve and fast.
Have a good day anyway.
There is the stack, static memory and the heap
Not exactly. C standard defines automatic storage, static storage and dynamic storage. The stack is a possible (in fact the common) implementation for automatic storage, as is the heap for dynamic storage. This definitions acts on the lifetime of variables:
automatic objects have their life bound by their containing block
static objects come to life at the beginning of the program, and their life extends to the end of the program
dynamic variables are created by malloc (and associated) function(s) from the standard library and shall be destroyed by free.
Provided these rules are obeyed, an implementation if free to physically store objects where it wants to. In particular, some implementations were known to store automatic arrays on the heap and automatically destroy them at block exit
Static memory can't overflow since it's set for all static variables
In the time of MS/DOS, the small memory model required all static variable to fit in a single segment of 64 kbytes. If you wanted more, you could not compile in that mode. The error could be a compile error in one single compilation unit caused the error, or a link error if only the total size exceeded 64k
stack is set to ~1MB Windows or ~8MB Linux
Compiler options allow to change the size of the stack on common compilers
Now for your questions:
does a global non static array of structs go on the stack?
it can depend on implementation, provided it is automatically destroyed when leaving block, it can be stored on the heap
What if I have a static array in file b where there is no main? Goes on to static memory right?
Yes it has to be static, whether the translation unit contains a main or not
And what is if a function has a local static variable with initialized value? Is it initialized everytime i call the function?
as it is static, it is created and initialized at the beginning of the program, and keeps its value through other calls. It is not reinitialized on following calls
and do functions take from the stack?
What do you mean here? Common implementation do use the stack for the function return address and all its automatic variables
How to avoid exhausting the stack in large programs with long lifed stack variables and plenty of them?
You can increase the stack size at compile time, or use a different design. For example iterative algorithms are less stack consuming than recursive ones
And most of all, where do string literals go?
A string litteral has static duration. Some implementation store them in a read only segment, but there is no requirement for it. Simply it is undefined behaviour to try to modify a string litteral.
Where is the difference in terms of memory management?
char arr[3] = {'A', 'B', 'C'};
char arr[3] = "ABC";
char *arr = "ABC";
First and second ones both declare a non const array of 3 characters initialized with A, B and C, and no terminating null.
Third one is quite different: it declares a pointer to a (null terminated) string litteral. That means that arr[0] = 'X'; is undefined behaviour because it modifies a litteral. And sizeof(arr) the length of the string litteral but sizeof(char *).
Is it good to include string literals anyway or rather read them from a file or whatnot?
I cannot understand the question. A string litteral is available inside the program. A string stored in a file requires access to the file. So at least the file name has to be a string litteral.
This question already has answers here:
Determine size of dynamically allocated memory in C
(15 answers)
Closed 5 years ago.
I have read in: How does free know how much to free? that when one has some memory allocation denoted with a pointer such as
float (*ptr)[10]=malloc(sizeof(float)*100)
for a 10x10 array, ptr has a "head" to it with "accounting" information telling of the "step size" and what not so that you can properly perform pointer arithmetic and use free and whatnot.
Is there a consistent (not architecture dependent) and reliable (defined behavior) that can allow one to get their hands on this information?
I have read elsewhere that the de facto way to track array length when there are casts and dynamic memory allocations about is to manually allocate a slot to store the size. This naturally leads me to believe the answer to my question is 'no' yet I think I'd rather not make assumptions or I'll get my own sort of memory leakage.
Converting comments into an answer.
There is no defined standard way to get at the 'size of the block of allocated memory'. Each implementation has to have a way of knowing the size of each block it allocates, but there's no way for a programmer using the implementation to know the size (in general).
So it is dependent on some number of things, but if all is known, system, architecture, compiler, you're saying there is no resource to find out how things are formatted in memory?
There is no standard (neither de jure nor de facto standard) way to get at the information about the size of a block of memory allocated. All else apart, the size allocated by the library is usually bigger than the size requested (definitely because of the housekeeping data, but even the data portion may be rounded up to a multiple of 8 or a multiple of 16) — should the code report the size requested or the size allocated?
And, as 1201ProgramAlarm noted, one option on open source systems is to look at the C library's implementation of malloc() and free() to see what it does and devise a mechanism to provide the answer to the programmer. However, any such research is specific to that system — different systems will do it differently, in general — and the whole idea runs into a stone wall if the system is a closed source system.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
free() only needs the pointer value to release allocated memory. Which means that C knows how big assigned memory blocks are. Then how come there isn't some built-in function to find the size of a pointer array?
I know the convention is to keep track of array sizes, but given that there already is some memory management happening natively, why don't we harness that to have a convenient size() function for arrays?
Such a function would be possible. The question is why the C standard doesn't require it.
The GNU C library implementation provides a function malloc_usable_size() that's similar to what you're suggesting. It returns the number of usable bytes for a malloc()ed pointer -- which may be greater than the requested size for various reasons.
It's not 100% clear why no such function is in the standard. It's not mentioned in the 1989 ANSI C Rationale or in the ISO C99 Rationale.
One thing to keep in mind is that an implementation doesn't need to keep track of the number of bytes requested by a malloc() call. It will often round up the request to some larger value, and it only needs to keep track of that.
For that matter, free() doesn't necessarily need to know the size of the block being deallocated. It might just call some lower-level function that doesn't provide that information. Or, for example, allocated blocks might b organized into linked lists, one list for each allocated size; free() might simply release a block from that list without having to know the size.
Finally, C programmers have gotten along without such a function for decades. Adding a requirement to provide it would impose some (probably fairly small) overhead on all implementations. I think the attitude is that you can simply remember how much memory you asked for, and use that information as needed.
If you allocate a single object:
some_type *ptr = malloc(sizeof *ptr);
then sizeof *ptr gives you the size of the object. If you allocate an array:
some_type *ptr = malloc(count * sizeof *ptr);
then sizeof *ptr only gives you the size of a single element of the allocated array -- but if you remember the value of count you can compute the total requested size easily enough.
Bottom line: The C standard could have required such a function, but it's not really necessary.
UPDATE: Kerrek SB makes an excellent point in a comment, one that I hadn't thought of. I'll take the liberty of summarizing it here.
A function that operates on an array via a pointer to its initial element (and there are a lot of such functions) shouldn't have to care how the array was allocated. The proposed size() function, like the GNU-specific malloc_usable_size(), works only when the argument points to a heap-allocated array. This means that the function either has to assume that the array is heap-allocated (and be right about that assumption!) or be given extra information. In the latter case, it might as well be given the size of the array, making size() superfluous.
free() may use internal data to reclaim the block of memory being released, but be aware that this data does not necessarily contain the exact size passed to malloc(), calloc(), or realloc() to allocate the block. The C Standard does not specify a function to retrieve this information.
Most malloc() implementations provide a non-standard function to retrieve the available size of the allocated block: in the Glibc, this function is size_t malloc_usable_size(void *ptr);. Other libraries may have a different function or no function at all to retrieve this information.
As for a generic solution to retrieve the size of the array to which you have a pointer, this is usually not possible. In efficient implementations, pointers do not carry size information. It is possible to implement fat pointers that would carry this information, but the whole system needs to be compiled this way. Some integrated compilers such as tcc support this approach to provide runtime pointer checking.
Because basically, the address will point on a chunk of memory, which contains meta-data (such as the size of the chunk). Freeing that entry will actually mark the block available (if the pointer is valid).
If the caller access that memory location afterward, that's undefined behaviour. So even from that point of view, free will have its job done.
free() only needs the pointer value,because you can only pass the pointer that malloc() return.The malloc() will write the size of this assign in the front address of return pointer.When you pass the pointer to free(),free() will read the size,so free() knows how many space to release.Therefor,there not used any function to find the size of a pointer array.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
how to allocate the dynamic memory with our own function
without using malloc(), how to allocate dynamic memory using C language.
You cannot get fresh heap memory without some support from the underlying operating system. I am assuming you have a POSIX operating system, e.g. Linux.
You could define your own malloc, but (in a hosted C implementation) most library functions assume that it has the traditional semantics (two successive and successful calls to malloc without any free-s producing two unaliased pointers to distinct non-overlapping memory zones).
In practice, your system malloc is generally implemented by querying fresh segments -in multiples of 4Kbytes pages- from virtual memory in your address space with a system call like mmap(2). But your standard C library malloc tries hard to reuse previously free-d memory zones before calling mmap, and it allocates some "large" (e.g. 128Kbytes or 1Mbytes) memory chunks using mmap and organize it as a set of memory zones (details are complex, since most malloc implementations are optimized for actual common use-cases). Quite often, malloc handles small allocations differently than large ones.
Occasionally (but most often not) a malloc implementation might release memory to the kernel using e.g. munmap but this does not happen often in practice. So in practice a process which has malloc-ed a lot of memory in many small zones and have free-d almost all of them still keep a lot of memory (to be able to reuse it without any mmap)
If you do not want to use malloc() provided by the library, you will have to implement your own memory manager, but I don't see any reason for doing so. That code is thoroughly tested and is in use for a long time.
You can implement your own sample memory manager based on the sbrk() system call.
I would recommend you to go through the following link.
Assuming you are on linux, check the link below:
http://www.ibm.com/developerworks/linux/library/l-memory/
There's no reason not to use malloc to dynamically allocate memory.
That said, you could define global char array with a very large size and write a series of functions to hand out space from that array. However, you need to keep track of what is/isn't available and be aware of alignments issues. And that's just scraping the surface.
Bottom line: use malloc.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I want to avoid memory allocations to construct the path for device
char *device;
const char *p;
char buf[PATH_MAX];
Witch method is memory less use
PATH_MAX
snprintf(buf, sizeof(buf) - 1, "/dev/%s", p);
device = &buf[0];
or
asprintf
asprintf(&device, "/dev/%s", p);
asprintf allocates the memory needed dynamically while the first method declares a (probably) larger buffer on the stack. So asprintf uses exactly as much bytes as needed and char buf[MAX] does not. But you have to make sure you free the memory allocated in asprintf yourself
The most significant difference is not the few bytes of memory, but where the memory is coming from and how you need to handle it afterwards.
buf is allocated on the stack, and if you return from the function your snippet is coming from, then device will not point to a valid memory location anymore. Corruption or a crash will be the most likely outcome. On the other hand, if you only use device inside the function, you won't have to free it afterwards. Also, stack-allocated memory is more efficient in general, although caching and optimizations may equal the picture.
asprintf allocates the memory from the heap, which means you can safely pass it around to all parts of your program as long as it runs. The downside is that you have to free it when you don't need it anymore or it will hang around forever and you have a memory leak (which might matter or not, depending on how many unfreed buffers are floating around). It's then your responsibility to free the memory.
But to answer your question: asprintf will allocate no more bytes than needed, and you don't have to specify a maximum size which might "feel" better. You probably shouldn't worry about a few bytes of stack memory extra though. How big is PATH_MAX? Half a kilobyte?