I want to make sure the return address of sbrk is within a certain specific range. I read somewhere that sbrk allocates from an area allocated at program initialization. So I'm wondering if there's anyway I can enforce the program initialization to allocate from a specific address? For example, with mmap, I'll be able to do so with MAP_FIXED_NOREPLACE . Is it possible to have something similar?
No, this is not possible. brk and sbrk refer to the data segment of the program, and that can be loaded at any valid address that meets the needs of the dynamic linker. Different architectures can and do use different addresses, and even machines of the same architecture can use different ranges depending on the configuration of the kernel. Using a fixed address or address range is extremely nonportable and will make your program very brittle to future changes. I fully expect that doing this will cause your program to break in the future simply by upgrading libc.
In addition, modern programs are typically compiled as position-independent executables so that ASLR can be used to improve security. Therefore, even if you knew the address range that was used for one invocation of your program, the very next invocation of your program might use a totally different address range.
In addition, you almost never want to invoke brk or sbrk by hand. In almost all cases, you will want to use the system memory allocator (or a replacement like jemalloc), which will handle this case for you. For example, glibc's malloc implementation, like most others, will allocate large chunks of memory using mmap, which can significantly reduce memory usage in long-running programs, since these large chunks can be freed independently. The memory allocator also may not appreciate you changing the size of the data segment without consulting it.
Finally, in case you care about portability to other Unix systems, not all systems even have brk and sbrk. OpenBSD allocates all memory using mmap which improves security by expanding the use of ASLR (at the cost of performance).
If you absolutely must use a fixed address or address range and there is no alternative, you'll need to use mmap to allocate that range of memory.
Can I use functions malloc and delete in embedded C? For example, I have one function, where was created pointer on structure with function malloc. This function return address in ram and I can use this . After exit from my function, where memory was allocated, this pointer will be deleted or this memory reserved for this, while not will be function delete terminated ?
Typedef struct {
Char varA;
Char varB
} myStruct ;
Void myfunc ( void)
{
myStruct * ptrStruct = ( myStruct *) malloc ( sizeof (myStruct)) ;
// Code here
//........
return ;
}
Generally, you shouldn't be using malloc in embedded systems, because doing so doesn't make any sense as explained here. In particular, it doesn't make any sense what-so-ever to use it on bare metal systems.
The only place where it makes sense to use dynamic memory allocation is large hosted, multi-process systems where multiple processes share the same RAM. If your definition of an embedded system is an Android smart phone or a portable PC, then yes it is fine to use malloc.
If you find yourself using it anywhere else, it almost certainly means that your program design is fundamentally flawed and also that you don't know how a heap works.
In addition, almost every embedded systems programming standard bans dynamic memory allocation.
There is nothing specific about embedded systems that prevent the use of dynamic memory.
However you may need to provide support for it in a number of ways, for example:
You need to ensure that the linker allocates sufficient space for the dynamic heap. Some linker scripts may already automatically allocate all remaining memory to the heap after stack and any other reserved allocations.
You may need to implement low level stubs to allow the library to access heap memory - for example in the newlib library, you need to implement sbrk_r() for malloc() etc. to work correctly.
In a multi-threaded system you may need to implement mutex stubs to ensure safe heap allocation. If the library does not provide such stubs, then malloc()/free() etc. will not be safe to use in such an environment, and you should write wrapper functions that assert the locks externally.
There are a number of reasons however why you might choose to avoid using dynamic memory (or at least standard library implemented dynamic memory) in an embedded system however:
Standard allocation schemes have non-deterministic timing unsuited to hard-real-time systems.
You need to handle the possibility of allocation failure gracefully for every allocation. Handling a potential non-deterministic run-time error safely is more complex than simply having the compiler tell you have insufficient memory at build time.
You need to guard against memory leaks; true of any system, but with no OS to manage memory exhaustion and kill a leaking process how will your system behave?
The standard library heap management may not be thread-safe without mutex stubs or wrapper functions.
Bugs that corrupt the heap are unlikely to affect execution immediately, often only causing an observable failure when a new heap operation is performed, resulting in non-deterministic behaviour at a time and location unrelated to the actual cause - making them very hard to diagnose. Again this is true of any system, but the debug facilities in a cross-hosted embedded system are often less sophisticated that on a self-hosted system.
Yes, you can use malloc in embedded C. Some embedded systems have its own encapsulated memory allocation APIs. malloc() is the C lib API.
The memory is allocated from heap, a dedicated memory range defined by system designer. If you did not free the allocated memory after your function exits, the allocated memory is reserved and other processes cannot use it. Typically, it is memory leak. If you free the allocated memory but you still use the pointer after that, it is a wild pointer and will cause unknown behaviour.
When the OS loads a process into memory it initializes the stack pointer to the virtual address it has decided where the stack should go in the process's virtual address space and program code uses this register to know where stack variables are. My question is how does malloc() know at what virtual address the heap starts at? Does the heap always exist at the end of the data segment, if so how does malloc() know where that is? Or is it even one contiguous area of memory or just randomly interspersed with other global variables in the data section?
malloc implementations are dependent on the operating system; so is the process that they use to get the beginning of the heap. On UNIX, this can be accomplished by calling sbrk(0) at initialization time. On other operating systems the process is different.
Note that you can implement malloc without knowing the location of the heap. You can initialize the free list to NULL, and call sbrk or a similar function with the allocation size each time a free element of the appropriate size is not found.
This only about Linux implementations of malloc
Many malloc implementations on Linux or Posix use the mmap(2) syscall to get some quite big range of memory. then they can use munmap(2) to release it.
(It looks like sbrk(2) might not be used a lot any more; in particular, it is not ASLR friendly and might not be multi-thread friendly)
Both these syscalls may be quite expansive, so some implementations ask memory (using mmap) in quite large chunks (e.g. in chunk of one or a few megabytes). Then they manage free space as e.g. linked lists of blocks, etc. They will handle differently small mallocs and large mallocs.
The mmap syscall usually does not start giving memory range at some fixed pieces (notably because of ASLR.
Try on your system to run a simple program printing the result of a single malloc (of e.g. 128 int-s). You probably will observe different addresses from one run to the next (because of ASLR). And strace(1)-ing it is very instructive. Try also cat /proc/self/maps (or print the lines of /proc/self/maps inside your program). See proc(5)
So there is no need to "start" the heap at some address, and on many systems that does not make even any sense. The kernel is giving segments of virtual addresses at random pages.
BTW, both GNU libc and musl libc are free software. You should look inside the source code of their malloc implementation. I find that source code of musl libc is very readable.
On Windows, you use the Heap functions to get the process heap memory. The C runtime will allocate memory blocks on the heap using HeapAlloc and then use that to fulfil malloc requests.
I have a couple of questions.
Suppose a program is compiled using 2 object files. Each uses malloc and free in most of their functions. But these object files were generated at different times and happen to be using different malloc implementations. Let's say the implementations share variable names and function names. Will the program work fine or not? Why?
If a program has object file 1 and 2, code from object file 1 call malloc and allocates some memory then frees it. Now code from object file 2 calls malloc. Can it use the memory that was freed? How does it work underneath?
Trying to provide a useful answer, even though it's far from complete.
Part 1.
First, it's hard enough to link the program with two implementations of malloc sharing function names: duplicate definitions usually cause linker errors. I can see how we manage to do it using GNU binutils, and there probably are some equivalent tricks for other toolchains. For the rest of the answer, let's assume we managed to link two implementations successfully. (It's usually a good thing that you get linker errors instead of mixing two implementations, possibly even introducing malloc/free asymmetry which has almost no chance to work).
Let's also assume that memory allocated with one particular implementation is always freed using free from the same implementation. Otherwise, it's virtually guaranteed to fail.
Two implementations may work together, or they may interfere, depending on how they request more memory from the OS when their local heaps run out of space. MS Windows has a system interface for managing heaps, and two different mallocs are likely to be built on top of it; then nothing prevents them from working together. Implementations requesting memory with sbrk-like call will work together if they're both ready that someone else will request sbrk increase independently of malloc. I'd expect that malloc from glibc won't fail here, but I'm not really sure.
Part 2.
If the implementation used by object 1 is able to return memory to OS, memory can be reused by the implementation called by object 2. That is, memory reuse may happen but it's less likely than when a single implementation is used.
The possibility of returning memory to OS depends on malloc/free implementation, and may also depend on allocated chunk size and various system settings. For example, glibc uses anonymous mmap for large chunks of memory, and these chunks are unmapped when freed.
I am writing C for an MPC 555 board and need to figure out how to allocate dynamic memory without using malloc.
Typically malloc() is implemented on Unix using sbrk() or mmap(). (If you use the latter, you want to use the MAP_ANON flag.)
If you're targetting Windows, VirtualAlloc may help. (More or less functionally equivalent to anonymous mmap().)
Update: Didn't realize you weren't running under a full OS, I somehow got the impression instead that this might be a homework assignment running on top of a Unix system or something...
If you are doing embedded work and you don't have a malloc(), I think you should find some memory range that it's OK for you to write on, and write your own malloc(). Or take someone else's.
Pretty much the standard one that everybody borrows from was written by Doug Lea at SUNY Oswego. For example glibc's malloc is based on this. See: malloc.c, malloc.h.
You might want to check out Ralph Hempel's Embedded Memory Manager.
If your runtime doesn't support malloc, you can find an open source malloc and tweak it to manage a chunk of memory yourself.
malloc() is an abstraction that is use to allow C programs to allocate memory without having to understand details about how memory is actually allocated from the operating system. If you can't use malloc, then you have no choice other than to use whatever facilities for memory allocation that are provided by your operating system.
If you have no operating system, then you must have full control over the layout of memory. At that point for simple systems the easiest solution is to just make everything static and/or global, for more complex systems, you will want to reserve some portion of memory for a heap allocator and then write (or borrow) some code that use that memory to implement malloc.
An answer really depends on why you might need to dynamically allocate memory. What is the system doing that it needs to allocate memory yet cannot use a static buffer? The answer to that question will guide your requirements in managing memory. From there, you can determine which data structure you want to use to manage your memory.
For example, a friend of mine wrote a thing like a video game, which rendered video in scan-lines to the screen. That team determined that memory would be allocated for each scan-line, but there was a specific limit to how many bytes that could be for any given scene. After rendering each scan-line, all the temporary objects allocated during that rendering were freed.
To avoid the possibility of memory leaks and for performance reasons (this was in the 90's and computers were slower then), they took the following approach: They pre-allocated a buffer which was large enough to satisfy all the allocations for a scan-line, according to the scene parameters which determined the maximum size needed. At the beginning of each scan-line, a global pointer was set to the beginning of the scan line. As each object was allocated from this buffer, the global pointer value was returned, and the pointer was advanced to the next machine-word-aligned position following the allocated amount of bytes. (This alignment padding was including in the original calculation of buffer size, and in the 90's was four bytes but should now be 16 bytes on some machinery.) At the end of each scan-line, the global pointer was reset to the beginning of the buffer.
In "debug" builds, there were two scan buffers, which were protected using virtual memory protection during alternating scan lines. This method detects stale pointers being used from one scan-line to the next.
The buffer of scan-line memory may be called a "pool" or "arena" depending on whome you ask. The relevant detail is that this is a very simple data structure which manages memory for a certain task. It is not a general memory manager (or, properly, "free store implementation") such as malloc, which might be what you are asking for.
Your application may require a different data structure to keep track of your free storage. What is your application?
You should explain why you can't use malloc(), as there might be different solutions for different reasons, and there are several reasons why it might be forbidden or unavailable on small/embedded systems:
concern over memory fragmentation. In this case a set of routines that allocate fixed size memory blocks for one or more pools of memory might be the solution.
the runtime doesn't provide a malloc() - I think most modern toolsets for embedded systems do provide some way to link in a malloc() implementation, but maybe you're using one that doesn't for whatever reason. In that case, using Doug Lea's public domain malloc might be a good choice, but it might be too large for your system (I'm not familiar with the MPC 555 off the top of my head). If that's the case, a very simple, custom malloc() facility might be in order. It's not too hard to write, but make sure you unit test the hell out of uit because it's also easy to get details wrong. For example, I have a set of very small routines that use a brain dead memory allocation strategy using blocks on a free list (the allocator can be compile-time configured for first, best or last fit). I give it an array of char at initialization, and subsequent allocation calls will split free blocks as necessary. It's nowhere near as sophisticated as Lea's malloc(), but it's pretty dang small so for simple uses it'll do the trick.
many embedded projects forbid the use of dynamic memory allocation - in this case, you have to live with statically allocated structures
Write your own. Since your allocator will probably be specialized to a few types of objects, I recommend the Quick Fit scheme developed by Bill Wulf and Charles Weinstock. (I have not been able to find a free copy of this paper, but many people have access to the ACM digital library.) The paper is short, easy to read, and well suited to your problem.
If you turn out to need a more general allocator, the best guide I have found on the topic of programming on machines with fixed memory is Donald Knuth's book The Art of Computer Programming, Volume 1. If you want examples, you can find good ones in Don's epic book-length treatment of the source code of TeX, TeX: The Program.
Finally, the undergraduate textbook by Bryant and O'Hallaron is rather expensive, but it goes through the implementation of malloc in excruciating detail.
Write your own. Preallocate a big chunk of static RAM, then write some functions to grab and release chunks of it. That's the spirit of what malloc() does, except that it asks the OS to allocate and deallocate memory pages dynamically.
There are a multitude of ways of keeping track of what is allocated and what is not (bitmaps, used/free linked lists, binary trees, etc.). You should be able to find many references with a few choice Google searches.
malloc() and its related functions are the only game in town. You can, of course, roll your own memory management system in whatever way you choose.
If there are issues allocating dynamic memory from the heap, you can try allocating memory from the stack using alloca(). The usual caveats apply:
The memory is gone when you return.
The amount of memory you can allocate is dependent on the maximum size of your stack.
You might be interested in: liballoc
It's a simple, easy-to-implement malloc/free/calloc/realloc replacement which works.
If you know beforehand or can figure out the available memory regions on your device, you can also use their libbmmm to manage these large memory blocks and provide a backing-store for liballoc. They are BSD licensed and free.
FreeRTOS contains 3 examples implementations of memory allocation (including malloc()) to achieve different optimizations and use cases appropriate for small embedded systems (AVR, ARM, etc). See the FreeRTOS manual for more information.
I don't see a port for the MPC555, but it shouldn't be difficult to adapt the code to your needs.
If the library supplied with your compiler does not provide malloc, then it probably has no concept of a heap.
A heap (at least in an OS-less system) is simply an area of memory reserved for dynamic memory allocation. You can reserve such an area simply by creating a suitably sized statically allocated array and then providing an interface to provide contiguous chunks of this array on demand and to manage chunks in use and returned to the heap.
A somewhat neater method is to have the linker allocate the heap from whatever memory remains after stack and static memory allocation. That way the heap is always automatically as large as it possibly can be, allowing you to use all available memory simply. This will require modification of the application's linker script. Linker scripts are specific to the particular toolchain, and invariable somewhat arcane.
K&R included a simple implementation of malloc for example.