My Question is related to the memory layout in embedded system
I learned that when we flash(or burn) a executable file it sits either in ROM or FLASH depending on the hardware we use.
But i also learned from the memory layout of a c-program that program segment contains the .text area ( i.e compiled code)
My question is :
1) Is it the same code what we burn in flash/ROM sits in RAM(as depicted in .text area of program code)
2) Two copies are created one in flash/ROM and RAM ??
1) Depending on your hardware, the .text (and other segments) may be accessible directly in flash/ROM, or (in the case of serial flash), it may need to be copied into RAM to be executable.
2) The version in flash/ROM is the only version, UNTIL execution starts. Then (depending on the answers in 1), some start up code MAY copy the ROM into RAM to execute it, OR it may execute directly from the flash/ROM. Once executing, the C start up code MAY copy (or initialise) some of the non-code segments into RAM (e.g. .data, .bss etc).
Older, slower processors, may execute from ROM (think 8086/6502 era), whereas more modern processors (Pentium+ era, FPGA etc) would run incredibly slowly if executing from flash, so they will copy the executable into RAM (and even then, the currently executing code will be cached into the processor itself, so there may be a 3rd copy of the code).
Related
Let's say I have SOC that have different processors (embedded SOC) and different memory regions. let's say it has internal ram for 3 different processors it has and an external ddr ram, let's call them sram1,sram2,sram3, and ddr1.
let's say I write some code and write a linker that uses all 4 different memories (and perhaps few sections per memory). I think what it means at this stage is that linker assumes your code is running at those addresses and resolves addresses based on them. But in order for this code to actually run, it has to be put in the correct addresses and then run. and I'm confused about this part, who makes sure that the code is copied into the right memory addresses before it's run ? is it the initial software that is running ?
In my case, the initial code (bootloader) is loaded by ROM. the ROM looks for an address in a specidfied format (specific to my processors) and copies the code from boot media (sd card) there. which is all the code in one region (let's say sram1). this makes sense, few hundred KBs of bootloader code gets copied in one of the regions which matches the linker file. What confuses me is after this : let's say my bootloader wants to load another app, and as I mentioned, that app uses all 4 different memory regions. is it the duty of bootloader to make sure the code is copied from sd card to the correct memory regions before passing control to it ? I think it should be, but then the bootloader should completely know how you have linked your app with the linker script, and I think this doesn't make sense. Can you enlighten me of how it works ?
When we write a particular code in C, that code gets allocated to either data memory or code memory. When are those memory initialised, at run time or compile time. Any possible explanation of why they are initialised that way?
Most of .code comes from the binary image (as you might expect), the loader does make some changes before the program runs (writing the actual addresses of imported functions to an import table for example). On a system with hardware level memory control the pages that .code is loaded to will be marked read-only (and executable if the hardware gives that level of control).
.data also comes from the program binary, but those pages are marked read-write and non-executable (except for .rodata which is read-only)
.bss does not come from the binary, it simply gets allocated and initialized to 0 (this is the stack lives.
After finishing the load the real program entry point runs (not main), this sets up the environment, runs initializers calls main runs destructors and any final tear-down code the platform needs.
All globally initialised values are stored in .data segment .i.e. initialised data segment and uninitialised values are stored in bss and compiler initialise those uninitiated values to zero automatically in bss. Then why data segment is separated as .data and bss.
whether it has an advantage or not ? or any benefit
The C programming language (it is a specification written in English) does not know about .bss or .data or data segments. Check that by reading n1570 (the latest draft of C11).
In some cases (e.g. embedded computing) you don't have any data segment. For example when you cross-compile for an Arduino, the resulting code gets uploaded to the flash memory of that microcontroller (and data is in RAM, which your program would perhaps explicitly clear).
(most of my answer below is focused on Linux, but you could adapt it to other OSes)
Regarding the data segment on Unix-like systems like Linux, read more the specification of ELF. It is convenient to avoid spending file space in the executable file for something (the .bss) which is all zeros. The kernel code for execve(2) is able to set up an initial virtual address space with some cleared data (which is not mapped to some file segment). See also mmap(2) & elf(5) & ld-linux(8). Try some cat /proc/$$/maps command to understand the virtual address space of your shell. See proc(5).
So the executable contains segments (some of which are all zeros and don't take any disk space), and is constructed by the linker -which also handle relocations- from several object files provided by the compiler from source code files. On Linux, use objdump(1) and readelf(1) (and nm(1)...) to explore and inspect ELF executables and ELF object files.
BTW a cleared data segment don't need to be fetched from disk by the virtual memory subsystem, and that might make things slightly faster. The kernel would just clear a page in RAM when it is paged in.
So the .bss exist to avoid wasting disk space in executables (and to speedup initializing of zeroed data). Obviously, some mutable data is explicitly initialized to non-zero content (and that needs to sit in .data and takes some disk space in the executable). Constant immutable read-only data goes into .rodata (into the text segment, generally shared by several processes running the same program)
You could configure your linker (e.g. with some linker script) to put all the data (even the cleared ones) in some explicit data segment (but doing so is just wasting disk space)...... Historically, Unix have been developed on machines with little but costly disk space (so wasting it was unthinkable at that time, hence the need of .bss; today you care less!)
Read Levine's book Linkers and Loaders for more, and Advanced Linux Programming and Operating Systems : Three Easy Pieces.
I just learned about different memory segments like: Text, Data, Stack and Heap. My question is:
1- Where the boundaries between these sections are defined? Is it in Compiler or OS?
2- How the compiler or OS know which addresses belong to each section? Should we define it anywhere?
This answer is from the point of view of a more special-purpose embedded system rather than a more general-purpose computing platform running an OS such as Linux.
Where the boundaries between these sections are defined? Is it in Compiler or OS?
Neither the compiler nor the OS do this. It's the linker that determines where the memory sections are located. The compiler generates object files from the source code. The linker uses the linker script file to locate the object files in memory. The linker script (or linker directive) file is a file that is a part of the project and identifies the type, size and address of the various memory types such as ROM and RAM. The linker program uses the information from the linker script file to know where each memory starts. Then the linker locates each type of memory from an object file into an appropriate memory section. For example, code goes in the .text section which is usually located in ROM. Variables go in the .data or .bss section which are located in RAM. The stack and heap also go in RAM. As the linker fills one section it learns the size of that section and can then know where to start the next section. For example, the .bss section may start where the .data section ended.
The size of the stack and heap may be specified in the linker script file or as project options in the IDE.
IDEs for embedded systems typically provide a generic linker script file automatically when you create a project. The generic linker file is suitable for many projects so you may never have to customize it. But as you customize your target hardware and application further you may find that you also need to customize the linker script file. For example, if you add an external ROM or RAM to the board then you'll need to add information about that memory to the linker script so that the linker knows how to locate stuff there.
The linker can generate a map file which describes how each section was located in memory. The map file may not be generated by default and you may need to turn on a build option if you want to review it.
How the compiler or OS know which addresses belong to each section?
Well I don't believe the compiler or OS actually know this information, at least not in the sense that you could query them for the information. The compiler has finished its job before the memory sections are located by the linker so the compiler doesn't know the information. The OS, well how do I explain this? An embedded application may not even use an OS. The OS is just some code that provides services for an application. The OS doesn't know and doesn't care where the boundaries of memory sections are. All that information is already baked into the executable code by the time the OS is running.
Should we define it anywhere?
Look at the linker script (or linker directive) file and read the linker manual. The linker script is input to the linker and provides the rough outlines of memory. The linker locates everything in memory and determines the extent of each section.
For your Query :-
Where the boundaries between these sections are defined? Is it in Compiler or OS?
Answer is OS.
There is no universally common addressing scheme for the layout of the .text segment (executable code), .data segment (variables) and other program segments. However, the layout of the program itself is well-formed according to the system (OS) that will execute the program.
How the compiler or OS know which addresses belong to each section? Should we define it anywhere?
I divided your this question into 3 questions :-
About the text (code) and data sections and their limitation?
Text and Data are prepared by the compiler. The requirement for the compiler is to make sure that they are accessible and pack them in the lower portion of address space. The accessible address space will be limited by the hardware, e.g. if the instruction pointer register is 32-bit, then text address space would be 4 GiB.
About Heap Section and limit? Is it the total available RAM memory?
After text and data, the area above that is the heap. With virtual memory, the heap can practically grow up close to the max address space.
Do the stack and the heap have a static size limit?
The final segment in the process address space is the stack. The stack takes the end segment of the address space and it starts from the end and grows down.
Because the heap grows up and the stack grows down, they basically limit each other. Also, because both type of segments are writeable, it wasn't always a violation for one of them to cross the boundary, so you could have buffer or stack overflow. Now there are mechanism to stop them from happening.
There is a set limit for heap (stack) for each process to start with. This limit can be changed at runtime (using brk()/sbrk()). Basically what happens is when the process needs more heap space and it has run out of allocated space, the standard library will issue the call to the OS. The OS will allocate a page, which usually will be manage by user library for the program to use. I.e. if the program wants 1 KiB, the OS will give additional 4 KiB and the library will give 1 KiB to the program and have 3 KiB left for use when the program ask for more next time.
Most of the time the layout will be Text, Data, Heap (grows up), unallocated space and finally Stack (grows down). They all share the same address space.
The sections are defined by a format which is loosely tied to the OS. For example on Linux you have ELF and on Mac OS you have Mach-O.
You do not define the sections explicitly as a programmer, in 99.9% of cases. The compiler knows what to put where.
I'm searching for a way to see the RAM usage of my application running on an at32uc3b0512.
arv32-size.exe foo.elf tells me:
text data bss dec hex filename
263498 11780 86524 361802 5854a foo.elf
According to 'google', RAM usage is .data + .bss. But .data + .bss is already (11780+86524)/1024 = 96kb, which would mean that my RAM is full (at32uc3b0512 -> 96kb SRAM). But the application works as desired. Am I wrong???
The chip you are using has 96kB of RAM and that is also the sum of your .bss and .data sections. This does not mean that all of your RAM is being used up, rather it is merely showing how the RAM is being allocated.
The program on MCU is usually located in FLASH
this is not true if you have some OS present
and load program to memory on runtime from somewhere like SD card
not all MCU's can do that
I suspect that is not your case
The program Flash is 512 KByte big (I guess from your IC's number)
The SDRAM is used for C engine/OS,stack and heap
your chip has 96 KByte
the C engine is something like OS handling
dynamic allocations,heap,stack,subroutine calls
and including RTL used during compilation
and of coarse dummy Interrupt sub routines for unused interrupts...
When you compile program to ELF/HEX what ever
the compiler/linker tells you only
how big the program code and data is (located in program FLASH memory)
how big static variables you have
the rest is unknown until runtime itself
So if you need to know how big chunk of memory you take
then you need to extract it from runtime
by some RTL call to get memory status
or by estimating it yourself based on knowledge of
what your program does
how much of dynamic memory is used
heap/stack trashing/usage
recursions level, etc...
Or you can try to increasingly allocate memory until you hit out of memory
and count how big chunk you allocated altogether
then release it of coarse
the used memory is then ~ 96KB - altogether_allocated_memory
(+/-) granularity ...