During startup tests I am required to test all RAM locations using a galpat test, I have wrote the function to do this but run into a problem that the functions variables exist in RAM and therefore get mashed as part of the test.
What would be the best way around this?
A possible approach could be - especially taking care of the processor stack, .data and .bss is something you can avoid - but there is no easy way to have C work without a proper stack:
Write your code that it exclusively uses ROM code and stack variables.
Have your startup code (that would normally be written in assembler anyhow) allocate the stack in the upper half of memory, test the lower half
move (copy) the stack from upper memory to already tested areas (can be done in C)
Reset the stack pointer to point to the copied stack (involves assembler coding)
Do the rest of the memtest (can be done in C again)
(This assumes your code runs from ROM, which it would normally do at such early point in the start-up). In case there is any memory failure in areas where you allocate the stack, your code will simply crash (What it does before that - Reactor meltdown... - depends on the application).
When moving the stack around outside C's control, you should be very careful what you actually store there - Pointers to stack variables will become invalid - or rather undefined - once you've moved the stack. Simple scalar variables and pointers to outside the stack as typically used in a RAM test should work, however.
You could try and declare your variables as register to try and keep RAM usage as low as possible - But you can't force C to put certain variables into registers, and a good C compiler will put them there anyhow.
Whether this is any better than writing the whole memtest in assembler (you'd need to do the stack adjustments in assembler anyhow, as there is no means to move the processor stack around in C) I dare to challenge. I don't see much of a point here using C on this low level, especially as assembler could run a memtest routine completely from registers, without using any RAM. This makes it much more immune to any RAM problem. A RAM testing routine shouldn't rely on working memory.
The bottom line is you cannot, you have two choices, you can either only test part of the ram or part of the ram at a time which means you are not doing a complete address test. Or you dont run from the ram you are testing, which is basically the rule if you really want to test the ram. So you have to run from rom with out using a stack in the ram under test or you use another ram, perhaps there is a cache somewhere that can be used direct access to give you a little ram.
Testing half or some other fraction at a time, which is not a complete test, but better than not testing part of it at all, can be done with either a position independent module or with multiple compiles of the test that are position dependent. No reason for the stack to be an issue, the rom based code copying and jumping to the code under test can set the stack pointer based on the fraction under test or not under test, and then repeat. Treat the module like a function not like an entire program and the preserving the stack or "moving the test" problem goes away it returns back to the rom based code which can relaunch further tests.
One crazy way to do it would be to attempt to turn on the I cache get the test code into the cache (before it hits itself), and then blast away at the ram including the code behind the ram. (cant have stack) I would only try this as a fun experiment but not for anything real. Lots of problems to solve with an approach like this.
My approach would be this:
Do the test before anything is initialized (variables, stack). In gcc you can: void RAM_test(void) _attribute_ ((section (".init0")));
Write it in assembly. Ensure that the test does not use/store any variables in the RAM, only uses processor registers.
Store the result somewhere so you can use it later in normal program.
If you do have ROM space and can afford the time at boot time, I would implement the test in assembly or in a naked C function using directives to put your variables in registers so that no RAM is consumed as part of the test. This is going to be pretty architecture and compiler specific, though, and neither of those were mentioned.
Related
I need to check the RAM of a MCU at startup using a checkerboard-like algorithm. I do not want to lose whatever data is already in RAM, and also i do not know how not to affect the variables im using to perform this algorithm.
I was thinking something like:
for (position=0; position< 4096; position++)
{
*Temporal = 0x5555;
if(*Temporal != 0x5555) Error = TRUE;
*Temporal = 0xAAAA;
if(*Temporal != 0xAAAA) Error= TRUE;
Temporal +=1;
}
should i modify the linker to know where Temporal and Error are being placed?
Use the modifier "register" when declaring your pointers (as in "register int *"). Registers are a special segment of memory inside the processor core that (usually) does not count as part of RAM, so any changes to them don't count as RAM read/writes. Some exceptions exist; for instance, in AVR microcontroles, the first 32 bytes of RAM are "faked" into registers.
Your question probably got a downvote for the lack of clarity, and the question about retaining whatever was in RAM being easily solved by most beginner C programmers (just copy the contents of the pointer into a temp variable before testing it, and copying back after the end of the test). Also, you're not performing a Checksum: you're just making a Memory Test. These are very different, separate things.
You need to ensure the memory that you are testing is in a different location than the memory containing the program you are running. You may be able to do this by running directly out of flash, or whatever other permanent storage you have connected. If not, you need to do something with the link map to ensure proper memory segmentation.
Inside the test function, ussing register as MVittiS suggests is a good idea. Another alternate is to use a global variable mapped to a different segment than the one under test.
You may want to read this article about memory testing to understand the limitations of the test you propose, how memory can fail, and what you should be testing for.
I don't know (not enough details) but your memory test may be a cache test and might not test any RAM at all.
Your memory test (if it does test memory) is also very badly designed. For example, you could chop (or short out) all of the address lines and all data lines except for the least significant 2 data lines, and even though there'd be many extremely serious faults the test will still pass.
My advice would be to read something like this web page about memory testing, just to get some ideas and background information: http://www.esacademy.com/en/library/technical-articles-and-documents/miscellaneous/software-based-memory-testing.html
In general, for non-destructive tests you'd copy whatever you want to keep somewhere else, then do the test, then copy the data back. It's very important that "somewhere else" is tested first (you don't want to copy everything into faulty RAM and then copy it back).
I would also recommend using assembly (instead of C) to ensure no unwanted memory accesses are made; including your stack.
If your code is in RAM that needs to be tested then you will probably need 2 copies of your RAM testing code. You'd use the second copy of your code when you're testing the RAM that contains the first copy of your code. If your code is in ROM then it's much easier (but you still need to worry about your stack).
I'm trying to get a ram-resident image to checksum itself, which is proving easier said than done.
The code is first compiled on a cross development platform, generating an .elf output. A utility is used to strip out the binary image, and that image is burned to flash on the target platform, along with the image size. When the target is started, it copies the binary to the correct region of ram, and jumps to it. The utility also computes a checksum of all the words in the elf that are destined for ram, and that too is burned into the flash. So my image theoretically could checksum its own ram resident image using the a-priori start address and the size saved in flash, and compare to the sum saved in flash.
That's the theory anyway. The problem is that once the image begins executing, there is change in the .data section as variables are modified. By the time the sum is done, the image that has been summed is no longer the image for which the utility calculated the sum.
I've eliminated change due to variables defined by my application, by moving the checksum routine ahead of all other initializations in the app (which makes sense b/c why run any of it if an integrity check fails, right?), but the killer is the C run time itself. It appears that there are some items relating to malloc and pointer casting and other things that are altered before main() is even entered.
Is the entire idea of self-checksumming C code lame? If there was a way to force app and CRT .data into different sections, I could avoid the CRT thrash, but one might argue that if the goal is to integrity check the image before executing (most of) it, that initialized CRT data should be part of that. Is there a way to make code checksum itself in RAM like this at all?
FWIW, I seem stuck with a requirement for this. Personally I'd have thought that the way to go is to checksum the binary in the flash, before transfer to ram, and trust the loader and the ram. Paranoia has to end somewhere right?
Misc details: tool chain is GNU, image contains .text, .rodata and .data as one contiguously loaded chunk. There is no OS, this is bare metal embedded. Primary loader essentially memcpy's my binary into ram, at a predetermined address. No relocations occur. VM is not used. Checksum only needs testing once at init only.
updated
Found that by doing this..
__attribute__((constructor)) void sumItUp(void) {
// sum it up
// leave result where it can be found
}
.. that I can get the sum done before almost everything except the initialization of the malloc/sbrk vars by the CRT init, and some vars owned by "impure.o" and "locale.o". Now, the malloc/sbrk value is something I know from the project linker script. If impure.o and locale.o could be mitigated, might be in business.
update
Since I can control the entry point (by what's stated in flash for the primary loader), it seems the best angle of attack now is to use a piece of custom assembler code to set up stack and sdata pointers, call the checksum routine, and then branch into the "normal" _start code.
If the checksum is done EARLY enough, you could use ONLY stack variables, and not write to any data-section variables - that is, make EVERYTHING you need to perform the checksumming [and all preceding steps to get to that point] ONLY use local variables for storing things in [you can read global data of course].
I'm fairly convinced that the right way is to trust the flash & loader to load what is in the flash. If you want to checksum the code, sure, go and do that [assuming it's not being modified by the loader of course - for example runtime loading of shared libraries or relocation of the executable itself, such as random virtual address spaces and such]. But the data loaded from flash can't be relied upon once execution starts properly.
If there is a requirement from someone else that you should do this, then please explain to them that this is not feasible to implement, and that "the requirement, as it stands" is "broken".
I'd suggest approaching this like an executable packer, like upx.
There are several things in the other answers and in your question that, for lack of a better term, make me nervous.
I would not trust the loader or anything in flash that wasn't forced on me.
There is source code floating around on the net that was used to secure one of, I think, HTCs recent phones. Look around on forum.xda-developers.com and see if you can find it and use it for an example.
I would push back on this requirement. Cellphone manufacturers spend a lot of time on keeping their images locked down and, eventually, all of them are beaten. This seems like a vicious circle.
Can you use the linker script to place impure.o and locale.o before or after everything else, allowing you to checksum everything but those and the malloc/sbrk stuff? I'm guessing malloc and sbrk are called in the bootloader that loads your application, so the thrash caused by those cannot be eliminated?
It's not an answer to just tell you to fight this requirement, but I agree that this seems to be over-thought. I'm sure you can't go into any amount of detail, but I'm assuming the spec authors are concerned about malicious users/hackers, rather than regular memory corruption due to cosmic rays, etc. In this case, if a malicious user/hacker can change what's loaded into RAM, they can just change your checksumming routine (which is itself running from RAM, correct?) to always return a happy status, no matter how well the checksum routine they aren't running anymore is designed.
Even if they are concerned about regular memory corruption, this checksum routine would only catch that if the error occurred during the original copy to memory, which is actually the least likely time such an error would occur, simply because the system hasn't been running long enough to have a high probability of a corruption event.
In general, what you want to do is impossible, since on many (most?) platforms the program loader may "relocate" some program address constants.
Can you update the loader to perform the checksum test on the flash resident binary image, before it is copied to ram?
Is there a tool to where I have spills in my c code?
I mean see what block of code potentially make a register move to memory.
EDIT: what is a spill:
In the process of compiling your code at some point you will have to do register allocation. The compiler will do an interference graph ( "variables" are nodes and they are connected if they are alive at the same time ). From this point there is a linear process that will do graph coloring: for each variable assign a register that wont interfere with other variables... If you don't have enough register to color the graph the algorithm will fail
and a variable(register) will be spilled ( moved to memory ).
From a software engineering point of view, this mean you should always minimize a variable live so you can minimize the chance of having a spill.
When you want to optimize code you should look for those kinds of things since a spill will give an extra time to read/write memory. I was looking for a tool or a compiler flag that could tell me where is spill so I can optimize.
I'm aware of no such tool.
Because decisions about spills vary from compiler to compiler, and version of the compiler and even by settings within a given version of a given compiler, any such tool would have to be tightly coupled to a compiler and would likely only support one.
On the other hand, you can always look at the generated assembly yourself and see if a given variable is spilled or not.
Generally either disassemble or compile to assembler instead of an object.
For specific compilers like gcc and llvm (where you have the source and can easily re-build the compiler), modify the compiler to print some sort of output to indicate how many times it had to spill, as you call it, to memory. Perhaps as you find the register allocation routine, you may find that the compiler already has such output. Personally I just disassemble or compile to assembler.
A generic assembler analysis tool is possible, but is it worth the effort? You would want to know where function/optimization boundaries are. You would want to distinguish volatile variables, or hardware registers where the write to ram was intentional. You could just look for stack based writes only. Or look for cases where there is a write to the stack that is not a push, where the register is destroyed on the next instruction. Actually it would be pretty easy to search for writes to a stack pointer relative address, with the next instruction destroying the register, with that stack based relative address being read back in a relatively nearby execution path where the stack frame has not been cleaned up in that execution path. Do I know of such a tool? Nope.
for a long time, I am thinking and studying output of C language compiler in assembler form, as well as CPU architecture. I know this may be silly to you, but it seems to me that something is very ineffective. Please, don´t be angry if I am wrong, and there is some reason I do not see for all these principles. I will be very glad if you tell me why is it designed this way. I actually truly believe I am wrong, I know the genius minds of people which get PCs together knew a reason to do so. What exactly, do you ask? I´ll tell you right away, I use C as a example:
1: Stack local scope memory allocation:
So, typical local memory allocation uses stack. Just copy esp to ebp and than allocate all the memory via ebp. OK, I would understand this if you explicitly need allocate RAM by default stack values, but if I do understand it correctly, modern OS use paging as a translation layer between application and physical RAM, when address you desire is further translated before reaching actual RAM byte. So why don´t just say 0x00000000 is int a,0x00000004 is int b and so? And access them just by mov 0x00000000,#10? Because you wont actually access memory blocks 0x00000000 and 0x00000004 but those your OS set the paging tables to. Actually, since memory allocation by ebp and esp use indirect addressing, "my" way would be even faster.
2: Variable allocation duplicity:
When you run application, Loader load its code into RAM. When you create variable, or string, compiler generates code that pushes these values on the top o stack when created in main. So there is actual instruction for do so, and that actual number in memory. So, there are 2 entries of the same value in RAM. One in form of instruction, second in form of actual bytes in the RAM. But why? Why not to just when declaring variable count at which memory block it would be, than when used, just insert this memory location?
How would you implement recursive functions? What you are describing is equivalent to using global variables everywhere.
That's just one problem. How can you link to a precompiled object file and be sure it won't corrupt the memory of your procedures?
Because C (and most other languages) support recursion, so a function can call itself, and each call of the function needs separate copies of any local variables. Also, on most current processors, your way would actually be slower -- indirect addressing is so common that processors are optimized for it.
You seem to want the behavior of C (or at least that C allows) for string literals. There are good and bad points to this, such as the fact that even though you've defined a "variable", you can't actually modify its contents (without affecting other variables that are pointing at the same location).
The answers to your questions are mostly wrapped up in the different semantics of different storage classes
Google "data segment"
Think about the difference in behavior between global and local variables.
Think about how constant and non-constant variables have different requirements when functions are called repeatedly (or as Mehrdad says, recursively)
Think about the difference between static and non static automatic variables again in the context of multiple or recursive calls.
Since you are comparing assembler and c (which are very close together from an architectural standpoint), I'm inclined to say that you're describing micro-optimization, which is meaningless unless you profile the code to see if it performs better.
In general, programming languages are evolving towards a more declarative style (i.e. telling the computer what you want done, rather than how you want it done). When you program in an imperative language (like assembly or c), you specify in extreme detail how you want the problem solved. This gives the compiler little room to make optimization decisions on your behalf.
However, as the languages become more declarative, the compilers are getting smarter, because we are giving them the room they need to make more intelligent performance optimizations.
If every function would put its first variable at offset 0 and so on then you would have to change the memory mapping each time you enter a function (you could not allocate all variables to unique addresses if you want recursion). This is doable, but with current hardware it's very slow. Furthermore, the address translation performed by the virtual memory is not free either, it's actually quite complicated to implement this efficiently.
Addressing off ebp (or any other register) costs having a mux (to select the register) and an adder (to add the offset to the register). The time taken for this can often be overlapped with other operations.
If you want to be able to modify the static value you have to copy it to the stack. If you don't (saying it's 'const') then a good C compiler will no copy it to the stack.
I know this is more "heavy" question, but I think its interesting too. It was part of my previous questions about compiler functions, but back than I explained it very badly, and many answered just my first question, so ther it is:
So, if my knowledge is correct, modern Windows systems use paging as a way to switch tasks and secure that each task has propriate place in memory. So, every process gets its own place starting from 0.
When multitasking goes into effect, Kernel has to save all important registers to the task´s stack i believe than save the current stack pointer, change page entry to switch to another proces´s physical adress space, load new process stack pointer, pop saved registers and continue by call to poped instruction pointer adress.
Becouse of this nice feature (paging) every process thinks it has nice flat memory within reach. So, there is no far jumps, far pointers, memory segment or data segment. All is nice and linear.
But, when there is no more segmentation for the process, why does still compilers create variables on the stack, or when global directly in other memory space, than directly in program code?
Let me give an example, I have a C code:int a=10;
which gets translated into (Intel syntax):mov [position of a],#10
But than, you actually ocupy more bytes in RAM than needed. Becouse, first few bytes takes the actuall instruction, and after that instruction is done, there is new byte containing the value 10.
Why, instead of this, when there is no need to switch any segment (thus slowing the process speed) isn´t just a value of 10 coded directly into program like this:
xor eax,eax //just some instruction
10 //the value iserted to the program
call end //just some instruction
Becouse compiler know the exact position of every instruction, when operating with that variable, it would just use it´s adress.
I know, that const variables do this, but they are not really variables, when you cannot change them.
I hope I eplained my question well, but I am still learning English, so forgive my sytactical and even semantical errors.
EDIT:
I have read your answers, and it seems that based on those I can modify my question:
So, someone told here that global variable is actually that piece of values attached directly into program, I mean, when variable is global, is it atached to the end of program, or just created like the local one at the time of execution, but instead of on stack on heap directly?
If the first case - attached to the program itself, why is there even existence of local variables? I know, you will tell me becouse of recursion, but that is not the case. When you call function, you can push any memory space on stack, so there is no program there.
I hope you do understand me, there always is ineficient use of memory, when some value (even 0) is created on stack from some instruction, becouse you need space in program for that instruction and than for the actual var. Like so: push #5 //instruction that says to create local variable with integer 5
And than this instruction just makes number 5 to be on stack. Please help me, I really want to know why its this way. Thanks.
Consider:
local variables may have more than one simultaneous existence if a routine is called recursively (even indirectly in, say, a recursive decent parser) or from more than one thread, and these cases occur in the same memory context
marking the program memory non-writable and the stack+heap as non-executable is a small but useful defense against certain classes of attacks (stack smashing...) and is used by some OSs (I don't know if windows does this, however)
Your proposal doesn't allow for either of these cases.
So, there is no far jumps, far pointers, memory segment or data segment. All is nice and linear.
Yes and no. Different program segments have different purposes - despite the fact that they reside within flat virtual memory. E.g. data segment is readable and writable, but you can't execute data. Code segment is readable and executable, but you can't write into it.
why does still compilers create variables on the stack, [...] than directly in program code?
Simple.
Code segment isn't writable. For safety reasons first. Second,
most CPUs do not like to have code segment being written into as it
breaks many existing optimization used to accelerate execution.
State of the function has to be private to the function due to
things like recursion and multi-threading.
isn´t just a value of 10 coded directly into program like this
Modern CPUs prefetch instructions to allow things like parallel execution and out-of-order execution. Putting the garbage (to CPU that is the garbage) into the code segment would simply diminish (or flat out cancel) the effect of the techniques. And they are responsible for the lion share of the performance gains CPUs had showed in the past decade.
when there is no need to switch any segment
So if there is no overhead of switching segment, why then put that into the code segment? There are no problems to keep it in data segment.
Especially in case of read-only data segment, it makes sense to put all read-only data of the program into one place - since it can be shared by all instances of the running application, saving physical RAM.
Becouse compiler know the exact position of every instruction, when operating with that variable, it would just use it´s adress.
No, not really. Most of the code is relocatable or position independent. The code is patched with real memory addresses when OS loads it into the memory. Actually special techniques are used to actually avoid patching the code so that the code segment too could be shared by all running application instances.
The ABI is responsible for defining how and what compiler and linker supposed to do for program to be executable by the complying OS. I haven't seen the Windows ABI, but the ABIs used by Linux are easy to find: search for "AMD64 ABI". Even reading the Linux ABI might answer some of your questions.
What you are talking about is optimization, and that is the compiler's business. If nothing ever changes that value, and the compiler can figure that out, then the compiler is perfectly free to do just what you say (unless a is declared volatile).
Now if you are saying that you are seeing that the compiler isn't doing that, and you think it should, you'd have to talk to your compiler writer. If you are using VisualStudio, their address is One Microsoft Way, Redmond WA. Good luck knocking on doors there. :-)
Why isn´t just a value of 10 coded directly into program like this:
xor eax,eax //just some instruction
10 //the value iserted to the program
call end //just some instruction
That is how global variables are stored. However, instead of being stuck in the middle of executable code (which is messy, and not even possible nowadays), they are stored just after the program code in memory (in Windows and Linux, at least), in what's called the .data section.
When it can, the compiler will move variables to the .data section to optimize performance. However, there are several reasons it might not:
Some variables cannot be made global, including instance variables for a class, parameters passed into a function (obviously), and variables used in recursive functions.
The variable still exists in memory somewhere, and still must have code to access it. Thus, memory usage will not change. In fact, on the x86 ("Intel"), according to this page the instruction to reference a local variable:
mov eax, [esp+8]
and the instruction to reference a global variable:
mov eax, [0xb3a7135]
both take 1 (one!) clock cycle.
The only advantage, then, is that if every local variable is global, you wouldn't have to make room on the stack for local variables.
Adding a variable to the .data segment may actually increase the size of the executable, since the variable is actually contained in the file itself.
As caf mentions in the comments, stack-based variables only exist while the function is running - global variables take up memory during the entire execution of the program.
not quite sure what your confusion is?
int a = 10; means make a spot in memory, and put the value 10 at the memory address
if you want a to be 10
#define a 10
though more typically
#define TEN 10
Variables have storage space and can be modified. It makes no sense to stick them in the code segment, where they cannot be modified.
If you have code with int a=10 or even const int a=10, the compiler cannot convert code which references 'a' to use the constant 10 directly, because it has no way of knowing whether 'a' may be changed behind its back (even const variables can be changed). For example, one way 'a' can be changed without the compiler knowing is, if you have a pointer which points 'a'. Pointers are not fixed at runtime, so the compiler cannot determine at compile time whether there will be a pointer which will point to and modify 'a'.