I have this issue which is probably something simple but I cannot get around it.
I deflate with a python script (import zlib) and I do the inflate on an embedded platform using zlib version 1.2.11, January 15th, 2017.
For testing purposes, I've created a constant table in ROM, which is the deflated data created by the script.
The z_stream structure is called test_stream.
If I point the test_stream.next_in to the constant table/buffer in ROM, the inflate works fine, the data is recovered OK, no problem.
But if I copy from ROM into a RAM buffer and pass that RAM buffer to test_stream.next_in, the inflateInit(&test_stream) returns Z_DATA_ERROR (-3) and the test_stream.msg contains: " -- missing end-of-block" or "invalid code lengths set".
Obviously I've checked that the RAM buffer is copied properly from ROM.
Am I missing something here, does the RAM buffer needs some alignment or something else?
I know it seems weird but any idea would be appreciated.
No, there are no alignment requirements. You must be giving it the wrong address, or not properly copying the compressed data.
Related
I need to create a ROM mask that provides some functions. However, it should be possible to overwrite the functions for providing firmware patches. Therefore, a patch table should be located in a Flash memory that may be overwritten by later firmware upgrades, whereas the main part of the firmware is located in mask ROM and cannot be modified later.
Does anyone have any idea how this is done? what is the best practice to create patch tables?
Patch table - Basically your ROM has one more built in jumps to a RAM (Flash) adddresses built into it. The flash always has some sort of jump back to the loction just after your ROM jump call to return to your original code. You now have the ability to change your program's behavior from RAM. This assumes you are allowed to run code from RAM of course. If not, then only data tables can be changed on the fly.
Now, just having a single jump on startup allows you to modify starting state code like version numbers or other global/constant data, but that may not be enough. You can add another jump to flash that is run every "so often" to your code as well so that it can update program state after it is running - something like once a vblank or once a application loop.
The above should give you enough lee-way to change data that has changed over time since release or perhaps to fix a slight logic error, but it won't allow you to whole-sale change functions. To do that, you need more code. And that more code depends on just how much RAM you can use.
For instance, if you had enough RAM, and running from Flash RAM didn't hurt perforamnce too much (and is allowed), you could make things very flexible by copying all your key ROM functions into RAM on startup, then calling your RAM patch jump described above, which would allow new code stored elsewhere in your RAM patch to overwrite any previously copied code. If you took this approach, you would also want to make sure that you left some extra space around your original functions to allow new padded functions a bit of room to grow. The original code to be copied to RAM could be stored compressed as well to save ROM space. This allows you to change anything after the fact. It also introduces a very easy way to exploit your code if something else is allowed to write to RAM so beware.
Hope this helps.
I need to check the RAM of a MCU at startup using a checkerboard-like algorithm. I do not want to lose whatever data is already in RAM, and also i do not know how not to affect the variables im using to perform this algorithm.
I was thinking something like:
for (position=0; position< 4096; position++)
{
*Temporal = 0x5555;
if(*Temporal != 0x5555) Error = TRUE;
*Temporal = 0xAAAA;
if(*Temporal != 0xAAAA) Error= TRUE;
Temporal +=1;
}
should i modify the linker to know where Temporal and Error are being placed?
Use the modifier "register" when declaring your pointers (as in "register int *"). Registers are a special segment of memory inside the processor core that (usually) does not count as part of RAM, so any changes to them don't count as RAM read/writes. Some exceptions exist; for instance, in AVR microcontroles, the first 32 bytes of RAM are "faked" into registers.
Your question probably got a downvote for the lack of clarity, and the question about retaining whatever was in RAM being easily solved by most beginner C programmers (just copy the contents of the pointer into a temp variable before testing it, and copying back after the end of the test). Also, you're not performing a Checksum: you're just making a Memory Test. These are very different, separate things.
You need to ensure the memory that you are testing is in a different location than the memory containing the program you are running. You may be able to do this by running directly out of flash, or whatever other permanent storage you have connected. If not, you need to do something with the link map to ensure proper memory segmentation.
Inside the test function, ussing register as MVittiS suggests is a good idea. Another alternate is to use a global variable mapped to a different segment than the one under test.
You may want to read this article about memory testing to understand the limitations of the test you propose, how memory can fail, and what you should be testing for.
I don't know (not enough details) but your memory test may be a cache test and might not test any RAM at all.
Your memory test (if it does test memory) is also very badly designed. For example, you could chop (or short out) all of the address lines and all data lines except for the least significant 2 data lines, and even though there'd be many extremely serious faults the test will still pass.
My advice would be to read something like this web page about memory testing, just to get some ideas and background information: http://www.esacademy.com/en/library/technical-articles-and-documents/miscellaneous/software-based-memory-testing.html
In general, for non-destructive tests you'd copy whatever you want to keep somewhere else, then do the test, then copy the data back. It's very important that "somewhere else" is tested first (you don't want to copy everything into faulty RAM and then copy it back).
I would also recommend using assembly (instead of C) to ensure no unwanted memory accesses are made; including your stack.
If your code is in RAM that needs to be tested then you will probably need 2 copies of your RAM testing code. You'd use the second copy of your code when you're testing the RAM that contains the first copy of your code. If your code is in ROM then it's much easier (but you still need to worry about your stack).
I'm trying to get a ram-resident image to checksum itself, which is proving easier said than done.
The code is first compiled on a cross development platform, generating an .elf output. A utility is used to strip out the binary image, and that image is burned to flash on the target platform, along with the image size. When the target is started, it copies the binary to the correct region of ram, and jumps to it. The utility also computes a checksum of all the words in the elf that are destined for ram, and that too is burned into the flash. So my image theoretically could checksum its own ram resident image using the a-priori start address and the size saved in flash, and compare to the sum saved in flash.
That's the theory anyway. The problem is that once the image begins executing, there is change in the .data section as variables are modified. By the time the sum is done, the image that has been summed is no longer the image for which the utility calculated the sum.
I've eliminated change due to variables defined by my application, by moving the checksum routine ahead of all other initializations in the app (which makes sense b/c why run any of it if an integrity check fails, right?), but the killer is the C run time itself. It appears that there are some items relating to malloc and pointer casting and other things that are altered before main() is even entered.
Is the entire idea of self-checksumming C code lame? If there was a way to force app and CRT .data into different sections, I could avoid the CRT thrash, but one might argue that if the goal is to integrity check the image before executing (most of) it, that initialized CRT data should be part of that. Is there a way to make code checksum itself in RAM like this at all?
FWIW, I seem stuck with a requirement for this. Personally I'd have thought that the way to go is to checksum the binary in the flash, before transfer to ram, and trust the loader and the ram. Paranoia has to end somewhere right?
Misc details: tool chain is GNU, image contains .text, .rodata and .data as one contiguously loaded chunk. There is no OS, this is bare metal embedded. Primary loader essentially memcpy's my binary into ram, at a predetermined address. No relocations occur. VM is not used. Checksum only needs testing once at init only.
updated
Found that by doing this..
__attribute__((constructor)) void sumItUp(void) {
// sum it up
// leave result where it can be found
}
.. that I can get the sum done before almost everything except the initialization of the malloc/sbrk vars by the CRT init, and some vars owned by "impure.o" and "locale.o". Now, the malloc/sbrk value is something I know from the project linker script. If impure.o and locale.o could be mitigated, might be in business.
update
Since I can control the entry point (by what's stated in flash for the primary loader), it seems the best angle of attack now is to use a piece of custom assembler code to set up stack and sdata pointers, call the checksum routine, and then branch into the "normal" _start code.
If the checksum is done EARLY enough, you could use ONLY stack variables, and not write to any data-section variables - that is, make EVERYTHING you need to perform the checksumming [and all preceding steps to get to that point] ONLY use local variables for storing things in [you can read global data of course].
I'm fairly convinced that the right way is to trust the flash & loader to load what is in the flash. If you want to checksum the code, sure, go and do that [assuming it's not being modified by the loader of course - for example runtime loading of shared libraries or relocation of the executable itself, such as random virtual address spaces and such]. But the data loaded from flash can't be relied upon once execution starts properly.
If there is a requirement from someone else that you should do this, then please explain to them that this is not feasible to implement, and that "the requirement, as it stands" is "broken".
I'd suggest approaching this like an executable packer, like upx.
There are several things in the other answers and in your question that, for lack of a better term, make me nervous.
I would not trust the loader or anything in flash that wasn't forced on me.
There is source code floating around on the net that was used to secure one of, I think, HTCs recent phones. Look around on forum.xda-developers.com and see if you can find it and use it for an example.
I would push back on this requirement. Cellphone manufacturers spend a lot of time on keeping their images locked down and, eventually, all of them are beaten. This seems like a vicious circle.
Can you use the linker script to place impure.o and locale.o before or after everything else, allowing you to checksum everything but those and the malloc/sbrk stuff? I'm guessing malloc and sbrk are called in the bootloader that loads your application, so the thrash caused by those cannot be eliminated?
It's not an answer to just tell you to fight this requirement, but I agree that this seems to be over-thought. I'm sure you can't go into any amount of detail, but I'm assuming the spec authors are concerned about malicious users/hackers, rather than regular memory corruption due to cosmic rays, etc. In this case, if a malicious user/hacker can change what's loaded into RAM, they can just change your checksumming routine (which is itself running from RAM, correct?) to always return a happy status, no matter how well the checksum routine they aren't running anymore is designed.
Even if they are concerned about regular memory corruption, this checksum routine would only catch that if the error occurred during the original copy to memory, which is actually the least likely time such an error would occur, simply because the system hasn't been running long enough to have a high probability of a corruption event.
In general, what you want to do is impossible, since on many (most?) platforms the program loader may "relocate" some program address constants.
Can you update the loader to perform the checksum test on the flash resident binary image, before it is copied to ram?
I have what seems to be a simple problem, and I am sure that I have solved it before, but I am having trouble finding the answer again so hopefulley there is someone out there who can point me in the right direction
I have an area of memory that i would like to "reserve" for a specific variable in my code.
I know I can do this by editing the linker script, and removing my block of data from the rom section, to ensure that program code is not written the specific address - but rather than editing the script I remember doing this in c code a long time ago. I did it using a pragma or something similiar and from memory it looked something like this (the code is not right, but this is vaguely what i remember!)
"#pragma _address #0x00040000
char mydata[1024]; "
Which would reserve 1024 bytes of data at the address 0x00040000.
Does anyone know the correct format for the above code?
For context - I am writing embedded C applications, using flash based microprocessors. The area is flash and I would like to store flash configuration data (generated by my program) there. To use the flash I need to align my data in the correct block, so i need to specify the address. What I would like to do is reserve an area of flash that wont be filled with my program code, so that I can fill it with my configuration data.
Thanks for your help!
Stevo
The #pragma directive has implementation-defined keywords, so your answer depends on your compiler. Recommend consulting your compiler's manual.
I want to dump all global and static variables to a file and load them back on the next program invocation. A solution I thought of is to dump the .data segment to a file. But .data segment on a 32bit machine spans over 2^32 address space (4GB). In which part of this address space the variables reside? How do I know which part of the .data segment I should dump?
And when loading the dumped file, I guess that since the variables are referenced by offset in the data segment, it will be safe to just memcpy the whole dump to the alleged starting point of the "variables area". Please correct me if I am wrong.
EDIT
A good start is this question.
Your problem is how to find the beginning and the end of the data segment. I am not sure how to do this, but I could give you a couple of ideas.
If all your data are relatively self-contained, (they are declared within the same module, not in separate modules,) you might be able to declare them within some kind of structure, so the beginning will be the address of the structure, and the end will be some variable that you will declare right after the structure. If I remember well, MASM had a "RECORD" directive or something like that which you could use to group variables together.
Alternatively, you may be able to declare two additional modules, one with a variable called "beginning" and another with a variable called "end", and make sure that the first gets linked before anything else, and the second gets linked after everything else. This way, these variables might actually end up marking the beginning and the end of the data segment. But I am not sure about this, I am just giving you a pointer.
One thing to remember is that your data will inevitably contain pointers, so saving and loading all your data will only work if the OS under which you are running can guarantee that your program will always be loaded in the same address. If not, forget it. But if you can have this guarantee, then yes, loading the data should work. You should not even need a memcpy, just set the buffer for the read operation to be the beginning of the data segment.
The state of an entire program can be very complicated, and will not only involve variables but values in registers. You'll almost certainly be better off keeping track of what data you want to store and then storing it to a file yourself. This can be relatively painless with the right setup and encapsulation. Then when you resume the application, read in the program state and resume.
Assuming you are using gnu tools (gcc, binutils) if you look at the linker scripts the embedded folks use like the gba developers and microcontroller developers using roms (yagarto or devkit-arm for example). In the linker script they surround the segments of interest with variables that they can use elsewhere in their code. For rom based software for example you specify the data segment with a ram AT rom or rom AT ram in the linker script meaning link as if the data segment is in ram at this address space, but also link the data itself into rom at this address space, the boot code then copies the .data segment from the rom to the ram using these variables. I dont see why you couldnt do the same thing to have the compiler/linker tools tell you where stuff is then runtime use those variables to grab the data from memory and save it somewhere to hybernate or shut down and then restore that data from wherever. The variables you use to perform the restore of course should not be part of the .data segment or you trash the variables you are using to restore the segment.
In response to your header question, on Windows, the location and size of the data and bss segments can be obtained from the in-memory PE header. How that is laid out and how to parse it is documented in this specification:
http://msdn.microsoft.com/en-us/windows/hardware/gg463119
I do not believe that there is a guarantee that with every execution you will have the sam sequence of variables, hence the offsets may point to the wrong content.