I am using Codewarrior 8.3 (IDE version 5.9) to program a 56f8367 DSC.
I am using respected third party software, so I imagine that they know what they are doing and don't want to mess with their code too much, but they are playing around with passing void * parameters and it is something I am not totally familiar with.
So I have this function:
static void T_CALLBACK _transmit(
void *pContext,
TX_DATA *pTxDescriptor)
{
CONTEXT *pLinkContext = (CONTEXT *)pContext;
...
}
which is being called through a function pointer. When I stop the processor just before this function call, I can see the address pointed to by pContext is 0x1000, but after it is cast here, the address pointed to by pLinkContext is 0x0800. This, obviously causes problems because we start writing and reading from a different part of memory.
There is something weird going on with the byte addressing/alignment where it is getting "shifted" over 1 bit. I see what is going wrong, I just don't understand why or, more importantly, how to solve the problem.
What should I be looking for?
(Editing to add the call per comment request) - although, I'm not sure how much it will help considering everything is buried in structures and is being called through a function pointer. I can say that "pTprtContext->tmw.pChannel->pLinkContext" is of a different type than CONTEXT, pLinkContext does match up with the beginning of CONTEXT, so I think they are just trying to insert it in there.
static void T_LOCAL _transmitNextFrame(
D_CONTEXT *pTprtContext)
{
...
/* Transmit frame */
pTprtContext->t.pChannel->pLink->pLinkTransmit(
pTprtContext->t.pChannel->pLinkContext, &pTprtContext->linkTxDescriptor);
}
You say "shifted over by 1 byte," but it is actually only one bit, that is, the number is divided by 2.
This is usually the result of using a byte address in one context and a (2-byte) word address in another context. They probably refer to the same address.
Does this help you decipher it?
I use CodeWarrior compiler for an HC12 family's 16-bit microcontroller. With this compiler, I can choose a few memory models which change (among other several things) how many bytes pointers are. More specifically, +small+ memory model uses __near 16-bit pointers, whereas +large+ model makes use of __far 24-bit pointers.
If your code is compiled with a different memory model than your third party software's and the compiler does not warn you, I guess you may get wierd result.
Related
I am working on a C file register program that handles arbitrary generic data so the user needs to supply functions to be used, these functions are saved in function pointer in the register struct and work nicely. But I need to be able to run these functions again when the program is restarted ideally without the user needing the supply them again. I serialize important data about the register structure and write it into a header.
I was wondering how I can save the functions there too, a compiled c function is just raw binary data, right? So there must be a way to store it into a file and load the function pointers from the content in the file, but I am not sure how to this. Can someone point me in the right direction?
I am assuming it's possible to do this is C since it allows you to do pretty much anything but I might be missing something, can I do this without system calls at all? Or if not what would be the simplest way to do this in posix?
The functions are supplied when creating the register or creating new secondary indexes:
registerHandler* createAndOpenRecordFile(
int overwrite, char *filename, int keyPos, fn_keyCompare userCompare, fn_serialize userSerialize, fn_deserialize userDeserialize, int type, ...)
And saved as functions pointers:
typedef void* (*fn_serialize)(void*);
typedef void* (*fn_deserialize)(void*);
typedef int (*fn_keyCompare) (const void *, const void *);
typedef struct {
...
fn_serialize encode;
fn_deserialize decode;
fn_keyCompare compare;
} registerHandler;
While your logic makes some sort of sense, things much, much more complex than that. My answer is going to contain most of the comments already made here, only in answer form...
Let's assume that you have a pointer to a function. If that function has a jump instruction in it, that jump instructions could jump to an absolute address. That means that when you deserialize the function, you have to have a way to force it to be loaded into the same address, so that the absolute jump jumps to the correct address.
Which brings us to the next point. Given that your question is tagged with posix, there is no POSIX-compliant way to load code into a specific address, there's MAP_FIXED, but it's not going to work unless you write your own dynamic linker. Why does that matter? because the function's assembly code might reference the function's start address, for various reasons, most prominent of which is if the function itself gives its own address as an argument to another function.
Which actually brings us to our next point. If the serialized function calls other functions, you'd have to serialize them too. But that's the "easy" part. The hard part is if the function jumps into the middle of another function rather than call the other function, which could happen e.g. as a result of tail-call optimization. That means you have to serialize everything the function jumps into (recursively), but if the function jumps to 0x00000000ff173831, how many bytes will you serialize from that address?
For that matter, how do you know when any function ends in a portable way?
Even worse, are you even guaranteed that the function is contiguous in memory? Sure, all existing, sane hardware OS memory managers and hardware architectures make it contiguous in memory, but is it guaranteed to be so 1 year from now?
Yet another issue is: What if the user passes a different function based on something dynamic? i.e. if the environment variable X is true, we want function x(), otherwise we want y()?
We're not even going to think about discussing portability across hardware architectures, operating systems, or even versions of the same hardware architecture.
But we are going to talk about security. Assuming that you no longer require the user to give you a pointer to their code, which might have had a bug that they fixed in a new version, you'll continue to use the buggy version until the user remembers to "refresh" your data structures with new code.
And when I say "bug" above, you should read "security vulnerability". If the vulnerable function you're serializing launches a shell, or indeed refers to anything outside the processes, it becomes a persistent exploit.
In short, there's no way to do what you want to do in a sane and economic way. What you can do, instead, is to force the user to package these functions for you.
The most obvious way to do it is asking them to pass a filename of a library which you then open with dlopen().
Another way to do it is pass something like a Lua or JavaScript string and embed an engine to execute these strings as code.
Yet another way is to pass paths to executables, and execute these when the data needs to be processed. This is what git does.
But what you should probably do is just require that the user always passes these functions. Keep it simple.
Is it more cache friendly to use a local copy of an argument than using the pointer argument directly? Will the overhead of copying outweigh the performance gained?
I guess it depends on the size of the copied data.
void foo(struct data *p) {
/* Do stuff with p ... */
}
// VS
void bar(struct data *p) {
struct data copy = *p;
/* Do stuff with copy ... */
*p = copy;
}
I'm asserting that "Do stuff" pushes a lot of other local variables on the stack.
Edit: Also, the data is supposed to be altered/initialized by the function.
Edit: Is this something the compiler might optimize?
Take a look at your operating systems ABI.
Let's assume for the moment that you're running a 64 bit Linux system. A pointer is an integer type and the first six integer types are passed by registers during your function call - no cache involvement so far.
If you go for your copy you assign extra RAM-space. In this case your cache is part of your whole program.
I would go for the first one as it passes your pointer information directly and does not assign extra RAM.
But: If you like to measure your program. Go for valgrind (still assuming you're running linux).
Code both variants and do valgrind --tool=cachegrind myfile and compare the results. Though I really doubt that there's a difference on modern CPUs...
there are so many variables involved here it's very difficult to consider them all, but let me try and outline a few important aspects:
whether you use the pointer or use a local copy instead should not make a difference from the cash memory's point of view: modern caches based architectures will have the data in the cache by the time you need it (unless your data structure is larder than a cache line - which is very unlikely)
There is a very good chance that the generated code in both cases is 95% the same: when you read or write a field in the from your data structure, the compiler reads it locally (either in a register or places it on the stack) anyway. The difference would come from reading/writing all data fields or just some of them.
Considering a modern day parallel CPU architecture the overhead might not even be there, actually in some cases, because the compiler is able to group the instructions better, the generated code for the situation where you use a local copy might be faster.
I'm new to programming small/medium memory models CPUs. I am working with an embedded processor that has 256KB of flash code space contained in addresses 0x00000 to 0x3FFFF, and with 20KB of RAM contained in addresses 0xF0000 to 0xFFFFF. There are compiler options to choose between small, medium, or large memory models. I have medium selected. My question is, how does the compiler differentiate between a code/flash address and a RAM address?
Take for example I have a 1 byte variable at RAM address 10, and I have a const variable at the real address 10. I did something like:
value = *((unsigned char *)10);
How would the compiler choose between the real address 10 or the (virtual?) address 10. I suppose if I wanted to specify the value at real address 10 I would use:
value = *((const unsigned char *)10);
?
Also, can you explain the following code which I believe is related to the answer:
uint32_t var32; // 32 bit unsigned integer.
unsigned char *ptr; // 2 byte pointer.
ptr = (unsigned char *)5;
var32 = (uint32_t)ptr;
printf("%lu", var32)
The code prints 983045 (0xf0005 hex). It seems unrealistic, how can a 16 bit variable return a value greater than what 16 bits can store?
Read your compiler's documentation to find out details about each memory model.
It may have various sorts of pointer, e.g. char near * being 2-byte, and char far * being 4-byte. Alternatively (or as well as), it might have instructions for changing code pages which you'd have to manually invoke.
how can a 16 bit variable return a value greater than what 16 bits can store?
It can't. Your code converts the pointer to a 32-bit int. , and 0xF0005 can fit in a 32-bit int. Based on your description, I'd guess that char * is only pointing to the data area, and you would use a different sort of pointer to point to the code area.
I tried to comment on Matt's answer but my comment was too long, and I think it might be an answer, so here's my comment:
I think this is an answer, I'm really looking for more details though. I've read the manual but it doesn't have much information on the topic. You are right, the compiler has near/far keywords you can use to manually specify the address (type?). I guess the C compiler knows if a variable is a near or far pointer, and if it's a near pointer it generates instructions that map the 2 byte near pointer to a real address; and these generated mapping instructions are opaque to the C programmer. That would be my only guess. This is why the pointer returns a value greater than its 16 bit value; the compiler is mapping the address to an absolute address before it stores the value in var32. This is possible because 1) the RAM addresses begin at 0xF0000 and end at 0xFFFFF, so you can always map a near address to its absolute address by or'ing the address with 0xF0000, and 2) there is no overlap between a code (far) pointer and a near pointer or'd with 0xF0000. Can anyone confirm?
My first take would be read the documentation, however as I had seen, it was already done.
So my assumption would be that you somehow got to work for example on a large existing codebase which was developed with a not too widely supported compiler on a not too well known architecture.
In such a case (after all my attempts with acquiring proper documentation failed) my take would be generating assembler outputs for test programs, and analysing those. I did this a while ago, so it is not from thin air (it was a 8051 PL/M compiler running on an MDS-70, which was emulated by a DOS based emulator from the late 80s, for which DOS was emulated by DOSBox - yes, and for the huge codebase we needed to maintain we couldn't get around this mess).
So build simple programs which would do something with some pointers, compile those without optimizations to assembly (or request an assembly dump, whatever the compiler can do for you), and understand the output. Try to cover all pointer types and memory models you know of in your compiler. It will clarify what is happening, and hopefully the existing documentations will also help once you understand their gaps this way. Finally, don't stop at understanding just enough for the immediate problem, try to document the gaps properly, so later you won't need to redo the experiments to figure out things you once almost done.
I have a function and i'm accessing a struct's members a lot of times in it.
What I was wondering about is what is the good practice to go about this?
For example:
struct s
{
int x;
int y;
}
and I have allocated memory for 10 objects of that struct using malloc.
So, whenever I need to use only one of the object in a function, I usually create (or is passed as argument) pointer and point it to the required object (My superior told me to avoid array indexing because it adds a calculation when accessing any member of the struct)
But is this the right way? I understand that dereferencing is not as expensive as creating a copy, but what if I'm dereferencing a number of times (like 20 to 30) in the function.
Would it be better if i created temporary variables for the struct variables (only the ones I need, I certainly don't use all the members) and copy over the value and then set the actual struct's value before returning?
Also, is this unnecessary micro optimization? Please note that this is for embedded devices.
This is for an embedded system. So, I can't make any assumptions about what the compiler will do. I can't make any assumptions about word size, or the number of registers, or the cost of accessing off the stack, because you didn't tell me what the architecture is. I used to do embedded code on 8080s when they were new...
OK, so what to do?
Pick a real section of code and code it up. Code it up each of the different ways you have listed above. Compile it. Find the compiler option that forces it to print out the assembly code that is produced. Compile each piece of code with every different set of optimization options. Grab the reference manual for the processor and count the cycles used by each case.
Now you will have real data on which to base a decision. Real data is much better that the opinions of a million highly experience expert programmers. Sit down with your lead programmer and show him the code and the data. He may well show you better ways to code it. If so, recode it his way, compile it, and count the cycles used by his code. Show him how his way worked out.
At the very worst you will have spent a weekend learning something very important about the way your compiler works. You will have examined N ways to code things times M different sets of optimization options. You will have learned a lot about the instruction set of the machine. You will have learned how good, or bad, the compiler is. You will have had a chance to get to know your lead programmer better. And, you will have real data.
Real data is the kind of data that you must have to answer this question. With out that data nothing anyone tells you is anything but an ego based guess. Data answers the question.
Bob Pendleton
First of all, indexing an array is not very expensive (only like one operation more expensive than a pointer dereference, or sometimes none, depending on the situation).
Secondly, most compilers will perform what is called RVO or return value optimisation when returning structs by value. This is where the caller allocates space for the return value of the function it calls, and secretly passes the address of that memory to the function for it to use, and the effect is that no copies are made. It does this automatically, so
struct mystruct blah = func();
Only constructs one object, passes it to func for it to use transparently to the programmer, and no copying need be done.
What I do not know is if you assign an array index the return value of the function, like this:
someArray[0] = func();
will the compiler pass the address of someArray[0] and do RVO that way, or will it just not do that optimisation? You'll have to get a more experienced programmer to answer that. I would guess that the compiler is smart enough to do it though, but it's just a guess.
And yes, I would call it micro optimisation. But we're C programmers. And that's how we roll.
Generally, the case in which you want to make a copy of a passed struct in C is if you want to manipulate the data in place. That is to say, have your changes not be reflected in the struct it self but rather only in the return value. As for which is more expensive, it depends on a lot of things. Many of which change implementation to implementation so I would need more specific information to be more helpful. Though, I would expect, that in an embedded environment you memory is at a greater premium than your processing power. Really this reads like needless micro optimization, your compiler should handle it.
In this case creating temp variable on the stack will be faster. But if your structure is much bigger then you might be better with dereferencing.
i have the following code:
void print(const char* str){
system_call(4,1,str,strlen(str)); }
void foo2(void){ print("goo \n");}
void buz(void){ ...}
int main(){
char buf[256];
void (*func_ptr)(void)=(void(*)(void))buf;
memcpy(buf,foo2, ((void*)buz)-((void*)foo2));
func_ptr();
return 0;
}
the question is, why will this code fall?
the answer was, something about calling a function not via pointer is to a relative address, but i havent been able to figure out whats wrong here? which line is the problematic one?
thank you for your help
Well to begin with, there is nothing which says that foo2() and buz() must be next to each other in memory. And for another, as you guess, the code must be relative for stunts like that to work. But most of all, it is not allowed by the standard.
As Chris Luts referred to, stack (auto) variables are not executable on many operating systems, to protect from attacks.
The first two lines in your main() function are problematic.
Line 1. (void(*)(void))buf
converting buf to a function pointer is undefined
Line 2. ((void*)buz)-((void*)foo2)
subtraction of pointers is undefined unless the pointers point within the same array.
Also, Section 5.8 Functions of H&S says "Although a pointer to a function is often assumed to be the address of the function's code in memory, on some computers a function pointer actually points to a block of information needed to invoke the function."
First and foremost, C function pointers mechanism is for equal-signature function calling abstraction. This is powerful and error-prone enough without these stunts.
I can't see an advantage/sense in trying to copying code from one place to another. As some have commented, it's not easy to tell the amount of relativeness/rellocatable code within a C function.
You tried copying the code of a function onto a data memory region. Some microcontrollers would just told you "Buzz off!". On machine architectures that have data/program separated memories, given a very understanding compiler (or one that recognizes data/code modifiers/attributes), it would compile to the specific Code-Data Move instructions. It seams it would work... However, even in data/code separated memory archs, data-memory instruction execution is not possible.
On the other hand, in "normal" data/code shared memory PCs, likely it would also not work because data/code segments are declared (by the loader) on the MMU of the processor. Depending on the processor and OS, attempts to run code on data segments, is a segmentation fault.