This question already has answers here:
Closed 14 years ago.
Is it possible to write a C function that does the following?
Allocate a bunch of memory in the heap
Writes machine code in it
Executes those machines instructions
Of course, I would have to restore the state of the stack to what it was prior to the execution of those machine instructions manually, but I want to know if this is feasible in first place.
It's certainly possible. For various reasons, we've spent a lot of effort of the last 30-40 years trying to make it as difficult as possible, but it is possible. In most systems now, there are hardware and software mechanisms that attempt to protect data space from being executed.
The basics, though, are fairly straightforward: you construct a piece of code, and assemble it, either by hand or4 via a compiler. You then need a fragment of code space, so you insert the code into your program
unsigned int prgm[] = { 0x0F, 0xAB, 0x9A ... }; // Random numbers, just as an example
since you wanted to use the heap you need to malloc the space
void * myspace ;
if((myspace= malloc(sizeof(prgm))) != NULL) {
memcpy(myspace, pgrm, sizeof(pgrm));
} else { // allocation error
}
Now, what you need is a way to get the program counter to point to that chunk of data that is also your chunk of code. Here's where you need a little craftiness. Setting the program counter is no big deal; that's just a JUMP instruction for your underlying machine. But how to do that?
One of the easiest ways is by purposefully messing with the stack. The stack, again conceptually, looks something like this (the details depend on both your OS and compiler pairs, and on your hardware):
| subroutine return addr |
| parameters ... |
| automatic variables |
The basic trick here is to sneakily get the address of your code into the return address; when a routine returns, it basically jumps to that return addrfess. If you can fake it out, the PC will be set to where you like.
So, what you need is a routine, let's call it "goThere()"
void goThere(void * addr){
int a ; // observe above; this is the first space
// on the stack following the parameters
int * pa; // so we use it's address
pa = (&a - (sizeof(int)+(2*sizeof(void*))) ; // so use the address
// but back up by the size of an int, the pointer on the
// stack, and the return address
// Now 'pa' points to the routine's return add on the stack.
*pa = addr; // sneak the address of the new code into return addr
return ; // and return, tricking it into "returning"
// to the address of your special code block
}
Will it work? Well, maybe, depending on the hardware and OS. Most modern OS's will protect the heap (via memory mapping or similar) from the PC moving into it. This is a useful thing for security purposes, because we'd just as well not let you take that kind of complete control.
This is very similar to this question :)
Read calling code stored in the heap from vc++. On posix, mprotect seems to be appropriate (look into man mprotect):
char *mem = malloc(sizeof(code));
mprotect(mem, sizeof(code), PROT_READ|PROT_WRITE|PROT_EXEC);
memcpy(mem, code, sizeof(code));
// now arrange some code to jump to mem. But read the notes here on casting
// from void* to a function pointer:
// http://www.opengroup.org/onlinepubs/009695399/functions/dlsym.html
However, it says:
Whether PROT_EXEC has any effect different from PROT_READ is architecture- and kernel version-dependent. On some hardware architectures (e.g., i386), PROT_WRITE implies PROT_READ.
So better, first check whether on your operation system, that works.
RE: manually restoring the stack
If you follow the calling conventions used by your platform / compiler inside the machine code you generate, then you shouldn't have to do any manual stack restoring. The compiler will do that for you, when you do
*pfunc(args)
it should add any appropriate pre or post call stack manipulation steps that are necessary.
Just make sure that you follow the right conventions inside the generated code, however.
Related
I was wondering what is the equivalent of the command Peek and Poke (Basic and other variants) in C. Something like PeekInt, PokeInt (integer).
Something that deals with memory banks, I know in C there are lots of ways to do this and I am trying to port a Basic program to C.
Is this just using pointers? Do I have to use malloc?
Yes, you will have to use pointers. No, it does not necessarily involve malloc.
If you have a memory address ADDRESS and you want to poke a byte value into it, in C that's
char *bytepointer = ADDRESS;
*bytepointer = bytevalue;
As a concrete example, to poke the byte value 0h12 to memory address 0h3456, that would look like
char *bytepointer = 0x3456;
*bytepointer = 0x12;
If on the other hand you want to poke in an int value, that looks like
int *intpointer = ADDRESS;
*intpointer = intvalue;
"Peeking" is similar:
char fetched_byte_value = *bytepointer;
int fetched_int_value = *intpointer;
What's happening here is basically that these pointer variables like bytepointer and intpointer contain memory addresses, and the * operator accesses the value that the pointer variable points to. And this "access" can be either for the purpose of fetching (aka "peeking"), or storing ("poking").
See also question 19.25 in the C FAQ list.
Now, with that said, I have two cautions for you:
Pointers are an important but rather deep concept in C, and they can be tricky to understand at first. If you've never used a pointer, if you haven't gotten to the Pointers chapter in a good C textbook, the superficial introduction I've given you here isn't going to be enough, and there are all sorts of pitfalls waiting for you if you don't learn about them first.
Setting a pointer to point to an absolute, numeric memory address, and then accessing that location, is a low-level, high-powered technique that ends up being rather rare these days. It's useful (and still commonly used) if you're doing embedded programming, if you're using a microcontroller to access some hardware, or perhaps if you're writing an operating system. In more ordinary, "applications" level programming, on the other hand, there just aren't any memory addresses which you're allowed to access in this way.
What values do you want to peek or poke, to accomplish what? What is that Basic program supposed to do? If you're trying to, for example, poke values into display memory so they show up on the screen, beware that on a modern operating system (that is, anything newer than MS-DOS), it doesn't work that way any more.
Yes, its exactly pointers.
to read from the location pointed by some pointer code value = *pointer. (peek)
to write to that location, code *pointer = value. (poke)
notice the asterisk sign before pointer.
malloc is used when you need to allocate runtime memory in heap, but its not the only way to get a memory pointer. you can get a pointer simply by using & before name of a variable.
int value = 2;
int *myPointer = &value;
remember to free any memory allocated with malloc after you are done with it. See also the valgrind tool.
also I recommend to stick with non malloc solution because of its simplicity and easy maintenance.
you can call a function many time to allocate new memory for its local variables and work with them.
The concept of peek/poke (to directly access memory) in C is irrelevant - you probably don't need them and if you do you really should ask a specific question about the code you are trying to write. If you are building 32 or 64 bit code for a modern platform, peek() / poke() are largely irrelevant in any event and unlikely to do what you expect - unless you expect a memory protection fault! You will be digging out some antique compiler if you are building 16-bit x86 "real-mode" code.
C is a systems-level language and as such operating directly on memory fundamental to the language. It is also a compiled rather than interpreted language and you would normally access a memory location through a variable/symbol and let the linker resolve/locate its address. However in some cases (in embedded systems code or kernel level device drivers for example) you might need to access a specific physical address which you might do as follows:
#include <stdint.h>
const uint16_t* const KEYBOARD_STATE_FLAGS_ADDR = ((uint16_t*)0x0417) ;
...
// Peek keyboard state
keyboard_flags = *KEYBOARD_STATE_FLAGS_ADDR ;
If you really want functions to observe/modify arbitrary address locations then for example:
uint16_t peek16( uint32_t addr )
{
return *((uint16_t*)addr) ;
}
void poke16( uint32_t addr, uint16_t value )
{
*((uint16_t*)addr) = value ;
}
... and perhaps corresponding implementations for 8, 32 and even 64 bit access. However good luck avoiding a SEG-FAULT exception on a modern system where memory is virtualised and protected - as I said it largely makes no sense.
Something that deals with memory banks,
I assume given the [qbasic] tag that you are referring to 16-bit x86 segment:offset addressing rather than "memory banks" - that is an architecture specific thing, irrelevant on modern systems. If you really need to deal with that (i.e. are targeting 16-bit x86 C code), you need to mention that - it is a very different issue, specific to a particular obsolete architecture. You would also need to specify what compiler you are using, because it involves non-standard compiler extensions (and to be frank, techniques I have long forgotten; it would be software archaeology).
Is this just using pointers?
Essentially yes, in C pointers are addresses, but in 16-Bit x86 code there is the added complication of the segmented memory architecture and the concept of near and far pointers. If you are porting this code to a modern system, it is unlikely to be a simple matter of porting peek/poke commands verbatim - they are unlikely to work. If you are are just translating 16-bit QBasic code to 16-bit C - why?!
Do I have to use malloc?
No - or at least that depends on the semantics of the code you are porting, but unlikely, and it is not relevant to peek/poke specifically. malloc allocates memory from a system heap and provides its address - which is non-deterministic. peek/poke are a means of accessing specific memory addresses.
In the end it is probably not a matter of directly implementing the peek/poke commands in the code you are porting, and whether that would work or not would depend on the platform and architecture you are porting the code to. Most likely you would be better off looking at the higher level semantics of the code you are porting and implementing that in a manner that suits the platform you are targeting or better be platform independent and therefore portable.
Basically when porting you should ask your self what does this bit of code do, and port that rather than a line-by-line translation. And that consideration should be done at as high a level as possible to make the best use of the target language an/or its library. peek/poke were typically used for either accessing machine features no exposed through some higher lever interface, or for directly accessing hardware or video-buffers. On a modern platform much of that is either unnecessary or won't work - and probably both.
When I try to google this, all I find is stuff about getting and setting the stack limit, such as -[NSThread stackSize], but that's NOT what I want. I want to know how much memory is in actually in use on the stack in the current thread, or equivalently how much stack space remains available.
I'm hoping to figure out a stack overflow in a crash report submitted by a user. In my previous experience, a stack overflow has usually been caused by an infinite recursion, but not this time. So I'm wondering if some of my C++ functions are really using a heck of a lot more stack space than they should.
A comment suggested that I get the stack pointer at the start of the thread, and compare its value later. I happened across the question Print out value of stack pointer. It has several answers:
(The accepted answer) Take the address of a local variable.
Use a little assembly language to get the value of the stack pointer register.
Use the function __builtin_frame_address(0) in GCC or Clang.
I tried those techniques (Apple Clang, macOS 11.2). Methods 2 and 3 produced similar results, but method 1 produced absurdly different results. For one thing, method 1 gives values that increase as you go deeper into a call chain, while the others give values that decrease. What's up with this, are there two different kinds of stacks?
If you are trying to do that, I guess you want to know how much memory are you using to guess the optimum number of threads you can create of some kind.
The answer is not easy, as you normally don't have access to the stack pointer. But I'll try to devise a solution for you that will not require to access the stack pointer, while it requires to use a global variable per thread.
The idea is to force a parameter to be in the stack. Even if the ABI in your system uses register to pass parameters, if you save the address of a parameter (the actual parameter variable) into some local variable, and then after that you call a function, that takes a parameter (the type doesn't matter, as you are going to use it's address to compare both):
static char *initial_stack_pseudo_addr;
size_t save_initial_stack(char dumb)
{
/* the & operator forces dumb to be implemented in the stack */
initial_stack_pseudo_addr = &dumb;
}
size_t how_much_stack(int dumb)
{
return initial_stack_pseudo_addr - &dumb;
}
So when you start the thread, you call save_initial_stack(0);. When you want to know how much stack you have consumed, just can do the following:
size_t stack_size = how_much_stack(0);
printf("at this point I have %zi bytes of stack\n", stack_size);
Basically, what you have done is to calculate how many bytes are between the address of the local parameter of the call to save_initial_stack() to the address of the local parameter of the call you do now to get the stack size. This is approximate, but the stack changes too quick to have a precise idea.
The following example will illustrate the thing. A recursive function is called after setting the initial pointer value, then at each recursive call the current size of the stack (approximate) is computed and printed, and a new recursive call is made. The program should run until the process gets a stack overflow.
#include <stdio.h>
char *stack_at_start;
void save_stack_pointer(char dumb)
{
stack_at_start = &dumb;
}
size_t get_stack_size(char dumb)
{
return stack_at_start - &dumb;
}
void recursive()
{
printf("Stack size: %zi\n", get_stack_size(0));
recursive();
}
int main()
{
save_stack_pointer(0);
recursive();
}
As i am relatively new to C , i have to use for one of my projects the following:
i must declare some global variables which have to be stored every time the program runs at the same memory address.
I did some read and i found that is i declare it "static" it will be stored at the same memory location.
But my question is: can i indicate the program where to store that variable or not.
For example : int a to be stored at 0xff520000. Can this thing be done or not? i have searched here but did not found any relevant example. If their is some old post regarding this, please be so kind to share the link .
Thank you all in advance.
Laurentiu
Update: I am using a 32uC
In your IDE there will be a memory map available through some linker file. It will contain all addresses in the program. Read the MCU manual to see at which addresses there is valid memory for your purpose, then reserve some of that memory for your variable. You have to read the documentation of your specific development platform.
Next, please note that it doesn't make much sense to map variables at specific addresses unless they are either hardware registers or non-volatile variables residing in flash or EEPROM.
If the contents of such a memory location will change during execution, because it is a register, or because your program contains a bootloader/NVM programming algorithm changing NVM memory cells, then the variables must be declared as volatile. Otherwise the compiler will break your code completely upon optimization.
The particular compiler most likely has a non-standard way to allocate variables at specific addresses, such as a #pragma or sometimes the weird, non-standard # operator. The only sensible way you can allocate a variable at a fixed location in standard C, is this:
#define MY_REGISTER (*(volatile uint8_t*)0x12345678u)
where 0x12345678 is the address where 1 byte of that is located. Once you have a macro declaration like this, you can use it as if it was a variable:
void func (void)
{
MY_REGISTER = 1; // write
int var = MY_REGISTER; // read
}
Most often you want these kind of variables to reside in the global namespace, hence the macro. But if you for some reason want the scope of the variable to be reduced, then skip the macro and access the address manually inside the code:
void func (void)
{
*(volatile uint8_t*)0x12345678u = 1; // write
int var = *(volatile uint8_t*)0x12345678u; // read
}
You can do this kind of thing with linker scripts, which is quite common in embedded programming.
On a Linux system you might never get the same virtual address due to address space randomization (a security feature to avoid exploits that would rely on knowing the exact location of a variable like you describe).
If it's just a repeatable pointer you want, you may be able to map a specific address with mmap, but that's not guaranteed.
Like was mentioned in other answers - you can't.
But, you can have a workaround. If it's ok for the globals to be initialized in the main(), you can do something of this kind:
int addr = 0xff520000;
int main()
{
*((int*)addr) = 42;
...
return 0;
}
Note, however, that this is very dependent on your system and if running in protected environment, you'll most likely get a runtime crash. If you're in embedded/non-protected environment, this can work.
No you cannot tell it explicitly where to store a variable in memory. Mostly because on modern systems you have many things done by the system in regards to memory, that is out of your control. Address Layout Randomization is one thing that comes to mind that would make this very hard.
according your compiler if you use XC8 Compiler.
Simply you can write int x # 0x12 ;
in this line you set x in the memory location 0x12
Not at the C level. If you work with assembly language, you can directly control the memory layout. But the C compiler does this for you. You can't really mess with it.
Even with assembly, this only controls the relative layout. Virtual memory may place this at any (in)convenient physical location.
You can do this with some compiler extensions, but it's probably not what you want to do. The operating system handles your memory and will put things where it wants. How do you even know that the memory address you want will be mapped in your program? Ignore everything in this paragraph if you're on an embedded platform, then you should read the manual for that platform/compiler or at least mention it here so that people can give a more specific answer.
Also, static variables don't necessarily have the same address when the program runs. Many operating systems use position independent executables and randomize the address space on every execution.
You can declare a pointer to a specific memory address, and use the contents of that pointer as a variable I suppose:
int* myIntPointer = 0xff520000;
While studying tsr programming i have seen the use of certain code which i cannot understand..
The example cede part is(in c):
(please explain the bolded sections)
#include "dos.h"
#include"stdio.h"
void interrupt our();
void interrupt (*prev)();
char far *scr=(char far*)0xB8000000L;
int ticks;
unsigned char color;
main()
{
prev=getvect(8); // <<<<<
setvect(8,our); // <<<<<
keep(0,10000); // <<<<<
}
You would partially understand this code if you read the answer i posted to your similar question on TSR
How to write a TSR which changes case of characters
The most important things here are
Far pointer: Since 16 bit DOS used segment offset addressing scheme, your normal near pointer could not access memory beyond 64K of it's allocated segment. You have to read details to understand it.
Video memory address: This B8000000 is the address for which you need far pointer. The special thing about this address is, that starting from this location bytes (equal to the resolution of screen * 2) are copied directly into video memory.
So if you assign a character to a pointer address after indirection it will be printed on screen
Something like
char * far p = 0xB8000000;
*p = 'a'; // this would actually print a on screen at left top
Loop forward to get to the rest of the screen.
There was a c book by yashwant kanetkar which had a good deal of reference for this. I remember using it in my undergrad many years ago.
The rest of them are just indexing api's in dos.h. Why don't you go through their description and get back here if you don't understand any?
This program installs an interrupt handler. It uses interrupt number 8, the system timer interrupt. This was a common practice to use this interrupt to "continuously" do stuff on a machine running DOS.
prev=getvect(8);
This line gets the interrupt vector, that is, a pointer to a function that the system calls 18 times per second.
setvect(8,our);
This line sets the interrupt vector, that is, tells the system to call this function, instead of the old function, 18 times per second. Note that to avoid a crash, the new function must call the old function, in addition to its main purpose (which seems to be changing the case of characters).
keep(0,10000);
This line makes the program with exit code 0 (a conventional value for success) and tells DOS to leave 10000 bytes (or maybe 16-byte units? Unlikely; I don't remember) in RAM. This is unlike normal completion of program (exit(0)), where DOS marks all RAM previously occupied by the program as free.
A common cause of a crash in a TSR program is caused by the absence of keep at the end. DOS releases the memory occupied the by the code of the function our, and in the next 1/18 of a second, a random piece of code is executed.
See Int 21/AH=31h for more information.
Please note also that the parameter to keep should be calculated by manipulating some addresses, so that you don't take too much memory, and on the other hand, take enough memory to contain the code of the function our, which performs the stuff you need. The value 10000 is just an example.
Say I have two instances of an application, with the same inputs and same execution sequence. Therefore, one instance is a redundant one and is used for comparing data in memory with the other instance, as a kind of error detection mechanism.
Now, I want all memory allocations and deallocations to happen in exactly the same manner in the two processes. What is the easiest way to achieve that? Write my own malloc and free? And what about memories allocated with other functions such as mmap?
I'm wondering what you are trying to achieve. If your process is deterministic, then the pattern of allocation / deallocation should be the same.
The only possible difference could be the address returned by malloc. But you should probably not depend on them (the easiest way being not using pointers as key map or other data structure). And even then, there should only be difference if the allocation is not done through sbrk (the glibc use anonymous mmap for large allocations), or if you are using mmap (as by default the address is selected by the kernel).
If you really want to have exactly the same address, one option is to have a large static buffer and to write a custom allocator that does use memory from this buffer. This has the disadvantage of forcing you to know beforehand the maximum amount of memory you'll ever need. In a non-PIE executable (gcc -fno-pie -no-pie), a static buffer will have the same address every time. For a PIE executable you can disable the kernel's address space layout randomization for loading programs. In a shared library, disabling ASLR and running the same program twice should lead to the same choices by the dynamic linker for where to map libraries.
If you don't know before hand the maximum size of the memory you want to use, or if you don't want to recompile each time this size increase, you can also use mmap to map a large anonymous buffer at a fixed address. Simply pass the size of the buffer and the address to use as parameter to your process and use the returned memory to implement your own malloc on top of it.
static void* malloc_buffer = NULL;
static size_t malloc_buffer_len = 0;
void* malloc(size_t size) {
// Use malloc_buffer & malloc_buffer_len to implement your
// own allocator. If you don't read uninitialized memory,
// it can be deterministic.
return memory;
}
int main(int argc, char** argv) {
size_t buf_size = 0;
uintptr_t buf_addr = 0;
for (int i = 0; i < argv; ++i) {
if (strcmp(argv[i], "--malloc-size") == 0) {
buf_size = atoi(argv[++i]);
}
if (strcmp(argv[i], "--malloc-addr") == 0) {
buf_addr = atoi(argv[++i]);
}
}
malloc_buffer = mmap((void*)buf_addr, buf_size, PROT_WRITE|PROT_READ,
MAP_FIXED|MAP_PRIVATE, 0, 0);
// editor's note: omit MAP_FIXED since you're checking the result anyway
if (malloc_buffer == MAP_FAILED || malloc_buffer != (void*)but_addr) {
// Could not get requested memory block, fail.
exit(1);
}
malloc_size = buf_size;
}
By using MAP_FIXED, we are telling the kernel to replace any existing mappings that overlap with this new one at buf_addr.
(Editor's note: MAP_FIXED is probably not what you want. Specifying buf_addr as a hint instead of NULL already requests that address if possible. With MAP_FIXED, mmap will either return an error or the address you gave it. The malloc_buffer != (void*)but_addr check makes sense for the non-FIXED case, which won't replace an existing mapping of your code or a shared library or anything else. Linux 4.17 introduced MAP_FIXED_NOREPLACE which you can use to make mmap return an error instead of memory at the wrong address you don't want to use. But still leave the check in so your code works on older kernels.)
If you use this block to implement your own malloc and don't use other non-deterministic operation in your code, you can have complete control of the pointer values.
This suppose that your pattern usage of malloc / free is deterministic. And that you don't use libraries that are non-deterministic.
However, I think a simpler solution is to keep your algorithms deterministic and not to depend on addresses to be. This is possible. I've worked on a large scale project were multiple computer had to update state deterministically (so that each program had the same state, while only transmitting inputs). If you don't use pointer for other things than referencing objects (most important things is to never use pointer value for anything, not as a hash, not as a key in a map, ...), then your state will stay deterministic.
Unless what you want to do is to be able to snapshot the whole process memory and do a binary diff to spot divergence. I think it's a bad idea, because how will you know that both of them have reached the same point in their computation? It is much more easier to compare the output, or to have the process be able to compute a hash of the state and use that to check that they are in sync because you can control when this is done (and thus it become deterministic too, otherwise your measurement is non-deterministic).
What is not deterministic is not only malloc but mmap (the basic syscall to get more memory space; it is not a function, it is a system call so is elementary or atomic from the application's point of view; so you cannot rewrite it within the application) because of address space layout randomization on Linux.
You could disable it with
echo 0 > /proc/sys/kernel/randomize_va_space
as root, or thru sysctl.
If you don't disable address space layout randomization you are stuck.
And you did ask a similar question previously, where I explained that your malloc-s won't always be deterministic.
I still think that for some practical applications, malloc cannot be deterministic. Imagine for instance a program having an hash-table keyed by the pid-s of the child processes it is launching. Collision in that table won't be the same in all your processes, etc.
So I believe you won't succeed in making malloc deterministic in your sense, whatever you'll try (unless you restrict yourself to a very narrow class of applications to checkpoint, so narrow that your software won't be very useful).
Simply put, as others have stated: if the execution of your program's instructions is deterministic, then memory returned by malloc() will be deterministic. That assumes your system's implementation doesn't have some call to random() or something to that effect. If you are unsure, read the code or documentation for your system's malloc.
This is with the possible exception of ASLR, as others have also stated. If you don't have root privileges, you can disable it per-process via the personality(2) syscall and the ADDR_NO_RANDOMIZE parameter. See here for more information on the personalities.
Edit: I should also say, if you are unaware: what you're doing is called bisimulation and is a well-studied technique. If you didn't know the terminology, it might help to have that keyword for searching.
When writing high-reliability code, the usual practise is to avoid malloc and other dynamic memory allocation. A compromise sometimes used is to do all such allocation only during system initialisation.
You can used shared memory to store your data. It will accessible from both processes and you can fill it in a deterministic way.