I'm trying to Implement simple OS and now have to implement memory management.
At first, we typed simple code code to check memory size as below.
What the problem i met is that the result of this function depends on increment size.
If I set increment to 1024, this function return 640Kb.
However, If I set increment to 1024*1024, this functinon return 120Mb.
(my system(bochs)'s memory set to 120MB.)
I checked the optimization option and A20 gate.
Anyone who knows why my function didn't work well?
unsigned int memtest_sub(unsigned int start, unsigned int end)
{
unsigned int i;
unsigned int* ptr;
unsigned int orgValue;
const unsigned int testValue = 0xbfbfbfbf;
for (i = start; i <= end; i += 1024*1024) {
ptr = (unsigned int*) i;
orgValue = *ptr;
*ptr = testValue;
if (*ptr != testValue) {
break;
}
*ptr = orgValue;
}
return i;
}
You can't do probes like that.
First the memory isn't necessarily contiguous as you've already discovered. It almost never is. The hole at 640k is for legacy reasons, but even further in the memory is usually split up. You have to ask your firmware for the memory layout.
Second some memory banks might be double mapped into the physical space and you'll end up in real trouble if you start using them. This isn't very common, but it's a real pain to deal with it.
Third, and probably most important, there are devices mapped into that space. By writing to random addresses you're potentially writing to registers of important hardware. Writing back whatever you read won't do you good because some hardware registers have side effects as soon as you write them. As a matter of fact, some hardware registers have side effects when you read them. Some of that hardware isn't necessarily protected and you might do permanent damage. I've bricked ethernet hardware in the past by having pointer errors in a 1:1 mapped kernel because the EEPROM/flash was unprotected. Other places you write to might actually change the layout of the memory itself.
Since you're most likely on i386 read this: http://wiki.osdev.org/Detecting_Memory_(x86)
Also, consider using a boot loader that detects memory for you and communicates that and other important information you need to know in a well defined API. The boot loader is better debugged with respect to all weird variants of hardware.
Following assignments are buggy:
ptr = (unsigned int*) i;
orgValue = *ptr;
*ptr = testValue;
ptr not pointing any valid memory, you can't treat i's value as address where you can perform some read-write operation - Undefined behaviour
Related
I need to allocate all the memory my application will use up front. And then whenever needed overwrite that memory with data I need to do computations on. The memory has to be allocated first before any computations because I'm trying to run a multi-threaded CUDA algorithm in parallel as explained in my question here (Multi-Threaded CPU CUDA application not asynchronous when calling CudaFree).
I thought I could allocate all the memory needed as a byte pointer and then store that pointer as a void pointer:
void * allocateMemory()
{
byte *mem;
int nbytes = 13107200;
mem = (byte *) malloc(nbytes);
return mem;
}
Later in my program I want to use the memory that's already allocated to store data. I don't know ahead of time what type the data will be but I know it's size won't go over the allocated limit.
void doSomething(void * mem)
{
int *a = (int*) mem;
for (int i = 0; i < 100; i++)
{
a[i] = i;
}
//do stuff
}
There are many other functions like doSomething(void * mem) above but that use type double or type float or maybe even type byte. I need to be able to overwrite the orignally allocated memory with whatever data type I need. The above code does not work because it says I can't deference a void pointer. It also says I attempted to read or write protected memory.
What is the proper way to do this? What is the best way to accomplish my goal of having all my memory allocated at the beginning and then used however necessary throughout? Thanks!
It sounds like you have two problems.
Cannot dereference a void pointer. Somewhere in your code you have used the result from allocateMemory() without a cast. The code you give is OK, but whatever line the compiler is flagging as wrong is not OK. For example, maybe you have:
void *foo = allocateMemory();
foo[42]; // compiler doesn't have a real type here - error
((int*)foo)[42]; // compiler happy
Attempted to access protected memory. Somewhere in your code you have an invalid pointer. The most likely cause is that allocateMemory() is returning NULL (which you are not checking for).
Your general approach seems OK to me; the issues you describe are related to details in your code, not the overall idea.
In my application, I have a nested pair of loops which follow similarly-nested linked lists in order to parse the data. I made a stupid blunder and cast one struct as the child struct, EG:
if (((ENTITY *) OuterEntityLoop->data)->visible == true) {
instead of:
if (((ENTITY_RECORD *) OuterEntityLoop->data)->entity->visible == true) {
This caused a problem where about 70% of runs would result in the application halting completely - not crashing, just sitting and spinning. Diagnostic printfs in program flow would fire in odd order or not at all, and though it spontaneously recovered a couple of times for the most part it broke the app.
So here's the thing. Even after paring down the logic inside to be absolutely it wasn't infinite looping based on a logic bug, to the point where the loop only contained my printf, it was still broken.
Thing two: when the struct was identified incorrectly, it still complained if I tried to access a nonexistent property even though it didn't have the extant property.
My questions are:
Why did this corrupt memory? Can simply reading garbage memory trash the program's control structures? If not, does this mean I still have a leak somewhere even though Electric Fence doesn't complain anymore?
I assume that the reason it complained about a nonexistent property is because it goes by the type definition given, not what's actually there. This is less questionable in my mind now that I've typed it out, but I'd like confirmation that I'm not off base here.
There's really no telling what will happen when a program accesses invalid memory, even for reading. On some systems, any memory read operation will either be valid or cause an immediate program crash, but on other systems it's possible that an erroneous read could be misinterpreted as a signal to do something. You didn't specify whether you're using a PC or an embedded system, but on embedded systems there are often many addresses by design which trigger various actions when they are read [e.g. dequeueing received data from a serial port, or acknowledging an interrupt]; an erroneous read of such an address might cause serial data to be lost, or might cause the interrupt controller to think an interrupt had been processed when it actually hadn't.
Further, in some embedded systems, an attempt to read an invalid address may have other even worse effects that aren't really by design, but rather by happenstance. On one system I designed, for example, I had to interface a memory device which was a little slow to get off the bus following a read cycle. Provided that the next memory read was performed from a memory area which had at least one wait sate or was on a different bus, there would be no problem. If code which was running in the fast external memory partition tried to read that area, however, the failure of the memory device to get off the bus quickly would corrupt some bits of the next fetched instruction. The net effect of all this was that accessing the slow device from code located in some places was no problem, but accessing it--intentionally or not--from code located in the fast partition would cause weird and non-reproduceable failures.
welcome to C, where the power of casting, allows you to make any piece of memory look like any object you want, but at your own risk. If the thing you cast is not really an object of that type, and that type contains a pointer to something else, you run the risk of crashing. Since even attempting to read random memory that has not been actually mapped into a processes virtual memory address space can cause a core or reading from the certain areas of memory that do not have read permission will also cause a core, like the NULL pointer.
example:
#include <stdio.h>
#include <stdlib.h>
struct foo
{
int x;
int y;
int z;
};
struct bar
{
int x;
int y;
struct foo *p;
};
void evil_cast(void *p)
{
/* hmm... maybe this is a bar pointer */
struct bar *q = (struct bar *)p;
if (q != NULL) /* q is some valid pointer */
{
/* as long as q points readable memory q->x will return some value, */
/* this has a fairly high probability of success */
printf("random pointer to a bar, x value x(%d)\n", q->x);
/* hmm... lets use the foo pointer from my random bar */
if (q->p != NULL)
{
/* very high probabilty of coring, since the likely hood that a */
/* random piece of memory contains a valid address is much lower */
printf("random value of x from a random foo pointer, from a random bar pointer x value x(%d)\n", q->p->x);
}
}
}
int main(int argc, char *argv[])
{
int *random_heap_data = (int *)malloc(1024); /* just random head memory */
/* setup the first 5 locations to be some integers */
random_heap_data[0] = 1;
random_heap_data[1] = 2;
random_heap_data[2] = 3;
random_heap_data[3] = 4;
random_heap_data[4] = 5;
evil_cast(random_heap_data);
return 0;
}
Let's assume that there is a function store_at(int) which is supposed to store the passed number in a given hexa location as shown below:
void store_at(int val)
{
int *ptr;
ptr = (int *)0x261;
// logic goes here
return;
}
How do we write logic to store val at the given Hex location (0x261 In this case)?
Does saying *ptr = val; work? I vaguely remember reading somewhere that this is not allowed in C.
*ptr = val; works. But you have to make sure this address is allocated and even more, accessible. Without knowing for what you are programming C, I could suggest some ways of prevention on accessing addresses you don't have permission. So it pretty much depends on the architecture and/or operational system you're using.
For example, in ATMEGA32 microcontroller, you don't have any limitation regarding the access of the main memory for it. You can read, write and execute code from/for it:
PORTB = 1;
// Knowing that PORTB is stored at 0x38, you can do the equivalent:
*((unsigned int *)0x0038) = 1;
But that's on embedded systems. Now if you want total access for a memory space (as long as it's in your application sandbox), you can use VirtualProtect for Windows and mprotect for Linux:
int val = 123;
DWORD oldprotection;
VirtualProtect((LPVOID)0x261, sizeof(int), PAGE_EXECUTE_READWRITE, &oldprotection);
*(int *)0x261 = val;
And here is the types of protection you can use with it: Memory Protection Constants.
And a mprotect example:
int val = 123;
mprotect((const void *)(((int)(0x261) / PAGESIZE) * PAGESIZE), sizeof(int), PROT_WRITE | PROT_READ | PROT_EXEC);
*(int *)0x261 = val;
Note that this mprotect example is untested, you may need to increase the size for protection or some other things.
The division by PAGESIZE there is just a trick to align the address correctly. Also note that your address is invalid for Linux, as its division will lead to 0 if PAGESIZE is greater than it (the same as "it will be").
According to the syntax for accessing a address using a pointer, all of these work:
*(int *)0x261 = val;
int *ptr = (int *)0x261;
*ptr = val;
Yes, expression *ptr = val (and even more, *(int *)0x261 = val; ) is perfectly valid in C. But then you're facing technical limitations of runtime environments.
Modern operating systems usually run processes in a sandbox of virtual memory (so processes can't access and spoil memory of some other process) and technically the virtual memory of a process looks like a set of regions which you can access, some in readonly way, some does not allow executing code from here and so on. When you try to access non-available VM region, you'll get SIGSEGV on Unix-like systems or Access Violation error on Windows systems, the same for writing to a read-only memory region and trying to execute code in region where it's prohibited by operating system (for example, you can see virtual memory mappings for a linux process with pid in /proc/$PID/maps.
Memory of a process is usually managed by the operating system (you get new memory from the heap using OS-provided functions like malloc(), calloc(); the stack memory regions are allocated by the OS at process startup), so in user-space programming you virtually never need to reference data by literal pointer.
Another possible environments are kernel-space or bare-metal C programs, where you have all the physical memory available to you, but still you must be aware of what you accessing (it may be ports, a gap in the physical memory, it may be reserved by hardware and so on). Programming such environments is an advanced topic and needs good C experience.
I want to essentially do the following (which is probably dangerous and what not) just for the heck of it:
int main() {
int x = 0x00ff00ff;
printf("Value at addr x: %x\n",*x);
return 0;
}
Basically take a look at the contents of a certain address in my machine. Maybe write to it. I'm guessing I'm not allowed to do the latter.
The error I get is error: invalid type argument of 'unary *'.
Is there any way to do this?
You need a pointer:
int *x = (int*)0x00ff00ff;
And you're right, it's probably not a good idea, unless you know that 0x00ff00ff is a valid address of some sort. It's not actually undefined behaviour since the standard says you can't dereference illegal addresses but then states that "illegal" includes things like:
addresses of freed heap objects.
NULL pointers.
wrong alignment.
but doesn't explicitly list arbitrary pointer values, since that would make memory-mapped I/O in embedded systems problematic.
For example, you may control a UART (universal asynchronous receiver/transmitter, basically a serial port device) in an embedded system by reading or writing known memory-mapped I/O addresses:
#define UART_READ_READY ((char*)0xff00)
#define UART_READ_CLEAR ((char*)0xff01)
#define UART_DATA ((char*)0xff02)
char getUartCharWithWait (unsigned int tries) {
char retChar;
unsigned int limit;
// Keep looping until character available, at least for a while.
limit = tries;
while (*UART_READ_READY == 0)
if (limit-- == 0)
return '\0';
// Get character, tell UART to clear it, then return it.
retChar = *UART_DATA;
*UART_CLEAR = 1;
return retChar;
}
In this example, you have code like:
retChar = *UART_DATA;
which will read a byte (C char) from "memory" address 0xff02, which will actually be from a device monitoring the address bus and intercepting specific addresses.
You are getting the previously mentioned error because there is no way to dereference an int, making x a pointer-to-int will yield the "correct" result (ie. it will be able to compile).
int * x = (int*)0x00ff00ff;
"It works, IT WORKS! Or er.. I mean, it compiles. Now, what's a segfault?"
Or maybe, I shouldn't cast. Here's what I'm doing:
I'm writing a piece of code that links a Linux device driver to a higher level library. The authors of the library use void * (under a new name via typedef) to store handles to an implementation specific object that describes a communication channel.
The driver I want to connect with the library uses int to store handles to its channels (because they are file descriptors as returned by calls to open()). So, in my code, I get void * passed in from the library and need to call stuff from the driver using an int and vice versa. I. e.:
// somewhere in the library ...
typedef void* CAN_HANDLE;
// ... in my code
CAN_HANDLE canOpen_driver(s_BOARD *board)
{
int fd;
// ...
fd = open(busname, O_RDWR);
// ...
return (CAN_HANDLE) fd; // <-- not safe, especially not when converting back.
}
The adapters that others have written actually store some struct etc. somewhere and just cast between pointers, so no size issues arise. In my case, I don't really want to manage file descriptors, as the OS already does.
On my PC, I think the pointer is larger than the int, so I could bit-twiddle my way out of this, but the code goes into embedded systems, too, and I'm not experienced enough to make any assumptions about the size of types on those machines.
Edit: Of course you don't need a struct, you can just allocate memory for a plain int.
CAN_HANDLE canOpen_driver(s_BOARD *board)
{
int *fd = malloc(sizeof(int));
if (fd)
{
// ...
*fd = open(busname, O_RDWR);
// ...
return (CAN_HANDLE) fd;
}
// failure
return NULL;
}
This assumes there's a matching call to clean up. Something like:
void canClose_driver(CAN_HANDLE handle)
{
int *fd = handle;
free(fd);
}
Depending on the architecture, you might get away with that. If I understand correctly, the driver never actually uses the void* that you provide to it. It simply stores it to pass it back to your code later on.
Based on that assumption, as long as sizeof(void*) >= sizeof(int), it will be safe to cast between those types because you are sure that it is really a int.
If you cannot guarantee the size condition, or do not want to rely on a hack, you should allocate memory for the int and return the address of that memory. You might use malloc() or allocate a int in a fixed-size array, for example. The downside is that you will need to free that memory when it is no longer needed. I imagine that the driver has some kind of notification that signals your code when the data structure is no longer needed.
Generally on embedded systems, the following CPU models are most common:
Data bus Address bus
8 bit 16 bit
8 bit 16+8 bit (banking)
16 bit 16 bit
16 bit 16+8 bit (banking)
32 bit 32 bit
Generally, the address bus will always be >= than the data bus. I can't think of any CPU where the data bus would be larger than the address bus.
Here's a somewhat dirty trick that may or may not solve the issue:
typedef union
{
CAN_HANDLE handle;
long value;
} CAN_HANDLE_t;
This should be fairly portable, even though you will likely have to adapt this union to the specific system (far pointers etc).