I'm running a program called VMdriver5.exe (left side of image I attached), that creates a process of another program called VMmapper.exe. Inside the VMdriver5.c code, it gets its PID (using GetCurrentProcessId()) and passes it's PID to VMmapper.exe upon creation, such that VMmapper shows the memory layout of VMdriver5.exe so I can see the virtual memory options.
I was wondering if I could get some help, because, when I call VirtualAlloc() on certain memory addresses, it works fine. However, as you can see in the image below, there is a region of memory with 503,808 bytes free, and I attempt to reserve 65,536 bytes of that space using VirtualAlloc(), but it fails for some reason. I pass the memory address 00185000 as one of the parameters.
The code I'm using is this:
lpvAddr = VirtualAlloc(vmAddress, units << 16, MEM_RESERVE, flProtect); // only works with PAGE_READONLY
if(lpvAddr == NULL) {
printf("Case 1: Reserve a region, VirtualAlloc() failed. Error: %ld\n", GetLastError());
}
else {
printf("Committed %lu bytes at address 0x%lp\n",dwPageSize, lpvAddr); // %lu = long unsigned decimal integer
}
break;
I would appreciate any help and can provide any more code or info if needed. Thanks guys and happy easter.
Related
On trying to deal with heaps in WinApi I've got some strange results for HeapAlloc's actions. Lets consider the following code. The problem is that according to Microsoft Documentation on Windows API(next - Doc), I have to get two Success strings printed to the console. By I get an Error when trying to run this code with the Debud option in MSVC 2013. But the strangest thing is that when I try to run this code without Debug option, or run the compiled .exe file, I get the correct result.
#include <Windows.h>
#include <stdio.h>
int main()
{
LPSYSTEM_INFO sys;
HANDLE hNewHeap;
LPVOID ptr;
sys = (LPSYSTEM_INFO) HeapAlloc(GetProcessHeap(),
0,
sizeof(SYSTEM_INFO));
GetSystemInfo(sys);
printf("Page size: %u\n", sys->dwPageSize);//Here we get the
//'Page size: 4096' string
//printed to the console
hNewHeap = HeapCreate(0, 1, 1);
//That's easy. We create new heap object, getting its HADNLE descriptor.
//According to Doc, the initial heap size is set to page size, which is
//4096 on my computer, like maximum heap size is also done. So the heap
//size now is 4096.
ptr = HeapAlloc(hNewHeap, 0, 2624); //Here we allocate the memory
//block in our new heap, that might have 2624 bytes size.
if ( ptr ) printf("Success!\n");//Here we check if the HeapAlloc functio
//worked correctly and print the appropriate string.
else printf("Error!\n");
//On this time we get 'Success' string printed to the console and free
//allocated memory block
if ( ptr ) HeapFree(hNewHeap, 0, ptr);
ptr = HeapAlloc(hNewHeap, 0, 2525);//Here we try to allocate the memory
//block, which size is 2526. And, like previous time, we expect to get
//'Success'.
if ( ptr ) printf("Success!\n");
else printf("Error!\n");
//But we get 'Error' here!!!
if ( ptr ) HeapFree(hNewHeap, 0, ptr);
HeapDestroy(hNewHeap);
system("pause");
};
If you try the same with any less than 2624 number, you will not get an 'Error'. If you try to do this with any more that 2625 number, you WILL get an 'Error'. But we get an 'Error' only when the Debug option is on.
Can somebody explain me why is happening so?
P.S.: Sorry for bad English.
P.S.: The strange is also the number 2625 that do not correspond to any function or application size and that sometimes I get correct result, that is after restarting studio or making some changes in the code.(But only sometimes)
You are creating a fixed size heap. The documentation says:
The HeapCreate function rounds dwMaximumSize up to a multiple of the system page size and then reserves a block of that size in the process's virtual address space for the heap.
So your heap has a fixed size of 4096 bytes.
Subsequently your allocation from this heap fails because the heap is not large enough for the block being allocated. Again from the documentation:
The system uses memory from the private heap to store heap support structures, so not all of the specified heap size is available to the process. For example, if the HeapAlloc function requests 64 kilobytes (K) from a heap with a maximum size of 64K, the request may fail because of system overhead.
It is these heap support structures that are causing your confusion. You think that there is sufficient space in the heap for your second allocation, but you are not accounting for the heap support structures.
The documentation for HeapAlloc tells you to call GetExceptionCode on failure. Expect that to return STATUS_NO_MEMORY.
How do i know the amount of memory used . i.e RAM usage ?
int main()
{
int i=0;
for(i=0;i<100;i++)
{
printf("%d\n",i);
}
return 0;
}
I want to write a code which calculate the amount of memory used by this program . May be like-
int main()
{
int i=0;
for(i=0;i<100;i++)
{
printf("%d\n",i);
}
printf("Amount of memory consumed=%f",SOME_FUNCTION());
return 0;
}
The getrusage system call will return a handful of information for the current process, among which is the "resident set size":
struct rusage usage;
if (!getrusage(RUSAGE_SELF, &usage)) {
printf("Maximum resident set size (KB): %ld\n", usage.ru_maxrss);
} else {
perror("getrusage");
}
This size equates to the amount of memory that is physically wired to the process and not the entire size of the virtual address space, parts of which might be paged-out or never loaded.
It's not easy to check how much memory your program uses on Linux system. but most likely what you want to check is the value of VmRSS in /proc/[pid]/status (or second column of /proc/[pid]/statm). VmRSS ("resident set size") is the amount of memory your process currently uses.
Other than that you may be interested in VmSize from /proc/[pid]/status (or first column of /proc/[pid]/statm). This is total memory your process uses, including memory swapped out, memory used by shared libraries, memory-mapped resources (which, in general, don't consume real RAM).
To get PID of your process, use getpid(). From within your process you could also check /proc/self/status.
A simple approach will to create a wrapper function for memory allocation and freeing and call the wrapper and put memory usage information in that. This can only be used in case of dynamic memory allocaion. e.g
#define ALLOC 1
#define FREE 2
mem_op(void * pointer,int size,int operation)
{
switch(operation)
{
static int mem_used;
case ALLOC:
// call malloc or alloc
mem_used = mem_used+size;
break;
case FREE:
// call free
mem_used = mem_used-size;
break;
}
I have some data stored in a FLASH memory that I need to access with C pointers to be able to make a non-Linux graphics driver work (I think this requirement is DMA related, not sure). Calling read works, but I don't want to have intermediate RAM buffers between the FLASH and the non-Linux driver.
However, just creating a pointer and storing the address that I want on it is making Linux emit an exception about invalid access on me.
void *ptr = 0xdeadbeef;
int a = *ptr; // invalid access!
What am I missing here? And could someone point me to a material to make this concepts clear for me?
I'm reading about mmap but I'm not sure that this is what I need.
The problem you have is that linux runs your program in a virtual address space. So every address you use directly in the code (like 0xdeadbeef) is a virtual address that gets translated by the memory management unit into a physical address which is not necessarily the same as your virtual address. This allows easy separation of multiple independent processes and other stuff like paging, etc.
The problem is now, that in your case no physical address is mapped to the virtual address 0xdeadbeef causing the kernel to abort execution.
The call mmap you already found asks the kernel to assign a specific file (from a specific offset) to a virtual address of your process. Note that the returning address of mmap could be a completely different address. So don't make any assumptions about the virtual address you get.
Therefore there are examples with mmap and /dev/mem out there where the offset for the memory device is the physical address. After the kernel was able to assign the file from the offset you gave to a virtual address of your process you can access the memory area asif it were a direct access.
After you don't need the area anymore don't forget to munmap the area. Otherwise you'll cause something similar to a memory leak.
One problem with the /dev/mem method is that the user running the process needs access to this device. This could introduce a security issue (e.g. Samsung recently introduced such a security hole in their hand held devices)
A more secure way is the way described in a article i found (The Userspace I/O HOWTO) as you still have control about the memory areas accessable by the user's process.
You need to access the memory differently. Basically you need to open /dev/mem and use mmap(). (as you suggested). Simple example:
int openMem(unsigned int address, unsigned int size)
{
int mmapFD;
int page_size;
unsigned int page_start_address;
/* Minimum page size for the mmapped region. */
mask = size - 1;
/* Get the page size. */
page_size = (int) sysconf(_SC_PAGE_SIZE);
/* We have to map shared memory to beginning of memory page so adjust
* memory address accordingly. */
page_start_address = address - (address % page_size);
/* Open the file that will be mapped. */
if((mmapFD = open("/dev/mem", (O_RDWR | O_SYNC))) == -1)
{
printf("Opening shared memory device failed\n");
return -1;
}
mmap_base_address = mmap(0, size, (PROT_READ|PROT_WRITE), MAP_SHARED, mmapFD, (off_t)page_start_address & ~mask);
if(mmap_base_address == MAP_FAILED)
{
printf("Mapping memory failed\n");
return -1;
}
return 0;
}
unsigned int *getAddress(unsigned int address)
{
unsigned int log_address;
log_address = (int)((off_t)mmap_base_address + ((off_t)address & mask));
return (unsigned int*)log_address;
}
...
result = openMem(address, 0x10000);
if (result < 0)
return result;
target_address = getValue(address);
*(unsigned int*)target_address = value;
This would set "value" to "address".
You need to call ioremap - something like:
void *myaddr = ioremap(0xdeadbeef, size);
where size is the size of your memory region. You probably want to use a page-aligned address for the first argument, e.g. 0xdeadb000 - but I expect your actual device isn't at "0xdeadbeef" anyways.
Edit: The call to ioremap must be done from a driver!
I am trying to scan the address space to find my chunks of memory that have read/write permissions. It is acceptable to check a single address per page as each page have the same permissions. I know I should be getting Segmentation Fault: 11 when trying to write to a piece of memory I shouldn't be able to. This happens when I am trying to access higher addresses but when I am in the lower portion, say 0x00000100, I get the Bus error: 10.
NOTE: The code is compiled with the -m32 flag so it simulates a 32 bit machine.
ALSO NOTE: The memory for chunk_list has already been malloc'ed before this function is called.
I have copied the code below:
#include <signal.h>
#include <stdio.h>
#include <unistd.h>
#include "memchunk.h"
int get_mem_layout (struct memchunk *chunk_list, int size)
{
//grab the page size
long page = sysconf(_SC_PAGESIZE);
printf("The page size for this system is %ld bytes\n", page);
//page size is 4069 bytes
//test printing the number of words on a page
long words = page / 4;
printf("Which works out to %ld words per page\n", words);
//works out to 1024 words a page
//1024 = 0x400
//create the addy pointer
//start will be used after bus error: 10 is solved
void *start;
char * currAddy;
currAddy = (char*)0x01000000;
//someplace to store the addy to write to
//char * testWrite;
//looping through the first size pages
int i;
for(i = 0; i < size; i++){
//chunk start - wrong addy being written just testing
chunk_list[i].start = currAddy;
printf("addy is %p\n",currAddy);
sleep(1);
//try and write to the current addy
//testWrite = currAddy;
//*testWrite = 'a';
*currAddy = '1';
//+= 0x400 to get to next page
currAddy += 0x400;
}
//while loop here for all the addys - not there yet because still dealing with bus error: 10
return 0;
}
Any help would be greatly appreciated. I also left some other attempts at it commented out in the code, still all produce a bus error: 10 in the lower portion of the memory space.
EDIT: I will be dealing with seg faults using signals. I know how to deal with the seg fault, so is there a way to handle a bus error: 10 using signals as well?
Reading from or writing to unmapped memory is supposed to cause a bus fault. To discover whether a memory is there, install a handler for SEGFAULTs to react accordingly.
In a Linux SE (Security Enhanced) process, the program sections are loaded at randomized locations to frustrate viruses being able to rely on stable addresses.
In most virtual memory systems, a non-mapped space is usually left from address zero up a ways so attempts to dereference a NULL pointer or a structure based on a NULL pointer cause an exception. In the 1980s, the blank space was often 64K to 256K. On modern architectures, 16M is a reasonable choice to detect NULL-based accesses.
On many virtual memory systems, there is a system call to obtain per process mapped memory locations. On Linux, inspect the contents of /proc/self/maps.
I have plenty of RAM, however, after starting and finishing a large number of processes, it seems that most of the applications' virtual memory has been paged to disk, and switching to any of the older processes requires a very long time to load the memory back into RAM.
Is there a way, either via Windows API or via kernel call, to get Windows to unpage all (or as much as possible) memory? Maybe by stepping through the list of running processes and get the memory manager to unpage each process's memory?
Update 3: I've uploaded my complete program to github.
OK, based on the replies so far, here's a naive suggestion for a tool that tries to get all applications back into physical memory:
Allocate a small chunk of memory X, maybe 4MB. (Should it be non-pageable?)
Iterate over all processes:
For each process, copy chunks of its memory to X.
(Possibly suspending the process first?)
Suppose you have 2GB of RAM, and only 1GB is actually required by processes. If everything is in physical memory, you'd only copy 256 chunks, not the end of the world. At the end of the day, there's a good chance that all processes are now entirely in the physical memory.
Possible convenience and optimisation options:
Check first that the total required space is no more than, say, 50% of the total physical space.
Optionally only run on processes owned by the current user, or on a user-specified list.
Check first whether each chunk of memory is actually paged to disk or not.
I can iterate over all processes using EnumProcesses(); I'd be grateful for any suggestions how to copy an entire process's memory chunk-wise.
Update: Here is my sample function. It takes the process ID as its argument and copies one byte from each good page of the process. (The second argument is the maximal process memory size, obtainable via GetSystemInfo().)
void UnpageProcessByID(DWORD processID, LPVOID MaximumApplicationAddress, DWORD PageSize)
{
MEMORY_BASIC_INFORMATION meminfo;
LPVOID lpMem = NULL;
// Get a handle to the process.
HANDLE hProcess = OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_VM_READ, FALSE, processID);
// Do the work
if (NULL == hProcess )
{
fprintf(stderr, "Could not get process handle, skipping requested process ID %u.\n", processID);
}
else
{
SIZE_T nbytes;
unsigned char buf;
while (lpMem < MaximumApplicationAddress)
{
unsigned int stepsize = PageSize;
if (!VirtualQueryEx(hProcess, lpMem, &meminfo, sizeof(meminfo)))
{
fprintf(stderr, "Error during VirtualQueryEx(), skipping process ID (error code %u, PID %u).\n", GetLastError(), processID);
break;
}
if (meminfo.RegionSize < stepsize) stepsize = meminfo.RegionSize;
switch(meminfo.State)
{
case MEM_COMMIT:
// This next line should be disabled in the final code
fprintf(stderr, "Page at 0x%08X: Good, unpaging.\n", lpMem);
if (0 == ReadProcessMemory(hProcess, lpMem, (LPVOID)&buf, 1, &nbytes))
fprintf(stderr, "Failed to read one byte from 0x%X, error %u (%u bytes read).\n", lpMem, GetLastError(), nbytes);
else
// This next line should be disabled in the final code
fprintf(stderr, "Read %u byte(s) successfully from 0x%X (byte was: 0x%X).\n", nbytes, lpMem, buf);
break;
case MEM_FREE:
fprintf(stderr, "Page at 0x%08X: Free (unused), skipping.\n", lpMem);
stepsize = meminfo.RegionSize;
break;
case MEM_RESERVE:
fprintf(stderr, "Page at 0x%08X: Reserved, skipping.\n", lpMem);
stepsize = meminfo.RegionSize;
break;
default:
fprintf(stderr, "Page at 0x%08X: Unknown state, panic!\n", lpMem);
}
//lpMem = (LPVOID)((DWORD)meminfo.BaseAddress + (DWORD)meminfo.RegionSize);
lpMem += stepsize;
}
}
CloseHandle(hProcess);
}
Question: Does the region by whose size I increment consist of at most one page, or am I missing pages? Should I try to find out the page size as well and only increment by the minimum of region size and page size? Update 2: Page size is only 4kiB! I changed the above code to increment only in 4kiB steps. In the final code we'd get rid of the fprintf's inside the loop.
Well, it isn't hard to implement yourself. Use VirtualQueryEx() to discover the virtual addresses used by a process, ReadProcessMemory() to force the pages to get reloaded.
It isn't likely to going to make any difference at all, it will just be your program that takes forever to do its job. The common diagnostic for slow reloading of pages is a fragmented paging file. Common on Windows XP for example when the disk hasn't been defragged in a long time and it was allowed to fill close to capacity frequently. The SysInternals' PageDefrag utility can help fix the problem.
No, windows provides no such feature natively. Programs such as Cacheman and RAM IDLE accomplish this by simply allocating a large chunk of RAM, forcing other things to page to disk, which effectively accomplishes what you want.