On POSIX systems, I am able to use the mmap function to read the contents of a file faster than getline, getc, etc. This is important in the program that I am developing as it is expected to read very large files into memory; iteratively collecting lines using getline is too costly. Portability is also a requirement of my software, so if I use mmap, I need to find a way to memory map files using the WinApi, as I'd rather not compile through cygwin/msys. From a cursory search I identified this MSDN article which describes very briefly a way to map files into memory, however, from trawling through documentation I can't make head nor tails of how to actually implement it, and I'm stuck on finding example snippets of code, like there are for POSIX mmap.
How do I use the WinApi's memory mapping options to read a file into a char*?
How do I use the WinApi's memory mapping options to read a file into a
char*?
Under Windows, when you map a file in memory, you get a pointer to the memory location where the first byte of the file has been mapped. You can cast that pointer to whatever datatype you like, including char*.
In other words, it is Windows which decide where the mapped data will be in memory. You cannot provide a char* and expect Windows will load data there.
This means that if you already have a char* and want the data from the file in the location pointed by that char*, then you have to copy it. Not a good idea in terms of performances.
Here is a simple program dumping a text file by mapping the file into memory and then displaying all ASCII characters. Tested with MSVC2019.
#include <stdio.h>
#include <Windows.h>
int main(int argc, char *argv[])
{
TCHAR *lpFileName = TEXT("hello.txt");
HANDLE hFile;
HANDLE hMap;
LPVOID lpBasePtr;
LARGE_INTEGER liFileSize;
hFile = CreateFile(lpFileName,
GENERIC_READ, // dwDesiredAccess
0, // dwShareMode
NULL, // lpSecurityAttributes
OPEN_EXISTING, // dwCreationDisposition
FILE_ATTRIBUTE_NORMAL, // dwFlagsAndAttributes
0); // hTemplateFile
if (hFile == INVALID_HANDLE_VALUE) {
fprintf(stderr, "CreateFile failed with error %d\n", GetLastError());
return 1;
}
if (!GetFileSizeEx(hFile, &liFileSize)) {
fprintf(stderr, "GetFileSize failed with error %d\n", GetLastError());
CloseHandle(hFile);
return 1;
}
if (liFileSize.QuadPart == 0) {
fprintf(stderr, "File is empty\n");
CloseHandle(hFile);
return 1;
}
hMap = CreateFileMapping(
hFile,
NULL, // Mapping attributes
PAGE_READONLY, // Protection flags
0, // MaximumSizeHigh
0, // MaximumSizeLow
NULL); // Name
if (hMap == 0) {
fprintf(stderr, "CreateFileMapping failed with error %d\n", GetLastError());
CloseHandle(hFile);
return 1;
}
lpBasePtr = MapViewOfFile(
hMap,
FILE_MAP_READ, // dwDesiredAccess
0, // dwFileOffsetHigh
0, // dwFileOffsetLow
0); // dwNumberOfBytesToMap
if (lpBasePtr == NULL) {
fprintf(stderr, "MapViewOfFile failed with error %d\n", GetLastError());
CloseHandle(hMap);
CloseHandle(hFile);
return 1;
}
// Display file content as ASCII charaters
char *ptr = (char *)lpBasePtr;
LONGLONG i = liFileSize.QuadPart;
while (i-- > 0) {
fputc(*ptr++, stdout);
}
UnmapViewOfFile(lpBasePtr);
CloseHandle(hMap);
CloseHandle(hFile);
printf("\nDone\n");
}
Related
I have mapped a file of unknown size (around 4-6 GiB) in Windows platform and got a pointer to the start of the file data returned from the MapFileView function. But how can I know that I have reached the end of the file when I access the data using the pointer sequentially?
Here is the code I have so far written and it successfully maps the file and returns the pointer:
#include <Windows.h>
#include <stdio.h>
#include <inttypes.h>
int main()
{
HANDLE hFile = CreateFile("Test.bin",
GENERIC_READ | GENERIC_WRITE,
0,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL);
if (!hFile)
{
printf("Could not create file (%lu).\n", GetLastError());
exit(1) ;
}
HANDLE hMapFile = CreateFileMappingA(hFile,
NULL,
PAGE_READWRITE,
0,
0,
NULL);
if (!hMapFile)
{
printf("Could not create file mapping object (%lu).\n", GetLastError());
CloseHandle(hFile);
exit(1);
}
int32_t* pBuf = (int32_t*) MapViewOfFile(hMapFile,
FILE_MAP_ALL_ACCESS,
0,
0,
0);
if (!pBuf)
{
printf("Could not map file (%lu).\n", GetLastError());
CloseHandle(hFile);
CloseHandle(hMapFile);
exit(1);
};
UnmapViewOfFile(pBuf);
CloseHandle(hFile);
CloseHandle(hMapFile);
exit(0);
}
So I wanted to read equal sized different parts of the file simultaneously in multiple threads. I believe mapped file is the right choice for this purpose. Advice about any other faster and possible approaches is highly appreciated.
I have researched some similar questions in the forum and I suppose this is the closest topic I could find:
Read all contents of memory mapped file or Memory Mapped View Accessor without knowing the size of it
But this answer is using C# and is not written using the WinAPI, therefore, I couldn't understand their process.
Thanks in advance :)
Call GetFileSizeEx to get the size of a file, and use this in combination with the base address and the current read address to determine where the end address is.
I created a DLL injector program using C and a DLL. When I am trying to run the program the target process crash (I tried a notepad and cmd). I am compiling the injector as 64 bit and the DLL as well. the program and the DLL compiled with Visual Studio.
After some checks a saw if I remove the CreateRemoteThread The program will not crash, and the DLL injected (of course without executing the DLL).
I tried to use RUNDLL32.exe to check if the DLL is the problem and I have been able to see the message box.
Injector:
#include <Windows.h>
#include <stdio.h>
int main() {
// The DLL path we want to inject and the target process id.
LPCSTR dllpath = "C:\\Users\\....\\hello-world.dll";
int processID = 2980;
printf("#### Starting ####\n");
// Open target process handle
HANDLE hProcess = OpenProcess(PROCESS_ALL_ACCESS, FALSE, processID);
if (hProcess == NULL) {
//close handles
CloseHandle(hProcess);
printf("[!] Unable to find the target process id: %d\n" , processID);
return 1;
}
printf("[+] Open target process handle\n");
// Getting targt memory address for the dll path
LPVOID dllpathMemoryAddr = VirtualAllocEx(hProcess, NULL, strlen(dllpath) + 1, MEM_COMMIT | MEM_RESERVE, PAGE_EXECUTE_READWRITE);
if (dllpathMemoryAddr == NULL) {
//close handles
CloseHandle(hProcess);
printf("[!] Unable to get memory address of target process for the dllpath");
return 1;
}
printf("[+] Allocate the memory address to store the dllpath\n");
// Writing the dll path to the target memory address
BOOL succeedWrite = WriteProcessMemory(hProcess, dllpathMemoryAddr, dllpath, strlen(dllpath) + 1, NULL);
if (!succeedWrite) {
//close handles
CloseHandle(hProcess);
printf("[!] Unable to write to the memory address of target process the dllpath\n");
return 1;
}
printf("[+] Writed the dllpath to memory\n");
// Getting LoadLibreryA address
FARPROC loadLibAddr =
(GetModuleHandle(TEXT("kernel32.dll")), "LoadLibraryA");
if (loadLibAddr == NULL) {
// free the memory
VirtualFreeEx(hProcess, dllpathMemoryAddr, 0, MEM_RELEASE);
//close handles
CloseHandle(hProcess);
printf("[!] Unable to get the memory address of LoadLibraryA function\n");
return 1;
}
printf("[+] Allocate the memory address to LoadLibraryA function\n");
// Create remote thread on the remote process to load the dll
HANDLE rThread = CreateRemoteThread(hProcess, 0, 0, (LPTHREAD_START_ROUTINE)loadLibAddr, dllpathMemoryAddr, 0, NULL);
if (rThread == NULL) {
// free the memory
VirtualFreeEx(hProcess, dllpathMemoryAddr, 0, MEM_RELEASE);
//close handles
CloseHandle(hProcess);
printf("[!] Unable to create thread to execute the LoadLibraryA function\n the error: %u\n", GetLastError());
return 1;
}
printf("[+] Created remote thread to execute the dll\n");
// Waiting to opertion to complete
WaitForSingleObject(rThread, INFINITE);
// free the memory
VirtualFreeEx(hProcess, dllpathMemoryAddr, 0, MEM_RELEASE);
//close handles
CloseHandle(hProcess);
CloseHandle(rThread);
printf("#### DLL INJECTED ####\n");
return TRUE;
}
DLL
#define WIN32_LEAN_AND_MEAN
#include <stdio.h>
#include <windows.h>
extern "C" __declspec(dllexport)
BOOL APIENTRY DllMain(HMODULE hModule,
DWORD ul_reason_for_call,
LPVOID lpReserved) {
switch (ul_reason_for_call) {
case DLL_PROCESS_ATTACH:
MessageBox(NULL, "Hello world!", "Hello World!", NULL);
break;
case DLL_THREAD_ATTACH:
case DLL_THREAD_DETACH:
case DLL_PROCESS_DETACH:
break;
}
return TRUE;
}
output:
#### Starting ####
[+] Open target process handle
[+] Allocate the memory address to store the dllpath
[+] Writed the dllpath to memory
[+] Allocate the memory address to LoadLibraryA function
[+] Created remote thread to execute the dll
#### DLL INJECTED ####
If I understand the comments of the OP correctly, the main problem was that the OP had originally used the following line (before editing the question):
BOOL succeedWrite = WriteProcessMemory(hProcess, dllpathMemoryAddr, dllpath, strlen(dllpath), NULL);
By writing strlen(dllpath) instead of strlen(dllpath) + 1 bytes, the OP did not write the terminating null character of the string into the remote process. Therefore, the remote process likely crashed when that non-terminated string was passed as a function parameter to LoadLibraryA.
As pointed out in the comments section, there were also some other problems with the code, but it is unlikely that these were the cause of the crash.
I have 2 processes, 1 process needs to write to a shared memory segment, the other process needs to read from the shared memory segment. The writing process only writes to shared mem, the reading process only reads.
Writing process
HANDLE hMapFile;
LPCTSTR pBuf;
char* Message = "test";
int MessageSize = 5;
hMapFile = CreateFileMappingA(INVALID_HANDLE_VALUE, NULL, PAGE_EXECUTE_READWRITE, 0, Size, SegmentName);
if (hMapFile == NULL)
{
printf("Could not create file mapping object (%d).\n", GetLastError());
return NULL;
}
pBuf = (LPTSTR)MapViewOfFile(hMapFile, FILE_MAP_ALL_ACCESS, 0, 0, Size);
if (pBuf == NULL)
{
printf("Could not map view of file (%d).\n", GetLastError());
//CloseHandle(hMapFile);
return NULL;
}
CopyMemory((PVOID)pBuf, Message, MessageSize);
Shortly after my reading process will read the message like so
HANDLE hMapFile;
LPCTSTR pBuf;
hMapFile = OpenFileMapping(FILE_MAP_ALL_ACCESS, FALSE, SegmentName);
if (hMapFile == NULL)
{
printf("Could not open file mapping object MINER PROC (%d).\n", GetLastError());
exit(0);
//return RetVal;
}
RetVal = (LPTSTR)MapViewOfFile(hMapFile, FILE_MAP_ALL_ACCESS, 0, 0, Size);
if (RetVal == NULL)
{
printf("Could not map view of file (%d).\n", GetLastError());
CloseHandle(hMapFile);
exit(0);
//return RetVal;
}
UnmapViewOfFile(hMapFile);
CloseHandle(hMapFile);
This works perfectly for a 1 time write then read. I need the writing process to be able to change the values.
I've tried this in the writing process
memset((PVOID)pBuf, '\0', strlen(OriginalMessage));
CopyMemory(pBuf, "newmessage",
strlen("newmessage") + 1);
But this gives me random crashes in the writing process, although the reading process seems to be able to read the changes fine.
I've also tried to close the segment like this from the writing process immediately after CopyMemory() call
UnmapViewOfFile(pBuf);
CloseHandle(hMapFile);
But then the memory disappears and the reading process cannot read the segment name. I also tried calling UnmapViewOfFile() & CloseHandle() then creating a new memory segment with the same name and I keep getting random crashes after that.
What is the proper way to change a shared memory value of a given shared memory segment name without crashes? Thanks.
EDIT - I'd like to add that I tested this code using the described methods (creating new shared memory segments) on 4 Win10 machine, 2 Win7 machines and I got no crashes. The crashes only occur on a Win8.1 machine. it's quite odd.
EDIT2 - also, all functions are successful in both the reading and writing processes when the crash occurs.
I am new in programming, especially in Windows System Programming and I 'm reading a relevant book. Currently I 'm playing arround with GetFileSizeEx, SetFilePointer and SetFilePointerEx in order to get the file size of a file.
I have created this code that works until line 65 where I can't get SetFilePointerEx to work to get the size.
#include <Windows.h>
#include <tchar.h>
#include <stdio.h>
#define BUFF_SIZE 0x100
// program to test file size
int _tmain(DWORD argc, LPTSTR argv[])
{
HANDLE hIn;
HANDLE hOut;
LARGE_INTEGER liSize;
LONG lSize, lDistance = 0;
TCHAR szMsgGetFile[BUFF_SIZE];
TCHAR szMsgSetFile[BUFF_SIZE];
DWORD nIn;
LARGE_INTEGER liPt;
PLARGE_INTEGER pLi;
pLi = &liPt;
SecureZeroMemory(&liSize, sizeof(LARGE_INTEGER));
SecureZeroMemory(&pLi, sizeof(LARGE_INTEGER));
SecureZeroMemory(szMsgGetFile, _tcslen(szMsgGetFile));
SecureZeroMemory(szMsgSetFile, _tcslen(szMsgSetFile));
//get input and output handles
hIn = CreateFile(argv[1], GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
if (hIn == INVALID_HANDLE_VALUE)
_tprintf(_T("[ERROR] CreateFile to get file input handle failed. Error code %d.\n"), GetLastError());
hOut = CreateFile(_T("CONOUT$"), GENERIC_WRITE, 0, NULL, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
if (hOut == INVALID_HANDLE_VALUE)
_tprintf(_T("[ERROR] CreateFile to get file output handle failed. Error code %d.\n"), GetLastError());
//get the size of the file with GetFileSizeEx, acquired from hIn that is argv1
if (!GetFileSizeEx(hIn, &liSize))
_tprintf(_T("[ERROR] GetFileSizeEx failed. Error code %d\n"), GetLastError());
//get the size of the file with SetFilePointer
//You can obtain the file length by specifying a zero-length move from the end of
//file, although the file pointer is changed as a side effect
lSize = SetFilePointer(hIn, lDistance, NULL, FILE_END);
if (lSize == INVALID_SET_FILE_POINTER)
_tprintf(_T("[ERROR] SetFilePointer failed. Error code %d\n"), GetLastError());
//output the size with WriteConsole (and sprintf)
//and with _tprintf. Notice the usage of the liSize LARGE_INTEGER
_stprintf_s(szMsgGetFile, BUFF_SIZE, "[*] GetFileSizeEx (WriteConsole): The size is %I64d Bytes.\n", liSize.QuadPart);
if (!WriteConsole(hOut, szMsgGetFile, _tcslen(szMsgGetFile), &nIn, NULL))
_tprintf(_T("[ERROR] WriteConsole failed. Error code %d\n"), GetLastError());
_tprintf(_T("[*] GetFileSizeEx (tprintf): The size is %I64d Bytes.\n"), liSize.QuadPart);
//output the size with WriteConsole (and sprintf)
//and _tprintf
_stprintf_s(szMsgSetFile, BUFF_SIZE, "[*] SetFilePointer (WriteConsole): The size is %ld Bytes.\n", lSize);
if (!WriteConsole(hOut, szMsgSetFile, _tcslen(szMsgSetFile), &nIn, NULL))
_tprintf(_T("[ERROR] WriteConsole failed. Error code %d\n"), GetLastError());
_tprintf(_T("[*] SetFilePointer (tprintf): The size is %ld Bytes.\n"), lSize);
//get the size of the file with SetFilePointerEx
//Determine a file’s size by positioning 0 bytes from the end and using the file
//pointer value returned by SetFilePointerEx.
SecureZeroMemory(&liPt, sizeof(LARGE_INTEGER));
SetFilePointerEx(hIn, liPt, pLi, FILE_END);
_tprintf(_T("[*] SetFilePointerEx: %lld Bytes.\n"), pLi->QuadPart);
return 0;
}
MSDN says that
You can use SetFilePointerEx to determine the length of a file. To do this, use FILE_END for dwMoveMethod and seek to location zero. The file offset returned is the length of the file.
However, SetFilePointerEx is of type BOOL. The "Windows System Programming" book says that the "Determine a file’s size by positioning 0 bytes from the end and using the file pointer value returned by SetFilePointerEx.". I am guessing that this parameter is the _Out_opt_ PLARGE_INTEGER lpNewFilePointer according to MSDN.
I would like help on how to get the file size of the file by using SetFilePointerEx.
You have a number of errors in your code. Here's an example of SetFilePointerEx that works. In general, Win32 functions don't allocate the memory to store their output (some do). It's up to the caller to allocate memory. In this case, the memory for the output of SetFilePointerEx is allocated on the stack by declaring size2 to be a LARGE_INTEGER. A pointer to that LARGE_INTEGER is then provided to SetFilePointerEx.
auto hIn = CreateFile(_T("C:\\foo"), GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, nullptr);
/* check for errors */
LARGE_INTEGER size;
GetFileSizeEx(hIn, &size);
/* check for errors */
LARGE_INTEGER size2;
LARGE_INTEGER offset;
ZeroMemory(&offset, sizeof offset);
SetFilePointerEx(hIn, offset, &size2, FILE_END);
/* check for errors */
Optionally, DWORD dwFileSize = GetFileSize(hFile, NULL); can get the size of the file opened by HANDLE hFile = CreateFileA/W(...);.
I'm trying to get to work ReadFile function. Here's my code:
#define BUFFERSIZE 5
int main(int argc, char* argv[])
{
OVERLAPPED overlapIn = {};
HANDLE tHandle;
char buf[BUFFERSIZE] = {};
DWORD lpNumberOfBytesRead;
tHandle = CreateFile(
L"\\\\.\\D:",
GENERIC_READ,
FILE_SHARE_READ,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL);
if (tHandle == INVALID_HANDLE_VALUE)
{
DWORD error = GetLastError();
assert(0);
}
if (ReadFile(tHandle, &buf, BUFFERSIZE - 1, &lpNumberOfBytesRead, NULL) == 0)
{
int error = GetLastError();
printf("Terminal failure: Unable to read from disk.\n GetLastError=%d\n", error);
CloseHandle(tHandle);
return 1;
}
The GetLastError function returns code 87, which is ERROR_INVALID_PARAMETER.
It's clear that one of the parameters is wrong, but I have no idea which one, since I tried to do everything like it's written in the documentation.
This is described in the documentation for CreateFile:
Volume handles can be opened as noncached at the discretion of the particular file system, even when the noncached option is not specified in CreateFile. You should assume that all Microsoft file systems open volume handles as noncached.
The MSDN article on File Buffering describes the requirements for noncached handles:
File access sizes, including the optional file offset in the OVERLAPPED structure, if specified, must be for a number of bytes that is an integer multiple of the volume sector size. For example, if the sector size is 512 bytes, an application can request reads and writes of 512, 1,024, 1,536, or 2,048 bytes, but not of 335, 981, or 7,171 bytes.
File access buffer addresses for read and write operations should be physical sector-aligned, which means aligned on addresses in memory that are integer multiples of the volume's physical sector size. Depending on the disk, this requirement may not be enforced.
Rigorous code should check the sector size for the file system in question, then use this approach to allocate the memory. However, in my experience, the sector size has always been less than or equal to the allocation granularity, so you can get away with just using VirtualAlloc() to allocate a memory block.
buffer size needs to be aligned with hdd sectorsize
WIN32_FIND_DATA atr = {0};
DWORD BYTES_PER_SECTOR;
char path[MAX_PATH];
/* get path length current dir */
const size_t len = GetCurrentDirectory(0, 0);
/* set path to path char array */
GetCurrentDirectory(len, path);
/* windows function to get disk details */
GetDiskFreeSpace(NULL, NULL, &BYTES_PER_SECTOR, NULL, NULL);
/* find first file in dir */
find = FindFirstFile(path, &atr);
for(;find != INVALID_HANDLE_VALUE;){
/* get the file size */
DWORD filesize = atr.nFileSizeLow;
if(atr.nFileSizeHigh > 0){
filesize = atr.nFileSizeHigh;
filesize = (filesize << 31);
filesize = atr.nFileSizeLow;
}
/* sector size aligned file size */
size_t buffer_size = ((BYTES_PER_SECTOR + ((filesize + BYTES_PER_SECTOR)-1)) & ~(BYTES_PER_SECTOR -1));
/* create buffer */
DWORD buffer[buffer_size];
/* create a new file or open an existing file */
handle = CreateFile(&path[0], GENERIC_READ | GENERIC_WRITE, 0 , NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL | FILE_FLAG_NO_BUFFERING, NULL))!=INVALID_HANDLE_VALUE)
/* read the file in to buffer */
ReadFile(handle, (void*)&buffer, buffer_size, &bytesread, NULL)
if(FindNextFile(find, &atr)==0){ printf("last file processed, leaving\n");break;};
}
CloseHandle(file);
FindClose(find);