Just trying to experiment with reflective DLL loading, so I wrote a simple messagebox:
package main
import "C"
import (
"unsafe"
"syscall"
)
//export OnProcessAttach
func OnProcessAttach() {
const (
NULL = 0
MB_OK = 0
)
caption := "Hola"
title := "desdegoo"
ret, _, _ := syscall.NewLazyDLL("user32.dll").NewProc("MessageBoxW").Call(
uintptr(NULL),
uintptr(unsafe.Pointer(syscall.StringToUTF16Ptr(caption))),
uintptr(unsafe.Pointer(syscall.StringToUTF16Ptr(title))),
uintptr(MB_OK))
if ret != 1 {
return
}
return
}
func main() {}
I generated a dll (just a messagebox with cgo/golang), using the following command
go build --buildmode=c-shared main.go
When loading the dll with LoadLibrary() and running the exported function OnProcessAttach it works (pops the message box), but when trying to achieve DLL reflective loading, by resolving the relocations and resolving the IAT it just don't work.
Seems like performing the base relocations and the IAT sets to null sections on .rdata that are used for initializing the go runtime (which initializes in the entrypoint from NT headers)
This is the piece of code I'm using for resolving the imports:
// resolve base relocations
IMAGE_DATA_DIRECTORY relocations = ntHeaders->OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_BASERELOC];
DWORD_PTR relocationTable = relocations.VirtualAddress + (DWORD_PTR)dllBase;
DWORD relocationsProcessed = 0;
while (relocationsProcessed < relocations.Size)
{
PBASE_RELOCATION_BLOCK relocationBlock = (PBASE_RELOCATION_BLOCK)(relocationTable + relocationsProcessed);
relocationsProcessed += sizeof(BASE_RELOCATION_BLOCK);
DWORD relocationsCount = (relocationBlock->BlockSize - sizeof(BASE_RELOCATION_BLOCK)) / sizeof(BASE_RELOCATION_ENTRY);
PBASE_RELOCATION_ENTRY relocationEntries = (PBASE_RELOCATION_ENTRY)(relocationTable + relocationsProcessed);
for (DWORD i = 0; i < relocationsCount; i++)
{
relocationsProcessed += sizeof(BASE_RELOCATION_ENTRY);
if (relocationEntries[i].Type == 0)
{
continue;
}
DWORD_PTR relocationRVA = relocationBlock->PageAddress + relocationEntries[i].Offset;
DWORD_PTR addressToPatch = 0;
ReadProcessMemory(GetCurrentProcess(), (LPCVOID)((DWORD_PTR)dllBase, relocationRVA), &addressToPatch, sizeof(DWORD_PTR), NULL);
addressToPatch += deltaImageBase;
memcpy((PVOID)((DWORD_PTR)dllBase + relocationRVA), &addressToPatch, sizeof(DWORD_PTR));
}
}
// resolve IAT
PIMAGE_IMPORT_DESCRIPTOR importDescriptor = NULL;
IMAGE_DATA_DIRECTORY importsDirectory = ntHeaders->OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_IMPORT];
importDescriptor = (PIMAGE_IMPORT_DESCRIPTOR)(importsDirectory.VirtualAddress + (DWORD_PTR)dllBase);
LPCSTR libraryName = "";
HMODULE library = NULL;
while (importDescriptor->Name != NULL)
{
libraryName = (LPCSTR)importDescriptor->Name + (DWORD_PTR)dllBase;
library = LoadLibraryA(libraryName);
if (library)
{
PIMAGE_THUNK_DATA thunk = NULL;
thunk = (PIMAGE_THUNK_DATA)((DWORD_PTR)dllBase + importDescriptor->FirstThunk);
while (thunk->u1.AddressOfData != NULL)
{
if (IMAGE_SNAP_BY_ORDINAL(thunk->u1.Ordinal))
{
LPCSTR functionOrdinal = (LPCSTR)IMAGE_ORDINAL(thunk->u1.Ordinal);
thunk->u1.Function = (DWORD_PTR)GetProcAddress(library, functionOrdinal);
}
else {
PIMAGE_IMPORT_BY_NAME functionName = (PIMAGE_IMPORT_BY_NAME)((DWORD_PTR)dllBase + thunk->u1.AddressOfData);
DWORD_PTR functionAddress = (DWORD_PTR)GetProcAddress(library, functionName->Name);
thunk->u1.Function = functionAddress;
}
++thunk;
}
}
importDescriptor++;
}
After doing so I resolve the EAT looking for the OnProcessAttach function, running it directly doesn't work obviously because the go runtime isn't initialized, but trying to initialize it crashes the program because the above stated. It gives a EXCEPTION_ACCESS_VIOLATION because tries to read a chunk of bytes that gets nullified.
Dissasembly of the entrypoint:
mov rax, qdword ptr ds:[address]
mov dword ptr ds:[rax]
Following the address in dump it looks null
00 00 00 00 00 00 00 00 00 [..]
While the original dll has indeed values
90 2B C5 EA 01 [...]
I know I'm setting to null those bytes on .rdata, but can't figure out why that's happening when I perform the relocations, maybe the Go runtime doesn't fit for what I'm trying to do? Or is something else?
Related
File hierarchy:
src:
main.c
main.html
main.js
out:
main.wasm -> compiled file
main.c:
#define WASM_EXPORT __attribute__((visibility("default")))
WASM_EXPORT
float power(float number,float power){
float res = number;
for (int i = 0; i < power - 1 ; i++) {
res = res * number;
}
return res;
}
main.js:
fetch('../out/main.wasm').then(response =>
response.arrayBuffer()
).then(bytes =>
WebAssembly.instantiate(bytes)
).then(results => {
instance = results.instance;
document.getElementById("container").textContent = instance.exports.power(2,8);
}).catch(console.error);
error:
TypeError: instance.exports.power is not a function
Just started wasm & can't figure out why the power function is not visible
How are you compiling your C to wasm BTW?
In general its not enough to simply set the visibility. You either need to pass --export=power to the linker (most likely via -Wl,--export=power passed to the compiler), or set the 'export_name' attribute: __attribute__((export_name("power")))
I've been stuck on a problem for some time now and I'm not getting anywhere. May one of you have mercy and help me.
I have transcribed some header files with c2nim. Debugging and many methods seem to work fine, but there is one section where I just can't get anywhere. The section I want to map looks like
C
typedef struct AVFormatContext {
...
AVStream **streams;
...
}
...
AVFormatContext *pFormatCtxInCam = NULL;
...
pFormatCtxInCam = avformat_alloc_context();
ret = avformat_open_input(&pFormatCtxInCam, "video=Venus USB2.0 Camera", inFrmt, &inOptions);
for (i = 0; i < pFormatCtxInCam->nb_streams; i++)
if (pFormatCtxInCam->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO)
video_stream_idx_cam = i;
Nim
type AVFormatContext* {.bycopy.} = object
...
streams*: ptr ptr AVStream
...
var pFormatCtxInCam: ptr AVCodecContext = nil
pFormatCtxInCam = avformat_alloc_context()
ret = avformat_open_input(pFormatCtxInCam.addr, "video=Venus USB2.0 Camera", inFrmt, inOptions.addr)
-> Problem
-> How can I access Streams as array ???
I can understand the basics in C and most of the memory layout, but here I reach my limits.
I try to cast in some ways.
Examples:
var streams = cast[array[0..0, ptr AVStream]](pFormatCtxInCam[].streams[].addr)
var streams = cast[array[0..0, AVStream]](pFormatCtxInCam[].streams[].addr)
var streams = cast[ref array[0..0, ptr AVStream]](pFormatCtxInCam[].streams)
var streams = cast[array[0..5, AVStream]](pFormatCtxInCam[].streams)
var streams = cast[ptr UncheckedArray[ptr AVStream]](pFormatCtxInCam[].streams)
var streams = cast[ptr UncheckedArray[ptr AVStream]](pFormatCtxInCam[].streams[])
var streams = cast[ptr UncheckedArray[ptr AVStream]](pFormatCtxInCam[].streams[][])
As you can see, I've tried a lot of things because I just don't know what to do...
-> How can I access ptr ptr in nim as an array?
Thx for help
var streams = cast[ptr UncheckedArray[ptr AVStream]](pFormatContext[].streams)
That seems to be the RIGHT answer. It is a pointer to an array of pointers to the elements.
I'm trying to remap large [UInt8] into [UInt16] and so far my solution works like this:
//Function that converts [UInt8] into [UInt16]
func foo(arr: [UInt8])->[UInt16]{
//Split into even and odds
let even = stride(from: 0, to: arr.count, by: 2).map { arr[$0] }
let odd = stride(from: 1, to: arr.count, by: 2).map { arr[$0] }
//pair each even with the next odd
let paired=Array(zip(even, odd))
//reduce UInt8 pairs to UInt16
return paired.reduce([UInt16]()) { (acc, curr) -> [UInt16] in
let u16 = UnsafePointer([curr.0, curr.1]).withMemoryRebound(to: UInt16.self, capacity: 1) {
$0.pointee
}
var newV = acc
newV.append(u16)
return newV
}
}
The above works but its very inefficient. The reduce function is where most of the computation time takes place.
I was wondering if it would be possible to use the UnsafePointer.withMemoryRebound remapping directly.
I tried:
let test : [UInt16] = UnsafePointer([UInt8(0), UInt8(1),UInt8(0), UInt8(1)]).withMemoryRebound(to: [UInt16].self, capacity: 2) {
$0.pointee
}
resulting in:
Execution interrupted. Enter code to recover and continue.
Enter LLDB commands to investigate (type :help for assistance.)
Process 57635 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
frame #0: 0x00007fff7d038d5f libobjc.A.dylib`objc_retain + 15
libobjc.A.dylib`objc_retain:
-> 0x7fff7d038d5f <+15>: movq (%rdi), %rax
0x7fff7d038d62 <+18>: movabsq $0x7ffffffffff8, %rcx ; imm = 0x7FFFFFFFFFF8
0x7fff7d038d6c <+28>: andq %rax, %rcx
0x7fff7d038d6f <+31>: testb $0x2, 0x20(%rcx)
Target 0: (repl_swift) stopped.
Maybe I misunderstood how it supposed to work. Can it be done? Is there a better way to do it?
My suggestion is to create a Data instance and remap the bytes with withUnsafeBytes
let arr : [UInt8] = [0x31, 0x32, 0x33, 0x34]
let data = Data(arr)
let u16 = data.withUnsafeBytes {
[UInt16](UnsafeBufferPointer(start: $0, count: data.count/MemoryLayout<UInt16>.stride))
} // [12849, 13363]
Access to the array's element storage is done with the withUnsafeBytes() method, which calls the closure with an UnsafeRawBufferPointer. This pointer can now be bound to a buffer pointer of the desired type UInt16. Finally the new array is created – that is the only place where data is copied.
let uint8Array: [UInt8] = [1, 2, 3, 4]
let uint16Array = uint8Array.withUnsafeBytes { ptr in
Array(ptr.bindMemory(to: UInt16.self))
}
print(uint16Array) // [513, 1027]
I am trying to convert below code to Swift:
{
// Set up the variables
double totalUsedMemory = 0.00;
mach_port_t host_port;
mach_msg_type_number_t host_size;
vm_size_t pagesize;
// Get the variable values
host_port = mach_host_self();
host_size = sizeof(vm_statistics_data_t) / sizeof(integer_t);
host_page_size(host_port, &pagesize);
vm_statistics_data_t vm_stat;
// Check for any system errors
if (host_statistics(host_port, HOST_VM_INFO, (host_info_t)&vm_stat, &host_size) != KERN_SUCCESS) {
// Error, failed to get Virtual memory info
return -1;
}
// Memory statistics in bytes
natural_t usedMemory = (natural_t)((vm_stat.active_count +
vm_stat.inactive_count +
vm_stat.wire_count) * pagesize);
natural_t allMemory = [self totalMemory];
return usedMemory;
}
My Swift code is:
{
// Set up the variables
var totalUsedMemory: Double = 0.00
var host_port: mach_port_t
var host_size: mach_msg_type_number_t
var pagesize:vm_size_t
// Get the variable values
host_port = mach_host_self()
host_size = mach_msg_type_number_t(MemoryLayout<vm_statistics_data_t>.stride / MemoryLayout<integer_t>.stride)
// host_size = sizeof(vm_statistics_data_t) / sizeof(integer_t);
host_page_size(host_port, &pagesize);
var vm_stat: vm_statistics_data_t ;
// Check for any system errors
if (host_statistics(host_port, HOST_VM_INFO, (host_info_t)&vm_stat, &host_size) != KERN_SUCCESS) {
// Error, failed to get Virtual memory info
return -1;
}
// Memory statistics in bytes
var usedMemory: Int64 = (Int64)((vm_stat.active_count + vm_stat.inactive_count + vm_stat.wire_count) * pagesize);
return usedMemory;
}
I am getting these 2 errors:
**Binary operator '&' cannot be applied to operands of type '(host_info_t).Type' (aka 'UnsafeMutablePointer.Type') and 'vm_statistics_data_t' (aka 'vm_statistics')
in this statement
host_statistics(host_port, HOST_VM_INFO, (host_info_t)&vm_stat, &host_size)
And
Binary operator '*' cannot be applied to operands of type 'UInt32' and 'vm_size_t' (aka 'UInt')**
in this statement -
var usedMemory: Int64 = (Int64)((vm_stat.active_count + vm_stat.inactive_count + vm_stat.wire_count) * pagesize);
Swift is a lot more strict about pointer types than C is, which can make it a real pain to interact with functions like this that expect you to pass pointers to types other than the actual type of the thing you're trying to pass to the function. So I agree with the commenters that you're probably better off leaving this function in (Objective-)C. However, if you absolutely have to convert to Swift, you're probably going to have to do something like this:
// Initialize a blank vm_statistics_data_t
var vm_stat = vm_statistics_data_t()
// Get a raw pointer to vm_stat
let err: kern_return_t = withUnsafeMutableBytes(of: &vm_stat) {
// Bind the raw buffer to Int32, since that's what host_statistics
// seems to want a pointer to.
let boundBuffer = $0.bindMemory(to: Int32.self)
// Call host_statistics, and return its status out of the closure.
return host_statistics(host_port, HOST_VM_INFO, boundBuffer.baseAddress, &host_size)
}
// Now take a look at what we got and compare it against KERN_SUCCESS
if err != KERN_SUCCESS {
// Error, failed to get Virtual memory info
return -1;
}
I have a third-party C library I am using to write an R extension. I am required to create a few structs defined in the library (and initialize them) I need to maintain them as part of an S4 object (think of these structs as defining to state of a computation, to destroy them would be to destroy all remaining computation and the results of all that has been already computed).
I am thinking of creating a S4 object to hold pointers these structs as void* pointers but it is not at all clear how to do so, what would be the type of the slot?
As pointed out by #hrbrmstr, you can use the externalptr type to keep such objects "alive", which is touched on in this section of Writing R Extensions, although I don't see any reason why you will need to store anything as void*. If you don't have any issue with using a little C++, the Rcpp class XPtr can eliminate a fair amount of the boilerplate involved with managing EXTPTRSXPs. As an example, assume the following simplified example represents your third party library's API:
#include <Rcpp.h>
#include <stdlib.h>
typedef struct {
unsigned int count;
double total;
} CStruct;
CStruct* init_CStruct() {
return (CStruct*)::malloc(sizeof(CStruct));
}
void free_CStruct(CStruct* ptr) {
::free(ptr);
::printf("free_CStruct called.\n");
}
typedef Rcpp::XPtr<CStruct, Rcpp::PreserveStorage, free_CStruct> xptr_t;
When working with pointers created via new it is generally sufficient to use Rcpp::XPtr<SomeClass>, because the default finalizer simply calls delete on the held object. However, since you are dealing with a C API, we have to supply the (default) template parameter Rcpp::PreserveStorage, and more importantly, the appropriate finalizer (free_CStruct in this example) so that the XPtr does not call delete on memory allocated via malloc, etc., when the corresponding R object is garbage collected.
Continuing with the example, assume you write the following functions to interact with your CStruct:
// [[Rcpp::export]]
xptr_t MakeCStruct() {
CStruct* ptr = init_CStruct();
ptr->count = 0;
ptr->total = 0;
return xptr_t(ptr, true);
}
// [[Rcpp::export]]
void UpdateCStruct(xptr_t ptr, SEXP x) {
if (TYPEOF(x) == REALSXP) {
R_xlen_t i = 0, sz = XLENGTH(x);
for ( ; i < sz; i++) {
if (!ISNA(REAL(x)[i])) {
ptr->count++;
ptr->total += REAL(x)[i];
}
}
return;
}
if (TYPEOF(x) == INTSXP) {
R_xlen_t i = 0, sz = XLENGTH(x);
for ( ; i < sz; i++) {
if (!ISNA(INTEGER(x)[i])) {
ptr->count++;
ptr->total += INTEGER(x)[i];
}
}
return;
}
Rf_warning("Invalid SEXPTYPE.\n");
}
// [[Rcpp::export]]
void SummarizeCStruct(xptr_t ptr) {
::printf(
"count: %d\ntotal: %f\naverage: %f\n",
ptr->count, ptr->total,
ptr->count > 0 ? ptr->total / ptr->count : 0
);
}
// [[Rcpp::export]]
int GetCStructCount(xptr_t ptr) {
return ptr->count;
}
// [[Rcpp::export]]
double GetCStructTotal(xptr_t ptr) {
return ptr->total;
}
// [[Rcpp::export]]
void ResetCStruct(xptr_t ptr) {
ptr->count = 0;
ptr->total = 0.0;
}
At this point, you have done enough to start handling CStructs from R:
ptr <- MakeCStruct() will initialize a CStruct and store it as an externalptr in R
UpdateCStruct(ptr, x) will modify the data stored in the CStruct, SummarizeCStruct(ptr) will print a summary, etc.
rm(ptr); gc() will remove the ptr object and force the garbage collector to run, thus calling free_CStruct(ptr) and destroying the object on the C side of things as well
You mentioned the use of S4 classes, which is one option for containing all of these functions in a single place. Here's one possibility:
setClass(
"CStruct",
slots = c(
ptr = "externalptr",
update = "function",
summarize = "function",
get_count = "function",
get_total = "function",
reset = "function"
)
)
setMethod(
"initialize",
"CStruct",
function(.Object) {
.Object#ptr <- MakeCStruct()
.Object#update <- function(x) {
UpdateCStruct(.Object#ptr, x)
}
.Object#summarize <- function() {
SummarizeCStruct(.Object#ptr)
}
.Object#get_count <- function() {
GetCStructCount(.Object#ptr)
}
.Object#get_total <- function() {
GetCStructTotal(.Object#ptr)
}
.Object#reset <- function() {
ResetCStruct(.Object#ptr)
}
.Object
}
)
Then, we can work with the CStructs like this:
ptr <- new("CStruct")
ptr#summarize()
# count: 0
# total: 0.000000
# average: 0.000000
set.seed(123)
ptr#update(rnorm(100))
ptr#summarize()
# count: 100
# total: 9.040591
# average: 0.090406
ptr#update(rnorm(100))
ptr#summarize()
# count: 200
# total: -1.714089
# average: -0.008570
ptr#reset()
ptr#summarize()
# count: 0
# total: 0.000000
# average: 0.000000
rm(ptr); gc()
# free_CStruct called.
# used (Mb) gc trigger (Mb) max used (Mb)
# Ncells 484713 25.9 940480 50.3 601634 32.2
# Vcells 934299 7.2 1650153 12.6 1308457 10.0
Of course, another option is to use Rcpp Modules, which more or less take care of the class definition boilerplate on the R side (using reference classes rather than S4 classes, however).