How can I link my dmd-compiled program with a specific version of the druntime?
I need this when experimenting with a new GC. GC's are defined in druntime.
See also: https://forum.dlang.org/post/dahmztupviwgazbzevps#forum.dlang.org
Related
Is there a way to produce a C static library from Go code, but without Go runtime function definitions?
Rationale:
Project A creates a C static library with go build -buildmode=c-archive, libA.a .
Works well: project B uses pure C and is able to easily create an executable, statically linking with libA.a, all is fine.
Problem 1: project C happens to also use Go, but would like to use libA.a as a regular C library. Now it has a link problem: the Go runtime functions such as e.g. _cgo_panic are now defined both in project C runtime (as it uses Go) and in libA.a.
Problem 2: project D uses pure C, same as B. Yet it wants to use two different libraries from project A, e.g. libA.a and some libA2.a. Sadly, it does not link either, because Go runtime functions are now defined in both libA.a and libA2.a.
The problems faced by project C and project D could be easily resolved if project A could produce its libraries without the Go runtime definitions inside. Project C could just link with libA.a. Project D would link with libA.a, libA2.a and some libGo.a that would contain definitions of all the Go runtime stuff.
What I tried:
Using linker flags at 'project C' level, such as -Wl,--allow-multiple-definition. Now its build fails with a cryptic message 'function symbol table not sorted by program counter'.
Manually removing go.o from 'libA.a' (as it's just an "ar" archive): didn't work as 'go.o' also contained implementations of my exported functions, so I've removed too much.
Using go build -buildmode=c-shared. As expected, it produces a dynamic library which uses another format, so I could not directly use it as a static library.
Any solution at the client side (such as finding a proper way to ignore duplicate definitions at the link stage for project C) would be also considered a valid answer.
I can also accept a negative answer (no solution) if it provides enough evidence.
Update: see a related question Is there a way to include multiple c-archive packages in a single binary
With the current implementation it's not going to work to use -buildmode=c-archive multiple times and put the results into multiple shared libraries, as you've discovered. The essential problem is that there has to be only one Go runtime, but you have multiple runtimes. When using -buildmode=c-archive there's no way to isolate the different runtimes.
The -buildmode=c-shared libraries differ from buildmode=c-archive in that they are built with -Bsymbolic which forces all their local references to be local. The effect is that we have multiple Go runtimes, but they don't refer to each other so there is no confusion.
You could try adding -Wl,-Bsymbolic to build each shared library that includes Go code in c-archive if your C code doesn't mind being linked with -Bsymbolic.
I wish you luck.
Recently I've got interested in learning the heap management of C (the malloc module). I want to break the malloc source files(e.g., malloc.c, arena.c) into smaller files so it's easier to read and study for me. I'm using the glibc 2.23 and have successfully built it locally (in a separate "build" folder) on Ubuntu 14.04 following the instructions on wiki.
As my initial attempt, I put the __malloc_assert into the files massert.h and massert.c but then realized I have no idea how to add them to the makefiles so they can get compiled and linked.
Since I moved __malloc_assert out of malloc.c, I got the link errors when running make again, which was expected:
/home/mvs/git/glibc/build/libc_pic.os: In function `detach_arena':
/home/mvs/git/glibc/malloc/arena.c:629: undefined reference to `__malloc_assert'
/home/mvs/git/glibc/build/libc_pic.os: In function `mremap_chunk':
/home/mvs/git/glibc/malloc/malloc.c:2832: undefined reference to `__malloc_assert'
/home/mvs/git/glibc/malloc/malloc.c:2813: undefined reference to `__malloc_assert'
/home/mvs/git/glibc/malloc/malloc.c:2812: undefined reference to `__malloc_assert'
/home/mvs/git/glibc/malloc/malloc.c:2830: undefined reference to `__malloc_assert'
/home/mvs/git/glibc/build/libc_pic.os:/home/mvs/git/glibc/malloc/malloc.c:2776: more undefined references to `__malloc_assert' follow
I thought I should look at how the malloc/malloc.c is used in the makefiles, but I couldn't find where it is used. I'm mainly looking at the following files:
glibc/Makeconfig
glibc/Makefile
glibc/Rules
glibc/malloc/Makefile
Alternatively, I've searched the makefile on libc-help mailing list and looked at all the results but didn't find one that matches what I want. Two of the threads, "glibc + add new function" and "Adding a function to glibc?", were talking about adding a new function to the library, which is not my case (I'm not adding new function but merely restructuring the code).
I'm kind of new to the makefile system and still reading the GNU makefile manual, but thought shooting an email here may get me out of the struggle more quickly.
Thanks!
You need to add massert (not massert.c) to the routines variable in malloc/Makefile.
There are several such variables: routines is for libc it self, but there is also libm-routines for libm, and so on.
By default, these source files are built for all variants: static (.o), shared (.os), profiling (.op, profiling builds are disabled by default though). Some special functions are only intended for static builds (.oS, they go into libc_nonshared.a) and are listed in static-only-routines as well. Specific build targets can be excluded using the elide-routines.os variable.
I got to know of a way to print the source code of a running code in C using the __FILE__ macro. As such I can seek the location and use putchar() to alter the contents of the file.
Is it possible to dynamically change the running code using this method?
Is it possible to dynamically change the running code using this method ?
No, because once a program is compiled it no longer depends on the source file.
If you want learn how to alter the behavior of an process that is already running from within the process itself, you need to learn about assembly for the architecture you're using, the executable file format on your system, and the process API on your system, at the very least.
As most other answers are explaining, in practical terms, most C implementations are compilers. So the executable that is running has only an indirect (and delayed) relation with the source code, because the source code had to be processed by the compiler to produce that executable.
Remember that a programming language is (not a software but...) a specification, written in some report. Read n1570, draft specification of C11. Most implementations of C are command-line compilers (e.g. GCC & Clang/LLVM in the free software realm), even if you might find interpreters.
However, with some operating systems (notably POSIX ones, such as MacOSX and Linux), you could dynamically load some plugin. Or you could create, in some other way (such as JIT compilation libraries like libgccjit or LLVM or libjit or GNU lightning), a fresh function and dynamically get a pointer to it (and that is not stricto sensu conforming to the C standard, where a function pointer should point to some existing function of your program).
On Linux, you might generate (at runtime of your own program, linked with -rdynamic to have its names usable from plugins, and with -ldl library to get the dynamic loader) some C code in some temporary source file e.g. /tmp/gencode.c, run a compilation (using e.g. system(3) or popen(3)) of that emitted code as a /tmp/gencode.so plugin thru a command like e.g. gcc -O1 -g -Wall -fPIC -shared /tmp/gencode.c -o /tmp/gencode.so, then dynamically load that plugin using dlopen(3), find function pointers (from some conventional name) in that loaded plugin with dlsym(3), and call indirectly that function pointer. My manydl.c program shows that is possible for many hundred thousands of generated C files and loaded plugins. I'm using similar tricks in my GCC MELT. See also this and that. Notice that you don't really "self-modify" C code, you more broadly generate additional C code, compile it (as some plugin, etc...), and then load it -as an extension or plugin- then use it.
(for pragmatical reasons including ease of debugging, I don't recommend overwriting some existing C file, but just emitting new C code in some fresh temporary .c file -from some internal AST-like representation- that you would later feed to the compiler)
Is it possible to dynamically change the running code?
In general (at least on Linux and most POSIX systems), the machine code sits in a read-only code segment of the virtual address space so you cannot change or overwrite it; but you can use indirection thru function pointers (in your C code) to call newly loaded code (e.g. from dlopen-ed plugins).
However, you might also read about homoiconic languages, metaprogramming, multi-staged programming, and try to use Common Lisp (e.g. using its SBCL implementation, which compile to machine code at every REPL interaction and at every eval). I also recommend reading SICP (an excellent and freely available introduction to programming, with some chapters related to metaprogramming approaches)
PS. Dynamic loading of plugins is also possible in Windows -which I don't know- with LoadLibrary, but with a very different (and incompatible) model. Read Levine's linkers and loaders.
A computer doesn't understand the code as we do. It compiles or interprets it and loads into memory. Our modification of code is just changing the file. One needs to compile it and link it with other libraries and load it into memory.
ptrace() is a syscall used to inject code into a running program. You can probably look into that and achieve whatever you are trying to do.
Inject hello world in a running program. I have tried and tested this sometime before.
I am working on a program in C, and I am having trouble with libconfig.h. Because of this, I think if I could have my program download an external function from the Internet (using libcurl.h) and have my program treat it as a compiled and linked shared object, that would be perfect. It would need to work on all desktop platforms (Windows, Mac, and Linux), so no .dll's, and would have to be downloaded by the program, treated as a function, and then get deleted by the program. So, my question is: is that possible in C?
The reason that I need to download it separately is because the function would need to be updated regularly, and requiring the user to download a new version of the program regularly would defeat the purpose of the program.
Well the closest to what you ask for would be this
Download .so/.dll using curl
Dynamically load .so/.dll into your process
set up function pointer in your process to point to a function in .so/.dll
On Windows:
HMODULE handle = LoadLibrary("mylib.dll");
if (handle)
myfunc = GetProcAddress(handle, "myfunc");
To unload call
FreeLibrary(handle)
It decreases ref count, and the DLL is actually unloaded when ref count hits 0.
On Linux, check this post:
How do I load a shared object in C++?
You can't just treat it as compiled; you would have to do one of two things:
Actually compile it on the fly, then load it as a dynamic library, which requires ensuring that there is a compiler on the system and will probably cause an unholy mess of errors on the user end.
Build your own C parser to interpret the external function, which is no small feat.
Far simpler solution: just write a function that works and compile platform-specific versions of it into your binary (or a library, if you prefer) before shipping the product.
You could link a Python interpreter into your program and have it execute a Python version of your function.
This approach would actually work with different languages, such as Java, Ruby, etc.
I know that when you compile and link a C program, you link it with
C library
C runtime startup code
I wonder if I write a program (in a new language, or just C without linking to this code) and link it directly to a C code shared library (say zlib or gsl or fftw or something) and omitting the C library and C startup code (assuming my program will load the external lib itself using its magic), will this "just work"?
I know there is some initialization code in the CRT startup, so I wonder how I can call the required functions without having my application itself depend on a C library: so loading the external C library will at that point call the necessary initialization code (if any, this is the question), and otherwise just load the OS libraries/interfaces.
The reason I ask is that I want to write a language with a Standard library that hooks into the OS API directly, unlike most C++ implementations, that are built on top of the C library.
Take a look here https://blogs.oracle.com/ksplice/entry/hello_from_a_libc_free
So you can startup your program without depend to any library included libc, then libraries can be load and use as needed later.
I have used C shared libraries from a number of other languages. Whether you must explicitly initialize the shared library depends on the library. Normally, it will be implicitly initialized on load, but some libraries require extra initialization. Read the documentation.
And the code of my program (C or other language) must of course be initialized too, but that is what the compiler/linker usually take care of, by linking to the startup code by default.