In runtime I'm trying to recover an address of a function that is not exported but is available through shared library's symbols table and therefore is visible to the debugger.
I'm working on advanced debugging procedure that needs to capture certain events and manipulate runtime. One of the actions requires knowledge of an address of a private function (just the address) which is used as a key elsewhere.
My current solution calculates offset of that private function relative to a known exported function at build time using nm. This solution restricts debugging capabilities since it depends on a particular build of the shared library.
The preferable solution should be capable of recovering the address in runtime.
I was hoping to communicate with the attached debugger from within the app, but struggle to find any API for that.
What are my options?
In runtime I'm trying to recover an address of a function that is not exported but is available through shared library's symbols table and therefore is visible to the debugger.
Debugger is not a magical unicorn. If the symbol table is available to the debugger, it is also available to your application.
I need to recover its address by name using the debugger ...
That is entirely wrong approach.
Instead of using the debugger, read the symbol table for the library in your application, and use the info gained to call the target function.
Reading ELF symbol table is pretty easy. Example. If you are not on ELF platform, getting equivalent info should not be much harder.
In lldb you can quickly find the address by setting a symbolic breakpoint if it's known to the debugger by whatever means:
b symbolname
If you want call a non exported function from a library without a debugger attached there are couple of options but each will not be reliable in the long run:
Hardcode the offset from an exported library and call exportedSymbol+offset (this will work for a particular library binary version but will likely break for anything else)
Attempt to search for a binary signature of your nonexported function in the loaded library. (slightly less prone to break but the binary signature might always change)
Perhaps if you provide more detailed context what are you trying achieve better options can be considered.
Update:
Since lldb is somehow aware of the symbol I suspect it's defined in Mach-O LC_SYMTAB load command of your library. To verify that you could inspect your lib binary with tools like MachOView or MachOExplorer . Or Apple's otool or Jonathan Levin's jtool/jtool2 in console.
Here's an example from very 1st symbol entry yielded from LC_SYMTAB in MachOView. This is /usr/lib/dyld binary
In the example here 0x1000 is virtual address. Your library most likely will be 64bit so expect 0x10000000 and above. The actual base gets randomized by ASLR, but you can verify the current value with
sample yourProcess
yourProcess being an executable using the library you're after.
The output should contain:
Binary Images:
0x10566a000 - 0x105dc0fff com.apple.finder (10.14.5 - 1143.5.1) <3B0424E1-647C-3279-8F90-4D374AA4AC0D> /System/Library/CoreServices/Finder.app/Contents/MacOS/Finder
0x1080cb000 - 0x1081356ef dyld (655.1.1) <D3E77331-ACE5-349D-A7CC-433D626D4A5B> /usr/lib/dyld
...
These are the loaded addresses 0x100000000 shifted by ASLR. There might be more nuances how exactly those addresses are chosen for dylibs but you get the idea.
Tbh I've never needed to find such address programmatically but it's definitely doable (as /usr/bin/sample is able to do it).
From here to achieve something practically:
Parse Mach-o header of your lib binary (check this & this for starters)
Find LC_SYMTAB load command
Find your symbol text based entry and find the virtual address (the red box stuff)
Calculate ASLR and apply the shift
There is some C Apple API for parsing Mach-O. Also some Python code exists in the wild (being popular among reverse engineering folks).
Hope that helps.
Related
I am trying, as an exercise, to make some sort of custom profiler for binaries in C, using the ptrace api. I assume all binaries to be traced have been statically linked, I have access to tools such as nm(1), objdump and readelf and use a Linux, x86, 32 bit system.
In the current phase I am trying to create a dynamic call tree/graph (relative calls only) of the traced process and include the total number of instructions executed in each function call. In order to do that, I tried to:
Retrieve all user defined symbols in the ELF file using nm(1) as well as their addresses
Use ptrace to step through the code and identify call and ret restructions
After each call, use the rip register to figure out the address of the current instruction and within which function this instruction is; thus deducing the corresponding symbol name.
My question is relative to this last point. I was wondering if there is a way, using the ptrace api, to identify the call instruction as well as the address of the function to which the execution will jump; or even better directly the symbol name which corresponds to this function.
I have tried reading the documentation for ptrace but it is, at least for me, far from clear. Is there a standard approach to what I am trying to do? Is my approach maybe completely wrong?
As far as I know compiler convert source code to machine code. But this code do not have any OS-related sections and linker add them to file.
But is it's possible to make some executable without linker?
Answering your question very literally - yes, it is possible to make an executive file without a linker: you don't need a compiler or linker to generate machine code. Binaries are a series of opcodes and relevant information (offsets, addresses etc). If you open a binary editor then type out some opcodes and make a program. Save and run it.
Of course the binary will be processor specific, just as if you had compiled a binary (native) executive. Here's a reference to the Intel x86 opcodes.
http://ref.x86asm.net/coder32.html.
If you're however asking, "Can I compile a source file directly into an executive file without a linker?" then speaking purely: no - unless the compiler has aspects of a linker integrated within it. The compiler generates intermediate objects that are passed on to the linker to "link" them into a binary such as a library or executive. Without the link step the pipeline is not complete.
Let's first make a statement that is to be considered true, compilers do not generate machine code that can be immediately executed (JIT's do, but lets ignore that).
Instead they generate files (object, static, dynamic, executable) which describe what they contains as well as groups of symbols. Symbols can be global variables or functions.
But symbols just like the file itself contain metadata. This metadata is very important. See the machine code stored in a symbol is the raw instructions for the target architecture but it does not know where memory is stored.
While modern CPU's give each process its own address space, a symbol may not land and probably won't land in the same address twice. In very recent times this is a security measure, but in past its so that dynamic linking works correctly.
So when the OS loads up an executable or shared library it can place it wherever it wants and by doing so make it not repeatable. Otherwise we'd all have to start caring and saying "this file contains 100% of the code I intend to execute". Usually on load the raw binary in the symbol table get transformed by patching it with the symbol locations in RAM. Making everything just work.
In summary the compiler emits files that allow for dynamic patching of assembly
prior to execution. If it didn't, we would be living in a very restrictive and problematic world.
Linkers even have scripts to change how they operate. They are a very complex and delicate piece of software required to make our programs work.
Have a read of the PE-COFF and ELF standards if you want to get an idea of just how complex those formats really are.
I'm writing some C code to hook some function of .so ELF (shared-library) loaded into memory.
My C code should be able to re-direct an export function of another .so library that was loaded into the app/program's memory.
Here's a bit of elaboration:
Android app will have multiple .so files loaded. My C code has to look through export function that belongs to another shared .so library (called target.so in this case)
This is not a regular dlsym approach because I don't just want address of a function but I want to replace it with my own fuction; in that: when another library makes the call to its own function then instead my hook_func gets called, and then from my hook_func I should call the original_func.
For import functions this can work. But for export functions I'm not sure how to do it.
Import functions have the entries in the symbol table that have corresponding entry in relocation table that eventually gives the address of entry in global offset table (GOT).
But for the export functions, the symbol's st_value element itself has address of the procedure and not GOT address (correct me if I'm wrong).
How do I perform the hooking for the export function?
Theoretically speaking, I should get the memory location of the st_value element of dynamic symbol table entry ( Elf32_Sym ) of export function. If I get that location then I should be able to replace the value in that location with my hook_func's address. However, I'm not able to write into this location so far. I have to assume the dynamic symbol table's memory is read-only. If that is true then what is the workaround in that case?
Thanks a lot for reading and helping me out.
Update: LD_PRELOAD can only replace the original functions with my own, but then I'm not sure if there any way to call the originals.
In my case for example:
App initializes the audio engine by calling Audio_System_Create and passes a reference of AUDIO_SYSTEM object to Audio_System_Create(AUDIO_SYSTEM **);
AUDIO API allocates this struct/object and function returns.
Now if only I could access that AUDIO_SYSTEM object, I would easily attach a callback to this object and start receiving audio data.
Hence, my ultimate goal is to get the reference to AUIOD_SYSTEM object; and in my understanding, I can only get that if I intercept the call where that object is first getting allocated through Audio_System_Create(AUIOD_SYSTEM **).
Currently there is no straight way to grab the output audio at android. (all examples talk about recording audio that comes from microphone only)
Update2:
As advised by Basile in his answer, I made use of dladdr() but strangely enough it gives me the same address as I pass to it.
void *pFunc=procedure_addr; //procedure address calculated from the st_value of symbol from symbol table in ELF file (not from loaded file)
int nRet;
// Lookup the name of the function given the function pointer
if ((nRet = dladdr(pFunc, &DlInfo)) != 0)
{
LOGE("Symbol Name is: %s", DlInfo.dli_sname);
if(DlInfo.dli_saddr==NULL)
LOGE("Symbol Address is: NULL");
else
LOGE("Symbol Address is: 0x%x", DlInfo.dli_saddr);
}
else
LOGE("dladdr failed");
Here's the result I get:
entry_addr =0x75a28cfc
entry_addr_through_dlysm =0x75a28cfc
Symbol Name is: AUDIO_System_Create
Symbol Address is: 0x75a28cfc
Here address obtained through dlysm or calculated through ELF file is the address of procedure; while I need the location where this address itself is; so that I can replace this address with my hook_func address. dladdr() didn't do what I thought it will do.
You should read in details Drepper's paper: how to write shared libraries - notably to understand why using LD_PRELOADis not enough. You may want to study the source code of the dynamic linker (ld-linux.so) inside your libc. You might try to change with mprotect(2) and/or mmap(2) and/or mremap(2) the relevant pages. You can query the memory mapping thru proc(5) using /proc/self/maps & /proc/self/smaps. Then you could, in an architecture-specific way, replace the starting bytes (perhaps using asmjit or GNU lightning) of the code of original_func by a jump to your hook_func function (which you might need to change its epilogue, to put the overwritten instructions -originally at original_func- there...)
Things might be slightly easier if original_func is well known and always the same. You could then study its source and assembler code, and write the patching function and your hook_func only for it.
Perhaps using dladdr(3) might be helpful too (but probably not).
Alternatively, hack your dynamic linker to change it for your needs. You might study the source code of musl-libc
Notice that you probably need to overwrite the machine code at the address of original_func (as given by dlsym on "original_func"). Alternatively, you'll need to relocate every occurrence of calls to that function in all the already loaded shared objects (I believe it is harder; if you insist see dl_iterate_phdr(3)).
If you want a generic solution (for an arbitrary original_func) you'll need to implement some binary code analyzer (or disassembler) to patch that function. If you just want to hack a particular original_func you should disassemble it, and patch its machine code, and have your hook_func do the part of original_func that you have overwritten.
Such horrible and time consuming hacks (you'll need weeks to make it work) make me prefer using free software (since then, it is much simpler to patch the source of the shared library and recompile it).
Of course, all this isn't easy. You need to understand in details what ELF shared objects are, see also elf(5) and read Levine's book: Linkers and Loaders
NB: Beware, if you are hacking against a proprietary library (e.g. unity3d), what you are trying to achieve might be illegal. Ask a lawyer. Technically, you are violating most abstractions provided by shared libraries. If possible, ask the author of the shared library to give help and perhaps implement some plugin machinery in it.
For example, I have a function func():
int func (int a, int b) {return a + b;}
Now I want write it to a file, so that I can use the system-call mmap to load it with PROT_EXEC and I can call it from another program.What should I do for it?
If you know what signature you need and a static library or the location of a shared library at compile time, you probably just want to include the header and link against the output library. If you want to invoke a function dynamically, you probably want dlopen / dlsym (UNIX) or LoadLibrary / GetProcAddress (Windows) for loading the libary dynamically and retrieving the address of the function by name.
Note that the cases where you actually need to load a library dynamically (at least explicitly) are pretty rare. This is often used for modular architectures (e.g. "plugins" or "extensions") where individual pieces of the application are distributed separately (which can be achieved more securely using IPC rather than dynamic loading... see my note below). Or for cases where your application is not allowed to include dependencies statically and needs to conditionally supply behavior based on the existence of certain library dependencies in the environment in which it happens to be executing. In most cases, though, you'll simply want to include a header that declares the symbols you need and compile for each target platform (possibly using #if...#else macros if there are symbols that vary across OSes or OS versions).
From a stability, security, and code complexity standpoint, I personally recommend that you avoid dynamic library loading. For core system functionality, it's reasonable to link against a dynamic library, but you'll want to do it in a way where the burden of dynamic loading is entirely on your toolchain (i.e. you shouldn't need to call dlopen or LoadLibrary explicitly). For other functionality, it is almost always better to statically link (assuming you distribute updates when there are security fixes for your dependencies), since this will avoid you getting broken by incompatible version updates and also prevent your users from experiencing dependency hell (you require version A but some other application requires version B); modular architectures are often better (and more securely) achieved through inter-process communication (IPC), since dynamically loaded libraries live in the process of the program that loads them (thereby giving them access to the entire process's virtual memory space), whereas with interprocess-communication, each component would be a separate process, and individual components would only have access to information that was given to it explicitly by the calling process, which would make it more difficult for a malicious component to steal data from the caller or other components or to produce instability.
The sanest thing if you want this to actually be used in the real world is probably to just compile the source as part of your program on each platform, like a regular function.
Next best is probably a separate process that you talk to rather than merge with.
Semi-sane (but still not a great choice, see our discussion in the other answer) would be making the shared library, like Michael Aaron Safyan said.
But if you want to know how it works just because - say, you want to write your own dynamic linker, or are doing some kind of runtime code generation like a JIT compiler, or if you just wanna know - you can make a raw code file.
To use it, what we'd have to do is similar to what the linker does - load the code at a particular address that it is made to work on and run it. There is position independent code that can run at any address, too.
Let's first get our function compiled and linked, then output into a raw image for a certain address. Assume the function is func in the file func.c and we're using gcc on Linux. (A Windows compiler would have similar options - gcc on Windows is exactly the same, I believe, but something like Digital Mars's C compiler does it differently with the linker command being /BINARY for instance)
Anyway, here's what I ran:
gcc -c func.c # makes func.o
ld func.o --oformat=binary -e func -o func.binary
This generates a file called func.binary. You can disassemble it most easily with ndisasm -b 64 func.binary (or -b 32 if you compiled the C in 32 bit mode) to confirm it looks right - I see an add instruction there, so looks good to me.
If you loaded that and mmaped then called it... it should work.
Problems will be quick to come up though:
If there's more than one function in that file, they'll all be squished together.
The addresses they try to use to call each other may be totally wrong.
Global variables and other static data will be messed up.
And there's more. The operating system uses more complex file formats for executables and libraries for a reason!
To go to the next step, you could consider writing an ELF or PE loader which reads that metadata off a standard file. Of course, once you get into much of this, you'll be doing exactly what the OS provides with dlopen and LoadLibrary.... so unless the goal is to just learn about the guts, just call those functions and call it done!
My embedded projects have a post-process step that replaces a value in the executable with the CRC of (some sections of) the flash. This step can only be done after linking since that is the first opportunity to CRC the image. In the past the file format was COFF, and I have created a custom tool to do the patching.
The development tool has switched to ELF, so I need to re-implement the CRC patcher. Before I do, I thought I'd look for an existing tool to do this. The compiler is based on gcc, but I can't see any combination of ld and nm and readelf that can do the job. A Google search has not been fruitful.
My present tool uses nm to find the address to patch, and calls the patcher with the address, the expected value (to prevent overwriting the wrong data), and the new CRC value. The CRC is calculated on a "hex" format of the executable (that I also patch) so fortunately I don't have to redo that part.
I can implement this with libelf and custom code again, but before I do, does it already exist?
Is there a better way to accomplish my goal of putting a CRC of the executable into the executable so it's available to the application?
If I've understood what you're trying to do correctly, I think the following would work:
nm gives you the runtime virtual address of the location you want to patch;
readelf -S gives you both the runtime virtual address and the offset within the file for the beginning of each section;
stitching the two together (e.g. with a few lines of your favourite scripting language) gives the offset within the file to patch.
I'm not sure if this would work, but you might be able to arrange it so that the CRC location within your object file were to be set to the address of an external symbol X. That external symbol might then be satisfied by a last linking step by linking in an elf file that did nothing but specify that X's address was the CRC that you have calculated.
This is still pretty hacky, and I'm not sure if it's easily do-able (since it is such an abuse of the tools).