I'm new to Hex-ray. While studying deep inside each functions, I arouse one question that can't find answer through googling or searching. In ida generated comment in function Pseudocode mode(F5),
It says like this:
char __userpurge function#<al>(int a1, ... )
{
HANDLE v2; // eax#1
DWORD v3; // eax#4
void *4; // eax#6
LARGE_INTEGER FileSize; // [esp+4h] [ebp-8h]#2
}
Can anyone tell me the meaning about eax#1, eax#4 these kind of stuffs?
In addition to that, I'm also wonder what [esp+4h] [ebp-8h]#2 means.
This question was already answered in Igor Skochinsky's answer on Stack Exchange Reverse Engineering:
The numbers denote the basic block number in the microcode where the
variable was first used. Microcode basic blocks do not necessarily
match the basics block in the disassembly, because during optimization
steps the microcode basic blocks may be split, merged, added, or
removed.
You can see some samples of microcode in Ilfak's whitepaper on the
decompiler
or in some of the old blog posts:
1
2 (block number is the first number on
the line).
It is recommended to ask questions about reverse engineering in SE.RE community.
Related
I want to obfuscate a particular string in the binary of a C program to make it harder to analyze. I know this will not prevent someone from seeing the string if running it in a debugger. Yes, this is merely obfuscation.
Every instance of obfuscation triggers a discussion saying it has no value whatsoever. So did this one! I am aware that a capable and determined attacker will be able to recover the string. For the sake of the argument let's say I'm writing a game for X year olds and the string to be hidden is a URL to be called only once they beat the game and their name will be added to the hall of fame. It's reasonable to assume that most X year olds will not have skills that go beyond opening the binary file in a hex editor. Thanks!
Is there some elegant way to do the hiding at compile time, perhaps using the C preprocessor and a macro?
What i have seen so far is a suggestion by Yuri Slobodyanyuk resulting in this:
#define HIDE_LETTER(a) (a) + 0x50
#define UNHIDE_STRING(str) do { char * ptr = str ; while (*ptr) *ptr++ -= 0x50; } while(0)
...
char str1[] = { HIDE_LETTER('s'), HIDE_LETTER('e'), HIDE_LETTER('c'), HIDE_LETTER('r'), HIDE_LETTER('e'),
HIDE_LETTER('t'), '\0' };
UNHIDE_STRING(str1); // unmangle the string in-place
It works but it's a bit ugly. 🙂 Perhaps someone knows a better solution?
I'm fine with something that is gcc specific.
PS: For C++ there is a solution by Adam Yaxley on github but I'm looking for C, not C++. And there's a solution with a little helper program at https://github.com/TwizzyIndy/hkteam_obfuscator
First, be aware that your issue is probably better covered by some legal approach (a contract reviewed by a paid lawyer) than by technical means.
Your approach is similar to Caesar cypher (which has been broken thousands of years ago: insight: compute frequencies of letters; in human English, e is the most frequent one). Even the German Enigma machine did a lot better in WW2. Read about the works of Alan Turing during WW2 (his team broke the Enigma machine encryption).
Is there some elegant way to do it at compile time, perhaps using the C preprocessor and a macro?
No, there is not
(and mathematical proofs of that exist in the literature, covered by books related to Frama-C or cybersecurity or Coq proof assistant; be aware of Rice's theorem; Read also Berto-Caseran book on Interactive Theorem Proving and Software Development ISBN 3-540-20854-2)
The argument of such a proof is based on cardinality. You could also use a probabilistic approach: store in your program some cryptic hashcode (e.g. computed by crypt(3) at build time) and ask from user input a secret key, etc...
Any professional hacker will be technically able (perhaps after weeks of work) to find your "secret" string. Or colleagues working on or with BinSec.
However, you could write some metaprogram generating your obfuscated string as C code (to be #include-d at compile time), and add into your program some deobfuscation routine.
I'm fine with something that is gcc specific.
On large programs, consider developing your GCC plugin (perhaps starting with Bismon). See also the DECODER project.
Be however aware of Rice's theorem. Read about P vs NP problem.
Consider also generating some C code (maybe some #include-d header) with tools like GPP.
Code obfuscation is a topic which has conferences. Did you attend any of them? Many papers exist in ACM conferences.
There could be also legal issues (perhaps related to GDPR). You should contact your lawyer. In France, see article 323 du Code Pénal.
If your code runs on a computer connected to the Internet and interacting with a user, consider a SaaS approach: you could ask some money with a VISA card at every run (or once a month).... Your bank will sell you appropriate software and permissions.
I'm writing a game for 8 year olds and the string to be hidden is a URL to be called only once they beat the game and their name will be added to the hall of fame. It's reasonable to assume that most 8 year olds will not have skills that go beyond opening the binary file in a hex editor.
I now no 8 years old kid able to do that, and those who do deserves to be added to your hall of fame. If indeed you are coding a game, I recommend putting the URL as clear text.
NB. The old XPM program could be inspirational, and so can be RefPerSys and Jacques Pitrat's last book Artificial Beings, the conscience of a conscious machine (ISBN-13: 978-1848211018). Feel free to contact me by email basile#starynkevitch.net (home) or basile.starynkevitch#cea.fr (office, at CEA LIST) for more.
PS. Consider of course starting your PhD on that topic! In France, at ENS or Ecole Polytechnique. There are interesting related talks at College de France. In Germany, Frauhaufer CyberSecurity lab. Probably, the Bundeswehr will fund your research in Germany (but I have no connections there), and also ITEA4. Of course, you will spend three or four years full-time to find a good enough solution. Please publish papers on arxiv.
How about this:
#define STRING "Obfuscated"
#define Makestr(i) string[i] = STRING[i]
char string[11];
Makestr(6); Makestr(5);
Makestr(9); Makestr(7);
Makestr(0); Makestr(3);
Makestr(2); Makestr(4);
Makestr(1); Makestr(8);
Makestr(10);
This will typically compile to the equivalent of
string[6] = 97; string[5] = 99;
string[9] = 100; string[7] = 116;
string[0] = 79; string[3] = 117;
string[2] = 102; string[4] = 115;
string[1] = 98; string[8] = 101;
string[10] = 0;
If you look at the object file using strings or a hex editor, it won't even be obvious that there's a string at all. (But if you step through the code in a debugger, you'd be able to suss out what it was doing soon enough. No way around that, really.)
You could also perturb the individual characters, as in your original question:
#define Makestr(i) string[i] = STRING[i] + 0x50
Me, I'd worry about overflow, so I'd probably do
#define Makestr(i) string[i] = STRING[i] ^ 0x55
Now you get the equivalent of
string[6] = 177;
or
string[6] = 52;
, etc.
Obviously in these cases you have to additionally unhide the
constructed string at run time, of course.
With clang I had to use -O to force it to collapse the constants and not emit the original string in the object file; with gcc it worked right away.
If your string is longer, the randomly-shuffled sequence of Makestr calls could get pretty unwieldy, though.
I changed the obfuscation to just flip bit 7.
Also i couldn't find a pretty way to do the encoding in the C preprocessor cpp at compile time.
I ended up encoding the string using this shell onliner
tr \\000-\\377 \\200-\\377\\0-\\177|od -t x1 -A none|sed -e 's/ /\\x/g'
and sticking the result into the C source:
#include <stdio.h>
#include <string.h>
/* flip bit 7 in string using shell commands
tr \\000-\\377 \\200-\\377\\0-\\177|od -t x1 -A none|sed -e 's/ /\\x/g'
*/
int main() {
char secret[] = "\xce\xef\xf4\xa0\xf5\xf3\xe9\xee\xe7\xa0\xf4\xe8"
"\xe5\xa0\xf0\xf2\xe5\xf0\xf2\xef\xe3\xe5\xf3\xf3\xef\xf2"
"\xa0\xba\xad\xa8";
for (int i = 0; secret[i]; i++)
secret[i] ^= 1 << 7; // flip bit 7
printf("%s\n",secret);
}
I will leave this question as unanswered for now in the hope that someone finds a one-step solution instead of this two-step approach.
I need an opinion from somebody who has some experince with assuring file integrity.
I am trying to protect the integrity of my file with a crc checksum. My primary goal is to make harder bypassing a licence file check (which consist in disassembling the executable and removing a conditional jump).
I came up with the following idea:
unsigned long crc_stored = 4294967295;
char* text_begin = (char*)0xffffffffffffffff;
char* text_end = (char*)0xffffffffffffffff;
int main(){
unsigned long crc = calc_checksum(text_begin, text_end);
if (crc == crc_stored)
//file is ok
}
I edit the .data section of the elf binary in the following way: text_begin and text_end will contain the begin and end address of the .text section, and crc_stored the crc checksum of the .text section.
I would like to know whether this is a proper way of doing this, or there are better methods?
Edit: Karoly Horvath has right. Let's say I use the crc check to decrypt some code. I would like to know which is the best way ro checksum protect the executable.
Olaf also has right. I can use a sha algorithm. The question is the same.
Edit2: please stop saying that any protection can bypassed. I know and I just want to make it harder. Please answer the question if you can.
Let me see. You have code that does this:
int main() {
if (!license_ok()) { exit(1); }
// do something useful
}
You are worried that someone will disassemble your code, and patch out the conditional jump, so you are proposing to change the code this way instead:
int main() {
if (calc_checksum() != stored_crc) { exit(1); }
if (!license_ok()) { exit(1); }
// do something useful
}
I hope you see that this "solution" is not really a solution at all (if someone is capable of patching out one conditional jump, surely he is just as capable of patching out two such jumps).
You can find ideas for a more plausible / robust solution in one of the many books on the subject.
Do not stop the programme from running.
If the license is wrong at the start some strange behaviour is likely after 1 to 5 minutes,
causing segfaults, wrong calculations, whatever.
But in some indirect ways. Like a 2nd thread that modifies calculations or changes a random bit in the stack of another thread if the license is wrong.
Also get a Map of yourself at runtime by /proc/self and run a checksum on your .text sections at runtime.
That way you also can find some runtime modifications.
But the bitter truth is,
if it is runnable then it is just a question of how much effort the attacker needs to get a unlicensed copy running. It is not important to make it unrunnable. Just the effort of getting it cracked must be bigger than the effort.
At work we have an Arduino sketch that gets changed periodically. In a nutshell, it communicates back and forth on a Serial port. For the most part our software development team controls the code; however, there are some other teams at our company that periodically make last minute changes to the sketch in order to accommodate specific client needs.
This has obviously been quite problematic because it means we might have different versions of our sketch deployed in different places without realizing it. Our software developers are very good at using source control but the other teams are not quite so disciplined.
One idea that was proposed was hard-coding a version number, so that a certain serial command would respond by reporting back the predefined version number. The trouble however is that our other teams might likewise fail to have the discipline to update the version number if they decide to make other changes.
Obviously the best solution involves cutting off the other team from making updates, but assuming that isn't possible for office politics reasons, I was wondering if there's any way to programmatically "reflect" on an Arduino sketch. Obviously a sketch is going to take up a certain number of bytes, and that sketch file is going to have a unique file hash. I was thinking if there was some way to either get the byte count, the file hash, or the last modified time as a preprocessor directive that can be injected into code that would be ideal. Something like this:
// pseudocode
const String SKETCH_FILE_HASH = #filehash;
const int SKETCH_FILE_SIZE = #filesize;
const int SKETCH_LAST_UPDATED = #modified;
But that's about as far as my knowledge goes with this. Is there any way to write custom preprocessor directives, or macros, for Arduino code? Specifically ones that can examine the sketch file itself? Is that even possible? Or is there some way that already exists to programmatically track changes in one way or another?
Risking an answer.
SKETCH_FILE_HASH : you would have to precompute externally and pass as a flag. I guess you're using the arduino IDE and this is not doable
SKETCH_FILE_SIZE: same answer
SKETCH_LAST_UPDATED: You can use __TIME__ to get a string containing compilation time.
What I would do, taking into account the polititc parts.
enmbed a keyword linked to your version control (e.g. svn:id for subversion, almost all VCS provide this)
embed compilation time
change the official build (the one the SW team controls) to use the actual toolchain and not the IDE and put it on a jenkins : you'll be able to use compilation flags!
embed a code like
#ifndef BUILD_TYPE
#define BUILD_TYPE "Unsupported"
#endif
On your continuous build process, use -DBUILD_TYPE="HEAD" or "Release"
I'm sorry I don't see a magicx wand solving your solution. I'd invest a lot into training on why version control can save you (seems you already have the war stories)
I was looking at this issue myself, and found this:
https://gist.github.com/jcw/1985789#file-bootcheck-ino
This is to look up the bootloader; but I'm thinking that something like this could be used for determining a signature of some sort for the code as a whole.
I did a quick experiment, where I added something like:
Serial.print("Other...");
Serial.println(CalculateChecksum(0, 2048));
in void setup(), and was able to get different values for the CRC, based on changing a tiny bit of code (a string).
This is not an explicit solution; I tried CalculateChecksum(0, 32767), and so on, and if I defined an integer like int a=101; and changed it to int a=102; the checksum was the same. Only when I changed a string (i.e., add a space) did this value change.
I'm not crystal clear on the way memory is allocated in the Arduino; I do know there is program memory (32,256 bytes) and global variable memory (2048 bytes), so I'm sure there is some way of doing this.
In another experiment, I used the pgm_read_byte() function, and if I create a simple memory dump function:
void MemoryDump (word addr, word size) {
word dataval = ~0;
// prog_uint8_t* p = (prog_uint8_t*) addr;
uint8_t* p = (uint8_t*) addr;
for (word i = 0; i < size; ++i)
{
dataval = pgm_read_byte(p++);
Serial.print(i);
Serial.print(" ->");
Serial.print(dataval,HEX);
Serial.print(" ");
Serial.print(dataval);
Serial.print(" ");
if(dataval>32)
{
Serial.print(char(dataval));
}
else
{
Serial.print("***");
}
Serial.print("\n");
}
}
... and I put in a line like:
Serial.println(F("12345fghijklmnopqrstuvwxyz"));
because the F() puts the string in program memory, you will see it there.
Reading the SRAM is a bit of an issue, as noted here:
http://forum.arduino.cc/index.php?topic=220125.0
I'm not a compiler god, so I don't know how stuff like a=101; looks to the compiler/IDE, or why this doesn't look different to the program memory area.
One last note:
http://playground.arduino.cc/Code/AvailableMemory
Those functions access SRAM, so perhaps, with a bit of tweaking, you could do a CRC on that memory, but it would seem a bit of an issue, since you have to be doing a computation with a variable... in SRAM! But if the code was identical, even if doing a computation like that, it might be possible. Again, I'm in deep water here, so if an AVR god has issue with this, please destroy this theory with an ugly fact!
I. Just implemented a kind of bitwise trie (based on nedtries), but my code does lot
Of memory allocation (for each node).
Contrary to my implemetation, nedtries are claimed to be fast , among othet things,
Because of their small number of memory allocation (if any).
The author claim his implementation to be "in-place", but what does it really means in this context ?
And how does nedtries achieve such a small number of dynamic memory allocation ?
Ps: I know that the sources are available, but the code is pretty hard to follow and I cannot figure how it works
I'm the author, so this is for the benefit of the many according to Google who are similarly having difficulties in using nedtries. I would like to thank the people here on stackflow for not making unpleasant comments about me personally which some other discussions about nedtries do.
I am afraid I don't understand the difficulties with knowing how to use it. Usage is exceptionally easy - simply copy the example in the Readme.html file:
typedef struct foo_s foo_t;
struct foo_s {
NEDTRIE_ENTRY(foo_t) link;
size_t key;
};
typedef struct foo_tree_s foo_tree_t;
NEDTRIE_HEAD(foo_tree_s, foo_t);
static foo_tree_t footree;
static size_t fookeyfunct(const foo_t *RESTRICT r)
{
return r->key;
}
NEDTRIE_GENERATE(static, foo_tree_s, foo_s, link, fookeyfunct, NEDTRIE_NOBBLEZEROS(foo_tree_s));
int main(void)
{
foo_t a, b, c, *r;
NEDTRIE_INIT(&footree);
a.key=2;
NEDTRIE_INSERT(foo_tree_s, &footree, &a);
b.key=6;
NEDTRIE_INSERT(foo_tree_s, &footree, &b);
r=NEDTRIE_FIND(foo_tree_s, &footree, &b);
assert(r==&b);
c.key=5;
r=NEDTRIE_NFIND(foo_tree_s, &footree, &c);
assert(r==&b); /* NFIND finds next largest. Invert the key function to invert this */
NEDTRIE_REMOVE(foo_tree_s, &footree, &a);
NEDTRIE_FOREACH(r, foo_tree_s, &footree)
{
printf("%p, %u\n", r, r->key);
}
NEDTRIE_PREV(foo_tree_s, &footree, &a);
return 0;
}
You declare your item type - here it's struct foo_s. You need the NEDTRIE_ENTRY() inside it otherwise it can contain whatever you like. You also need a key generating function. Other than that, it's pretty boilerplate.
I wouldn't have chosen this system of macro based initialisation myself! But it's for compatibility with the BSD rbtree.h so nedtries is very easy to swap in to anything using BSD rbtree.h.
Regarding my usage of "in place"
algorithms, well I guess my lack of
computer science training shows
here. What I would call "in place"
is when you only use the memory
passed into a piece of code, so if
you hand 64 bytes to an in place
algorithm it will only touch that 64
bytes i.e. it won't make use of
extra metadata, or allocate some
extra memory, or indeed write to
global state. A good example is an
"in place" sort implementation where
only the collection being sorted
(and I suppose the thread stack)
gets touched.
Hence no, nedtries doesn't need a
memory allocator. It stores all the
data it needs in the NEDTRIE_ENTRY
and NEDTRIE_HEAD macro expansions.
In other words, when you allocate
your struct foo_s, you do all the
memory allocation for nedtries.
Regarding understanding the "macro
goodness", it's far easier to
understand the logic if you compile
it as C++ and then debug it :). The
C++ build uses templates and the
debugger will cleanly show you state
at any given time. In fact, all
debugging from my end happens in a
C++ build and I meticulously
transcribe the C++ changes into
macroised C.
Lastly, before a new release, I
search Google for people having
problems with my software to see if
I can fix things and I am typically
amazed what someone people say about
me and my free software. Firstly,
why didn't those people having
difficulties ask me directly for
help? If I know that there is
something wrong with the docs, then
I can fix them - equally, asking on
stackoverflow doesn't let me know
immediately that there is a docs
problem bur rather relies on me to
find it next release. So all I would
say is that if anyone finds a
problem with my docs, please do
email me and say so, even if there
is a discussion say like here on
stackflow.
Niall
I took a look at the nedtrie.h source code.
It seems that the reason it is "in-place" is that you have to add the trie bookkeeping data to the items that you want to store.
You use the NEDTRIE_ENTRY macro to add parent/child/next/prev links to your data structure, and you can then pass that data structure to the various trie routines, which will extract and use those added members.
So it is "in-place" in the sense that you augment your existing data structures and the trie code piggybacks on that.
At least that's what it looks like. There's lots of macro goodness in that code so I could have gotten myself confused (:
In-place means you operate on the original (input) data, so the input data becomes the output data. Not-in-place means that you have separate input and output data, and the input data is not modified. In-place operations have a number of advantages - smaller cache/memory footprint, lower memory bandwidth, hence typically better performance, etc, but they have the disadvantage that they are destructive, i.e. you lose the original input data (which may or may not matter, depending on the use case).
In-place means to operate on the input data and (possibly) update it. The implication is that there no copying and/moving of the input data. This may result in loosing the input data original values which you will need to consider if it is relevant for your particular case.
I'm attempting to build a control-flow graph of the assembly results that are returned via a call to objdump -d . Currently the best method I've come up with is to put each line of the result into a linked list, and separate out the memory address, opcode, and operands for each line. I'm separating them out by relying on the regular nature of objdump results (the memory address is from character 2 to character 7 in the string that represents each line) .
Once this is done I start the actual CFG instruction. Each node in the CFG holds a starting and ending memory address, a pointer to the previous basic block, and pointers to any child basic blocks. I'm then going through the objdump results and comparing the opcode against an array of all control-flow opcodes in x86_64. If the opcode is a control-flow one, I record the address as the end of the basic block, and depending on the opcode either add two child pointers (conditional opcode) or one (call or return ) .
I'm in the process of implementing this in C, and it seems like it will work but feels very tenuous. Does anyone have any suggestions, or anything that I'm not taking into account?
Thanks for taking the time to read this!
edit:
The idea is to use it to compare stack traces of system calls generated by DynamoRIO against the expected CFG for a target binary, I'm hoping that building it like this will facilitate that. I haven't re-used what's available because A) I hadn't really though about it and B) I need to get the graph into a usable data structure so I can do path comparisons. I'm going to take a look at some of the utilities on the page you lined to, thanks for pointing me in the right direction. Thanks for your comments, I really appreciate it!
You should use an IL that was designed for program analysis. There are a few.
The DynInst project (dyninst.org) has a lifter that can translate from ELF binaries into CFGs for functions/programs (or it did the last time I looked). DynInst is written in C++.
BinNavi uses the ouput from IDA (the Interactive Disassembler) to build an IL out of control flow graphs that IDA identifies. I would also recommend a copy of IDA, it will let you spot check CFGs visually. Once you have a program in BinNavi you can get its IL representation of a function/CFG.
Function pointers are just the start of your troubles for statically identifying the control flow graph. Jump tables (the kinds generated for switch case statements in certain cases, by hand in others) throw a wrench in as well. Every code analysis framework I know of deals with those in a very heuristics-heavy approach. Then you have exceptions and exception handling, and also self-modifying code.
Good luck! You're getting a lot of information out of the DynamoRIO trace already, I suggest you utilize as much information as you can from that trace...
I found your question since I was interested in looking for the same thing.
I found nothing and wrote a simple python script for this and threw it on github:
https://github.com/zestrada/playground/blob/master/objdump_cfg/objdump_to_cfg.py
Note that I have some heuristics to deal with functions that never return, the gcc stack protector on 32bit x86, etc... You may or may not want such things.
I treat indirect calls similar to how you do (basically have a node in the graph that is a source when returning from an indirect).
Hopefully this is helpful for anyone looking to do similar analysis with similar restrictions.
I was also facing a similar issue in the past and wrote asm2cfg tool for this purpose: https://github.com/Kazhuu/asm2cfg. Tool has support for GDB disassembly and objdump inputs and spits out CFG as a dot or pdf.
Hopefully someone finds this helpful!