In the opensource code of OpenSSL(version 1.1.0e) I saw that some of the function definition is generated by the perl files present inside the folders.
In the build.info file present in each folders inside the crypto, they have written some lines to generate the .s from the corresponding .pl.
For example, for generating aes_p8_set_encrypt_key in crypto/aes/build.info:
GENERATE[aesp8-ppc.s]=asm/aesp8-ppc.pl $(PERLASM_SCHEME)
for generating OPENSSL_madd300_probe in crypto/build.info :
GENERATE[ppccpuid.s]=ppccpuid.pl $(PERLASM_SCHEME)
And also in the main Makefile(generated makefile), there are some lines as below:
crypto/aes/aes-x86_64.o: crypto/aes/aes-x86_64.s
$(CC) -I. -Icrypto/include -Iinclude $(CFLAGS) $(LIB_CFLAGS) -MMD -MF crypto/aes/aes-x86_64.d.tmp -MT $# -c -o $# crypto/aes/aes-x86_64.s
#touch crypto/aes/aes-x86_64.d.tmp
#if cmp crypto/aes/aes-x86_64.d.tmp crypto/aes/aes-x86_64.d > /dev/null 2> /dev/null; then \
rm -f crypto/aes/aes-x86_64.d.tmp; \
else \
mv crypto/aes/aes-x86_64.d.tmp crypto/aes/aes-x86_64.d; \
fi
Followed with :
crypto/aes/aes-x86_64.s: crypto/aes/asm/aes-x86_64.pl
CC="$(CC)" $(PERL) crypto/aes/asm/aes-x86_64.pl $(PERLASM_SCHEME) $#
Can anyone explain how the .s is generated from the .pl files?
I need to add them in my Makefile inside my project, to solve the undefined reference error coming for the functions whose definition is generated by the .pl file.
How the assembly file is generated from the Perl script in OpenSSL...
You are using the Cryptogams implementation of AES on Power8. Cryptogams is Andy Polyakov's project to provide high speed cryptography to other developers.
The assembly language file is generated with a xlat program. For the Power8 gear the file is ppc-xlate.pl in the perlasm directory. It is used by aesp8-ppc.pl in the crypto/aes/asm directory.
Here is how you translate it. I'm working from GCC112 on the compile farm, which is ppc64le.
git clone https://github.com/openssl/openssl.git
mkdir cryptogams
cp ./openssl/crypto/perlasm/ppc-xlate.pl cryptogams/
cp ./openssl/crypto/aes/asm/aesp8-ppc.pl cryptogams/
cd cryptogams/
chmod +x *.pl
./aesp8-ppc.pl ppc64le aesp8-ppc.s
aesp8-ppc.pl generates a pure assembly language source file so name the output file with the little *.s. Sometimes the translation includes C preprocessor statements and it needs a big *.S (but not in this case).
The second argument to aesp8-ppc.pl is called flavor. It is ppc64le in the commands above. Flavor does two things. First, it selects either 32-bit or 64-bit. Second, it selects either little-endian or big-endian. Be sure to get the flavor right.
According to Andy at ppc8 does not build on powerpc64 big-endian, big-endian PowerPC should use linux64, not linux64be. Little-endian needs linux64le, however.
Once you have aesp8-ppc.s you can compile and assemble the source file with GCC.
gcc -mcpu=power8 -c aesp8-ppc.s
And then:
$ objdump --disassemble aesp8-ppc.o
aesp8-ppc.o: file format elf64-powerpcle
...
0000000000000420 <aes_p8_set_decrypt_key>:
420: c1 ff 21 f8 stdu r1,-64(r1)
424: a6 02 48 7d mflr r10
428: 50 00 41 f9 std r10,80(r1)
42c: 75 fc ff 4b bl a0 <aes_p8_set_encrypt_key>
430: a6 03 48 7d mtlr r10
434: 00 00 03 2c cmpwi r3,0
438: 68 00 c2 40 bne- 4a0 <Ldec_key_abort>
43c: 36 20 07 55 rlwinm r7,r8,4,0,27
440: 10 ff 65 38 addi r3,r5,-240
444: 7e f8 08 55 rlwinm r8,r8,31,1,31
448: 14 3a a3 7c add r5,r3,r7
44c: a6 03 09 7d mtctr r8
...
At this point you have an object file, but you don't know the API signatures or how to use it. To find out what to do next you have to objdump and then grep the OpenSSL sources to see how they use it.
$ nm aesp8-ppc.o | grep ' T '
00000000000006c0 T aes_p8_cbc_encrypt
0000000000001140 T aes_p8_ctr32_encrypt_blocks
00000000000005c0 T aes_p8_decrypt
00000000000004c0 T aes_p8_encrypt
0000000000000420 T aes_p8_set_decrypt_key
00000000000000a0 T aes_p8_set_encrypt_key
0000000000001d00 T aes_p8_xts_decrypt
0000000000001a60 T aes_p8_xts_encrypt
You are interested in the four functions aes_p8_set_encrypt_key, aes_p8_set_decrypt_key, aes_p8_encrypt and aes_p8_decrypt. You will use the signatures you find to create a header file for your program.
I'll help you with the first one: aes_p8_set_encrypt_key.
$ cd openssl
# Find aes_p8_set_encrypt_key
$ grep -nIR aes_p8_set_encrypt_key
crypto/evp/e_aes.c:153:# define HWAES_set_encrypt_key aes_p8_set_encrypt_key
# Now look for HWAES_set_encrypt_key
$ grep -nIR HWAES_set_encrypt_key
...
crypto/evp/e_aes.c:2515:int HWAES_set_encrypt_key(const unsigned char *userKey, const int bits,
...
# Now find the complete HWAES_set_encrypt_key
$ cat -n crypto/evp/e_aes.c
...
2515 int HWAES_set_encrypt_key(const unsigned char *userKey, const int bits,
2516 AES_KEY *key);
Lather, rinse, repeat for AES_KEY, aes_p8_set_decrypt_key, aes_p8_encrypt and aes_p8_decrypt.
Eventually you will end up with a header like shown in Cryptogams | AES or Cryptogams | SHA on the OpenSSL wiki. Cryptogams | AES and Cryptogams | SHA are written for ARMv4 but the same applies to Power8 as well.
Andy dual licenses his work. One license is the OpenSSL license because Andy works for OpenSSL. The second license is a BSD style license that does not have the encumbrances of OpenSSL.
Andy's public source is at GitHub | dot-asm. Unfortunately, a lot of Andy's work has not been uploaded so you have to pull it from OpenSSL. And a lot it is not documented so you have to do a fair amount of poking and prodding in the OpenSSL sources.
As far as I know there are two places you can look for documentation on using Cryptogams and Power8 cryptography. First is the OpenSSL wiki pages Cryptogams | AES and Cryptogams | SHA. The tutorials are ARMv4 but it applies to Power 8, too. I wrote the wiki articles so errors and omissions are my mistakes.
The second place to look is GitHub and Noloader | POWER8 crypto. I help maintain Crypto++ and the POWER8 crypto book is my knowledge dump. The POWER8 book includes Cryptogams SHA for PowerPC in Chapter 7.
The POWER8 crypto book was written by Bill Schmidt and I because we could not find documentation when working with AES and SHA on Power8. Bill Schmidt works for IBM and even he could not get the docs. All we could find was a blog post from an IBM engineer that was grossly missing details.
Related
First time posting a question here, open to feedback.
I am trying to get an executable running in AWS lambda, however I am getting the error "cannot execute binary file" which is quite vague (more outputs below).
I am hoping to identify what could be wrong, and why.
As a disclaimer I am not very good at c or python.
Currently my process has been to write a hello world program in c:
#include <stdio.h>
int main() {
printf("Hello World");
return 0;
}
Create an AWS instance which matches the Python 3.8 lambda runtime (Amazon Linux 2)
(Tried on Amazon Linux 1 with the same results)
Install compiler tools
sudo yum install gcc
sudo yum install glibc-static
Compile the program to a static executable
gcc -static -static-libgcc -static-libstdc++ hello.c -o testings
Also tried with less flags with similar results.
gcc -static hello.c -o testings
Have also tried as a 32 bit binary using -m32 flag. Tried this as looking at the cpu information for the lambda runtime showed a 32 bit processor.
Verify it has compiled with static dependencies
file testings
with the results
testings: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.2.0, BuildID[sha1]=7167aa934c751be053864ee555be8f6a46e4d62e, not stripped
and
ldd testings
with the results
not a dynamic executable
I've copied it to my local machine, changed the file permissions to allow execution, zipped it and uploaded it to lambda.
Which contains two files.
testings (the compiled c program)
lambda_function.py which looks like
import json
from subprocess import run, call
def lambda_handler(event, context):
call(["ls -ll"], shell=True)
print("using run")
out = run(["./testings"], shell=True)
print("printing result")
print(out.returncode)
print("printing output")
print(out.stdout)
print("printing error")
print(out.stderr)
return {
'statusCode': 200,
'body': json.dumps('hello')
}
The output of running this function is:
Response:
{
"statusCode": 200,
"body": "\"hello\""
}
Request ID:
"fb640125-6698-4777-841a-47228d4da930"
Function Logs:
START RequestId: fb640125-6698-4777-841a-47228d4da930 Version: $LATEST
total 1092
-rwxrwxrwx 1 slicer 497 426 Jan 13 07:03 lambda_function.py
-rwxrwxrwx 1 slicer 497 1111526 Jan 13 06:32 testings
using run
/bin/sh: ./testings: cannot execute binary file
printing result
126
printing output
None
printing error
None
END RequestId: fb640125-6698-4777-841a-47228d4da930
REPORT RequestId: fb640125-6698-4777-841a-47228d4da930 Duration: 24.81 ms Billed Duration: 100 ms Memory Size: 512 MB Max Memory Used: 58 MB Init Duration: 113.42 ms
I can run the executable on the AWS instance where it was compiled and on my local machine (Ubuntu 18.04.3 LTS).
I have read quite a few similar questions (running executable in lambda) and am aware this is not a new question, however I haven't managed to find a solution that works for me ( I suspect I am misunderstanding something fundamental, as this is a very new domain for me.)
Additional Information:
using readelf to look at the binaries headers:
ELF Header:
Magic: 7f 45 4c 46 02 01 01 03 00 00 00 00 00 00 00 00
Class: ELF64
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - GNU
ABI Version: 0
Type: EXEC (Executable file)
Machine: Advanced Micro Devices X86-64
Version: 0x1
Entry point address: 0x400950
Start of program headers: 64 (bytes into file)
Start of section headers: 699480 (bytes into file)
Flags: 0x0
Size of this header: 64 (bytes)
Size of program headers: 56 (bytes)
Number of program headers: 6
Size of section headers: 64 (bytes)
Number of section headers: 33
Section header string table index: 32
Objdump can figure out when the binary is calling GLIBC functions, like printf:
$ objdump -d crackme03.64 -Mintel | grep printf
5c8: ff 25 12 0a 20 00 jmp QWORD PTR [rip+0x200a12] #200fe0 <__printf_chk#GLIBC_2.3.4>
However, looking at the same address, Radare is much less helpful:
[0x000005c8]> pd 2
/ (fcn) sub.__cxa_finalize_224_5c8 8
| sub.__cxa_finalize_224_5c8 ();
| ; CALL XREF from 0x000007bc (sym.main)
| ; CALL XREF from 0x00000809 (sym.main)
| 0x000005c8 ff25120a2000 jmp qword [reloc.__cxa_finalize_224] ; [0x200fe0:8]=0 LEA reloc.__cxa_finalize_224 ; reloc.__cxa_finalize_224
\ 0x000005ce 6690 nop
Is there a way to ask Radare to figure out what these functions correspond to?
It should work if you're using the newest version from git.
Radare2’s development is pretty quick – the project evolves every day, therefore it’s recommended to use the current git version. Update your version and it should solve your problem:
$ git clone https://github.com/radare/radare2.git
$ cd radare2
$ ./sys/install.sh
I have written simple Hello world program and compiled it with gcc-arm-linux-gnueabi compiler. It compiles well but when i try to execute it on ARM machine it complains "no such file or directory". I think gcc-arm-linux-gnueabi is for embedded Linux only due to e(mbedded)abi. Is it different from ARM Linux ABI?
Please help me to solve this problem
code is here
#include "stdio.h"
int main(void) {
printf("Hello world !\n");
return 0;
}
compiled as
arm-linux-gnueabi-gcc -Wall -o crosscomp hello.c
When i execute this crosscomp on target ARM machine error is crosscomp no such file or dir
EDIT When I was using arm-linux-gnueabi-gcc the entry point was not matching with the target machine entry point (readelf -l crosscom) but when I compiled with aarch64-linux-gnu-gcc entry point matched with target machine. But now error becomes permission denied on ./crosscomp. I tried with sudo which says crosscomp: no such command.
Note I posted same question on askubuntu https://askubuntu.com/questions/904685/cross-compilation-for-arm-error-no-such-file-or-directory but got no response.
The output of readelf is as below
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
Class: ELF64
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: AArch64
Version: 0x1
Entry point address: 0x400470
Start of program headers: 64 (bytes into file)
Start of section headers: 4488 (bytes into file)
Flags: 0x0
Size of this header: 64 (bytes)
Size of program headers: 56 (bytes)
Number of program headers: 8
Size of section headers: 64 (bytes)
Number of section headers: 29
Section header string table index: 26
This peculiar error message happens when the dynamic loader required by a particular executable is missing.
You can find out the name of the dynamic loader that you need by applying readelf to the problem executable. On my x86-64 Linux box, for example
$ readelf -l /bin/ls | grep 'program interpreter'
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
("program interpreter" is another name for "dynamic loader".)
So, run the above command on your crosscomp binary on your development box. (If you don't have readelf or you get error messages, try arm-linux-gnueabi-readelf.) The file named after "program interpreter:" needs to exist on your target ARM machine. If you don't know where to get it, please post the output of the above command + ls -l of the directory that should have the missing file in it.
I'm trying to compile a couple of programs for a little embedded device I own. It's a Little-endian MIPS (mipsel) processor. I retrieved this executable from it via telnet and the builtin ftp client:
root#debian-mipsel:/home/user/wansview/devel# readelf -h unzip1
ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: MIPS R3000
Version: 0x1
Entry point address: 0x401cc0
Start of program headers: 52 (bytes into file)
Start of section headers: 169960 (bytes into file)
Flags: 0x10001007, noreorder, pic, cpic, o32, mips2
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 6
Size of section headers: 40 (bytes)
Number of section headers: 24
Section header string table index: 23
root#debian-mipsel:/home/user/wansview/devel# file unzip1
unzip1: ELF 32-bit LSB executable, MIPS, MIPS-II version 1 (SYSV), dynamically linked (uses shared libs), stripped
I then downloaded the MIPSEL version of Debian and I'm running it in QEMU. When I run the retrieved program above I get:
root#debian-mipsel:/home/user/wansview/devel# ./unzip1
-bash: ./unzip1: No such file or directory
Which I understand means it's not the right platform. Stubbornly I compiled a little hello world nonetheless to compare the ELF and file info. My hello world runs fine in Debian MIPSEL but also returns No such file or directory on the embedded device. It's readelf and file output is strikingly similar though:
root#debian-mipsel:/home/user/wansview/devel# readelf -h hello
ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: MIPS R3000
Version: 0x1
Entry point address: 0x400740
Start of program headers: 52 (bytes into file)
Start of section headers: 3652 (bytes into file)
Flags: 0x10001005, noreorder, cpic, o32, mips2
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 10
Size of section headers: 40 (bytes)
Number of section headers: 36
Section header string table index: 35
root#debian-mipsel:/home/user/wansview/devel# file hello
hello: ELF 32-bit LSB executable, MIPS, MIPS-II version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.26, BuildID[sha1]=0xeb3877062337a3dfd15cc09305691685ac0e8c57, with unknown capability 0xf41 = 0x756e6700, with unknown capability 0x70100 = 0x1040000, stripped
I'm trying to better understand how my two systems differ and why the executables won't run on both. Are there any flags I could add to gcc to successfully compile for the embedded device?
More info about the device
# cat /proc/cpuinfo
system type : Ralink SoC
processor : 0
cpu model : MIPS 24K V4.12
BogoMIPS : 239.10
wait instruction : yes
microsecond timers : yes
tlb_entries : 32
extra interrupt vector : yes
hardware watchpoint : yes
ASEs implemented : mips16 dsp
VCED exceptions : not available
VCEI exceptions : not available
More info about Debian MIPSEL
(Binaries compiled on debian-mipsel won't run on the target embedded device)
root#debian-mipsel:/home/user/wansview/devel# cat /proc/cpuinfo
system type : MIPS Malta
processor : 0
cpu model : MIPS 24Kc V0.0 FPU V0.0
BogoMIPS : 1038.33
wait instruction : yes
microsecond timers : yes
tlb_entries : 16
extra interrupt vector : yes
hardware watchpoint : yes, count: 1, address/irw mask: [0x0ff8]
ASEs implemented : mips16
shadow register sets : 1
kscratch registers : 0
core : 0
VCED exceptions : not available
VCEI exceptions : not available
More info about Aboriginal Linux Mipsel
(Binaries compiled on Aboriginal Linux will run on the embedded device, and it can run binaries retrieved from the device. I'm not happy with it as it doesn't have make and other tools I need for compiling larger applications)
(mipsel:1) /home/wansview # cat /proc/cpuinfo
system type : MIPS Malta
machine : Unknown
processor : 0
cpu model : MIPS 24Kc V0.0 FPU V0.0
BogoMIPS : 1013.76
wait instruction : yes
microsecond timers : yes
tlb_entries : 16
extra interrupt vector : yes
hardware watchpoint : yes, count: 1, address/irw mask: [0x0ff8]
isa : mips1 mips2 mips32r1 mips32r2
ASEs implemented : mips16
shadow register sets : 1
kscratch registers : 0
core : 0
VCED exceptions : not available
VCEI exceptions : not available
LDD
Here's a screenshot with ldd ran against my hello world and against unzip1 on both aboriginal linux and debian mipsel. Aboriginal Linux runs applications retrieved from the device just fine, and if I compile under Aboriginal Linux I can run the resulting binary on the embedded device. The reason I'm not content with Aboriginal is that doesn't have GNU make and other useful tools for larger applications, and no easy way to get them there.
You clearly need a different toolchain. On your Debian-mipsel, your toolchain uses glibc while your target uses uClibc.
So, maybe you would like to generate it by yourself using Buildroot:
wget http://buildroot.uclibc.org/downloads/buildroot-2014.11.tar.gz
tar zxf http://buildroot.uclibc.org/downloads/buildroot-2014.11.tar.gz
cd buildroot-2014.11
A trick to preconfigure for mipsel, R1 without soft-float (my will, check yours):
cat <<_EOF > .config
BR2_HAVE_DOT_CONFIG=y
BR2_mipsel=y
BR2_ARCH="mipsel"
BR2_ENDIAN="LITTLE"
BR2_GCC_TARGET_ARCH="mips32"
BR2_GCC_TARGET_ABI="32"
BR2_ARCH_HAS_ATOMICS=y
BR2_mips_32=y
# BR2_MIPS_SOFT_FLOAT is not set
BR2_MIPS_OABI32=y
_EOF
Finish your choice in Buildroot menuconfig, but you can
also keep it like this with save and exit.
make menuconfig # tweak options at your will,
make -j8 # takes 8 minutes on my machine
Then, your compiler can be found in ./output/host/usr/bin
A real example:
echo '#include <stdio.h>
int main(int argc, char* argv[]) {
printf("Hello World.\n");
return 0;
}' > hello.c
And compile it with your brand-new uClibc GCC compiler
output/host/usr/bin/mipsel-buildroot-linux-uclibc-gcc -o hello hello.c
A glimpse into the hello program: (didn't had time to fix my ldd...)
$ file hello
hello: ELF 32-bit LSB executable, MIPS, MIPS32 version 1 (SYSV), dynamically linked (uses shared libs), not stripped
$ strings hello | grep "lib.*so*"
/lib/ld-uClibc.so.0
libgcc_s.so.1
libc.so.0
It's done with the toolchain and compile your program.
Now that you have time :-) see what Buildroot offers :
a complete distribution (in output/target/) for embedded systems in many architectures.
EDIT: Better chance to execute
You can statically link your program in order to maximize the chances to run your code on any target.
$ output/host/usr/bin/mipsel-linux-gcc -Wall -o hello -static hello.c
$ file ./hello
./hello: ELF 32-bit LSB executable, MIPS, MIPS32 version 1 (SYSV), dynamically linked (uses shared libs), not stripped
And now, as that static version doesn't depend anymore on any external lib (only uClibc, here), this MIPS executable can even run on my x86_64 machine (thanks to binfmt and Qemu) :
$ uname -mo
x86_64 GNU/Linux
$ ./hello
Hello World.
Cheers.
Maybe i'm wrong but a simple fix (but not elegant way) in such cases could be to link the missing files to the existing ones:
ln -s /lib/libc.so.6 /lib/libc.so.0
like in this situation:
https://dev.openwrt.org/ticket/3083
Is there any way to disassemble windows exe files in linux?
Would this involve Wine? If so, is there any windows debug/disassembly program available for linux?
And can this be done with gdb?
Thnx.
objdump -d works for me (I have version 2.24):
$ objdump -d ~/.wine/drive_c/Program\ Files\ \(x86\)/Internet\ Explorer/iexplore.exe |head
/home/user/.wine/drive_c/Program Files (x86)/Internet Explorer/iexplore.exe: file format pei-i386
Note that I tried other binaries, including ntoskrnl.exe and Steam.exe and they also work with objdump
Disassembly of section .text:
10001000 <.text>:
10001000: b8 01 00 00 00 mov $0x1,%eax
10001005: c2 04 00 ret $0x4
For me, gdb didn't work, even when running with wine:
/home/user/.wine/drive_c/Program Files (x86)/Internet Explorer/iexplore.exe: not an ELF binary... don't know how to load it
Check out IDA Debugger Used it in the past with success.