Compiling adding extra code in C to NASM, causing errors - c

The closet I have gotten to actually reasonable code is removing -m16 from gcc, but I get 64 bit intruction errors
I am trying to build C to .asm via gcc and objconv, but it outputs unnecessary code, that causes issues with other compiled code.
I also get an error while importing main.c:
main.asm:20: error: binary output format does not support external references
(Original nasm file extension: .s | C converted to nasm: .asm):
Expected output:
setc:
call draw_first_scene
call init_mouse
jmp enable_mouse
Output:
extern _GLOBAL_OFFSET_TABLE_
setc:
push bx
?_001:
db 66H, 0E8H
dd __x86.get_pc_thunk.bx-$-6H
db 66H, 81H, 0C3H
dd _GLOBAL_OFFSET_TABLE_-$-6H
dd 08EC8366H
db 66H, 0E8H
dd draw_first_scene-$-17H
db 66H, 0E8H
dd init_mouse-$-1DH
db 66H, 0E8H
dd enable_mouse-$-23H
dd 08C48366H, 0C3665B66H
__x86.get_pc_thunk.bx:
mov bx, word [si]
and al, 66H
ret
Removed -m16 output:
extern _GLOBAL_OFFSET_TABLE_
setc:
sub rsp, 8
xor eax, eax
call draw_first_scene
xor eax, eax
call init_mouse
xor eax, eax
add rsp, 8
jmp enable_mouse
main.c:
void setc() {
draw_first_scene();
init_mouse();
enable_mouse();
}
boot.s:
bits 16
org 0x7c00
; NASM
%include "bpb.s"
%include "mouse.s"
%include "draw.s"
%include "scene_draw.s"
%include "mouse_draw.s"
; C
%include "main.asm"
boot_start:
xor ax,ax
mov ds,ax
mov ss,ax
mov sp,0x7c00
cld
mov ax,12h
int 10h
jmp 0x0:setc
sti
.main_loop:
hlt
call draw_mouse
jmp .main_loop
.no_mouse:
mov si,noMouseMsg
call print_string
.err_loop:
hlt
jmp .err_loop
bpb.s:
bits 16
org 0x7c00
global bpb_disk_info
mov ax,07c0h
add ax,544
cli
mov ss,ax
mov sp,4096
sti
jmp boot_start
nop
bpb_disk_info:
OEMLabel: db "mkfs.fat"
BytesPerSector: dw 512
SectorsPerCluster: db 1
ReservedForBoot: dw 1
NumberOfFats: db 2
RootDirEntries: dw 224
LogicalSectors: dw 2880
MediumByte: db 0f0h
SectorsPerFat: dw 9
SectorsPerTrack: dw 18
Sides: dw 2
HiddenSectors: dd 0
LargeSectors: dd 0
DriveNo: dw 0
Signature: db 41
VolumeID: dd 0x2d7e5a1a
VolumeLabel: db " "
FileSystem: db "FAT12 "
My build.sh (I am running Linux on repl.it as I am on chromebook):
clear
set echo off
cleanup() {
{
rm -r $cur/node_modules/out
rm -r $cur/node_modules/temp
export temp=$prevtemp
export cur=$prevcur
} &>/dev/null
exit
}
trap cleanup 0
{
export prevtemp=$temp
export pervcur=$cur
export temp=""
export temp2=""
export cur="$(cd -- "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P)"
export cur=${cur%/node_modules}
cd $cur
mkdir nasm
mkdir node_modules
cd node_modules
mkdir out
mkdir temp
export temp=${cur}/node_modules/temp
cd $cur/src
} &> /dev/null
for i in $(find . -type f -print); do
if [[ "$i" == *.c ]]; then
export temp=${i/%.c/.o}
gcc -m16 -Wno-implicit-function-declaration -fno-asynchronous-unwind-tables -s -c -o $cur/node_modules/out${temp#.} $cur/src${i#.}
else
cp -f $i $cur/node_modules/out
fi
done
cd $cur/node_modules/out
for i in $(find . -type f -print); do
if [[ "$i" == *.o ]]; then
export temp=${i/%.o/.asm}
{
objconv -fnasm $cur/node_modules/out${i#.}
mv $cur/node_modules/out${temp#.} $cur/nasm${temp#.}
sed -i -e "/^SECTION ./d" $cur/nasm${temp#.}
sed -i -e "s/\s*;/;/g" $cur/nasm${temp#.}
sed -i -e "/^;/d" -e "s/^\([^;]*\);.*$/\1/" $cur/nasm${temp#.}
sed -i -e "/^global/d" -e "s/^\([^global]*\)global.*$/\1/" $cur/nasm${temp#.}
sed -i -e "/^default/d" -e "s/^\([^default]*\)default.*$/\1/" $cur/nasm${temp#.}
sed -i -e "/^extern/d" -e "s/^\([^extern]*\)extern.*$/\1/" $cur/nasm${temp#.}
sed -i -r "/^\s*$/d" $cur/nasm${temp#.}
sed -i -e "s/ / /g" $cur/nasm${temp#.}
sed -i "1s/^/extern _GLOBAL_OFFSET_TABLE_\n/" $cur/nasm${temp#.}
} &> /dev/null
fi
done
{
cd $cur/node_modules
} &> /dev/null
cd $cur/nasm
nasm -O0 -w+other -w+orphan-labels -f bin -o $cur/node_modules/out/boot.bin $cur/nasm/boot.s
cd $cur/node_modules/out
for i in $(find . -type f -print); do
if [[ "$i" == *.o ]]; then
rm $cur/node_modules/out${i#.}
fi
done
{
mkdosfs -C reliq.flp 1440
dd status=noxfer conv=notrunc if=boot.bin of=reliq.flp
rm reliq.iso
mkisofs -quiet -V 'RELIQ' -input-charset iso8859-1 -o reliq.iso -b reliq.flp ./
sleep 0.2
mv reliq.iso $cur
cd $cur
} &> /dev/null
{
qemu-system-x86_64 \
-netdev user,id=mynet0 \
-device ne2k_isa,netdev=mynet0 \
-drive file=reliq.iso,media=cdrom,cache=none \
-device sb16 \
-m 512M \
-cpu Cascadelake-Server \
-smp 1 \
-vga vmware \
-display gtk,grab-on-hover=on,zoom-to-fit=off \
-accel tcg \
-boot menu=on
} &> /dev/null

Related

Listing optimized binary function sizes in bytes

I am optimizing my code for an MCU, I want to see the overview of sizes of all functions in my C program including all libraries, like say:
$ objdump some arguments that I do not know | pipe through something | clean the result some how
_main: 300 bytes
alloc: 200 bytes
do_something: 1111 bytes
etc...
nm -S -t d a.out | grep function_name | awk '{gsub ("0*", "", $2); print $2}'
Or print a list of sorted sizes for each symbol:
nm -S --size-sort -t d a.out | awk '{gsub ("0*", "", $2); print $4 " " $2}'
nm list symbols from object files. We are using -S to print the size of defined symbols in the second column and -t d to make output in decimal format.
$ objdump a.out -t|grep "F .text"|cut -f 2-3
00000000000000b1 __slow_div
00000000000000d8 __array_compact_and_grow
00000000000000dc __log
00000000000000de __to_utf
00000000000000e9 __string_compact
00000000000000fe __gc_release
000000000000001d __gc_check
000000000000001f array_compact
00000000000001e8 __string_split
000000000000002a c_roots_clear
000000000000002f f
000000000000002f _start
000000000000002f wpr
000000000000003d array_get
Not perfect, sizes are in hex, but it's easy to write a script that will convert to decimals and sort arbitrarily.

Allwinner a64 - switch from aarch32 to aarch64 by warm reset

I want to deploy a simple bare metal software on the Pine64 board, hosting Allwinner A64 SoC. The configuration is following: when powered on, boot0 starts u-boot, which loads my hello.bin to RAM (0x40000000) and starts executing it. The thing is that it is in aarch32 execution state and I want aarch64.
I have found out a way how to do it as in this patch. Some background also on the wiki.
I have copied the code and the objdump -d hello.o returns identical results as in the link:
Disassembly of section .text:
00000000 <_reset>:
0: e59f0024 ldr r0, [pc, #36] ; 2c <_reset+0x2c>
4: e59f1024 ldr r1, [pc, #36] ; 30 <_reset+0x30>
8: e5801000 str r1, [r0]
c: f57ff04f dsb sy
10: f57ff06f isb sy
14: ee1c0f50 mrc 15, 0, r0, cr12, cr0, {2}
18: e3800003 orr r0, r0, #3
1c: ee0c0f50 mcr 15, 0, r0, cr12, cr0, {2}
20: f57ff06f isb sy
24: e320f003 wfi
28: eafffffe b 28 <_reset+0x28>
2c: 017000a0 .word 0x017000a0
30: 40008000 .word 0x40008000
It is supposed to perform a warm-reset and start executin at 0x40008000 in aarch64 execution state. But when running I am getting Undefined instruction error and it restarts in the same state and starts from 0x0.
## Starting application at 0x40000000 ...
undefined instruction
pc : [<40000018>] lr : [<7ff1d054>]
sp : 76eb8a90 ip : 00000030 fp : 7ff1d00c
r10: 00000002 r9 : 76ed0ea0 r8 : 7ffb5340
r7 : 77f1bd78 r6 : 40000000 r5 : 00000002 r4 : 77f1bd7c
r3 : 40000000 r2 : 77f1bd7c r1 : 40008000 r0 : 017000a0
Flags: nZCv IRQs on FIQs off Mode SVC_32
Resetting CPU ...
Why is that?
EDIT:
The first problem was noticed by #Frant below, the binary that should be linked with different .text section address, that is start from 0x40000000 instead of 0x0.
It also couldn't work loaded by u-boot, that is in EL2. In order to write to RMR one needs to be in EL3. This is possible with FEL method.
NOTE:
After facing this problem I was asking around for some help and apparently I was using an old way of flashing the board. Since some time Pine64 got much better support and now it is possible to boot it in two more convenient ways:
* mainline u-boot with atf, that will directly generate a binary one can flash to SD card, and drops you in EL2,
* using the sunxi-fel tool, as described below, which is very convenient if one does not want to re-flash SD card all the time, drops you in EL3 (WARNING: sunxi wiki is a bit misleading on the sunxi-fel command arguments, these one below worked for me).
My answer is an attempt to answer the following question: Does the aarch32 state-switching code you are using work ? The good new is that the code you are using works fine. The bad new is that something else may not work properly in your environment.This would not surprise me much given the terrible state of all Allwinner out-of-the box BSPs.
Since I did not know which exact versions of boot0 and u-boot you were using, I tested your code using Andre Przywara's FEL-capable SPL binaries for A64/H5 - see the FEL Booting section of the A64 entry for more details - and sunxi-fel:This does remove the boot0 and u-boot you are using as potential culprits.
The Minimal, Complete, and Verifiable example I built for testing your code requires:
Removing the SD card from the Pine64, so that it will enter the FEL mode at power-up,
A male-A to male-A USB 2.0 cable for connecting your PC to the upper USB host receptacle of the Pine64.
A bash script, build.sh, for building sunxi-tools, retrieving the FEL-capable SPL binaries,
rmr_switch.S, a version of rmr_switch.S minus comments plus a symbol to be pre-processed for setting the start address without having to modify the file all the time,
rmr_switch2.S, a version of the rmr_switch.S mentionned above, but using r0 and r1 the way they are being used in the patch you were referencing.
uart-aarch32.s, an aarch32 program displaying *** Hello from aarch32! *** on UART0,
uart-aarch64.s, an aarch64 program displaying *** Hello from aarch64! *** on UART0.
Here is the content for each of the required files:
build.sh:
#!/bin/bash
# usage:
# CROSS_COMPILE_AARCH64=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_aarch64-elf/bin/aarch64-elf- CROSS_COMPILE_AARCH32=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-eabi/bin/arm-eabi- ./build.sh
clear
CROSS_COMPILE_AARCH64=${CROSS_COMPILE_AARCH64:-/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_aarch64-elf/bin/aarch64-elf-}
CROSS_COMPILE_AARCH32=${CROSS_COMPILE_AARCH32:-/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-eabi/bin/arm-eabi-}
SOC=${SOC:-a64}
#AARCH32_START_ADDRESS=0x42000000
#AARCH64_START_ADDRESS=0x42010000
AARCH32_START_ADDRESS=0x40000000
AARCH64_START_ADDRESS=0x40008000
SUNXI_FEL=sunxi-tools/sunxi-fel
install_sunxi_tools()
{
if [ ! -f ${SUNXI_FEL} ]
then
git clone --branch v1.4.2 https://github.com/linux-sunxi/sunxi-tools
pushd sunxi-tools
make
popd
fi
}
retrieve_spl_aarch32()
{
if [ ! -f sunxi-a64-spl32-ddr3.bin ]
then
wget https://github.com/apritzel/pine64/raw/master/binaries/sunxi-a64-spl32-ddr3.bin
fi
if [ ! -f sunxi-h5-spl32-ddr3.bin ]
then
wget https://github.com/apritzel/pine64/raw/master/binaries/sunxi-h5-spl32-ddr3.bin
fi
}
test_aarch32()
{
# testing aarch32 program
PROGRAM=uart-aarch32.s
BASE=${PROGRAM%%.*}
${CROSS_COMPILE_AARCH32}gcc -O0 -nostdlib -nostartfiles -e ${AARCH64_START_ADDRESS} -Wl,-Ttext=${AARCH32_START_ADDRESS} -o ${BASE}.elf ${BASE}.s
${CROSS_COMPILE_AARCH32}objcopy --remove-section .note.gnu.build-id ${BASE}.elf
${CROSS_COMPILE_AARCH32}objcopy --remove-section .ARM.attributes ${BASE}.elf
${CROSS_COMPILE_AARCH32}objdump -D ${BASE}.elf > ${BASE}.lst
${CROSS_COMPILE_AARCH32}objcopy -O binary ${BASE}.elf ${BASE}.bin
${CROSS_COMPILE_AARCH32}objcopy ${BASE}.elf -O srec ${BASE}.srec
echo "------------------ test uart-aarch32 -----------------------------"
echo sudo ${SUNXI_FEL} spl sunxi-${SOC}-spl32-ddr3.bin
echo sudo ${SUNXI_FEL} write ${AARCH32_START_ADDRESS} uart-aarch32.bin
echo sudo ${SUNXI_FEL} exe ${AARCH32_START_ADDRESS}
echo "------------------------------------------------------------------"
}
test_aarch64()
{
# testing aarch64 program
PROGRAM=uart-aarch64.s
BASE=${PROGRAM%%.*}
${CROSS_COMPILE_AARCH64}gcc -O0 -nostdlib -nostartfiles -e ${AARCH64_START_ADDRESS} -Wl,-Ttext=${AARCH64_START_ADDRESS} -o ${BASE}.elf ${BASE}.s
${CROSS_COMPILE_AARCH64}objcopy --remove-section .note.gnu.build-id ${BASE}.elf
${CROSS_COMPILE_AARCH64}objcopy --remove-section .ARM.attributes ${BASE}.elf
${CROSS_COMPILE_AARCH64}objdump -D ${BASE}.elf > ${BASE}.lst
${CROSS_COMPILE_AARCH64}objcopy -O binary ${BASE}.elf ${BASE}.bin
${CROSS_COMPILE_AARCH64}objcopy ${BASE}.elf -O srec ${BASE}.srec
echo "------------------ test uart-aarch64 -----------------------------"
echo sudo ${SUNXI_FEL} spl sunxi-${SOC}-spl32-ddr3.bin
echo sudo ${SUNXI_FEL} write ${AARCH64_START_ADDRESS} uart-aarch64.bin
echo sudo ${SUNXI_FEL} reset64 ${AARCH64_START_ADDRESS}
echo "------------------------------------------------------------------"
}
test_rmr_switch()
{
# compiling rmr_switch.s
PROGRAM=rmr_switch.s
BASE=${PROGRAM%%.*}
rm -f ${BASE}.s
${CROSS_COMPILE_AARCH64}cpp -DAARCH64_START_ADDRESS=${AARCH64_START_ADDRESS} ${BASE}.S > ${BASE}.s
${CROSS_COMPILE_AARCH32}gcc -O0 -nostdlib -nostartfiles -e ${AARCH32_START_ADDRESS} -Wl,-Ttext=${AARCH32_START_ADDRESS} -o ${BASE}.elf ${BASE}.s
${CROSS_COMPILE_AARCH32}objcopy --remove-section .note.gnu.build-id ${BASE}.elf
${CROSS_COMPILE_AARCH32}objcopy --remove-section .ARM.attributes ${BASE}.elf
${CROSS_COMPILE_AARCH32}objdump -D ${BASE}.elf > ${BASE}.lst
${CROSS_COMPILE_AARCH32}objcopy -O binary ${BASE}.elf ${BASE}.bin
${CROSS_COMPILE_AARCH32}objcopy ${BASE}.elf -O srec ${BASE}.srec
echo "------------------ test rmr_switch uart-aarch64 ------------------"
echo sudo ${SUNXI_FEL} spl sunxi-${SOC}-spl32-ddr3.bin
echo sudo ${SUNXI_FEL} write ${AARCH32_START_ADDRESS} rmr_switch.bin
echo sudo ${SUNXI_FEL} write ${AARCH64_START_ADDRESS} uart-aarch64.bin
echo sudo ${SUNXI_FEL} exe ${AARCH32_START_ADDRESS}
echo "------------------------------------------------------------------"
}
test_rmr_switch2()
{
# compiling rmr_switch2.s
PROGRAM=rmr_switch2.s
BASE=${PROGRAM%%.*}
rm -f ${BASE}.s
${CROSS_COMPILE_AARCH64}cpp -DAARCH64_START_ADDRESS=${AARCH64_START_ADDRESS} ${BASE}.S > ${BASE}.s
${CROSS_COMPILE_AARCH32}gcc -O0 -nostdlib -nostartfiles -e ${AARCH32_START_ADDRESS} -Wl,-Ttext=${AARCH32_START_ADDRESS} -o ${BASE}.elf ${BASE}.s
${CROSS_COMPILE_AARCH32}objcopy --remove-section .note.gnu.build-id ${BASE}.elf
${CROSS_COMPILE_AARCH32}objcopy --remove-section .ARM.attributes ${BASE}.elf
${CROSS_COMPILE_AARCH32}objdump -D ${BASE}.elf > ${BASE}.lst
${CROSS_COMPILE_AARCH32}objcopy -O binary ${BASE}.elf ${BASE}.bin
${CROSS_COMPILE_AARCH32}objcopy ${BASE}.elf -O srec ${BASE}.srec
echo "------------------ test rmr_switch2 uart-aarch64 -----------------"
echo sudo ${SUNXI_FEL} spl sunxi-${SOC}-spl32-ddr3.bin
echo sudo ${SUNXI_FEL} write ${AARCH32_START_ADDRESS} rmr_switch2.bin
echo sudo ${SUNXI_FEL} write ${AARCH64_START_ADDRESS} uart-aarch64.bin
echo sudo ${SUNXI_FEL} exe ${AARCH32_START_ADDRESS}
echo "------------------------------------------------------------------"
}
# prerequisites
install_sunxi_tools
retrieve_spl_aarch32
# test
test_aarch32
test_aarch64
test_rmr_switch
test_rmr_switch2
rmr_switch.S:
.text
ldr r1, =0x017000a0 # MMIO mapped RVBAR[0] register
ldr r0, =AARCH64_START_ADDRESS # start address, to be replaced
str r0, [r1]
dsb sy
isb sy
mrc 15, 0, r0, cr12, cr0, 2 # read RMR register
orr r0, r0, #3 # request reset in AArch64
mcr 15, 0, r0, cr12, cr0, 2 # write RMR register
isb sy
1: wfi
b 1b
rmr_switch2.S:
.text
ldr r0, =0x017000a0 # MMIO mapped RVBAR[0] register
ldr r1, =AARCH64_START_ADDRESS # start address, to be replaced
str r1, [r0]
dsb sy
isb sy
mrc 15, 0, r0, cr12, cr0, 2 # read RMR register
orr r0, r0, #3 # request reset in AArch64
mcr 15, 0, r0, cr12, cr0, 2 # write RMR register
isb sy
1: wfi
b 1b
uart-aarch32.s:
.code 32
.text
ldr r1,=0x01C28000
ldr r2,=message
loop: ldrb r0, [r2]
add r2, r2, #1
cmp r0, #0
beq completed
strb r0, [r1]
b loop
completed: b .
.data
message:
.asciz "*** Hello from aarch32! ***"
.end
uart-aarch64.s:
.text
ldr x1,=0x01C28000
ldr x2,=message
loop: ldrb w0, [x2]
add x2, x2, #1
cmp w0, #0
beq completed
strb w0, [x1]
b loop
completed: b .
.data
message:
.asciz "*** Hello from aarch64! ***"
.end
Once all the files are in the same directory, the test procedure would be:
Execute build.sh: You can specify the SOC you are using A64 (default) or H5, and the aarch32/aarch64 toolchains in the command-line:
CROSS_COMPILE_AARCH64=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_aarch64-elf/bin/aarch64-elf- CROSS_COMPILE_AARCH32=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-eabi/bin/arm-eabi- ./build.sh
The output should look like this, (I removed harmless warnings):
------------------ test uart-aarch32 -----------------------------
sudo sunxi-tools/sunxi-fel spl sunxi-a64-spl32-ddr3.bin
sudo sunxi-tools/sunxi-fel write 0x40000000 uart-aarch32.bin
sudo sunxi-tools/sunxi-fel exe 0x40000000
------------------ test uart-aarch64 -----------------------------
sudo sunxi-tools/sunxi-fel spl sunxi-a64-spl32-ddr3.bin
sudo sunxi-tools/sunxi-fel write 0x40008000 uart-aarch64.bin
sudo sunxi-tools/sunxi-fel reset64 0x40008000
------------------ test rmr_switch uart-aarch64 ------------------
sudo sunxi-tools/sunxi-fel spl sunxi-a64-spl32-ddr3.bin
sudo sunxi-tools/sunxi-fel write 0x40000000 rmr_switch.bin
sudo sunxi-tools/sunxi-fel write 0x40008000 uart-aarch64.bin
sudo sunxi-tools/sunxi-fel exe 0x40000000
------------------ test rmr_switch2 uart-aarch64 -----------------
sudo sunxi-tools/sunxi-fel spl sunxi-a64-spl32-ddr3.bin
sudo sunxi-tools/sunxi-fel write 0x40000000 rmr_switch2.bin
sudo sunxi-tools/sunxi-fel write 0x40008000 uart-aarch64.bin
sudo sunxi-tools/sunxi-fel exe 0x40000000
------------------------------------------------------------------
Now, before entering the sunxi-fel commands required for each of the four tests, you need to unplug the Pine64 from its power source and from any USB host receptacle it may be plugged into (USB TTL uart, male-A to male-A USB cable). Reconnect the Pine64 to its power source, then re-plug USB cables.
lsusb should now display:
Bus 001 Device 016: ID 1f3a:efe8 Onda (unverified) V972 tablet in flashing mode
Output on the serial console for the four tests should be:
test uart-aarch32 (verifying an aarch32 program runs from 0x40000000):
U-Boot SPL 2018.01-00007-gdb0ecc9b42 (Feb 23 2018 - 00:50:52)
DRAM: 512 MiB
Trying to boot from FEL
*** Hello from aarch32! ***
test uart-aarch64 (verifying an aarch64 program runs from 0x40008000):
U-Boot SPL 2018.01-00007-gdb0ecc9b42 (Feb 23 2018 - 00:50:52)
DRAM: 512 MiB
Trying to boot from FEL
*** Hello from aarch64! ***
test test rmr_switch uart-aarch64 (running rmr_switch from 0x40000000, which will switch into aarch64 state and execute uart-aarch64 from 0x40008000):
U-Boot SPL 2018.01-00007-gdb0ecc9b42 (Feb 23 2018 - 00:50:52)
DRAM: 512 MiB
Trying to boot from FEL
*** Hello from aarch64! ***
test test rmr_switch2 uart-aarch64 (running rmr_switch2 from 0x40000000, which will switch into aarch64 state and execute uart-aarch64 from 0x40008000):
U-Boot SPL 2018.01-00007-gdb0ecc9b42 (Feb 23 2018 - 00:50:52)
DRAM: 512 MiB
Trying to boot from FEL
*** Hello from aarch64! ***
It is worth mentioning that those tests can be performed on Windows using Linaro mingw32 toolchains, a Windows version of sunxi-fel, and Zadig.
Bottom line, the code you were using seems to be working well, and the rmr_switch2.s code I assembled is the same (I guess) that the one you are using:
rmr_switch2.elf: file format elf32-littlearm
Disassembly of section .text:
40000000 <.text>:
40000000: e59f0024 ldr r0, [pc, #36] ; 4000002c <.text+0x2c>
40000004: e59f1024 ldr r1, [pc, #36] ; 40000030 <.text+0x30>
40000008: e5801000 str r1, [r0]
4000000c: f57ff04f dsb sy
40000010: f57ff06f isb sy
40000014: ee1c0f50 mrc 15, 0, r0, cr12, cr0, {2}
40000018: e3800003 orr r0, r0, #3
4000001c: ee0c0f50 mcr 15, 0, r0, cr12, cr0, {2}
40000020: f57ff06f isb sy
40000024: e320f003 wfi
40000028: eafffffd b 40000024 <.text+0x24>
4000002c: 017000a0 cmneq r0, r0, lsr #1
40000030: 40008000 andmi r8, r0, r0
The examples were was successfully tested on an H5-based OrangePI PC2. Command-line for running build.sh should be:
SOC=h5 CROSS_COMPILE_AARCH64=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_aarch64-elf/bin/aarch64-elf- CROSS_COMPILE_AARCH32=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-eabi/bin/arm-eabi- ./build.sh
Output for build.sh, and therefore sunxi-fel commands to be executed, will be slightly different, since a different, H5-specific, FEL-capable SPL will have to be used.
I noticed there is a small difference between the code you are using and rmr_switch2 code, but since it comes after the state switch/after wfi, it should not matter I guess - I am assuming the code you assembled was slightly different itself:
Yours (.o):
28: eafffffe b 28 <_reset+0x28>
Mine (.elf):
40000028: eafffffd b 40000024 <.text+0x24>
I hope this help.

Bash append a list to end element of array

I am attempting to add a list to the end of a bash array that already contains elements, the array contains {a..z} and I want to add {0..9} to the end of that list, I've attempted += and this doesn't work it clears the array in my case.
while [ ! $# -eq 0 ] # Argument selector for CLI input
do
case "$1" in
--num | -n)
chars=( {0..9} )
;;
--char | -c)
chars=( {a..z} )
;;
--upper-char | -C)
chars=( {A..Z} )
;;
--help | -h)
echo "Type the program name with an argument -n for numbers -c for lowercase char and -C for uppercase"
exit
;;
esac
case "$2" in
--num | -n)
chars[${#chars[#]}]=( {0..9} )
;;
--char | -c
chars[${#chars[#]}]=( {a..z} )
;;
--upper-char | -C)
chars[${#chars[#]}]=( {A..Z} )
;;
esac
shift
done
I'm hoping to do a third case statement once I've worked out how to append lists, best case scenario is not having to hard code the array every time I want to add items.
Appending to an array does work with +=.
To you it appeared to be overwriting because of some other mistake in the code. This should work:
while [ ! $# -eq 0 ] # Argument selector for CLI input
do
case "$1" in
--num | -n)
chars=( {0..9} )
;;
--char | -c)
chars=( {a..z} )
;;
--upper-char | -C)
chars=( {A..Z} )
;;
--help | -h)
echo "Type the program name with an argument -n for numbers -c for lowercase char and -C for uppercase"
exit
;;
esac
case "$2" in
--num | -n)
chars+=( {0..9} )
;;
--char | -c)
chars+=( {a..z} )
;;
--upper-char | -C)
chars+=( {A..Z} )
;;
esac
shift 2
done
When it didn't work for you,
I suspect you had just shift instead of shift 2,
and expected that script.sh -c -n would give you a..z and 0..9,
but it only gave 0..9.
What happened is after just one shift,
one argument still remained to process in the while loop,
and so in the next iteration chars got replaced with a chars=(...) statement.
I propose this alternative implementation:
chars=()
while [ $# != 0 ]; do
case "$1" in
--num | -n)
chars+=( {0..9} )
;;
--char | -c)
chars+=( {a..z} )
;;
--upper-char | -C)
chars+=( {A..Z} )
;;
--help | -h)
echo "Type the program name with an argument -n for numbers -c for lowercase char and -C for uppercase"
exit 1
;;
esac
shift
done

What is the performance difference between gawk and ....? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
This question has been discussed here on Meta and my answer give links to a test system to answer this.
The question often comes up about whether to use gawk or mawk or C or some other language due to performance so let's create a canonical question/answer for a trivial and typical awk program.
The result of this will be an answer that provides a comparison of the performance of different tools performing the basic text processing tasks of regexp matching and field splitting on a simple input file. If tool X is twice as fast as every other tool for this task then that is useful information. If all the tools take about the same amount of time then that is useful information too.
The way this will work is that over the next couple of days many people will contribute "answers" which are the programs to be tested and then one person (volunteers?) will test all of them on one platform (or a few people will test some subset on their platform so we can compare) and then all of the results will be collected into a single answer.
Given a 10 Million line input file created by this script:
$ awk 'BEGIN{for (i=1;i<=10000000;i++) print (i%5?"miss":"hit"),i," third\t \tfourth"}' > file
$ wc -l file
10000000 file
$ head -10 file
miss 1 third fourth
miss 2 third fourth
miss 3 third fourth
miss 4 third fourth
hit 5 third fourth
miss 6 third fourth
miss 7 third fourth
miss 8 third fourth
miss 9 third fourth
hit 10 third fourth
and given this awk script which prints the 4th then 1st then 3rd field of every line that starts with "hit" followed by an even number:
$ cat tst.awk
/hit [[:digit:]]*0 / { print $4, $1, $3 }
Here are the first 5 lines of expected output:
$ awk -f tst.awk file | head -5
fourth hit third
fourth hit third
fourth hit third
fourth hit third
fourth hit third
and here is the result when piped to a 2nd awk script to verify that the main script above is actually functioning exactly as intended:
$ awk -f tst.awk file |
awk '!seen[$0]++{unq++;r=$0} END{print ((unq==1) && (seen[r]==1000000) && (r=="fourth hit third")) ? "PASS" : "FAIL"}'
PASS
Here are the timing results of the 3rd execution of gawk 4.1.1 running in bash 4.3.33 on cygwin64:
$ time awk -f tst.awk file > /dev/null
real 0m4.711s
user 0m4.555s
sys 0m0.108s
Note the above is the 3rd execution to remove caching differences.
Can anyone provide the equivalent C, perl, python, whatever code to this:
$ cat tst.awk
/hit [[:digit:]]*0 / { print $4, $1, $3 }
i.e. find THAT REGEXP on a line (we're not looking for some other solution that works around the need for a regexp), split the line at each series of contiguous white space and print the 4th, then 1st, then 3rd fields separated by a single blank char?
If so we can test them all on one platform to see/record the performance differences.
The code contributed so far:
AWK (can be tested against gawk, etc. but mawk, nawk and perhaps others will require [0-9] instead of [:digit:])
awk '/hit [[:digit:]]*0 / { print $4, $1, $3 }' file
PHP
php -R 'if(preg_match("/hit \d*0 /", $argn)){$f=preg_split("/\s+/", $argn); echo $f[3]." ".$f[0]." ".$f[2];}' < file
shell
egrep 'hit [[:digit:]]*0 ' file | awk '{print $4, $1, $3}'
grep --mmap -E "^hit [[:digit:]]*0 " file | awk '{print $4, $1, $3 }'
Ruby
$ cat tst.rb
File.open("file").readlines.each do |line|
line.gsub(/(hit)\s[0-9]*0\s+(.*?)\s+(.*)/) { puts "#{$3} #{$1} #{$2}" }
end
$ ruby tst.rb
Perl
$ cat tst.pl
#!/usr/bin/perl -nl
# A solution much like the Ruby one but with atomic grouping
print "$4 $1 $3" if /^(hit)(?>\s+)(\d*0)(?>\s+)((?>[^\s]+))(?>\s+)(?>([^\s]+))$/
$ perl tst.pl file
Python
none yet
C
none yet
Applying egrep before awk gives a great speedup:
paul#home ~ % wc -l file
10000000 file
paul#home ~ % for i in {1..5}; do time egrep 'hit [[:digit:]]*0 ' file | awk '{print $4, $1, $3}' | wc -l ; done
1000000
egrep --color=auto 'hit [[:digit:]]*0 ' file 0.63s user 0.02s system 85% cpu 0.759 total
awk '{print $4, $1, $3}' 0.70s user 0.01s system 93% cpu 0.760 total
wc -l 0.00s user 0.02s system 2% cpu 0.760 total
1000000
egrep --color=auto 'hit [[:digit:]]*0 ' file 0.65s user 0.01s system 85% cpu 0.770 total
awk '{print $4, $1, $3}' 0.71s user 0.01s system 93% cpu 0.771 total
wc -l 0.00s user 0.02s system 2% cpu 0.771 total
1000000
egrep --color=auto 'hit [[:digit:]]*0 ' file 0.64s user 0.02s system 82% cpu 0.806 total
awk '{print $4, $1, $3}' 0.73s user 0.01s system 91% cpu 0.807 total
wc -l 0.02s user 0.00s system 2% cpu 0.807 total
1000000
egrep --color=auto 'hit [[:digit:]]*0 ' file 0.63s user 0.02s system 86% cpu 0.745 total
awk '{print $4, $1, $3}' 0.69s user 0.01s system 92% cpu 0.746 total
wc -l 0.00s user 0.02s system 2% cpu 0.746 total
1000000
egrep --color=auto 'hit [[:digit:]]*0 ' file 0.62s user 0.02s system 88% cpu 0.727 total
awk '{print $4, $1, $3}' 0.67s user 0.01s system 93% cpu 0.728 total
wc -l 0.00s user 0.02s system 2% cpu 0.728 total
versus:
paul#home ~ % for i in {1..5}; do time gawk '/hit [[:digit:]]*0 / { print $4, $1, $3 }' file > /dev/null; done
gawk '/hit [[:digit:]]*0 / { print $4, $1, $3 }' file > /dev/null 2.46s user 0.04s system 97% cpu 2.548 total
gawk '/hit [[:digit:]]*0 / { print $4, $1, $3 }' file > /dev/null 2.43s user 0.03s system 98% cpu 2.508 total
gawk '/hit [[:digit:]]*0 / { print $4, $1, $3 }' file > /dev/null 2.40s user 0.04s system 98% cpu 2.489 total
gawk '/hit [[:digit:]]*0 / { print $4, $1, $3 }' file > /dev/null 2.38s user 0.04s system 98% cpu 2.463 total
gawk '/hit [[:digit:]]*0 / { print $4, $1, $3 }' file > /dev/null 2.39s user 0.03s system 98% cpu 2.465 total
'nawk' is even slower!
paul#home ~ % for i in {1..5}; do time nawk '/hit [[:digit:]]*0 / { print $4, $1, $3 }' file > /dev/null; done
nawk '/hit [[:digit:]]*0 / { print $4, $1, $3 }' file > /dev/null 6.05s user 0.06s system 92% cpu 6.606 total
nawk '/hit [[:digit:]]*0 / { print $4, $1, $3 }' file > /dev/null 6.11s user 0.05s system 96% cpu 6.401 total
nawk '/hit [[:digit:]]*0 / { print $4, $1, $3 }' file > /dev/null 5.78s user 0.04s system 97% cpu 5.975 total
nawk '/hit [[:digit:]]*0 / { print $4, $1, $3 }' file > /dev/null 5.71s user 0.04s system 98% cpu 5.857 total
nawk '/hit [[:digit:]]*0 / { print $4, $1, $3 }' file > /dev/null 6.34s user 0.05s system 93% cpu 6.855 total
On OSX Yosemite
time bash -c 'grep --mmap -E "^hit [[:digit:]]*0 " file | awk '\''{print $4, $1, $3 }'\''' >/dev/null
real 0m5.741s
user 0m6.668s
sys 0m0.112s
first Idea
File.open("file").readlines.each do |line|
line.gsub(/(hit)\s[0-9]*0\s+(.*?)\s+(.*)/) { puts "#{$3} #{$1} #{$2}" }
end
Second idea
File.read("file").scan(/(hit)\s[[:digit:]]*0\s+(.*?)\s+(.*)/) { |f,s,t| puts "#{t} #{f} #{s}" }
Trying to get something able to compare answer I ended up creating a github repo here. Each push to this repo trigger a build on travis-ci which compose a markdown file pushed in turn to the gh-pages branch to update a web page with a view on the build results.
Anyone wishing to participate can fork the github repo, add tests and do a pull request wich I'll merge asap if it does not break the others tests.
mawk is slightly faster than gawk.
$ time bash -c 'mawk '\''/hit [[:digit:]]*0 / { print $4, $1, $3 }'\'' file | wc -l'
0
real 0m1.160s
user 0m0.484s
sys 0m0.052s
$ time bash -c 'gawk '\''/hit [[:digit:]]*0 / { print $4, $1, $3 }'\'' file | wc -l'
100000
real 0m1.648s
user 0m0.996s
sys 0m0.060s
(Only 1,000,000 lines in my input file. Best results of many displayed, though they were quite consistent.)
Here comes an equivalent in PHP:
$ time php -R 'if(preg_match("/hit \d*0 /", $argn)){$f=preg_split("/\s+/", $argn); echo $f[3]." ".$f[0]." ".$f[2];}' < file > /dev/null
real 2m42.407s
user 2m41.934s
sys 0m0.355s
compared to your awk:
$ time awk -f tst.awk file > /dev/null
real 0m3.271s
user 0m3.165s
sys 0m0.104s
I tried a different approach in PHP where I iterate trough the file manually, this makes things a lot faster but I'm still not impressed:
tst.php
<?php
$fd=fopen('file', 'r');
while($line = fgets($fd)){
if(preg_match("/hit \d*0 /", $line)){
$f=preg_split("/\s+/", $line);
echo $f[3]." ".$f[0]." ".$f[2]."\n";
}
}
fclose($fd);
Results:
$ time php tst.php > /dev/null
real 0m27.354s
user 0m27.042s
sys 0m0.296s

Getting multiple strings from SNMPWALK requests via Bash but unsure how to format for output

I haven't had too much experience with bash yet and I'm learning as I go. At work we're needing to use SNMP to pull data from wireless access points. I've been tasked to make a script to loop through a few thousand access points.
I'm pulling multiple strings of data by sequential requests via SNMPWALK. This is how I've set up my input:
IAP=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapName -Oqv)
MODEL=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::arrayModel -Oqv | cut -d ',' -f1)
CHANNEL=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapChannel -Oqv)
MAXHOUR=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapMaxStationsHour -Oqv)
MAXDAY=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapMaxStationsDay -Oqv)
MAXWEEK=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapMaxStationsWeek -Oqv)
MAXMONTH=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapMaxStationsMonth -Oqv)
MAXYEAR=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapMaxStationsYear -Oqv)
MAC=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapMacAddress -Oqv)
Here's an example of input as printed via echo:
iap1
iap2
iap3
iap4
iap5
iap6
iap7
iap8
XR4820
6
100
11
149
64
112
36
161
9
2
7
2
2
1
3
2
11
5
9
4
8
3
3
3
22
9
21
4
12
4
16
5
24
23
21
5
12
8
20
11
24
23
21
5
12
8
20
11
00:0f:7d:ea:b1:00-01
00:0f:7d:ea:b1:10-11
00:0f:7d:ea:b1:20-21
00:0f:7d:ea:b1:30-31
00:0f:7d:ea:b1:40-41
00:0f:7d:ea:b1:50-51
00:0f:7d:ea:b1:60-61
00:0f:7d:ea:b1:70-71
I need to take the data and output it to CSV. The problem is that I have to get them in order (ex. MODEL[0],CHANNEL[0],. . .,MAC[0],MODEL[1],CHANNEL[1], . . .). I can't seem to do this properly.
I made an attempt to use IFS as suggested by other answers in stackoverflow. I tried creating a for loop to then iterate through the elements:
IAP=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapName -Oqv)
if [ "$IAP" != "No Such Instance currently exists at this OID" ]
then
MODEL=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::arrayModel -Oqv | cut -d ',' -f1)
CHANNEL=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapChannel -Oqv)
IFS=' ' read -a CHANNEL <<< "$CHANNEL"
MAXHOUR=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapMaxStationsHour -Oqv)
IFS=' ' read -a MAXHOUR <<< "$MAXHOUR"
MAXDAY=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapMaxStationsDay -Oqv)
IFS=' ' read -a MAXDAY <<< "$MAXDAY"
MAXWEEK=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapMaxStationsWeek -Oqv)
IFS=' ' read -a MAXWEEK <<< "$MAXWEEK"
MAXMONTH=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapMaxStationsMonth -Oqv)
IFS=' ' read -a MAXMONTH <<< "$MAXMONTH"
MAXYEAR=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapMaxStationsYear -Oqv)
IFS=' ' read -a MAXYEAR <<< "$MAXYEAR"
MAC=$(snmpwalk -v 2c -c $COMMUNITY $DEVICE XIRRUS-MIB::iapMacAddress -Oqv)
IFS=' ' read -a MAC <<< "$MAC"
for ((i=0; i<${#IAP[#]}; ++i));
do
ia=${IAP[$i]}
m=${MAC[$i]}
c=${CHANNEL[$i]}
h=${MAXHOUR[$i]}
d=${MAXDAY[$i]}
w=${MAXWEEK[$i]}
mon=${MAXMONTH[$i]}
y=${MAXYEAR[$i]}
if [ -n "$m" ]
then
TEMP="$MODEL,$ia,$c,$h,$d,$w,$mon,$y"
if [ $i -gt 1 ]
then
OUTPUT=",$OUTPUT,$TEMP"
else
OUTPUT="$TEMP"
fi
fi
done
However, this is the output:
XR4820,iap1
iap2
iap3
iap4
iap5
iap6
iap7
iap8,11,8,11,11,11,11
While it should be:
iap1,6,9,11,22,24,24,iap2,100,2,5,9,23,23,iap3,...
Summarized:
My input is a handful of long strings that I need to parse and send the elements out to a .CSV file. They need to be in order, ie. get elements '0' from each, then elements '1' and so forth. I created a loop that isn't working but still prints some of the data. In my naive opinion the best solution is to convert each variable into an array and iterate through the elements in sequential order.
What is the proper way of taking each input variable and turning it into an array? The input for each variable can range from 2 to 16. I need to account for this dynamic sizing
Please look at my for loop. How can I simplify it while maintaining the output format? It's a mess from multiple changes
If I used IFS to take the data and place it into an array, what would I need to change?
You can make a variable (space delimited) an array byvar=( $var ), but I didn't get your questions completely.
If you edited your post and summarized the things that you want at the end of it, that would be helpful.
Edit (based on the comments):
In the case that your fields are seperated by '\n':
var1=`echo $var1 | tr "\n" " "`
var1=( $var1 )

Resources