Allwinner a64 - switch from aarch32 to aarch64 by warm reset - u-boot

Question

I want to deploy a simple bare metal software on the Pine64 board, hosting Allwinner A64 SoC. The configuration is following: when powered on, boot0 starts u-boot, which loads my hello.bin to RAM (0x40000000) and starts executing it. The thing is that it is in aarch32 execution state and I want aarch64.
I have found out a way how to do it as in this patch. Some background also on the wiki.
I have copied the code and the objdump -d hello.o returns identical results as in the link:
Disassembly of section .text:
00000000 <_reset>:
0: e59f0024 ldr r0, [pc, #36] ; 2c <_reset+0x2c>
4: e59f1024 ldr r1, [pc, #36] ; 30 <_reset+0x30>
8: e5801000 str r1, [r0]
c: f57ff04f dsb sy
10: f57ff06f isb sy
14: ee1c0f50 mrc 15, 0, r0, cr12, cr0, {2}
18: e3800003 orr r0, r0, #3
1c: ee0c0f50 mcr 15, 0, r0, cr12, cr0, {2}
20: f57ff06f isb sy
24: e320f003 wfi
28: eafffffe b 28 <_reset+0x28>
2c: 017000a0 .word 0x017000a0
30: 40008000 .word 0x40008000
It is supposed to perform a warm-reset and start executin at 0x40008000 in aarch64 execution state. But when running I am getting Undefined instruction error and it restarts in the same state and starts from 0x0.
## Starting application at 0x40000000 ...
undefined instruction
pc : [<40000018>] lr : [<7ff1d054>]
sp : 76eb8a90 ip : 00000030 fp : 7ff1d00c
r10: 00000002 r9 : 76ed0ea0 r8 : 7ffb5340
r7 : 77f1bd78 r6 : 40000000 r5 : 00000002 r4 : 77f1bd7c
r3 : 40000000 r2 : 77f1bd7c r1 : 40008000 r0 : 017000a0
Flags: nZCv IRQs on FIQs off Mode SVC_32
Resetting CPU ...
Why is that?
EDIT:
The first problem was noticed by #Frant below, the binary that should be linked with different .text section address, that is start from 0x40000000 instead of 0x0.
It also couldn't work loaded by u-boot, that is in EL2. In order to write to RMR one needs to be in EL3. This is possible with FEL method.
NOTE:
After facing this problem I was asking around for some help and apparently I was using an old way of flashing the board. Since some time Pine64 got much better support and now it is possible to boot it in two more convenient ways:
* mainline u-boot with atf, that will directly generate a binary one can flash to SD card, and drops you in EL2,
* using the sunxi-fel tool, as described below, which is very convenient if one does not want to re-flash SD card all the time, drops you in EL3 (WARNING: sunxi wiki is a bit misleading on the sunxi-fel command arguments, these one below worked for me).

Answer 1

My answer is an attempt to answer the following question: Does the aarch32 state-switching code you are using work ? The good new is that the code you are using works fine. The bad new is that something else may not work properly in your environment.This would not surprise me much given the terrible state of all Allwinner out-of-the box BSPs.
Since I did not know which exact versions of boot0 and u-boot you were using, I tested your code using Andre Przywara's FEL-capable SPL binaries for A64/H5 - see the FEL Booting section of the A64 entry for more details - and sunxi-fel:This does remove the boot0 and u-boot you are using as potential culprits.
The Minimal, Complete, and Verifiable example I built for testing your code requires:
Removing the SD card from the Pine64, so that it will enter the FEL mode at power-up,
A male-A to male-A USB 2.0 cable for connecting your PC to the upper USB host receptacle of the Pine64.
A bash script, build.sh, for building sunxi-tools, retrieving the FEL-capable SPL binaries,
rmr_switch.S, a version of rmr_switch.S minus comments plus a symbol to be pre-processed for setting the start address without having to modify the file all the time,
rmr_switch2.S, a version of the rmr_switch.S mentionned above, but using r0 and r1 the way they are being used in the patch you were referencing.
uart-aarch32.s, an aarch32 program displaying *** Hello from aarch32! *** on UART0,
uart-aarch64.s, an aarch64 program displaying *** Hello from aarch64! *** on UART0.
Here is the content for each of the required files:
build.sh:
#!/bin/bash
# usage:
# CROSS_COMPILE_AARCH64=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_aarch64-elf/bin/aarch64-elf- CROSS_COMPILE_AARCH32=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-eabi/bin/arm-eabi- ./build.sh
clear
CROSS_COMPILE_AARCH64=${CROSS_COMPILE_AARCH64:-/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_aarch64-elf/bin/aarch64-elf-}
CROSS_COMPILE_AARCH32=${CROSS_COMPILE_AARCH32:-/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-eabi/bin/arm-eabi-}
SOC=${SOC:-a64}
#AARCH32_START_ADDRESS=0x42000000
#AARCH64_START_ADDRESS=0x42010000
AARCH32_START_ADDRESS=0x40000000
AARCH64_START_ADDRESS=0x40008000
SUNXI_FEL=sunxi-tools/sunxi-fel
install_sunxi_tools()
{
if [ ! -f ${SUNXI_FEL} ]
then
git clone --branch v1.4.2 https://github.com/linux-sunxi/sunxi-tools
pushd sunxi-tools
make
popd
fi
}
retrieve_spl_aarch32()
{
if [ ! -f sunxi-a64-spl32-ddr3.bin ]
then
wget https://github.com/apritzel/pine64/raw/master/binaries/sunxi-a64-spl32-ddr3.bin
fi
if [ ! -f sunxi-h5-spl32-ddr3.bin ]
then
wget https://github.com/apritzel/pine64/raw/master/binaries/sunxi-h5-spl32-ddr3.bin
fi
}
test_aarch32()
{
# testing aarch32 program
PROGRAM=uart-aarch32.s
BASE=${PROGRAM%%.*}
${CROSS_COMPILE_AARCH32}gcc -O0 -nostdlib -nostartfiles -e ${AARCH64_START_ADDRESS} -Wl,-Ttext=${AARCH32_START_ADDRESS} -o ${BASE}.elf ${BASE}.s
${CROSS_COMPILE_AARCH32}objcopy --remove-section .note.gnu.build-id ${BASE}.elf
${CROSS_COMPILE_AARCH32}objcopy --remove-section .ARM.attributes ${BASE}.elf
${CROSS_COMPILE_AARCH32}objdump -D ${BASE}.elf > ${BASE}.lst
${CROSS_COMPILE_AARCH32}objcopy -O binary ${BASE}.elf ${BASE}.bin
${CROSS_COMPILE_AARCH32}objcopy ${BASE}.elf -O srec ${BASE}.srec
echo "------------------ test uart-aarch32 -----------------------------"
echo sudo ${SUNXI_FEL} spl sunxi-${SOC}-spl32-ddr3.bin
echo sudo ${SUNXI_FEL} write ${AARCH32_START_ADDRESS} uart-aarch32.bin
echo sudo ${SUNXI_FEL} exe ${AARCH32_START_ADDRESS}
echo "------------------------------------------------------------------"
}
test_aarch64()
{
# testing aarch64 program
PROGRAM=uart-aarch64.s
BASE=${PROGRAM%%.*}
${CROSS_COMPILE_AARCH64}gcc -O0 -nostdlib -nostartfiles -e ${AARCH64_START_ADDRESS} -Wl,-Ttext=${AARCH64_START_ADDRESS} -o ${BASE}.elf ${BASE}.s
${CROSS_COMPILE_AARCH64}objcopy --remove-section .note.gnu.build-id ${BASE}.elf
${CROSS_COMPILE_AARCH64}objcopy --remove-section .ARM.attributes ${BASE}.elf
${CROSS_COMPILE_AARCH64}objdump -D ${BASE}.elf > ${BASE}.lst
${CROSS_COMPILE_AARCH64}objcopy -O binary ${BASE}.elf ${BASE}.bin
${CROSS_COMPILE_AARCH64}objcopy ${BASE}.elf -O srec ${BASE}.srec
echo "------------------ test uart-aarch64 -----------------------------"
echo sudo ${SUNXI_FEL} spl sunxi-${SOC}-spl32-ddr3.bin
echo sudo ${SUNXI_FEL} write ${AARCH64_START_ADDRESS} uart-aarch64.bin
echo sudo ${SUNXI_FEL} reset64 ${AARCH64_START_ADDRESS}
echo "------------------------------------------------------------------"
}
test_rmr_switch()
{
# compiling rmr_switch.s
PROGRAM=rmr_switch.s
BASE=${PROGRAM%%.*}
rm -f ${BASE}.s
${CROSS_COMPILE_AARCH64}cpp -DAARCH64_START_ADDRESS=${AARCH64_START_ADDRESS} ${BASE}.S > ${BASE}.s
${CROSS_COMPILE_AARCH32}gcc -O0 -nostdlib -nostartfiles -e ${AARCH32_START_ADDRESS} -Wl,-Ttext=${AARCH32_START_ADDRESS} -o ${BASE}.elf ${BASE}.s
${CROSS_COMPILE_AARCH32}objcopy --remove-section .note.gnu.build-id ${BASE}.elf
${CROSS_COMPILE_AARCH32}objcopy --remove-section .ARM.attributes ${BASE}.elf
${CROSS_COMPILE_AARCH32}objdump -D ${BASE}.elf > ${BASE}.lst
${CROSS_COMPILE_AARCH32}objcopy -O binary ${BASE}.elf ${BASE}.bin
${CROSS_COMPILE_AARCH32}objcopy ${BASE}.elf -O srec ${BASE}.srec
echo "------------------ test rmr_switch uart-aarch64 ------------------"
echo sudo ${SUNXI_FEL} spl sunxi-${SOC}-spl32-ddr3.bin
echo sudo ${SUNXI_FEL} write ${AARCH32_START_ADDRESS} rmr_switch.bin
echo sudo ${SUNXI_FEL} write ${AARCH64_START_ADDRESS} uart-aarch64.bin
echo sudo ${SUNXI_FEL} exe ${AARCH32_START_ADDRESS}
echo "------------------------------------------------------------------"
}
test_rmr_switch2()
{
# compiling rmr_switch2.s
PROGRAM=rmr_switch2.s
BASE=${PROGRAM%%.*}
rm -f ${BASE}.s
${CROSS_COMPILE_AARCH64}cpp -DAARCH64_START_ADDRESS=${AARCH64_START_ADDRESS} ${BASE}.S > ${BASE}.s
${CROSS_COMPILE_AARCH32}gcc -O0 -nostdlib -nostartfiles -e ${AARCH32_START_ADDRESS} -Wl,-Ttext=${AARCH32_START_ADDRESS} -o ${BASE}.elf ${BASE}.s
${CROSS_COMPILE_AARCH32}objcopy --remove-section .note.gnu.build-id ${BASE}.elf
${CROSS_COMPILE_AARCH32}objcopy --remove-section .ARM.attributes ${BASE}.elf
${CROSS_COMPILE_AARCH32}objdump -D ${BASE}.elf > ${BASE}.lst
${CROSS_COMPILE_AARCH32}objcopy -O binary ${BASE}.elf ${BASE}.bin
${CROSS_COMPILE_AARCH32}objcopy ${BASE}.elf -O srec ${BASE}.srec
echo "------------------ test rmr_switch2 uart-aarch64 -----------------"
echo sudo ${SUNXI_FEL} spl sunxi-${SOC}-spl32-ddr3.bin
echo sudo ${SUNXI_FEL} write ${AARCH32_START_ADDRESS} rmr_switch2.bin
echo sudo ${SUNXI_FEL} write ${AARCH64_START_ADDRESS} uart-aarch64.bin
echo sudo ${SUNXI_FEL} exe ${AARCH32_START_ADDRESS}
echo "------------------------------------------------------------------"
}
# prerequisites
install_sunxi_tools
retrieve_spl_aarch32
# test
test_aarch32
test_aarch64
test_rmr_switch
test_rmr_switch2
rmr_switch.S:
.text
ldr r1, =0x017000a0 # MMIO mapped RVBAR[0] register
ldr r0, =AARCH64_START_ADDRESS # start address, to be replaced
str r0, [r1]
dsb sy
isb sy
mrc 15, 0, r0, cr12, cr0, 2 # read RMR register
orr r0, r0, #3 # request reset in AArch64
mcr 15, 0, r0, cr12, cr0, 2 # write RMR register
isb sy
1: wfi
b 1b
rmr_switch2.S:
.text
ldr r0, =0x017000a0 # MMIO mapped RVBAR[0] register
ldr r1, =AARCH64_START_ADDRESS # start address, to be replaced
str r1, [r0]
dsb sy
isb sy
mrc 15, 0, r0, cr12, cr0, 2 # read RMR register
orr r0, r0, #3 # request reset in AArch64
mcr 15, 0, r0, cr12, cr0, 2 # write RMR register
isb sy
1: wfi
b 1b
uart-aarch32.s:
.code 32
.text
ldr r1,=0x01C28000
ldr r2,=message
loop: ldrb r0, [r2]
add r2, r2, #1
cmp r0, #0
beq completed
strb r0, [r1]
b loop
completed: b .
.data
message:
.asciz "*** Hello from aarch32! ***"
.end
uart-aarch64.s:
.text
ldr x1,=0x01C28000
ldr x2,=message
loop: ldrb w0, [x2]
add x2, x2, #1
cmp w0, #0
beq completed
strb w0, [x1]
b loop
completed: b .
.data
message:
.asciz "*** Hello from aarch64! ***"
.end
Once all the files are in the same directory, the test procedure would be:
Execute build.sh: You can specify the SOC you are using A64 (default) or H5, and the aarch32/aarch64 toolchains in the command-line:
CROSS_COMPILE_AARCH64=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_aarch64-elf/bin/aarch64-elf- CROSS_COMPILE_AARCH32=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-eabi/bin/arm-eabi- ./build.sh
The output should look like this, (I removed harmless warnings):
------------------ test uart-aarch32 -----------------------------
sudo sunxi-tools/sunxi-fel spl sunxi-a64-spl32-ddr3.bin
sudo sunxi-tools/sunxi-fel write 0x40000000 uart-aarch32.bin
sudo sunxi-tools/sunxi-fel exe 0x40000000
------------------ test uart-aarch64 -----------------------------
sudo sunxi-tools/sunxi-fel spl sunxi-a64-spl32-ddr3.bin
sudo sunxi-tools/sunxi-fel write 0x40008000 uart-aarch64.bin
sudo sunxi-tools/sunxi-fel reset64 0x40008000
------------------ test rmr_switch uart-aarch64 ------------------
sudo sunxi-tools/sunxi-fel spl sunxi-a64-spl32-ddr3.bin
sudo sunxi-tools/sunxi-fel write 0x40000000 rmr_switch.bin
sudo sunxi-tools/sunxi-fel write 0x40008000 uart-aarch64.bin
sudo sunxi-tools/sunxi-fel exe 0x40000000
------------------ test rmr_switch2 uart-aarch64 -----------------
sudo sunxi-tools/sunxi-fel spl sunxi-a64-spl32-ddr3.bin
sudo sunxi-tools/sunxi-fel write 0x40000000 rmr_switch2.bin
sudo sunxi-tools/sunxi-fel write 0x40008000 uart-aarch64.bin
sudo sunxi-tools/sunxi-fel exe 0x40000000
------------------------------------------------------------------
Now, before entering the sunxi-fel commands required for each of the four tests, you need to unplug the Pine64 from its power source and from any USB host receptacle it may be plugged into (USB TTL uart, male-A to male-A USB cable). Reconnect the Pine64 to its power source, then re-plug USB cables.
lsusb should now display:
Bus 001 Device 016: ID 1f3a:efe8 Onda (unverified) V972 tablet in flashing mode
Output on the serial console for the four tests should be:
test uart-aarch32 (verifying an aarch32 program runs from 0x40000000):
U-Boot SPL 2018.01-00007-gdb0ecc9b42 (Feb 23 2018 - 00:50:52)
DRAM: 512 MiB
Trying to boot from FEL
*** Hello from aarch32! ***
test uart-aarch64 (verifying an aarch64 program runs from 0x40008000):
U-Boot SPL 2018.01-00007-gdb0ecc9b42 (Feb 23 2018 - 00:50:52)
DRAM: 512 MiB
Trying to boot from FEL
*** Hello from aarch64! ***
test test rmr_switch uart-aarch64 (running rmr_switch from 0x40000000, which will switch into aarch64 state and execute uart-aarch64 from 0x40008000):
U-Boot SPL 2018.01-00007-gdb0ecc9b42 (Feb 23 2018 - 00:50:52)
DRAM: 512 MiB
Trying to boot from FEL
*** Hello from aarch64! ***
test test rmr_switch2 uart-aarch64 (running rmr_switch2 from 0x40000000, which will switch into aarch64 state and execute uart-aarch64 from 0x40008000):
U-Boot SPL 2018.01-00007-gdb0ecc9b42 (Feb 23 2018 - 00:50:52)
DRAM: 512 MiB
Trying to boot from FEL
*** Hello from aarch64! ***
It is worth mentioning that those tests can be performed on Windows using Linaro mingw32 toolchains, a Windows version of sunxi-fel, and Zadig.
Bottom line, the code you were using seems to be working well, and the rmr_switch2.s code I assembled is the same (I guess) that the one you are using:
rmr_switch2.elf: file format elf32-littlearm
Disassembly of section .text:
40000000 <.text>:
40000000: e59f0024 ldr r0, [pc, #36] ; 4000002c <.text+0x2c>
40000004: e59f1024 ldr r1, [pc, #36] ; 40000030 <.text+0x30>
40000008: e5801000 str r1, [r0]
4000000c: f57ff04f dsb sy
40000010: f57ff06f isb sy
40000014: ee1c0f50 mrc 15, 0, r0, cr12, cr0, {2}
40000018: e3800003 orr r0, r0, #3
4000001c: ee0c0f50 mcr 15, 0, r0, cr12, cr0, {2}
40000020: f57ff06f isb sy
40000024: e320f003 wfi
40000028: eafffffd b 40000024 <.text+0x24>
4000002c: 017000a0 cmneq r0, r0, lsr #1
40000030: 40008000 andmi r8, r0, r0
The examples were was successfully tested on an H5-based OrangePI PC2. Command-line for running build.sh should be:
SOC=h5 CROSS_COMPILE_AARCH64=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_aarch64-elf/bin/aarch64-elf- CROSS_COMPILE_AARCH32=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-eabi/bin/arm-eabi- ./build.sh
Output for build.sh, and therefore sunxi-fel commands to be executed, will be slightly different, since a different, H5-specific, FEL-capable SPL will have to be used.
I noticed there is a small difference between the code you are using and rmr_switch2 code, but since it comes after the state switch/after wfi, it should not matter I guess - I am assuming the code you assembled was slightly different itself:
Yours (.o):
28: eafffffe b 28 <_reset+0x28>
Mine (.elf):
40000028: eafffffd b 40000024 <.text+0x24>
I hope this help.