I am trying to write a Qemu TCG plugin to recognize if an instruction is executed in user mode or in kernel mode for a basic program.
I understand that a full system emulation is needed for the same. I wrote a TCG plugin, registering two Qemu callbacks qemu_plugin_register_vcpu_syscall_cb and qemu_plugin_register_vcpu_syscall_ret_cb. I am using the target arm-softmmu for full system emulation (and then the corresponding qemu-system-arm). I can see that there are svc calls in the user mode but I do not get any kernel-mode instructions. Any idea on what might be happening?
I am running a C software on an ARM IP (module name is ARM926EJS) provided by Synopsis, and the entire tool I am using is Platform Architect also by Synopsis (based on TLM simulator, the hardware part is implemented in SystemC).
I received from my hardware module an address that the software is not supposed to access. Hardware modules cannot see the information of the software (or I may just not know), so I usually attach arm-none-eabi-gdb to the ARM IP during simulation to debug the software.
I have to check, but I believe that the ARM IP I am using does not support hardware watchpoint (using only one hardware watchpoint returns "too many hardware watchpoints"). I am trying software watchpoint by typing
set can-use-hw-watchpoints 0
I tried to see when the software accesses the strange address by typing
watch *(int*)(0x2056604+0xd0000000)
which the number is the address that I am receiving on the hardware. However, the program freezes, and the interrupt on gdb does not even work, I have to stop the simulation and delete the watchpoint to go on.
Strangely though, rwatch works correctly (although the problem persists since the address is accessed as "write"). It really would be beneficial to see which point in the software has caused the access to address 0x2056604+0xd0000000, but as watchpoints are not working for me, are there alternative ways to break on such access? Or is there a way to fix the freezing problem on software watchpoints?
Additional Note) I also tried to pause simulation on the memory access, then view the stack frame of the software. The software has strange stack and the gdb tells me that the stack is possibly corrupted. Output is the following..
#0 0x00002bf0 in ?? ()
#1 0x00001b94 in ?? ()
#2 0x00001b94 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
I am working on a project where I have a router with ARMv7 processor (Cortex A15) and OpenWRT OS. I have a shell on the router and can load kernel modules with insmod.
My goal is to write a kernel module in C which changes the HVBAR register and then executes the hvc instruction to get the processor in the hyp mode.
This is a scientific project where I want to check if I can place my own hypervisor on a running system. But before I start to write my own hypervisor I want to check if and how I can bring the processor in the hyp mode.
According to this picture take from armv7-a manual B.9.3.4 the system must be in insecure mode, not in user mode and the SCR.HCE bit must be set to 1.
My question is how I can prepare the processor with a C kernel module and inline assembly and then execute the hvc instruction. I want to do this with a kernel module because then I start in PL1. This pseudocode describes what I want to achieve:
call smc // to get in monitor mode
set SRC.HCE to 1 // to enable hvc instruction
set SRC.NS to 1 // to set the system to not secure
call hvc #0 // call the hvc instruction to produce a hypervisor exception
The easiest way to elevate privilege is to start off in the needed privilege mode already: You've a root shell. Is the boot chain verified? Could you replace bootloader or kernel, so your code naturally runs in PL2 (HYP) mode? If so, that's probably the easiest way to do it.
If you can't replace the relevant part of the boot chain, the details of writing the rootkit depend a lot on information about your system left out: In which mode is Linux started? Is KVM support enabled and active? Was PL2 initialized? Was it locked? Is there "secure" firmware you can exploit?
The objective is always the same: have HVBAR point at some code you can control and do a hvc. Depending on your environment, solutions may range from spraying as much RAM as possible with your code and hope (perhaps after some reboots) an uninitialized HVBAR would point at an instruction you control to inhibiting KVM from running and accessing the early hypervisor stub to install yourself instead.
Enumerating such exploits is a bit out of scope for a StackOverflow answer; this is rather dissertation material. Indeed, there's a doctoral thesis exactly on this topic:
Strengthening system security on the ARMv7 processor architecture with hypervisor-based security mechanisms
I have to check whether the ARM processor is in SVC or IRQ mode during kernel bootup. That is, I want to insert some code to check the ARM mode into start_kernel() function before the interrupts are enabled and after the interrupts are enabled.
I know that I need the SPSR or CPSR values(bits) to check the mode in ARM, but how can I write code for that in start_kernel function since the code for reading bits of CPSR/SPSR is in assembly? Where do I put the assembly code to check the ARM mode during bootup time? Is there any way to dump the SPSR/CPSR values?
I don't dare imagine why this should be a concern, but fortunately there's an easy answer:
It's in SVC mode.
The very first thing* the kernel entrypoint does is to forcibly switch into SVC mode and mask interrupts. To somehow be in the wrong mode by the time you reach C code in start_kernel way later would need the system to be unimaginably horribly broken. The only way I can conceive of that even being possible is if there is some Secure firmware running asynchronously (e.g. off a Secure timer interrupt) which deliberately corrupts the Non-secure state, and that's sufficiently ridiculous to discount.
* OK, the second thing if the kernel is built with KVM support and entered in HYP, but hey...
after the boot the ARM processor is in secure SVC mode during this mode you can access CPSR register(first 6 bits) to check in which mode however when the kernel is runing you are in user non-secure mode and in this mode non secure you can't access the the CPSR and SPSR register and also copro 15 register .
The only way is to write a code that generate an exception to switch monitor mode " using SMC assebmbly instruction" to jump to monitor secure mode then in that mode you reset the "NS" bit to switch to non-secure then you generate another exception to switch the SVC mode(SVC instruction assembly call) now you are in supervisor secure mode and you can then access CPSR and SPSR register
For example, virt-what shows if you are running inside hardware virtualization "sandbox".
How to detect if you are running in ARM "TrustZone" sandbox?
TrustZone maybe different than what you think. There is a continuum of modes. From 'a simple API of trusted functions' to 'dual OSs' running in each world.
If there was more context given to the question, it would be helpful. Is this for programatically determining or for reverse engineering considerations? For the current Linux user-space, the answer is no.
Summary
No current user space utility.
Time based analysis.
Code based analysis.
CPU exclusion and SCR.
ID_PRF1 bits [7:4].
virt-what is not a fool-proof way of discovering if you are running under a hyper-visor. It is a program written for linux user-space. Mostly, these are shell scripts which examine /proc/cpuinfo, etc. procfs is a pseudo-file system which runs code in the kernel context and reports to user space. There is no such detection of TrustZone in the main line ARM linux. By design, ARM has made it difficult to detect. An design intent is to have code in the normal world run unmodified.
Code analysis
In order to talk to the secure world, the normal world needs SMC instructions. If your user space has access to kernel code or the vmlinux image, you can try to analyze the code sections for an SMC instruction. However, this code maybe present in the image, but never active. At least this says whether the Linux kernel has some support for TrustZone. You could write a kernel module which would trap any execution of an SMC instruction, but there are probably better solutions.
Timing analysis
If an OS is running in the secure world, some time analysis would show that some CPU cycles have been stolen if frequency scaling is not active. I think this is not an answer in the spirit of the original question. This relies on knowing that the secure world is a full-blown OS with a timer (or at least pre-emptible interrupts).
CPU exclusion and SCR
The SCR (Secure configuration register) is not available in the normal world. From the ARM Cortex-A5 MPcore manual (pg4-46),
Usage constraints The SCR is:
• only accessible in privileged modes
• only accessible in Secure state.
An attempt to access the SCR from any state other than secure privileged
results in an Undefined instruction exception.
ID_PRF1 bits [7:4].
On some Cortex-A series, the instruction,
mrc p15, 0, r0, c0, c1, 1
will get a value where bits [7:4] indicate whether the CPU supports Security Extensions, also known as TrustZone. A non-zero value indicates it is supported. Many early CPUs may not support this CP15 register . So, it is much like the SCR and handling the undefined instruction. Also, it doesn't tell you that code is active in the TrustZone mode.
Summary
It is possible that you could write a kernel module which would try this instruction and handle the undefined exception. This would detect a normal versus secure world. However, you would have to exclude CPUs which don't have TrustZone at all.
If the device is not an ARMv6 or better, then TrustZone is impossible. A great deal of Cortex-A devices have TrustZone in the CPU, but it is not active.
The combined SMC test and a CPU id, is still not sufficient. Some boot loaders run in the secure world and then transition to the normal world. So secure is only active during boot.
Theoretically, it is possible to know, especially with more knowledge of the system. There maybe many signs, such as spurious interrupts from the GIC, etc. However, I don't believe that any user space linux tool exists as of Jan 2014. This is a typical war of escalation between virus/rootkit writers and malware detection software.TZ Rootkits
You have not specified any details of the processor (A8, A9, A15?) or the execution mode (user/kernel/monitor) from where you want to detect the processor state.
As per the ARM documentation, the current state of the processor as Secure (aka the TrustZone sandbox) or Non-secure can be detected by reading the Secure Configuration Register and checking for the NS bit.
To access the Secure Configuration Register: MRC p15, 0, <Rd>, c1, c1, 0
Bit 0 being set corresponds to the processor being in non-secure mode and vice-versa.
You can check the processor's datasheet, and find those registers which behaves different between Normal world and Secure world. Generally, in Secure World, when you read these registers you will just get null. But get data in Normal world. And also, some registers that you can just access in Secure world, if you are in Secure World, you can access it, but in Normal World your access will be rejected.
Any way, there are many ways to distinguish Normal World and Secure World. JUST READ THE DATASHEET IN DETAIL.