[ARM64] Trap and Forward CPU_ON psci smc

[Ozgur Karatas]

On Sat, Apr 16, 2022 at 2:27 PM Mushahid Hussain <mushi.shar at gmail.com>
wrote:

> Hi,
>

Hello,

I'm working on a hobby project: AARCH64 Hypervisor on Raspberry Pi 4b, to
> learn about kernel internals, boot process and ARM64 baremetal. I have a
> problem with trapping and forwarding a psci smc generated by linux kernel.
> I'll explain everything and what steps I have followed.
> I'm also using Trusted Firmware-A at EL3 using armstub config. More about
> TF-A here:
> https://trustedfirmware-a.readthedocs.io/en/latest/plat/rpi4.html
>
> I have posted this in TF-A mailing list as well. I would love to learn
> from experienced kernel and baremetal hackers here as well.
>
>
hm if i understood correctly you are using a trusted-software (TF-A)
distributed by ARM company and  dont forget its a secure monitor (smc)
version, called TEE for short and you can just monitoring mode, not REE
(rich execution environment) and while in this mode, you must first act on
trust-zone and an example and code for this is below:

https://trustedfirmware-a.readthedocs.io/en/latest/

Right now, I'm implementing SMC trapping. I can successfully forward almost
> all SMCs except for PSCI_CPU_ON_AARCH64. Linux makes these SMCs to bring up
> secondary CPUs during boot. Here's what I'm trying to do:
>
>    -  trap the PSCI_CPU_ON_AARCH64 SMC,
>    - preserve the entry_point address in global variable
>    - replace the entrypoint with my entrypoint and make the smc to
>    tf-a(or simply forward it.)
>    - when secondary cpus come online at the given address, where I set
>    their stack point and then eret the original address.
>
> Secondary cpus won't come online at the given address. Even if I don't
> change any arguments of CPU_ON smc and forward it as it is, the secondary
> cpus still won't come online. However, without trapping
> enabled(HCR_EL2.TSC=0), everything works fine.
>
> I tried to debug inside Trusted Firmware. I know that overall path for
> secondary CPU hotplug in is:
>
> CPU released from reset -> (ROM and possibly some other bootloader) ->
> bl31/aarch64/bl31_entrypoint.S:bl31_warm_entrypoint() ->
> lib/psci/psci_common.c:psci_warmboot_entrypoint() ->
> lib/psci/psci_on.c:psci_cpu_on_finish() -> rpi3_pwr_domain_on_finish()
>
>
although this is far from Linux kernel development and hacking, you
experience problem before sync please check here, people who write other
secure bootable usually use CPU register bits in low layer before sync. I
think this is known as trng and here after taking the CPU surface with the
getentropy function, mmio was written and the assembly boot bits were
assigned with the copy method.

https://git.trustedfirmware.org/TS/trusted-services.git/

Happy hacking!


I printed at all these points in Trusted Firmware with and without trapping
> enabled. Here's what I found: Nothing gets printed anywhere in that path if
> trapping is enabled. However, without trapping enabled, I can print
> anywhere even in bl31_entrypoint.S:bl31_warm_entrypoint().  What could be
> the problem?
>
> Here's my code:
> https://github.com/SikkiLadho/Leo/blob/4f272eff39934058a7f989c91aad82eab81041a1/src/exceptions.S#L39
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20220416/75535fcd/attachment.html>