[06/13] selftests/nolibc: allow specify a bios for qemu

Message ID 63bee74a7b3754a1b0e82bc57de52c18d2de003d.1684949268.git.falcon@tinylab.org
State New
Headers
Series tools/nolibc: riscv: Add full rv32 support |

Commit Message

Zhangjin Wu May 24, 2023, 5:52 p.m. UTC
  riscv qemu has a builtin bios (opensbi), but it may not match the latest
kernel and some old versions may hang during boot, let's allow user pass
a newer version to qemu via the -bios option.

we can use it like this:

    $ make run BIOS=/path/to/new-bios ...

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
  

Comments

Thomas Weißschuh May 26, 2023, 7 a.m. UTC | #1
On 2023-05-25 01:52:29+0800, Zhangjin Wu wrote:
> riscv qemu has a builtin bios (opensbi), but it may not match the latest
> kernel and some old versions may hang during boot, let's allow user pass
> a newer version to qemu via the -bios option.

Nitpick:

This seems very specific and hopefully only necessary temporarily.

Instead it could be changed to some generic mechanim like
"QEMU_ARGS_EXTRA"?

> we can use it like this:
> 
>     $ make run BIOS=/path/to/new-bios ...
> 
> Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
> ---
>  tools/testing/selftests/nolibc/Makefile | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
> index 9adc8944dd80..9213763ab3b6 100644
> --- a/tools/testing/selftests/nolibc/Makefile
> +++ b/tools/testing/selftests/nolibc/Makefile
> @@ -70,7 +70,8 @@ QEMU_ARGS_mips       = -M malta -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)"
>  QEMU_ARGS_riscv      = -M virt -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
>  QEMU_ARGS_s390       = -M s390-ccw-virtio -m 1G -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
>  QEMU_ARGS_loongarch  = -M virt -append "console=ttyS0,115200 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
> -QEMU_ARGS            = $(QEMU_ARGS_$(ARCH))
> +QEMU_ARGS_BIOS       = $(if $(BIOS),-bios $(BIOS))
> +QEMU_ARGS            = $(QEMU_ARGS_$(ARCH)) $(QEMU_ARGS_BIOS)
>  
>  # OUTPUT is only set when run from the main makefile, otherwise
>  # it defaults to this nolibc directory.
> -- 
> 2.25.1
>
  
Zhangjin Wu May 26, 2023, 10:25 a.m. UTC | #2
Hi, Thomas

> On 2023-05-25 01:52:29+0800, Zhangjin Wu wrote:
> > riscv qemu has a builtin bios (opensbi), but it may not match the latest
> > kernel and some old versions may hang during boot, let's allow user pass
> > a newer version to qemu via the -bios option.
> 
> Nitpick:
> 
> This seems very specific and hopefully only necessary temporarily.
>

RISC-V is such a new ISA and the Spec (especially the SBI) changes very
frequently ;-)

> Instead it could be changed to some generic mechanim like
> "QEMU_ARGS_EXTRA"?
>

Good point, will apply it.

Thanks,
Zhangjin

> > we can use it like this:
> > 
> >     $ make run BIOS=/path/to/new-bios ...
> > 
> > Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
> > ---
> >  tools/testing/selftests/nolibc/Makefile | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
> > index 9adc8944dd80..9213763ab3b6 100644
> > --- a/tools/testing/selftests/nolibc/Makefile
> > +++ b/tools/testing/selftests/nolibc/Makefile
> > @@ -70,7 +70,8 @@ QEMU_ARGS_mips       = -M malta -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)"
> >  QEMU_ARGS_riscv      = -M virt -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
> >  QEMU_ARGS_s390       = -M s390-ccw-virtio -m 1G -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
> >  QEMU_ARGS_loongarch  = -M virt -append "console=ttyS0,115200 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
> > -QEMU_ARGS            = $(QEMU_ARGS_$(ARCH))
> > +QEMU_ARGS_BIOS       = $(if $(BIOS),-bios $(BIOS))
> > +QEMU_ARGS            = $(QEMU_ARGS_$(ARCH)) $(QEMU_ARGS_BIOS)
> >  
> >  # OUTPUT is only set when run from the main makefile, otherwise
> >  # it defaults to this nolibc directory.
  
Conor Dooley May 26, 2023, 10:36 a.m. UTC | #3
On Fri, May 26, 2023 at 06:25:18PM +0800, Zhangjin Wu wrote:

> > On 2023-05-25 01:52:29+0800, Zhangjin Wu wrote:
> > > riscv qemu has a builtin bios (opensbi), but it may not match the latest
> > > kernel and some old versions may hang during boot, let's allow user pass
> > > a newer version to qemu via the -bios option.
> > 
> > Nitpick:
> > 
> > This seems very specific and hopefully only necessary temporarily.
> >
> 
> RISC-V is such a new ISA and the Spec (especially the SBI) changes very
> frequently ;-)


Huh. Could you please expand on which versions of QEMU will hang while
booting an upstream or stable kernel? Which kernels would be good to
know too.

Thanks,
Conor.
  
Zhangjin Wu May 26, 2023, 1:38 p.m. UTC | #4
Hi, Conor.

> 
> On Fri, May 26, 2023 at 06:25:18PM +0800, Zhangjin Wu wrote:
> 
> > > On 2023-05-25 01:52:29+0800, Zhangjin Wu wrote:
> > > > riscv qemu has a builtin bios (opensbi), but it may not match the latest
> > > > kernel and some old versions may hang during boot, let's allow user pass
> > > > a newer version to qemu via the -bios option.
> > >
> > > Nitpick:
> > >
> > > This seems very specific and hopefully only necessary temporarily.
> > >
> >
> > RISC-V is such a new ISA and the Spec (especially the SBI) changes very
> > frequently ;-)
> 
> Huh. Could you please expand on which versions of QEMU will hang while
> booting an upstream or stable kernel? Which kernels would be good to
> know too.
> 

As the cover letter listed (in the Environment section), the softwares we
used are:

    // higher qemu version is better, latest version is v8.0.0+
    $ qemu-system-riscv64  --version
    QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.18)
    Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

    // opensbi version, higher is better, must match kernel version and qemu version
    // rv64: used version is 1.2, latest is 1.2
    $ head -2 /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/run.out | tail -1
    OpenSBI v1.2-116-g7919530
    // rv32: used version is v0.9, latest is 1.2
    $ head -2 /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/run.out | tail -1
    OpenSBI v0.9-152-g754d511

The kernel version is the one this patchset based on (Willy's nolibc
repo), it is v6.4-rc1.

qemu v4.2.1 is the one systematically installed (/usr/bin) from the
qemu-system-misc package and used to test this patchset in my Ubuntu
20.04 based test docker image.

Just installed a v7.0.0 qemu from ppa:canonical-server/server-backports,
there is no default opensbi, and re-checked, there is one prebuilt
opensbi for rv64, but still no prebuilt opensbi for rv32.

    $ sudo add-apt-repository ppa:canonical-server/server-backports
    $ sudo apt install qemu-system-misc

    $ sudo apt install opensbi
    $ dpkg -S opensbi | grep -E "fw_*bin|elf"
    qemu-system-data: /usr/share/qemu/opensbi-riscv64-generic-fw_dynamic.elf
    opensbi: /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_dynamic.elf
    opensbi: /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf

    $ qemu-system-riscv32 -display none -no-reboot -kernel build/riscv32/virt/linux/v6.4-rc1/arch/riscv/boot/Image -serial stdio -M virt -append "console=ttyS0 panic=-1"
    qemu-system-riscv32: Unable to load the RISC-V firmware "opensbi-riscv32-generic-fw_dynamic.bin"

I used the one built myself, If not want to build such opensbi manually,
we can download one (1.2 currently) from qemu source code:

    https://gitlab.com/qemu-project/qemu/-/blob/master/pc-bios/opensbi-riscv32-generic-fw_dynamic.bin

Then, we can use it like this:

    $ qemu-system-riscv32 -display none -no-reboot -kernel build/riscv32/virt/linux/v6.4-rc1/arch/riscv/boot/Image -serial stdio -M virt -append "console=ttyS0 panic=-1" -bios /path/to/opensbi-riscv32-generic-fw_dynamic.bin

Will append this extra info in the commit message of the coming new
revision of this patch, thanks a lot.

The hang issue I mentioned may be using one of my older prebuilt version of
opensbi, I can not find which one it exactly is, so, please ignore that info,
will update that description too.

Btw, something not about this patch: qemu v8.0.0 seems not boot non-mmu
v6.3, both sides have issues, not dig into it carefully, so, not report
it yet.

Thanks,
Zhangjin

> Thanks,
> Conor.
  
Conor Dooley May 26, 2023, 3:08 p.m. UTC | #5
On Fri, May 26, 2023 at 09:38:25PM +0800, Zhangjin Wu wrote:
> Hi, Conor.
> 
> > 
> > On Fri, May 26, 2023 at 06:25:18PM +0800, Zhangjin Wu wrote:
> > 
> > > > On 2023-05-25 01:52:29+0800, Zhangjin Wu wrote:
> > > > > riscv qemu has a builtin bios (opensbi), but it may not match the latest
> > > > > kernel and some old versions may hang during boot, let's allow user pass
> > > > > a newer version to qemu via the -bios option.
> > > >
> > > > Nitpick:
> > > >
> > > > This seems very specific and hopefully only necessary temporarily.
> > > >
> > >
> > > RISC-V is such a new ISA and the Spec (especially the SBI) changes very
> > > frequently ;-)
> > 
> > Huh. Could you please expand on which versions of QEMU will hang while
> > booting an upstream or stable kernel? Which kernels would be good to
> > know too.
> > 
> 
> As the cover letter listed (in the Environment section), the softwares we
> used are:

Not super interested in those ones since they work ;)

> The kernel version is the one this patchset based on (Willy's nolibc
> repo), it is v6.4-rc1.
> 
> qemu v4.2.1 is the one systematically installed (/usr/bin) from the
> qemu-system-misc package and used to test this patchset in my Ubuntu
> 20.04 based test docker image.

Okay, in the context of RISC-V, that is pretty ancient ;)

> Just installed a v7.0.0 qemu from ppa:canonical-server/server-backports,
> there is no default opensbi, and re-checked, there is one prebuilt
> opensbi for rv64, but still no prebuilt opensbi for rv32.

Ah, I see.

> The hang issue I mentioned may be using one of my older prebuilt version of
> opensbi, I can not find which one it exactly is, so, please ignore that info,
> will update that description too.

Okay. If you do manage to reproduce it, LMK! I was/am just worried we
have some regressions because you should be able to keep booting with
those older opensbi versions, modulo some Kconfig changes - although if
it is something like qemu 4.2.1 specific I don't think I care all that
much about dinosaurs ;)

> Btw, something not about this patch: qemu v8.0.0 seems not boot non-mmu
> v6.3, both sides have issues, not dig into it carefully, so, not report
> it yet.

Cool. Feel free to CC me on whatever you discover. nommu gets little
enough testing in mainline, and even less in stable kernels. That reminds
me, I do need to add 32-bit nommu to the patchwork automation for
linux-riscv.

Thanks,
Conor.
  
Willy Tarreau May 28, 2023, 7:52 a.m. UTC | #6
On Fri, May 26, 2023 at 09:00:02AM +0200, Thomas Weißschuh wrote:
> On 2023-05-25 01:52:29+0800, Zhangjin Wu wrote:
> > riscv qemu has a builtin bios (opensbi), but it may not match the latest
> > kernel and some old versions may hang during boot, let's allow user pass
> > a newer version to qemu via the -bios option.
> 
> Nitpick:
> 
> This seems very specific and hopefully only necessary temporarily.
> 
> Instead it could be changed to some generic mechanim like
> "QEMU_ARGS_EXTRA"?

FWIW I, too, think that QEMU_ARGS_EXTRA could be very convenient for
various test cases on any architecture (change number of CPUs, RAM
size, boot options etc).

Willy
  

Patch

diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index 9adc8944dd80..9213763ab3b6 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -70,7 +70,8 @@  QEMU_ARGS_mips       = -M malta -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_riscv      = -M virt -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_s390       = -M s390-ccw-virtio -m 1G -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_loongarch  = -M virt -append "console=ttyS0,115200 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
-QEMU_ARGS            = $(QEMU_ARGS_$(ARCH))
+QEMU_ARGS_BIOS       = $(if $(BIOS),-bios $(BIOS))
+QEMU_ARGS            = $(QEMU_ARGS_$(ARCH)) $(QEMU_ARGS_BIOS)
 
 # OUTPUT is only set when run from the main makefile, otherwise
 # it defaults to this nolibc directory.