Help needed in getting kernel dump in QEMU VM

[Baoquan He]

Hi,

On 11/13/21 at 10:40am, Dongliang Mu wrote:
> Hi all KDUMP maintainers,
> 
> I would like to generate a kernel dump within QEMU VM.
> 
> 1. I reproduced the kernel crash [1] in QEMU VM. The QEMU startup
> script is as follows:
> 
> qemu-system-x86_64 \
>   -kernel $KERNEL/arch/x86/boot/bzImage \
>   -append "console=ttyS0 root=/dev/sda debug earlyprintk=serial slub_debug=QUZ"\
>   -hda $IMAGE/stretch.img \
>   -net user,hostfwd=tcp::10021-:22 -net nic \
>   -enable-kvm \
>   -nographic \
>   -m 2G \
>   -smp 2 \
>   -pidfile vm.pid \
>   2>&1 | tee vm.log
> 
> The stretch.img is generated by Syzkaller script [1]. -kernel option
> is convenient for loading any other kernels.
> 
> 2. As the .config already has the essential
> configuration(CONFIG_KEXEC, CONFIG_CRASH_DUMP, CONFIG_DEBUG_INFO), I
> did not change this configuration file.
> 
> 3. I installed kdump-tools crash kexec-tools makedumpfile
> linux-image-4.9.0-13-amd64 in the stretch.img. Here I installed
> linux-image-4.9.0-13-amd64 because there is no default kernel in /boot
> directory. And to make kdump-tools working, I modify
> /etc/default/kdump-tools in the following:
> 
> KDUMP_INITRD=/boot/initrd.img-4.9.0-13-amd64
> KDUMP_KERNEL=/boot/vmlinuz-4.9.0-13-amd64

What distros are you using? Asking this because I am sure you are not
using Fedora/RHEL OS. The implementation of kdump tools is different in
each distros, even though the mechanims in kdump code is the same.

When we try to get help from upstream, considering and asking good question
is very important for getting quick response and effective help.

Thanks
Baoquan

> 
> 4. I append "crashkernel=384M-:128M" to the command line in the
> startup script of QEMU.
> 
> 5. After rebooting, kdump service can start successfully, and the
> kdump-config shows:
> 
> root at syzkaller:~# kdump-config show
> DUMP_MODE:        kdump
> USE_KDUMP:        1
> KDUMP_SYSCTL:     kernel.panic_on_oops=1
> KDUMP_COREDIR:    /var/crash
> crashkernel addr: 0x77000000
>    /boot/vmlinuz-4.9.0-13-amd64
> kdump initrd:
>    /boot/initrd.img-4.9.0-13-amd64
> current state:    ready to kdump
> 
> kexec command:
>   /sbin/kexec -p --command-line="earlyprintk=serial oops=panic
> panic_on_warn=1 nmi_watchdog=panic panic=86400 net.ifnames=0
> sysctl.kernel.hung_task_all_cpu_backtrace=1 ima_policy=tcb
> kvm-intel.nested=1 nf-conntrack-ftp.ports=20000
> nf-conntrack-tftp.ports=20000 nf-conntrack-sip.ports=20000
> nf-conntrack-irc.ports=20000 nf-conntrack-sane.ports=20000
> vivid.n_devs=16 vivid.multiplanar=1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2
> netrom.nr_ndevs=16 rose.rose_ndevs=16 spec_store_bypass_disable=prctl
> numa=fake=2 nopcid dummy_hcd.num=8 binder.debug_mask=0
> rcupdate.rcu_expedited=1 root=/dev/sda console=ttyS0 vsyscall=native
> watchdog_thresh=55 workqueue.watchdog_thresh=140 console=ttyS0
> root=/dev/sda debug earlyprintk=serial slub_debug=QUZ irqpoll
> nr_cpus=1 nousb systemd.unit=kdump-tools.service
> ata_piix.prefer_ms_hyperv=0" --initrd=/boot/initrd.img-4.9.0-13-amd64
> /boot/vmlinuz-4.9.0-13-amd64
> 
> 6. When I execute the PoC, the current kernel crashes and then reboots
> into the dump-capture kernel. However, the kernel log shows, it is in
> emergency mode,
> 
> You are in emergency mode. After logging in, type "journalctl -xb" to view
> system logs, "systemctl reboot" to reboot, "systemctl default" or ^D to
> try again to boot into default mode.
> 
> Finally, I would like to ask several questions:
> 1) is the emergency mode due to the incorrect command line?
> 2) is this the right way to generate kernel dump from QEMU VM?
> 3) Any comments on the above procedures?
> 
> Thanks very much in advance.
> 
> [1] general protection fault in reiserfs_security_init
> (https://syzkaller.appspot.com/bug?id=8abaedbdeb32c861dc5340544284167dd0e46cde)
> 
> [2] https://github.com/google/syzkaller/blob/master/tools/create-image.sh
> 
> --
> My best regards to you.
> 
>      No System Is Safe!
>      Dongliang Mu