On Mon, Nov 15, 2021 at 3:27 PM Baoquan He <bhe@redhat.com> wrote:
Hi,
On 11/13/21 at 10:40am, Dongliang Mu wrote:
Hi all KDUMP maintainers,
I would like to generate a kernel dump within QEMU VM.
1. I reproduced the kernel crash [1] in QEMU VM. The QEMU startup script is as follows:
qemu-system-x86_64 \ -kernel $KERNEL/arch/x86/boot/bzImage \ -append "console=ttyS0 root=/dev/sda debug earlyprintk=serial slub_debug=QUZ"\ -hda $IMAGE/stretch.img \ -net user,hostfwd=tcp::10021-:22 -net nic \ -enable-kvm \ -nographic \ -m 2G \ -smp 2 \ -pidfile vm.pid \ 2>&1 | tee vm.log
The stretch.img is generated by Syzkaller script [1]. -kernel option is convenient for loading any other kernels.
2. As the .config already has the essential configuration(CONFIG_KEXEC, CONFIG_CRASH_DUMP, CONFIG_DEBUG_INFO), I did not change this configuration file.
3. I installed kdump-tools crash kexec-tools makedumpfile linux-image-4.9.0-13-amd64 in the stretch.img. Here I installed linux-image-4.9.0-13-amd64 because there is no default kernel in /boot directory. And to make kdump-tools working, I modify /etc/default/kdump-tools in the following:
KDUMP_INITRD=/boot/initrd.img-4.9.0-13-amd64 KDUMP_KERNEL=/boot/vmlinuz-4.9.0-13-amd64
What distros are you using? Asking this because I am sure you are not using Fedora/RHEL OS. The implementation of kdump tools is different in each distros, even though the mechanims in kdump code is the same.
I am using Debian stretch as the guest OS. So kdump-tools kexec makedumpfile is all from Debian.
When we try to get help from upstream, considering and asking good question is very important for getting quick response and effective help.
Thanks Baoquan
4. I append "crashkernel=384M-:128M" to the command line in the startup script of QEMU.
5. After rebooting, kdump service can start successfully, and the kdump-config shows:
root@syzkaller:~# kdump-config show DUMP_MODE: kdump USE_KDUMP: 1 KDUMP_SYSCTL: kernel.panic_on_oops=1 KDUMP_COREDIR: /var/crash crashkernel addr: 0x77000000 /boot/vmlinuz-4.9.0-13-amd64 kdump initrd: /boot/initrd.img-4.9.0-13-amd64 current state: ready to kdump
kexec command: /sbin/kexec -p --command-line="earlyprintk=serial oops=panic panic_on_warn=1 nmi_watchdog=panic panic=86400 net.ifnames=0 sysctl.kernel.hung_task_all_cpu_backtrace=1 ima_policy=tcb kvm-intel.nested=1 nf-conntrack-ftp.ports=20000 nf-conntrack-tftp.ports=20000 nf-conntrack-sip.ports=20000 nf-conntrack-irc.ports=20000 nf-conntrack-sane.ports=20000 vivid.n_devs=16 vivid.multiplanar=1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2 netrom.nr_ndevs=16 rose.rose_ndevs=16 spec_store_bypass_disable=prctl numa=fake=2 nopcid dummy_hcd.num=8 binder.debug_mask=0 rcupdate.rcu_expedited=1 root=/dev/sda console=ttyS0 vsyscall=native watchdog_thresh=55 workqueue.watchdog_thresh=140 console=ttyS0 root=/dev/sda debug earlyprintk=serial slub_debug=QUZ irqpoll nr_cpus=1 nousb systemd.unit=kdump-tools.service ata_piix.prefer_ms_hyperv=0" --initrd=/boot/initrd.img-4.9.0-13-amd64 /boot/vmlinuz-4.9.0-13-amd64
6. When I execute the PoC, the current kernel crashes and then reboots into the dump-capture kernel. However, the kernel log shows, it is in emergency mode,
You are in emergency mode. After logging in, type "journalctl -xb" to view system logs, "systemctl reboot" to reboot, "systemctl default" or ^D to try again to boot into default mode.
Finally, I would like to ask several questions: 1) is the emergency mode due to the incorrect command line? 2) is this the right way to generate kernel dump from QEMU VM? 3) Any comments on the above procedures?
Thanks very much in advance.
[1] general protection fault in reiserfs_security_init (https://syzkaller.appspot.com/bug?id=8abaedbdeb32c861dc5340544284167dd0e46cd...)
[2] https://github.com/google/syzkaller/blob/master/tools/create-image.sh
-- My best regards to you.
No System Is Safe! Dongliang Mu