Hi, I am not sure if this is the place to ask, if so please let me know where to do so. I am using a kernel module that I modified slightly to work on my current kernel version. It is running on a virtual machine and on a heavy load it goes into an infinite loop and freezes the system. I can tell that the virtual machine is using 100% of its assigned cpu. There are print statements almost everywhere in the module, but after a few seconds none get printed. The kernel has the following options already enabled: CONFIG_DETECT_SOFTLOCKUP and CONFIG_DETECT_HUNG_TASK. To my understanding the kernel is supposed to print the stack trace when a hung task is detected, however, nothing gets printed in kern.log. I am not sure if this is the place where the kernel would dump the stack or if the print process is affected by the hang since the print statements in the module are not being printed as well. I would like to know how to debug this issue. Any help is appreciated. Thank you.
IMO, I believe if you module is using/hanging all processors it's impossible to kernel use any to print stuffs. What about try Sysrq [1] ? I used it once in a power system plus xmon, but I'll be honest not sure how it works on x86. https://www.kernel.org/doc/Documentation/sysrq.txt On Fri, Aug 21, 2015 at 8:06 AM, Salam Farhat <salalimo@gmail.com> wrote:
Hi,
I am not sure if this is the place to ask, if so please let me know where to do so.
I am using a kernel module that I modified slightly to work on my current kernel version. It is running on a virtual machine and on a heavy load it goes into an infinite loop and freezes the system. I can tell that the virtual machine is using 100% of its assigned cpu. There are print statements almost everywhere in the module, but after a few seconds none get printed.
The kernel has the following options already enabled: CONFIG_DETECT_SOFTLOCKUP and
CONFIG_DETECT_HUNG_TASK. To my understanding the kernel is supposed to print the stack trace when a hung task is detected, however, nothing gets printed in kern.log. I am not sure if this is the place where the kernel would dump the stack or if the print process is affected by the hang since the print statements in the module are not being printed as well.
I would like to know how to debug this issue. Any help is appreciated.
Thank you.
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- ---------------------------------------------- Leônidas S. Barbosa (Kirotawa) blog: corecode.wordpress.com
On Fri, 21 Aug 2015 07:06:58 -0400, Salam Farhat said:
I am using a kernel module that I modified slightly to work on my current kernel version. It is running on a virtual machine and on a heavy load it goes into an infinite loop and freezes the system. I can tell that the virtual machine is using 100% of its assigned cpu.
Try giving the VM a second virtual CPU, if you have only one CPU and it gets hung sufficiently hard, the debugging stuff won't have a chance to run. Oh, and make sure you configure your kernel for SMP.
On Fri, Aug 21, 2015 at 8:33 AM, <Valdis.Kletnieks@vt.edu> wrote:
On Fri, 21 Aug 2015 07:06:58 -0400, Salam Farhat said:
I am using a kernel module that I modified slightly to work on my current kernel version. It is running on a virtual machine and on a heavy load it goes into an infinite loop and freezes the system. I can tell that the virtual machine is using 100% of its assigned cpu.
Try giving the VM a second virtual CPU, if you have only one CPU and it gets hung sufficiently hard, the debugging stuff won't have a chance to run.
Oh, and make sure you configure your kernel for SMP.
It has 4 cores and kernel is already configured for smp. Sysrq is a pretty cool option, but other than reboot almost no other commands run including 'e', 'f', ;i;. It seems I might have to resort to debugging by the process of elimination (removing code until it doesn't hang and so on.. ). Any other ideas I would appreciate it.
On 22 Aug, Salam Farhat wrote:
On Fri, Aug 21, 2015 at 8:33 AM, <Valdis.Kletnieks@vt.edu> wrote:
On Fri, 21 Aug 2015 07:06:58 -0400, Salam Farhat said:
I am using a kernel module that I modified slightly to work on my current kernel version. It is running on a virtual machine and on a heavy load it goes into an infinite loop and freezes the system. I can tell that the virtual machine is using 100% of its assigned cpu.
Try giving the VM a second virtual CPU, if you have only one CPU and it gets hung sufficiently hard, the debugging stuff won't have a chance to run.
Oh, and make sure you configure your kernel for SMP.
It has 4 cores and kernel is already configured for smp.
Sysrq is a pretty cool option, but other than reboot almost no other commands run including 'e', 'f', ;i;.
It seems I might have to resort to debugging by the process of elimination (removing code until it doesn't hang and so on.. ).
Any other ideas I would appreciate it. If you use qemu-kvm to run your VM, you could try to use gdb to debug your kernel running inside of VM.
man qemu: -s Shorthand for -gdb tcp::1234, i.e. open a gdbserver on TCP port 1234. -- Best regards, Andrey Skvortsov Secure eMail with gnupg: See http://www.gnupg.org/ PGP Key ID: 0x57A3AEAD
I used kgdb over serial port many years ago. Perhaps there's a version for ethernet. -- Fabien
participants (5)
-
Andrey Skvortsov -
Fabien R -
leo kirotawa -
Salam Farhat -
Valdis.Kletnieks＠vt.edu