RE: Debugging techniques inside kernel
Try using strace and see if all of the different crashes have the same or similar looking call stack. Here's a link on how to use it in case you didn't know already: http://stackoverflow.com/questions/174942/how-to-use-strace -----Original Message----- From: manty kuma Sent: 7/7/2013 8:06 PM To: Valdis Kletnieks Cc: kernelnewbies@kernelnewbies.org Subject: Re: Debugging techniques inside kernel Hi Valdis, Thanks for the info. I have serial logs (using minicom). This is not helping me much as randomly memory is getting crashed and everytime it changes. Even the logs before the crash are varying everytime. On Mon, Jul 8, 2013 at 11:49 AM, <Valdis.Kletnieks@vt.edu> wrote: On Mon, 08 Jul 2013 11:22:16 +0900, manty kuma said:
For ex : *CONFIG_SLUB_DEBUG_ON, *CONFIG_DMA_API_DEBUG
I am just told to use these configs and check. Not sure how to use them effectively. Please share if you are aware of these kind of configs and how to use them. To have them at one place it would be useful for debugging beginners like me.
Most _DEBUG config options are set-and-forget - you turn them on and there's no further configuration needed. They enable additional code that does additional sanity checking and if it finds a problem, it issues a printk() or a stack dump or similar which shows up in dmesg. If you're chasing memory corruption, it may be useful to use netconsole or a serial console to send out all the printk() output in real time, as often an error will be detected but the system will panic()/crash/hang in a way that your local syslog daemon is unable to write the message to disk. (I'm told that on newer UEFI based systems, you can use pstore in a similar fashion, but have not tried it myself yet).
On Mon, 08 Jul 2013 08:18:02 -0700, Anand Arumugam said:
Try using strace and see if all of the different crashes have the same or similar looking call stack.
strace is userspace, and *highly* unlikely to track down random memory corruption inside the kernel. Random kernel corruption usually means that something *else* stomped on the memory - which in turn means that the syscall that finally trips over the corrupt memory is probably not the one that did it. Not saying it can't possibly do so, but it's not where I'd start. Might want to run memtest86+ or similar, help rule out dodgy memory.
On Mon, Jul 8, 2013 at 11:19 PM, <Valdis.Kletnieks@vt.edu> wrote:
Might want to run memtest86+ or similar, help rule out dodgy memory.
+1 for the above idea... -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
On Mon, Jul 8, 2013 at 11:19 AM, <Valdis.Kletnieks@vt.edu> wrote:
On Mon, 08 Jul 2013 08:18:02 -0700, Anand Arumugam said:
Try using strace and see if all of the different crashes have the same or similar looking call stack.
strace is userspace, and *highly* unlikely to track down random memory corruption inside the kernel. Random kernel corruption usually means that something *else* stomped on the memory - which in turn means that the syscall that finally trips over the corrupt memory is probably not the one that did it.
Not saying it can't possibly do so, but it's not where I'd start.
Might want to run memtest86+ or similar, help rule out dodgy memory.
Yes, that's a good idea. I'll also add that just turning on SLAB DEBUG is a great test too. I've able to catch dodgy memory, that didn't get caught using memtest, but would fail after turning on slab debugging. That bit of code really hammers the memory. Not even the hardware diagnostics or memtester could catch that one ;-) Regards, - Joel Fernandes
participants (4)
-
Anand Arumugam -
Joel Fernandes -
Mulyadi Santosa -
Valdis.Kletnieks＠vt.edu