Getting stacktrace for userspace applications on crash(SIGSEGV)
Hi everybody, I'm debugging a library that implement some nasty protocol. This library is used by some JNI library which exports the native library to Java world. By the way I'm running on ARMv7. Well, some times, at random times, that library crashes. The problem is that I can't get stacktrace. Not from JVM's fatal errors logs, not from core dumps, not from libSegFault... I'm using "-g -rdynamic -funwind-tables -mapcs-frame" compiler options but still I can't get a stacktrace. I've tried using glibc backtrace too. What I don't tried yet: libunwind, valgrid and making kernel do the stacktrace. The cool part is that when I ran my library outside from the JVM (with native application) I do get backtraces! [Damn Java] I'm here to ask: Is it possible to make kernel print the stacktrace for some userspace application when it crashes? I saw USER_STACKTRACE_SUPPORT but can't find any help from it's Kconfig. I saw this https://kernelnewbies.org/FAQ/StackTrace and this http://processors.wiki.ti.com/index.php/Enabling_Stack_Dumping_in_Linux_Kern... but can't find "Kernel hacking -> Verbose kernel error messages" even with "Kernel hacking -> Kernel Debuggin" enabled. Any help is appreciated! Thanks !!! -- "Do or do not. There is no try" Yoda Master
Hi, Do u know where in the kernel it crashes and can add dump_stack() there and try ? I used dump_stack() in the past many times and it worked for me. IIRC, kernel hacking->Kernel debugging was selected in my kernekl config, and I did not had to to anything else Regards, Rami Rosen http://ramirose.wix.com/ramirosen On 9 September 2016 at 21:21, Daniel. <danielhilst@gmail.com> wrote:
Hi everybody,
I'm debugging a library that implement some nasty protocol. This library is used by some JNI library which exports the native library to Java world. By the way I'm running on ARMv7.
Well, some times, at random times, that library crashes. The problem is that I can't get stacktrace. Not from JVM's fatal errors logs, not from core dumps, not from libSegFault... I'm using "-g -rdynamic -funwind-tables -mapcs-frame" compiler options but still I can't get a stacktrace. I've tried using glibc backtrace too. What I don't tried yet: libunwind, valgrid and making kernel do the stacktrace. The cool part is that when I ran my library outside from the JVM (with native application) I do get backtraces! [Damn Java]
I'm here to ask: Is it possible to make kernel print the stacktrace for some userspace application when it crashes?
I saw USER_STACKTRACE_SUPPORT but can't find any help from it's Kconfig. I saw this https://kernelnewbies.org/FAQ/StackTrace and this http://processors.wiki.ti.com/index.php/Enabling_Stack_Dumping_in_Linux_Kern... but can't find "Kernel hacking -> Verbose kernel error messages" even with "Kernel hacking -> Kernel Debuggin" enabled.
Any help is appreciated! Thanks !!!
-- "Do or do not. There is no try" Yoda Master
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Hi Rami Thanks for your reply I'm asking if I can use kernel to dump a stacktrace for a userspace app instead of kernel code. I'm trying that as last resource since gdb, libSegFault and JVM's fatal error log had failed, Best regards, 2016-09-09 16:16 GMT-03:00 Rami Rosen <roszenrami@gmail.com>:
Hi, Do u know where in the kernel it crashes and can add dump_stack() there and try ? I used dump_stack() in the past many times and it worked for me. IIRC, kernel hacking->Kernel debugging was selected in my kernekl config, and I did not had to to anything else
Regards, Rami Rosen http://ramirose.wix.com/ramirosen
On 9 September 2016 at 21:21, Daniel. <danielhilst@gmail.com> wrote:
Hi everybody,
I'm debugging a library that implement some nasty protocol. This library is used by some JNI library which exports the native library to Java world. By the way I'm running on ARMv7.
Well, some times, at random times, that library crashes. The problem is that I can't get stacktrace. Not from JVM's fatal errors logs, not from core dumps, not from libSegFault... I'm using "-g -rdynamic -funwind-tables -mapcs-frame" compiler options but still I can't get a stacktrace. I've tried using glibc backtrace too. What I don't tried yet: libunwind, valgrid and making kernel do the stacktrace. The cool part is that when I ran my library outside from the JVM (with native application) I do get backtraces! [Damn Java]
I'm here to ask: Is it possible to make kernel print the stacktrace for some userspace application when it crashes?
I saw USER_STACKTRACE_SUPPORT but can't find any help from it's Kconfig. I saw this https://kernelnewbies.org/FAQ/StackTrace and this http://processors.wiki.ti.com/index.php/Enabling_Stack_Dumping_in_Linux_Kern... but can't find "Kernel hacking -> Verbose kernel error messages" even with "Kernel hacking -> Kernel Debuggin" enabled.
Any help is appreciated! Thanks !!!
-- "Do or do not. There is no try" Yoda Master
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- "Do or do not. There is no try" Yoda Master
My guess is that gdb is failing to walk the Java frames. Could you post what you see in the gdb when you ask for the backtrace? I can possibly try and help then. Regards, Harish
On 10-Sep-2016, at 12:46 AM, Rami Rosen <roszenrami@gmail.com> wrote:
Hi, Do u know where in the kernel it crashes and can add dump_stack() there and try ? I used dump_stack() in the past many times and it worked for me. IIRC, kernel hacking->Kernel debugging was selected in my kernekl config, and I did not had to to anything else
Regards, Rami Rosen http://ramirose.wix.com/ramirosen
On 9 September 2016 at 21:21, Daniel. <danielhilst@gmail.com> wrote: Hi everybody,
I'm debugging a library that implement some nasty protocol. This library is used by some JNI library which exports the native library to Java world. By the way I'm running on ARMv7.
Well, some times, at random times, that library crashes. The problem is that I can't get stacktrace. Not from JVM's fatal errors logs, not from core dumps, not from libSegFault... I'm using "-g -rdynamic -funwind-tables -mapcs-frame" compiler options but still I can't get a stacktrace. I've tried using glibc backtrace too. What I don't tried yet: libunwind, valgrid and making kernel do the stacktrace. The cool part is that when I ran my library outside from the JVM (with native application) I do get backtraces! [Damn Java]
I'm here to ask: Is it possible to make kernel print the stacktrace for some userspace application when it crashes?
I saw USER_STACKTRACE_SUPPORT but can't find any help from it's Kconfig. I saw this https://kernelnewbies.org/FAQ/StackTrace and this http://processors.wiki.ti.com/index.php/Enabling_Stack_Dumping_in_Linux_Kern... but can't find "Kernel hacking -> Verbose kernel error messages" even with "Kernel hacking -> Kernel Debuggin" enabled.
Any help is appreciated! Thanks !!!
-- "Do or do not. There is no try" Yoda Master
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Hi Harish, thank you for replying me :) Here it is a sample of what I see: (gdb) bt #0 0x76e5ed64 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x76e627f8 in __GI_abort () at abort.c:89 #2 0x76831b7c in ?? () from /opt/jdk1.8.0_60/jre/lib/arm/client/libjvm.so #3 0x76831b7c in ?? () from /opt/jdk1.8.0_60/jre/lib/arm/client/libjvm.so Backtrace stopped: previous frame identical to this frame (corrupt stack?) Best regards, 2016-09-09 17:00 GMT-03:00 Harish <harish.b310@gmail.com>:
My guess is that gdb is failing to walk the Java frames. Could you post what you see in the gdb when you ask for the backtrace? I can possibly try and help then.
Regards, Harish
On 10-Sep-2016, at 12:46 AM, Rami Rosen <roszenrami@gmail.com> wrote:
Hi, Do u know where in the kernel it crashes and can add dump_stack() there and try ? I used dump_stack() in the past many times and it worked for me. IIRC, kernel hacking->Kernel debugging was selected in my kernekl config, and I did not had to to anything else
Regards, Rami Rosen http://ramirose.wix.com/ramirosen
On 9 September 2016 at 21:21, Daniel. <danielhilst@gmail.com> wrote: Hi everybody,
I'm debugging a library that implement some nasty protocol. This library is used by some JNI library which exports the native library to Java world. By the way I'm running on ARMv7.
Well, some times, at random times, that library crashes. The problem is that I can't get stacktrace. Not from JVM's fatal errors logs, not from core dumps, not from libSegFault... I'm using "-g -rdynamic -funwind-tables -mapcs-frame" compiler options but still I can't get a stacktrace. I've tried using glibc backtrace too. What I don't tried yet: libunwind, valgrid and making kernel do the stacktrace. The cool part is that when I ran my library outside from the JVM (with native application) I do get backtraces! [Damn Java]
I'm here to ask: Is it possible to make kernel print the stacktrace for some userspace application when it crashes?
I saw USER_STACKTRACE_SUPPORT but can't find any help from it's Kconfig. I saw this https://kernelnewbies.org/FAQ/StackTrace and this http://processors.wiki.ti.com/index.php/Enabling_Stack_Dumping_in_Linux_Kern... but can't find "Kernel hacking -> Verbose kernel error messages" even with "Kernel hacking -> Kernel Debuggin" enabled.
Any help is appreciated! Thanks !!!
-- "Do or do not. There is no try" Yoda Master
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- "Do or do not. There is no try" Yoda Master
participants (3)
-
Daniel. -
Harish -
Rami Rosen