Hi, I am facing crash/hang with 2.6.32.60 Kernel (64bit). My maxcpus (CONFIG_NR_CPUS) count is 64 and below is the call trace. I see log saying "BUG: spinlock lockup on CPU#74". Is it a known bug? If its fixed let me know the patch. Any pointers really appreciated. Please note, if I change maxcpus=8 kernel boots successfully. Thanks Giri [Jan 26 06:41:35.439] [ 67.606000] Non-volatile memory driver v1.3^M [ 67.616000] Linux agpgart interface v0.103^M [ 67.637000] [drm] Initialized drm 1.1.0 20060810^M [ 67.652000] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled^M [ 68.092000] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A^M [Jan 26 06:41:36.679] [ 68.846000] serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A^M [ 68.910000] 00:03: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A^M [Jan 26 06:41:37.149] [ 69.316000] brd: module loaded^M [Jan 26 06:41:41.015] [ 73.183000] loop: module loaded^M [ 73.191000] lkdtm : Invalid command^M [ 73.200000] kgdb: Registered I/O driver kgdbts.^M [Jan 26 06:42:03.435] [ 73.552000] BUG: spinlock lockup on CPU#74, swapper/1, ffff880c9c1d2ac0^M <======= [ 73.552000] Pid: 1, comm: swapper Not tainted 2.6.32.60 #2^M [ 73.552000] Call Trace:^M [ 73.552000] <IRQ> [<ffffffff81196df0>] _raw_spin_lock+0x103/0x12b^M [ 73.552000] [<ffffffff81459c97>] _spin_lock+0x4f/0x62^M [ 73.552000] [<ffffffff8103c1a3>] ? scheduler_tick+0x49/0x166^M [ 73.552000] [<ffffffff8103c1a3>] scheduler_tick+0x49/0x166^M [ 73.552000] [<ffffffff8104e6de>] update_process_times+0x42/0x52^M [ 73.552000] [<ffffffff81061ef1>] tick_periodic+0x63/0x65^M [ 73.552000] [<ffffffff81061f11>] tick_handle_periodic+0x1e/0x6b^M [ 73.552000] [<ffffffff8101eb91>] smp_apic_timer_interrupt+0x84/0x97^M [ 73.552000] [<ffffffff8100c773>] apic_timer_interrupt+0x13/0x20^M [ 73.552000] <EOI> [<ffffffff8126a8a5>] ? kgdbts_break_test+0x1/0x22^M [ 73.552000] [<ffffffff8126a8a4>] ? kgdbts_break_test+0x0/0x22^M [ 73.552000] [<ffffffff8126b7d4>] ? configure_kgdbts+0x1ef/0x4bb^M [ 73.552000] [<ffffffff81177ade>] ? blk_register_region+0x28/0x2a^M [ 73.552000] [<ffffffff81b50f4c>] ? init_kgdbts+0x0/0x16^M [ 73.552000] [<ffffffff81b50f60>] ? init_kgdbts+0x14/0x16^M [ 73.552000] [<ffffffff8100905d>] ? do_one_initcall+0x57/0x157^M [ 73.552000] [<ffffffff81b24d88>] ? kernel_init+0x1f0/0x241^M [ 73.552000] [<ffffffff8100cc7a>] ? child_rip+0xa/0x20^M [ 73.552000] [<ffffffff8100c63c>] ? restore_args+0x0/0x30^M [ 73.552000] [<ffffffff81b24b98>] ? kernel_init+0x0/0x241^M [ 73.552000] [<ffffffff8100cc70>] ? child_rip+0x0/0x20^M [ 73.552000] sending NMI to all CPUs:^M [ 73.552000] NMI backtrace for cpu 74^M [ 73.552000] CPU 74:^M [ 73.552000] Modules linked in:^M [ 73.552000] Pid: 1, comm: swapper Not tainted 2.6.32.60 #2 UCSB-B420-M3^M [ 73.552000] RIP: 0010:[<ffffffff81187fa0>] [<ffffffff81187fa0>] delay_loop+0x20/0x2a^M [ 73.552000] RSP: 0018:ffff880c9c003da8 EFLAGS: 00000002^M [ 73.552000] RAX: 000000000001d185 RBX: 0000000000000000 RCX: 0000000000000030^M [ 73.552000] RDX: 000000000001d19a RSI: 0000000000000010 RDI: 000000000001d19b^M [ 73.552000] RBP: ffff880c9c003da8 R08: 000000000000000b R09: ffffffff81947ad0^M [ 73.552000] R10: 000000000000b4a0 R11: ffffffff81878da0 R12: 0000000000001000^M [ 73.552000] R13: 0000000000000092 R14: 0000000000000002 R15: ffffffff81947ac0^M [ 73.552000] FS: 0000000000000000(0000) GS:ffff880c9c000000(0000) knlGS:0000000000000000^M [ 73.552000] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b^M [ 73.552000] CR2: 0000000000000000 CR3: 0000000001001000 CR4: 00000000000406e0^M [ 73.552000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 000000000000 [Jan 26 06:42:04.041] 0000^M [ 73.552000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400^M [ 73.552000] Call Trace:^M [ 73.552000] <#DB[1]> <<EOE>> Pid: 1, comm: swapper Not tainted 2.6.32.60 #2^M [ 73.552000] Call Trace:^M [ 73.552000] <NMI> [<ffffffff8145b99d>] nmi_watchdog_tick+0x111/0x24d^M [ 73.552000] [<ffffffff8145aee1>] do_nmi+0xbd/0x269^M [ 73.552000] [<ffffffff8145ab1a>] nmi+0x1a/0x2c^M [ 73.552000] [<ffffffff81187fa0>] ? delay_loop+0x20/0x2a^M [ 73.552000] <<EOE>> <IRQ> [<ffffffff81188001>] __delay+0xa/0xc^M [ 73.552000] [<ffffffff8118803f>] __const_udelay+0x3c/0x3e^M [ 73.552000] [<ffffffff8101e898>] native_safe_apic_wait_icr_idle+0x31/0x44^M [ 73.552000] [<ffffffff8101f479>] default_send_IPI_mask_sequence_phys+0x49/0x110^M [ 73.552000] [<ffffffff810225f4>] physflat_send_IPI_mask+0x9/0xb^M [ 73.552000] [<ffffffff8102262b>] physflat_send_IPI_all+0x14/0x16^M [ 73.552000] [<ffffffff8101f8e6>] arch_trigger_all_cpu_backtrace+0x4e/0x7c^M [ 73.552000] [<ffffffff81196df5>] _raw_spin_lock+0x108/0x12b^M [ 73.552000] [<ffffffff81459c97>] _spin_lock+0x4f/0x62^M [ 73.552000] [<ffffffff8103c1a3>] ? scheduler_tick+0x49/0x166^M [ 73.552000] [<ffffffff8103c1a3>] scheduler_tick+0x49/0x166^M [ 73.552000] [<ffffffff8104e6de>] update_process_times+0x42/0x52^M [ 73.552000] [<ffffffff81061ef1>] tick_periodic+0x63/0x65^M [ 73.552000] [<ffffffff81061f11>] tick_handle_periodic+0x1e/0x6b^M [ 73.552000] [<ffffffff8101eb91>] smp_apic_timer_interrupt+0x84/0x97^M [ 73.552000] [<ffffffff8100c773>] apic_timer_interrupt+0x13/0x20^M [ 73.552000] <EOI> [<ffffffff8126a8a5>] ? kgdbts_break_test+0x1/0x22^M [ 73.552000] [<ffffffff8126a8a4>] ? kgdbts_break_test+0x0/0x22^M [ 73.552000] [<ffffffff8126b7d4>] ? configure_kgdbts+0x1ef/0x4bb^M [ 73.552000] [<ffffffff81177ade>] ? blk_register_region+0x28/0x2a^M [ 73.552000] [<ffffffff81b50f4c>] ? init_kgdbts+0x0/0x16^M [ 73.552000] [<ffffffff81b50f60>] ? init_kgdbts+0x14/0x16^M [ 73.552000] [<ffffffff8100905d>] ? do_one_initcall+0x57/0x157^M [ 73.552000] [<ffffffff81b24d88>] ? kernel_init+0x1f0/0x241^M [ 73.552000] [<ffffffff8100cc7a>] ? child_rip+0xa/0x20^M [ 73.552000] [<ffffffff8100c63c>] ? restore_args+0x0/0x30^M [ 73.552000] [<ffffffff81b24b98>] ? kernel_init+0x0/0x241^M [ 73.552000] [<ffffffff8100cc70>] ? child_rip+0x0/0x20^M
On Tue, Jan 27, 2015 at 12:06:56AM +0000, Giridhara RP (grp) wrote:
Hi,
I am facing crash/hang with 2.6.32.60 Kernel (64bit).
Note this is a _very_ old and obsolete kernel, please use a modern kernel release if you wish to get help from the community. If you are being forced to use this old version, then get support from the company you are paying to provide this kernel, as that is what their job is. Best of luck, greg k-h
Hi Greg, Thanks for the quick turnaround. Which is the stable version? Kernel 2.6.32.60 was used in RHEL 6.2 and we thought this kernel version is stable. Regards Giri -----Original Message----- From: Greg KH [mailto:greg@kroah.com] Sent: Tuesday, January 27, 2015 8:19 AM To: Giridhara RP (grp) Cc: kernelnewbies@kernelnewbies.org Subject: Re: Kernel 2.6.32.60 64bit Crash/Hung On Tue, Jan 27, 2015 at 12:06:56AM +0000, Giridhara RP (grp) wrote:
Hi,
I am facing crash/hang with 2.6.32.60 Kernel (64bit).
Note this is a _very_ old and obsolete kernel, please use a modern kernel release if you wish to get help from the community. If you are being forced to use this old version, then get support from the company you are paying to provide this kernel, as that is what their job is. Best of luck, greg k-h
On Tue, 27 Jan 2015 03:52:05 +0000, "Giridhara RP (grp)" said:
Thanks for the quick turnaround. Which is the stable version? Kernel 2.6.32.60 was used in RHEL 6.2 and we thought this kernel version is stable.
It's stable enough that RedHat can afford to keep supporting its paying customers. You probably will be quite disillusioned if you download the .src.rpm for the kernel and start counting %patch lines in the .SPEC file.:)
-----Original Message----- From: Valdis.Kletnieks@vt.edu [mailto:Valdis.Kletnieks@vt.edu] Sent: Tuesday, January 27, 2015 9:32 AM To: Giridhara RP (grp) Cc: Greg KH; kernelnewbies@kernelnewbies.org Subject: Re: Kernel 2.6.32.60 64bit Crash/Hung On Tue, 27 Jan 2015 03:52:05 +0000, "Giridhara RP (grp)" said:
Thanks for the quick turnaround. Which is the stable version? Kernel 2.6.32.60 was used in RHEL 6.2 and we thought this kernel version is stable.
It's stable enough that RedHat can afford to keep supporting its paying customers. You probably will be quite disillusioned if you download the .src.rpm for the kernel and start counting %patch lines in the .SPEC file.:) --------------- Hi Valdis, Which kernel version should I use to solve this crash/panic? Thanks Giri
On Tue, 27 Jan 2015 04:06:00 +0000, "Giridhara RP (grp)" said:
Which kernel version should I use to solve this crash/panic?
That will depend on exactly why you're still running a 2.6.32 kernel from 6 years ago. Remember - the fact it's all the way up to 2.6.32.65 doesn't mean you have all the bugfixes. It only means you have all the bugfixes that qualified for the -stable side of the fence (which is a small fraction of all the fixes that have happened in the past 6 years).
-----Original Message----- From: Valdis.Kletnieks@vt.edu [mailto:Valdis.Kletnieks@vt.edu] Sent: Tuesday, January 27, 2015 10:15 AM To: Giridhara RP (grp) Cc: Greg KH; kernelnewbies@kernelnewbies.org Subject: Re: Kernel 2.6.32.60 64bit Crash/Hung On Tue, 27 Jan 2015 04:06:00 +0000, "Giridhara RP (grp)" said:
Which kernel version should I use to solve this crash/panic?
That will depend on exactly why you're still running a 2.6.32 kernel from 6 years ago. Remember - the fact it's all the way up to 2.6.32.65 doesn't mean you have all the bugfixes. It only means you have all the bugfixes that qualified for the -stable side of the fence (which is a small fraction of all the fixes that have happened in the past 6 years). ----------------------- Hi Valdis, We were running 2.6.32.60 assuming it's a stable build and my assumption was wrong :(. Should I go ahead with stable 3.18.3 [2015-01-16 ]. Thanks Giri
Hi Giridhara, http://ark.intel.com/products/64622/Intel-Xeon-Processor-E5-4650-20M-Cache-2... Show it has only 8 core of CPU. Why are you setting the CONFIG_NR_CPUS=64. certainly it will have a issue in the kernel. -Anand Moon On Tuesday, January 27, 2015 10:27 AM, Giridhara RP (grp) <grp@cisco.com> wrote: -----Original Message----- From: Valdis.Kletnieks@vt.edu [mailto:Valdis.Kletnieks@vt.edu] Sent: Tuesday, January 27, 2015 10:15 AM To: Giridhara RP (grp) Cc: Greg KH; kernelnewbies@kernelnewbies.org Subject: Re: Kernel 2.6.32.60 64bit Crash/Hung On Tue, 27 Jan 2015 04:06:00 +0000, "Giridhara RP (grp)" said:
Which kernel version should I use to solve this crash/panic?
That will depend on exactly why you're still running a 2.6.32 kernel from 6 years ago. Remember - the fact it's all the way up to 2.6.32.65 doesn't mean you have all the bugfixes. It only means you have all the bugfixes that qualified for the -stable side of the fence (which is a small fraction of all the fixes that have happened in the past 6 years). ----------------------- Hi Valdis, We were running 2.6.32.60 assuming it's a stable build and my assumption was wrong :(. Should I go ahead with stable 3.18.3 [2015-01-16 ]. Thanks Giri _______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-----Original Message----- From: Anand Moon [mailto:moon.linux@yahoo.com] Sent: Tuesday, January 27, 2015 12:30 PM To: Giridhara RP (grp); Valdis.Kletnieks@vt.edu Cc: Greg KH; kernelnewbies@kernelnewbies.org Subject: Re: Kernel 2.6.32.60 64bit Crash/Hung Hi Giridhara, http://ark.intel.com/products/64622/Intel-Xeon-Processor-E5-4650-20M-Cache-2... Show it has only 8 core of CPU. Why are you setting the CONFIG_NR_CPUS=64. certainly it will have a issue in the kernel. -Anand Moon ------------------------------------------------------------------------------- Hi Anand, Agree it’s a 8 core processor, but Hyper threading is supported in this CPU. Thanks Giri On Tuesday, January 27, 2015 10:27 AM, Giridhara RP (grp) <grp@cisco.com> wrote: -----Original Message----- From: Valdis.Kletnieks@vt.edu [mailto:Valdis.Kletnieks@vt.edu] Sent: Tuesday, January 27, 2015 10:15 AM To: Giridhara RP (grp) Cc: Greg KH; kernelnewbies@kernelnewbies.org Subject: Re: Kernel 2.6.32.60 64bit Crash/Hung On Tue, 27 Jan 2015 04:06:00 +0000, "Giridhara RP (grp)" said:
Which kernel version should I use to solve this crash/panic?
That will depend on exactly why you're still running a 2.6.32 kernel from 6 years ago. Remember - the fact it's all the way up to 2.6.32.65 doesn't mean you have all the bugfixes. It only means you have all the bugfixes that qualified for the -stable side of the fence (which is a small fraction of all the fixes that have happened in the past 6 years). ----------------------- Hi Valdis, We were running 2.6.32.60 assuming it's a stable build and my assumption was wrong :(. Should I go ahead with stable 3.18.3 [2015-01-16 ]. Thanks Giri _______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Zitat von Anand Moon <moon.linux@yahoo.com>:
http://ark.intel.com/products/64622/Intel-Xeon-Processor-E5-4650-20M-Cache-2...
Show it has only 8 core of CPU. Why are you setting the CONFIG_NR_CPUS=64. certainly it will have a issue in the kernel. Why would that lead to issues? AFAIK all it does is set the size of the array which contains CPU data, setting it higher would only result in some kilobytes of memory being used...
Jan
On Tue, Jan 27, 2015 at 09:25:04AM +0100, Jan Niggemann wrote:
Zitat von Anand Moon <moon.linux@yahoo.com>:
http://ark.intel.com/products/64622/Intel-Xeon-Processor-E5-4650-20M-Cache-2...
Show it has only 8 core of CPU. Why are you setting the CONFIG_NR_CPUS=64. certainly it will have a issue in the kernel. Why would that lead to issues? AFAIK all it does is set the size of the array which contains CPU data, setting it higher would only result in some kilobytes of memory being used...
It is also the defined default in arch/x86/configs/x86_64_defconfig (on 3.18.3, but IIRC it's this way for quite a while). WKR Hinnerk
On Tue, Jan 27, 2015 at 03:52:05AM +0000, Giridhara RP (grp) wrote:
Hi Greg,
Thanks for the quick turnaround. Which is the stable version? Kernel 2.6.32.60 was used in RHEL 6.2 and we thought this kernel version is stable.
What do you mean by "stable"? Yes, it is "stable" if you have hardware that you bought before that kernel was released, and you know how to support a kernel on your own, but if you are doing this with newer hardware (as it seems you are), you need to use a much newer kernel. Again, the community can not help you out when you run old kernels like this, only a company that you pay support to can (like Red Hat, if you want to use their kernels, great, they will support you and you will pay for it.) So I strongly suggest, if you are doing this on your own, to use the 3.18 kernel release. Best of luck, greg k-h
participants (6)
-
Anand Moon -
Giridhara RP (grp) -
Greg KH -
Hinnerk van Bruinehsen -
Jan Niggemann -
Valdis.Kletnieks＠vt.edu