v3.15 kernel panic, bisecting didn't help

I can boot v3.15.0-rc8 but not v3.15.0 (full kernel panic below). I tried bisecting, I did git bisect v3.15 v.3.15-rc8 After 7 compile and reboot cycles I don't think I was able to find the bad commit. Every bisection booted with no problems. I ran "git bisect good" each time until I finally got to this: git bisect good 1860e379875dfe7271c649058aeddffe5afd9d0d is the first bad commit commit 1860e379875dfe7271c649058aeddffe5afd9d0d Author: Linus Torvalds <torvalds@linux-foundation.org> Date: Sun Jun 8 11:19:54 2014 -0700 Linux 3.15 :100644 100644 cdaa5b6a1c4d6e43e4c0c784831e1e3cb0d9fae5 6d1e304943a379c126c40e2aa1eb3bf4e872ecb1 M Makefile Which is just the version bump: http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=18... Also, this is my first bisection attempt, and on my 5th version the rc version jumped from 8 to 3 for some reason: ls -1|grep linux-image linux-image-3.15.0-1+_1_i386.deb <-- panics linux-image-3.15.0-rc3-5+_1_i386.deb <-- boots, rc3 ? linux-image-3.15.0-rc8-1+_1_i386.deb <-- boots linux-image-3.15.0-rc8-2+_1_i386.deb <-- boots linux-image-3.15.0-rc8-3+_1_i386.deb <-- boots linux-image-3.15.0-rc8-4+_1_i386.deb <-- boots linux-image-3.15.0-rc8-6+_1_i386.deb <-- boots linux-image-3.15.0-rc8-7+_1_i386.deb <-- boots Not sure if that rc3 jump is a sign of my novice git bisecting skills or what, but I don't know where to go from here. Any suggestions would be appreciated. Thanks, [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Initializing cgroup subsys cpuacct [ 0.000000] Linux version 3.15.0-1+ (root@mars) (gcc version 4.7.2 (Debian 4.7.2-5) ) #1 SMP Mon Jun 9 11:43:45 CDT 2014 [ 0.000000] e820: BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009dfff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009e000-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000003fffb9ff] usable [ 0.000000] BIOS-e820: [mem 0x000000003fffba00-0x000000003fffffff] ACPI data [ 0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000ffffffff] reserved [ 0.000000] Notice: NX (Execute Disable) protection missing in CPU! [ 0.000000] SMBIOS 2.3 present. [ 0.000000] e820: last_pfn = 0x3fffb max_arch_pfn = 0x100000 [ 0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106 [ 0.000000] found SMP MP-table at [mem 0x0009e140-0x0009e14f] mapped at [c009e140] [ 0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff] [ 0.000000] init_memory_mapping: [mem 0x37000000-0x373fffff] [ 0.000000] init_memory_mapping: [mem 0x30000000-0x36ffffff] [ 0.000000] init_memory_mapping: [mem 0x00100000-0x2fffffff] [ 0.000000] init_memory_mapping: [mem 0x37400000-0x377fdfff] [ 0.000000] RAMDISK: [mem 0x37844000-0x37c19fff] [ 0.000000] Allocated new RAMDISK: [mem 0x37428000-0x377fd57f] [ 0.000000] Move RAMDISK from [mem 0x37844000-0x37c1957f] to [mem 0x37428000-0x377fd57f] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x000FDFC0 000014 (v00 IBM ) [ 0.000000] ACPI: RSDT 0x3FFFFF80 000030 (v01 IBM SERONYXP 00001000 IBM 45444F43) [ 0.000000] ACPI: FACP 0x3FFFFF00 000074 (v01 IBM SERONYXP 00001000 IBM 45444F43) [ 0.000000] ACPI: DSDT 0x3FFFBA00 0042FA (v01 IBM SERTURQU 00001000 MSFT 0100000B) [ 0.000000] ACPI: FACS 0x3FFFFE00 000040 [ 0.000000] ACPI: APIC 0x3FFFFE40 000092 (v01 IBM SERONYXP 00001000 IBM 45444F43) [ 0.000000] ACPI: ASF! 0x3FFFFD80 00004B (v16 IBM SERONYXP 00000001 IBM 45444F43) [ 0.000000] 135MB HIGHMEM available. [ 0.000000] 887MB LOWMEM available. [ 0.000000] mapped low ram: 0 - 377fe000 [ 0.000000] low ram: 0 - 377fe000 [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x00001000-0x00ffffff] [ 0.000000] Normal [mem 0x01000000-0x377fdfff] [ 0.000000] HighMem [mem 0x377fe000-0x3fffafff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x00001000-0x0009dfff] [ 0.000000] node 0: [mem 0x00100000-0x3fffafff] [ 0.000000] Using APIC driver default [ 0.000000] ACPI: PM-Timer IO Port: 0x488 [ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x06] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x07] enabled) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x00] dfl dfl lint[0x1]) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x06] dfl dfl lint[0x1]) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] dfl dfl lint[0x1]) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x07] dfl dfl lint[0x1]) [ 0.000000] ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0]) [ 0.000000] IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15 [ 0.000000] ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16]) [ 0.000000] IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31 [ 0.000000] ACPI: IOAPIC (id[0x0c] address[0xfec02000] gsi_base[32]) [ 0.000000] IOAPIC[2]: apic_id 12, version 17, address 0xfec02000, GSI 32-47 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] e820: [mem 0x40000000-0xfebfffff] available for PCI devices [ 0.000000] setup_percpu: NR_CPUS:4 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] PERCPU: Embedded 12 pages/cpu @f6bf1000 s27520 r0 d21632 u49152 [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 260264 [ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.15.0-1+ root=/dev/mapper/g-root ro console=ttyS0,115200n8 console=tty0 [ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes) [ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) [ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) [ 0.000000] Initializing CPU#0 [ 0.000000] Initializing HighMem for node 0 (000377fe:0003fffb) [ 0.000000] Initializing Movable for node 0 (00000000:00000000) [ 0.000000] Memory: 1030380K/1048160K available (2660K kernel code, 229K rwdata, 880K rodata, 392K init, 340K bss, 17780K reserved, 139252K highmem) [ 0.000000] virtual kernel memory layout: [ 0.000000] fixmap : 0xfff67000 - 0xfffff000 ( 608 kB) [ 0.000000] pkmap : 0xff800000 - 0xffc00000 (4096 kB) [ 0.000000] vmalloc : 0xf7ffe000 - 0xff7fe000 ( 120 MB) [ 0.000000] lowmem : 0xc0000000 - 0xf77fe000 ( 887 MB) [ 0.000000] .init : 0xc13b2000 - 0xc1414000 ( 392 kB) [ 0.000000] .data : 0xc1299387 - 0xc13b0600 (1116 kB) [ 0.000000] .text : 0xc1000000 - 0xc1299387 (2660 kB) [ 0.000000] Checking if this processor honours the WP bit even in supervisor mode...Ok. [ 0.000000] SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] NR_IRQS:2304 nr_irqs:1024 16 [ 0.000000] Console: colour VGA+ 80x25 [ 0.000000] console [tty0] enabled [ 0.000000] console [ttyS0] enabled [ 0.000000] tsc: Fast TSC calibration using PIT [ 0.000000] tsc: Detected 1993.504 MHz processor [ 0.012005] Calibrating delay loop (skipped), value calculated using timer frequency.. 3987.00 BogoMIPS (lpj=7974016) [ 0.020007] pid_max: default: 32768 minimum: 301 [ 0.024019] ACPI: Core revision 20140424 [ 0.034484] ACPI: All ACPI Tables successfully acquired [ 0.041027] Security Framework initialized [ 0.044029] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes) [ 0.048008] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes) [ 0.052395] Initializing cgroup subsys devices [ 0.060007] Initializing cgroup subsys freezer [ 0.064007] Initializing cgroup subsys net_cls [ 0.068049] CPU: Physical Processor ID: 0 [ 0.072005] CPU: Processor Core ID: 0 [ 0.076006] mce: CPU supports 4 MCE banks [ 0.080016] CPU0: Thermal monitoring enabled (TM1) [ 0.084023] Last level iTLB entries: 4KB 64, 2MB 64, 4MB 64 [ 0.084023] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 64, 1GB 0 [ 0.084023] tlb_flushall_shift: 6 [ 0.088172] Freeing SMP alternatives memory: 16K (c1414000 - c1418000) [ 0.092142] Enabling APIC mode: Flat. Using 3 I/O APICs [ 0.100585] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.146513] smpboot: CPU0: Intel(R) Xeon(TM) CPU 2.00GHz (fam: 0f, model: 02, stepping: 07) [ 0.156000] Performance Events: Netburst events, Netburst P4/Xeon PMU driver. [ 0.158862] ... version: 0 [ 0.160004] ... bit width: 40 [ 0.164004] ... generic registers: 18 [ 0.168004] ... value mask: 000000ffffffffff [ 0.172004] ... max period: 0000007fffffffff [ 0.176004] ... fixed-purpose events: 0 [ 0.180004] ... event mask: 000000000003ffff [ 0.184602] x86: Booting SMP configuration: [ 0.188007] .... node #0, CPUs: #1 [ 0.016000] Initializing CPU#1 [ 0.288215] #2 [ 0.016000] Initializing CPU#2 [ 0.382274] #3 [ 0.016000] Initializing CPU#3 [ 0.478166] x86: Booted up 1 node, 4 CPUs [ 0.480006] smpboot: Total of 4 processors activated (15949.07 BogoMIPS) [ 0.485095] divide error: 0000 [#1] SMP [ 0.488000] Modules linked in: [ 0.488000] CPU: 2 PID: 6 Comm: kworker/u8:0 Not tainted 3.15.0-1+ #1 [ 0.488000] Hardware name: IBM eserver xSeries 335 -[867642X]-/, BIOS -[T2E114AUS-1.03]- 11/14/2002 [ 0.488000] task: f6469860 ti: f6484000 task.ti: f6484000 [ 0.488000] EIP: 0060:[<c10500b3>] EFLAGS: 00010046 CPU: 2 [ 0.488000] EIP is at find_busiest_group+0x18a/0x583 [ 0.488000] EAX: 00000000 EBX: f6485db0 ECX: 00000000 EDX: 00000000 [ 0.488000] ESI: 00000004 EDI: 00000000 EBP: f6485de8 ESP: f6485d14 [ 0.488000] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 [ 0.488000] CR0: 8005003b CR2: 00000068 CR3: 0141b000 CR4: 000007d0 [ 0.488000] Stack: [ 0.488000] 00000000 00000001 00000002 000006f2 00000000 c1413280 ffffffff 00000000 [ 0.488000] f6c0f280 00000000 f6485e34 f6403460 000006f2 000003ff 000003ff 000001ff [ 0.488000] 0000024d 00000002 00000001 00000000 00000001 00000000 00000000 00000000 [ 0.488000] Call Trace: [ 0.488000] [<c105059b>] load_balance+0xef/0x5cb [ 0.488000] [<c104b509>] ? sched_clock_local+0x11/0x139 [ 0.488000] [<c1050ffe>] pick_next_task_fair+0x28e/0x398 [ 0.488000] [<c1046ee9>] ? dequeue_task+0x8a/0x92 [ 0.488000] [<c1291b51>] __schedule+0x1f5/0x673 [ 0.488000] [<c1291e96>] ? __schedule+0x53a/0x673 [ 0.488000] [<c104967d>] ? default_wake_function+0xb/0xd [ 0.488000] [<c10549db>] ? __wake_up_common+0x34/0x5a [ 0.488000] [<c1292071>] schedule+0x56/0x58 [ 0.488000] [<c103dbbc>] worker_thread+0x244/0x27d [ 0.488000] [<c103d978>] ? manage_workers.isra.28+0x189/0x189 [ 0.488000] [<c1041668>] kthread+0x9f/0xa4 [ 0.488000] [<c1298241>] ret_from_kernel_thread+0x21/0x30 [ 0.488000] [<c10415c9>] ? kthread_freezable_should_stop+0x40/0x40 [ 0.488000] Code: 72 0e 00 3b 05 a0 d9 3a c1 89 c6 0f 8c 7b ff ff ff 8b 95 58 ff ff ff 8b 7b 14 8b 42 0c 31 d2 8b 48 04 8b 43 04 89 4b 10 c1 e0 0a <f7> f1 85 ff 89 85 38 ff ff ff 89 03 7 [ 0.488000] EIP: [<c10500b3>] find_busiest_group+0x18a/0x583 SS:ESP 0068:f6485d14 [ 0.488000] divide error: 0000 [#2] [ 0.488000] ---[ end trace 242397e5073f2949 ]--- [ 0.488000] SMP [ 0.488000] Modules linked in: [ 0.488000] CPU: 0 PID: 2 Comm: kthreadd Tainted: G D 3.15.0-1+ #1 [ 0.488000] Hardware name: IBM eserver xSeries 335 -[867642X]-/, BIOS -[T2E114AUS-1.03]- 11/14/2002 [ 0.488000] task: f64684e0 ti: f647c000 task.ti: f647c000 [ 0.488000] EIP: 0060:[<c104ccc9>] EFLAGS: 00010046 CPU: 0 [ 0.488000] EIP is at select_task_rq_fair+0x3f5/0x546 [ 0.488000] EAX: 00000000 EBX: f6403460 ECX: f6403480 EDX: 00000000 [ 0.488000] ESI: f643ca00 EDI: 00000000 EBP: f647df2c ESP: f647dec8 [ 0.488000] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 [ 0.488000] CR0: 8005003b CR2: ffe67000 CR3: 0141b000 CR4: 000007d0 [ 0.488000] Stack: [ 0.488000] f646f500 00000002 00000000 f647dedc c1087da7 f647df40 c102d562 f6403470 [ 0.488000] 00000070 00000008 00000000 00000379 f6bf4a80 00000000 c1413280 f646f500 [ 0.488000] 00000000 f64684e0 f6403420 00000000 00000000 00000000 f646f500 c12a14ec [ 0.488000] Call Trace: [ 0.488000] [<c1087da7>] ? perf_event_fork+0xf/0x11 [ 0.488000] [<c102d562>] ? copy_process.part.42+0x1068/0x1255 [ 0.488000] [<c10496e7>] wake_up_new_task+0x30/0xea [ 0.488000] [<c10415c9>] ? kthread_freezable_should_stop+0x40/0x40 [ 0.488000] [<c102d90c>] do_fork+0x10c/0x237 [ 0.488000] [<c10493a4>] ? do_set_cpus_allowed+0x2d/0x37 [ 0.488000] [<c10415c9>] ? kthread_freezable_should_stop+0x40/0x40 [ 0.488000] [<c102da51>] kernel_thread+0x1a/0x1f [ 0.488000] [<c1041ad5>] kthreadd+0xc7/0x10e [ 0.488000] [<c1298241>] ret_from_kernel_thread+0x21/0x30 [ 0.488000] [<c1041a0e>] ? kthread_create_on_cpu+0x44/0x44 [ 0.488000] Code: ff ff 8b 4d a0 01 45 f0 8b 45 b8 41 ba 04 00 00 00 e8 10 57 0f 00 3b 05 a0 d9 3a c1 89 c1 7c c8 8b 45 f0 31 d2 8b 4b 0c c1 e0 0a <f7> 71 04 83 7d c4 00 75 07 3b 45 dc 6 [ 0.488000] EIP: [<c104ccc9>] select_task_rq_fair+0x3f5/0x546 SS:ESP 0068:f647dec8 [ 0.488000] ---[ end trace 242397e5073f294a ]--- [ 0.488000] BUG: unable to handle kernel paging request at ffffffec [ 0.488000] IP: [<c1041927>] kthread_data+0xa/0xe [ 0.488000] *pde = 0141d067 *pte = 00000000 [ 0.488000] Oops: 0000 [#3] SMP [ 0.488000] Modules linked in: [ 0.488000] CPU: 2 PID: 6 Comm: kworker/u8:0 Tainted: G D 3.15.0-1+ #1 [ 0.488000] Hardware name: IBM eserver xSeries 335 -[867642X]-/, BIOS -[T2E114AUS-1.03]- 11/14/2002 [ 0.488000] task: f6469860 ti: f6484000 task.ti: f6484000 [ 0.488000] EIP: 0060:[<c1041927>] EFLAGS: 00010002 CPU: 2 [ 0.488000] EIP is at kthread_data+0xa/0xe [ 0.488000] EAX: 00000000 EBX: 00000002 ECX: 00000000 EDX: 00000002 [ 0.488000] ESI: 00000002 EDI: f6469860 EBP: f6485b2c ESP: f6485b24 [ 0.488000] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 [ 0.488000] CR0: 8005003b CR2: 00000014 CR3: 0141b000 CR4: 000007d0 [ 0.488000] Stack: [ 0.488000] c103dd15 f6469ad8 f6485ba0 c1291a39 f6485b48 c1413280 c1413280 f640e100 [ 0.488000] c10b6499 f6c0f280 f6469860 f6471b80 f6485b70 00000246 00406408 00000246 [ 0.488000] f6469860 f6469860 00000000 f6485b7c c105f724 00000000 f6485ba8 c102eba0 [ 0.488000] Call Trace: [ 0.488000] [<c103dd15>] ? wq_worker_sleeping+0xb/0x76 [ 0.488000] [<c1291a39>] __schedule+0xdd/0x673 [ 0.488000] [<c10b6499>] ? kmem_cache_free+0xd0/0xd9 [ 0.488000] [<c105f724>] ? call_rcu_sched+0xf/0x12 [ 0.488000] [<c102eba0>] ? release_task+0x387/0x399 [ 0.488000] [<c1292071>] schedule+0x56/0x58 [ 0.488000] [<c102feaf>] do_exit+0x75d/0x7b3 [ 0.488000] [<c128f935>] ? printk+0x17/0x19 [ 0.488000] [<c1294d00>] oops_end+0x8e/0x96 [ 0.488000] [<c1004094>] die+0x54/0x5c [ 0.488000] [<c12948e7>] do_trap+0x69/0xa4 [ 0.488000] [<c1002080>] ? math_state_restore+0x180/0x180 [ 0.488000] [<c10020ee>] do_divide_error+0x6e/0x78 [ 0.488000] [<c10500b3>] ? find_busiest_group+0x18a/0x583 [ 0.488000] [<c12946ce>] error_code+0x5a/0x60 [ 0.488000] [<c1002080>] ? math_state_restore+0x180/0x180 [ 0.488000] [<c10500b3>] ? find_busiest_group+0x18a/0x583 [ 0.488000] [<c105059b>] load_balance+0xef/0x5cb [ 0.488000] [<c104b509>] ? sched_clock_local+0x11/0x139 [ 0.488000] [<c1050ffe>] pick_next_task_fair+0x28e/0x398 [ 0.488000] [<c1046ee9>] ? dequeue_task+0x8a/0x92 [ 0.488000] [<c1291b51>] __schedule+0x1f5/0x673 [ 0.488000] [<c1291e96>] ? __schedule+0x53a/0x673 [ 0.488000] [<c104967d>] ? default_wake_function+0xb/0xd [ 0.488000] [<c10549db>] ? __wake_up_common+0x34/0x5a [ 0.488000] [<c1292071>] schedule+0x56/0x58 [ 0.488000] [<c103dbbc>] worker_thread+0x244/0x27d [ 0.488000] [<c103d978>] ? manage_workers.isra.28+0x189/0x189 [ 0.488000] [<c1041668>] kthread+0x9f/0xa4 [ 0.488000] [<c1298241>] ret_from_kernel_thread+0x21/0x30 [ 0.488000] [<c10415c9>] ? kthread_freezable_should_stop+0x40/0x40 [ 0.488000] Code: c4 3c 5b 5e 5f 5d c3 55 64 a1 ac e6 40 c1 8b 80 4c 02 00 00 89 e5 5d 8b 40 e4 c1 e8 02 83 e0 01 c3 55 8b 80 4c 02 00 00 89 e5 5d <8b> 40 ec c3 55 b9 04 00 00 00 89 e5 5 [ 0.488000] EIP: [<c1041927>] kthread_data+0xa/0xe SS:ESP 0068:f6485b24 [ 0.488000] CR2: 00000000ffffffec [ 0.488000] ---[ end trace 242397e5073f294b ]--- [ 0.488000] Fixing recursive fault but reboot is needed! [ 0.488051] divide error: 0000 [#4] SMP [ 0.492000] Modules linked in: [ 0.492000] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G D 3.15.0-1+ #1 [ 0.492000] Hardware name: IBM eserver xSeries 335 -[867642X]-/, BIOS -[T2E114AUS-1.03]- 11/14/2002 [ 0.492000] task: f6468000 ti: f644c000 task.ti: f644c000 [ 0.492000] EIP: 0060:[<c10500b3>] EFLAGS: 00010046 CPU: 0 [ 0.492000] EIP is at find_busiest_group+0x18a/0x583 [ 0.492000] EAX: 00000000 EBX: f644dc9c ECX: 00000000 EDX: 00000000 [ 0.492000] ESI: 00000004 EDI: ffffffff EBP: f644dd40 ESP: f644dc6c [ 0.492000] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 [ 0.492000] CR0: 8005003b CR2: ffe67000 CR3: 0141b000 CR4: 000007d0 [ 0.492000] Stack: [ 0.492000] b6cb9571 00000001 00000001 00000000 00000000 c1413280 00000000 00000000 [ 0.492000] f6c0f280 00000000 f644dd8c f6403460 00000000 00000000 00000000 00000000 [ 0.492000] 00000000 ffffffff 00000000 00000000 00000000 00000000 00000000 00000000 [ 0.492000] Call Trace: [ 0.492000] [<c105059b>] load_balance+0xef/0x5cb [ 0.492000] [<c104b509>] ? sched_clock_local+0x11/0x139 [ 0.492000] [<c1050ffe>] pick_next_task_fair+0x28e/0x398 [ 0.492000] [<c1046ee9>] ? dequeue_task+0x8a/0x92 [ 0.492000] [<c1291b51>] __schedule+0x1f5/0x673 [ 0.492000] [<c1291e96>] ? __schedule+0x53a/0x673 [ 0.492000] [<c104d006>] ? check_preempt_wakeup+0xf9/0x13f [ 0.492000] [<c1292071>] schedule+0x56/0x58 [ 0.492000] [<c12915ef>] schedule_timeout+0x17/0x10e [ 0.492000] [<c104738a>] ? check_preempt_curr+0x27/0x62 [ 0.492000] [<c10473d6>] ? ttwu_do_wakeup+0x11/0xba [ 0.492000] [<c129269c>] wait_for_common+0xc0/0xe3 [ 0.492000] [<c1049672>] ? try_to_wake_up+0x1aa/0x1aa [ 0.492000] [<c1292724>] wait_for_completion_killable+0x12/0x21 [ 0.492000] [<c10416f8>] kthread_create_on_node+0x8b/0xf9 [ 0.492000] [<c103ea61>] __alloc_workqueue_key+0x21a/0x302 [ 0.492000] [<c103d4f2>] ? process_scheduled_works+0x21/0x21 [ 0.492000] [<c13c3a16>] usermodehelper_init+0x1a/0x2a [ 0.492000] [<c13b2a65>] kernel_init_freeable+0xb6/0x19d [ 0.492000] [<c128d68f>] kernel_init+0x8/0xb3 [ 0.492000] [<c1298241>] ret_from_kernel_thread+0x21/0x30 [ 0.492000] [<c128d687>] ? rest_init+0x5f/0x5f [ 0.492000] Code: 72 0e 00 3b 05 a0 d9 3a c1 89 c6 0f 8c 7b ff ff ff 8b 95 58 ff ff ff 8b 7b 14 8b 42 0c 31 d2 8b 48 04 8b 43 04 89 4b 10 c1 e0 0a <f7> f1 85 ff 89 85 38 ff ff ff 89 03 7 [ 0.492000] EIP: [<c10500b3>] find_busiest_group+0x18a/0x583 SS:ESP 0068:f644dc6c [ 0.492000] ---[ end trace 242397e5073f294c ]--- [ 0.492000] divide error: 0000 [#5] SMP [ 0.492000] Modules linked in: [ 0.492000] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D 3.15.0-1+ #1 [ 0.492000] Hardware name: IBM eserver xSeries 335 -[867642X]-/, BIOS -[T2E114AUS-1.03]- 11/14/2002 [ 0.492000] task: f646b5a0 ti: f6490000 task.ti: f6490000 [ 0.492000] EIP: 0060:[<c10500b3>] EFLAGS: 00210246 CPU: 3 [ 0.492000] EIP is at find_busiest_group+0x18a/0x583 [ 0.492000] EAX: 00000000 EBX: f64bde38 ECX: 00000000 EDX: 00000000 [ 0.492000] ESI: 00000004 EDI: ffffffff EBP: f64bdedc ESP: f64bde08 [ 0.492000] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 [ 0.492000] CR0: 8005003b CR2: 00000000 CR3: 0141b000 CR4: 000007d0 [ 0.492000] Stack: [ 0.492000] 00000000 00000001 00000001 00000000 00000000 c1413280 00000000 00000001 [ 0.492000] f6c0f280 00000000 f64bdf28 f6403460 00000000 00000000 00000000 00000000 [ 0.492000] 00000000 ffffffff 00000000 00000000 00000000 00000000 00000000 00000000 [ 0.492000] Call Trace: [ 0.492000] [<c105059b>] load_balance+0xef/0x5cb [ 0.492000] [<c104d9bb>] ? update_blocked_averages+0x53f/0x547 [ 0.492000] [<c1050bac>] rebalance_domains+0x135/0x1ed [ 0.492000] [<c1050c92>] run_rebalance_domains+0x2e/0x10c [ 0.492000] [<c10312c5>] __do_softirq+0x91/0x174 [ 0.492000] [<c1031234>] ? __hrtimer_tasklet_trampoline+0x1a/0x1a [ 0.492000] [<c10032ff>] do_softirq_own_stack+0x1d/0x23 [ 0.492000] <IRQ> [ 0.492000] [<c10314e7>] irq_exit+0x34/0x75 [ 0.492000] [<c102148d>] smp_apic_timer_interrupt+0x33/0x3d [ 0.492000] [<c12944a5>] apic_timer_interrupt+0x2d/0x34 [ 0.492000] [<c104007b>] ? param_array_set+0x23/0xc0 [ 0.492000] [<c10076b5>] ? default_idle+0x5/0x7 [ 0.492000] [<c1007bcd>] arch_cpu_idle+0x9/0xb [ 0.492000] [<c10550c1>] cpu_startup_entry+0xe6/0x1c9 [ 0.492000] [<c101ffad>] start_secondary+0x1a6/0x1ab [ 0.492000] Code: 72 0e 00 3b 05 a0 d9 3a c1 89 c6 0f 8c 7b ff ff ff 8b 95 58 ff ff ff 8b 7b 14 8b 42 0c 31 d2 8b 48 04 8b 43 04 89 4b 10 c1 e0 0a <f7> f1 85 ff 89 85 38 ff ff ff 89 03 7 [ 0.492000] EIP: [<c10500b3>] find_busiest_group+0x18a/0x583 SS:ESP 0068:f64bde08 [ 0.492000] divide error: 0000 [#6] [ 0.492000] ---[ end trace 242397e5073f294d ]--- [ 0.492000] SMP [ 0.492000] Kernel panic - not syncing: Fatal exception in interrupt [ 0.492000] Modules linked in: [ 0.492000] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D 3.15.0-1+ #1 [ 0.492000] Hardware name: IBM eserver xSeries 335 -[867642X]-/, BIOS -[T2E114AUS-1.03]- 11/14/2002 [ 0.492000] task: f646abe0 ti: f648c000 task.ti: f648c000 [ 0.492000] EIP: 0060:[<c10500b3>] EFLAGS: 00210246 CPU: 1 [ 0.492000] EIP is at find_busiest_group+0x18a/0x583 [ 0.492000] EAX: 00000000 EBX: f649de38 ECX: 00000000 EDX: 00000000 [ 0.492000] ESI: 00000004 EDI: ffffffff EBP: f649dedc ESP: f649de08 [ 0.492000] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 [ 0.492000] CR0: 8005003b CR2: 00000000 CR3: 0141b000 CR4: 000007d0 [ 0.492000] Stack: [ 0.492000] 00000000 00000001 00000001 00000000 00000000 c1413280 00000000 00000001 [ 0.492000] f6c0f280 00000000 f649df28 f6403460 00000000 00000000 00000000 00000000 [ 0.492000] 00000000 ffffffff 00000000 00000000 00000000 00000000 00000000 00000000 [ 0.492000] Call Trace: [ 0.492000] [<c105059b>] load_balance+0xef/0x5cb [ 0.492000] [<c104d9bb>] ? update_blocked_averages+0x53f/0x547 [ 0.492000] [<c1050bac>] rebalance_domains+0x135/0x1ed [ 0.492000] [<c1050c92>] run_rebalance_domains+0x2e/0x10c [ 0.492000] [<c10312c5>] __do_softirq+0x91/0x174 [ 0.492000] [<c1031234>] ? __hrtimer_tasklet_trampoline+0x1a/0x1a [ 0.492000] [<c10032ff>] do_softirq_own_stack+0x1d/0x23 [ 0.492000] <IRQ> [ 0.492000] [<c10314e7>] irq_exit+0x34/0x75 [ 0.492000] [<c102148d>] smp_apic_timer_interrupt+0x33/0x3d [ 0.492000] [<c12944a5>] apic_timer_interrupt+0x2d/0x34 [ 0.492000] [<c104007b>] ? param_array_set+0x23/0xc0 [ 0.492000] [<c10076b5>] ? default_idle+0x5/0x7 [ 0.492000] [<c1007bcd>] arch_cpu_idle+0x9/0xb [ 0.492000] [<c10550c1>] cpu_startup_entry+0xe6/0x1c9 [ 0.492000] [<c101ffad>] start_secondary+0x1a6/0x1ab [ 0.492000] Code: 72 0e 00 3b 05 a0 d9 3a c1 89 c6 0f 8c 7b ff ff ff 8b 95 58 ff ff ff 8b 7b 14 8b 42 0c 31 d2 8b 48 04 8b 43 04 89 4b 10 c1 e0 0a <f7> f1 85 ff 89 85 38 ff ff ff 89 03 7 [ 0.492000] EIP: [<c10500b3>] find_busiest_group+0x18a/0x583 SS:ESP 0068:f649de08 [ 0.492000] Shutting down cpus with NMI [ 0.492000] ---[ end Kernel panic - not syncing: Fatal exception in interrupt -- Greg Donald