kernel 5.5 kernel and tg3 driver

sri bskmohan at gmail.com
Mon Oct 10 02:34:52 EDT 2011


Hello Experts,

I was using click-1.6.0 on centos 5.3 kernel and recently* ported that to
centos5.5 kernel.*.

After porting, am getting the (*low stack detected by irq handler)  and the
*following errors on the console frequently and causing the system
unresponsive totally.

Even after reboot, this issue is re-occurring some time after every 3 days.

System info:
kernel: centos 5.5 (2.6.194-el5)
kernel Modules using: click router
NIC driver: tg3 (broadcom)

Note: The above setup was working successfully for years earlier.

I suspect tg3 driver is doing something here but not sure of how to confirm
this.
Am looking for pointers to start with so that I can investigate. I tried to
do web search on this but not useful.
Also, would like to know any ways/tools to debug using the addresses given
in the stack trace.


Thanks in advance.


######################## *START OF LOGS* ############################
Oct  1 14:16:56 server03 kernel: low stack detected by irq handler
Oct  1 14:16:56 server03 kernel:  [<c0405e9b>] do_IRQ+0x86/0xc3
Oct  1 14:16:56 server03 kernel:  [<c04046e6>] common_interrupt+0x1a/0x20
Oct  1 14:16:56 server03 kernel:  [<c05a5c06>] neigh_lookup+0x23/0x7f
Oct  1 14:16:56 server03 kernel:  [<c05d6800>] arp_bind_neighbour+0x50/0x71
Oct  1 14:16:56 server03 kernel:  [<c05b7b44>] rt_intern_hash+0x372/0x480
Oct  1 14:16:56 server03 kernel:  [<c05b8352>] __ip_route_output_key+0x700/
0x79e
Oct  1 14:16:56 server03 kernel:  [<c05b83d4>]
__ip_route_output_key+0x782/0x79e
Oct  1 14:16:57 server03 kernel:  [<c05b8403>]
ip_route_output_flow+0x13/0x1d4
Oct  1 14:16:57 server03 kernel:  [<c05b85cd>] ip_route_output_key+0x9/0xb
Oct  1 14:16:57 server03 kernel:  [<c05bdd5d>] ip_send_reply+0xf8/0x209
Oct  1 14:16:57 server03 kernel:  [<c045f1a6>]
cache_alloc_refill+0x122/0x3e4
Oct  1 14:16:57 server03 kernel:  [<c05b7c2f>] rt_intern_hash+0x45d/0x480
Oct  1 14:16:57 server03 kernel:  [<c05cfc56>] tcp_v4_send_reset+0x104/0x11c
Oct  1 14:16:57 server03 kernel:  [<c05d0879>] tcp_v4_do_rcv+0x240/0x269
Oct  1 14:16:57 server03 kernel:  [<c05d28cb>] tcp_v4_rcv+0x7f7/0x848
Oct  1 14:16:57 server03 kernel:  [<c05bb173>] ip_local_deliver+0x157/0x1d0
Oct  1 14:16:57 server03 kernel:  [<c05baff2>] ip_rcv+0x3cb/0x3f5
Oct  1 14:16:57 server03 kernel:  [<c05a141d>]
__netif_receive_skb+0x3b3/0x3d5
Oct  1 14:16:57 server03 kernel:  [<fb1b121e>]
_ZN6ToHost4pushEiP6Packet+0xee/0x210 [click]
Oct  1 14:16:57 server03 kernel:  [<fb1ce85d>]
_ZN12EnsureEther24pushEiP6Packet+0x1d/0x20 [click]
Oct  1 14:16:57 server03 kernel:  [<fb1c0895>]
_ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
Oct  1 14:16:57 server03 kernel:  [<c05bb173>] ip_local_deliver+0x157/0x1d0
Oct  1 14:16:57 server03 kernel:  [<c05baff2>] ip_rcv+0x3cb/0x3f5
Oct  1 14:16:57 server03 kernel:  [<fb19e42d>]
_ZN4IPRw7Mapping9make_pairEiRK8IPFlowIDS3_iiPS0_S4_+0xad/0xd0 [click]
Oct  1 14:16:57 server03 kernel:  [<fb19fcfb>]
_ZN4IPRw7Pattern14create_mappingEiRK8IPFlowIDiiPNS_7MappingES5_RK7HashMapIS1_S5_E+0x1cb/0x1f0
[click]
Oct  1 14:16:57 server03 kernel:  [<fb218ce3>]
_ZN10IPRewriter13apply_patternEPN4IPRw7PatternEiRK8IPFlowIDii+0x153/0x160
[click]
Oct  1 14:16:57 server03 kernel:  [<fb218d9e>]
_ZN10IPRewriter4pushEiP6Packet+0xae/0x260 [click]
Oct  1 14:16:57 server03 kernel:  [<fb1ea535>]
_ZN11MacThrottle10l2PushProcEiP6Packet+0x55/0x3b0 [click]
Oct  1 14:16:57 server03 kernel:  [<fb1c0895>]
_ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
Oct  1 14:16:57 server03 kernel:  [<fb218ce3>]
_ZN10IPRewriter13apply_patternEPN4IPRw7PatternEiRK8IPFlowIDii+0x153/0x160
[click]
Oct  1 14:16:57 server03 kernel:  [<fb218d9e>]
_ZN10IPRewriter4pushEiP6Packet+0xae/0x260 [click]
Oct  1 14:16:58 server03 kernel:  [<fb1eab96>]
_ZN11MacThrottle4pushEiP6Packet+0x306/0x1890 [click]
Oct  1 14:16:58 server03 kernel:  [<c0417f22>] enqueue_task+0x29/0x39
Oct  1 14:16:58 server03 kernel:  [<c041805a>] __activate_task+0x4a/0x59
Oct  1 14:16:58 server03 kernel:  [<fb1c69d8>]
_ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
Oct  1 14:16:58 server03 kernel:  [<fb155806>]
_ZN7Element4pushEiP6Packet+0x26/0x40 [click]
Oct  1 14:16:58 server03 kernel:  [<fb207674>]
_ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct  1 14:16:58 server03 kernel:  [<fb1ee075>]
_ZN17NtlmLoginDetector4pushEiP6Packet+0x65/0x15d0 [click]
Oct  1 14:16:58 server03 kernel:  [<fb155806>]
_ZN7Element4pushEiP6Packet+0x26/0x40 [click]
Oct  1 14:16:58 server03 kernel:  [<fb207674>]
_ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct  1 14:16:58 server03 kernel:  [<fb21f881>]
_ZN15RecycledSkbPool7recycleEP7sk_buff+0x31/0xf0 [click]
Oct  1 14:16:58 server03 kernel:  [<fb1fbfdb>]
_ZN14PerfigoDiscard4pushEiP6Packet+0x4b/0x1c0 [click]
Oct  1 14:16:58 server03 kernel:  [<c05bb173>] ip_local_deliver+0x157/0x1d0
Oct  1 14:16:58 server03 kernel:  [<fb1d8e86>]
_ZN11HashIPTable4pushEiP6Packet+0xf6/0x2d0 [click]
Oct  1 14:16:58 server03 kernel:  [<c05a141d>]
__netif_receive_skb+0x3b3/0x3d5
Oct  1 14:16:58 server03 kernel:  [<fb1c69d8>]
_ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
Oct  1 14:16:58 server03 kernel:  [<fb192904>]
_ZN8IPFilter4pushEiP6Packet+0x64/0xf0 [click]
Oct  1 14:16:58 server03 kernel:  [<fb1e4517>]
_ZN12IPPortLookup4pushEiP6Packet+0x47/0x9e0 [click]
Oct  1 14:16:58 server03 kernel:  [<c043f334>] __do_IRQ+0x0/0x9f
Oct  1 14:16:58 server03 kernel:  [<c0405ecc>] do_IRQ+0xb7/0xc3
Oct  1 14:16:58 server03 kernel:  [<c043f334>] __do_IRQ+0x0/0x9f
Oct  1 14:16:58 server03 kernel:  [<fb207674>]
_ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct  1 14:16:58 server03 kernel:  [<fb1cd5fc>]
_ZN10DnsHandler17is_packet_allowedEPK6Packeti+0x8c/0x5e0 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1c69d8>]
_ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1c69d8>]
_ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
Oct  1 14:16:59 server03 kernel:  [<fb207674>]
_ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1cdbb8>]
_ZN10DnsHandler4pushEiP6Packet+0x68/0xb00 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1b1853>]
_ZN11ARPQuerier214send_query_forERK9IPAddressitit+0x133/0x3f0 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1b1dba>]
_ZN11ARPQuerier29handle_ipEP6Packet+0x2aa/0x5f0 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1b31bd>]
_ZN11ARPQuerier24pushEiP6Packet+0xcd/0x110 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1cdbb8>]
_ZN10DnsHandler4pushEiP6Packet+0x68/0xb00 [click]
Oct  1 14:16:59 server03 kernel:  [<fb207674>]
_ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1cdbb8>]
_ZN10DnsHandler4pushEiP6Packet+0x68/0xb00 [click]
Oct  1 14:16:59 server03 kernel:  [<fb207674>]
_ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1e234f>]
_ZN13IPFilterGroup4pushEiP6Packet+0x19f/0xa80 [click]
Oct  1 14:16:59 server03 kernel:  [<fb207674>]
_ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1fb79e>]
_ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct  1 14:16:59 server03 kernel:  [<fb207674>]
_ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct  1 14:16:59 server03 kernel:  [<fb207674>]
_ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1fb79e>]
_ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct  1 14:16:59 server03 kernel:  [<c05a5c59>] neigh_lookup+0x76/0x7f
Oct  1 14:16:59 server03 kernel:  [<fb1d3492>]
_ZN13HashIPLookup34pushEiP6Packet+0x112/0x1c0 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1e66cf>]
_ZN15LinearIPFilter24pushEiP6Packet+0x4f/0x3f0 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1b80e8>]
_ZN11ARPQuerier420handle_arp_retaggingEP6Packett+0x58/0x940 [click]
Oct  1 14:16:59 server03 kernel:  [<c05bf82d>] ip_output+0x1f0/0x227
Oct  1 14:16:59 server03 kernel:  [<fb1dd289>]
_ZN14HashMACLookup54pushEiP6Packet+0x249/0x890 [click]
Oct  1 14:16:59 server03 kernel:  [<fb1d45b7>]
_ZN13HashIPLookup54pushEiP6Packet+0xc7/0x110 [click]
Oct  1 14:17:00 server03 kernel:  [<fb1c0895>]
_ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
Oct  1 14:17:00 server03 kernel:  [<fb1fb79e>]
_ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct  1 14:17:00 server03 kernel:  [<fb1dcd23>]
_ZN14HashMACLookup514push_extern_ipEP6Packet+0x1b3/0x4d0 [click]
Oct  1 14:17:00 server03 kernel:  [<fb1d762e>]
_ZN12HashIPTable24pushEiP6Packet+0x18e/0x580 [click]
Oct  1 14:17:00 server03 kernel:  [<c05d065b>] tcp_v4_do_rcv+0x22/0x269
Oct  1 14:17:00 server03 kernel:  [<c05d28cb>] tcp_v4_rcv+0x7f7/0x848
Oct  1 14:17:00 server03 kernel:  [<fb1bfb12>]
_ZN11ARPQuerier415handle_responseEP6Packet+0x32/0x8f0 [click]
Oct  1 14:17:00 server03 kernel:  [<fb21f881>]
_ZN15RecycledSkbPool7recycleEP7sk_buff+0x31/0xf0 [click]
Oct  1 14:17:00 server03 kernel:  [<fb1fbfdb>]
_ZN14PerfigoDiscard4pushEiP6Packet+0x4b/0x1c0 [click]
Oct  1 14:17:00 server03 kernel:  [<fb192904>]
_ZN8IPFilter4pushEiP6Packet+0x64/0xf0 [click]
Oct  1 14:17:00 server03 kernel:  [<fb1c0895>]
_ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
Oct  1 14:17:00 server03 kernel:  [<fb1fb79e>]
_ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct  1 14:17:00 server03 kernel:  [<fb1fb79e>]
_ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct  1 14:17:00 server03 kernel:  [<fb1d762e>]
_ZN12HashIPTable24pushEiP6Packet+0x18e/0x580 [click]
Oct  1 14:17:00 server03 kernel:  [<c05d7777>] arp_rcv+0xd0/0xe1
Oct  1 14:17:00 server03 kernel:  [<c05a141d>]
__netif_receive_skb+0x3b3/0x3d5
Oct  1 14:17:00 server03 kernel:  [<c043f334>] __do_IRQ+0x0/0x9f
Oct  1 14:17:00 server03 kernel:  [<c043f334>] __do_IRQ+0x0/0x9f
Oct  1 14:17:00 server03 kernel:  [<c0405ecc>] do_IRQ+0xb7/0xc3
Oct  1 14:17:00 server03 kernel:  [<c0411f41>]
smp_apic_timer_interrupt+0x32/0x45
Oct  1 14:17:00 server03 kernel:  [<c040470b>]
apic_timer_interrupt+0x1f/0x24
Oct  1 14:17:00 server03 kernel:  [<c0405ecc>] do_IRQ+0xb7/0xc3
Oct  1 14:17:00 server03 kernel:  [<c0405ecc>] do_IRQ+0xb7/0xc3
Oct  1 14:17:01 server03 kernel:  [<fb1fe516>]
_ZN10Classifier4pushEiP6Packet+0x76/0xe0 [click]
Oct  1 14:17:01 server03 kernel:  [<fb155806>]
_ZN7Element4pushEiP6Packet+0x26/0x40 [click]
Oct  1 14:17:01 server03 kernel:  [<fb1ab2a5>]
_ZN10FromDevice8run_taskEP4Task+0xa5/0x120 [click]
Oct  1 14:17:01 server03 kernel:  [<fb16c127>]
_ZN12RouterThread6driverEv+0x177/0x290 [click]
Oct  1 14:17:01 server03 kernel:  [<fb21c822>] _Z11click_schedPv+0x82/0x130
[click]
Oct  1 14:17:01 server03 kernel:  [<fb21c7a0>] _Z11click_schedPv+0x0/0x130
[click]
Oct  1 14:17:01 server03 kernel:  [<c0404987>] kernel_thread_helper+0x7/0x10
Oct  1 14:17:01 server03 kernel:  =======================



#######################* END OF LOGS #*
#####################################################
-- 
--
 Krishna Mohan B


-- 
--
 Krishna Mohan B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20111010/b002ab4e/attachment-0001.html 


More information about the Kernelnewbies mailing list