<div dir="ltr"><div><div><div><div><div>HI All,<br></div>I facing CPU HALT issue on linux kernel 4.0.4 after running below mentioned command:<br></div>#hostapd -B /etc/hostapd.conf<br><br><br></div>dump_stack:<br><br>[ 335.262172] [<80013400>] (__irq_svc) from [<8043f524>] (_raw_spin_unlock_irqrestore+0x20/0x54)<br>[ 335.270975] [<8043f524>] (_raw_spin_unlock_irqrestore) from [<7f116820>] (ieee80211_wake_queues_by_reason+0x88/0x90 [mac80211])<br>[ 335.282669] [<7f116820>] (ieee80211_wake_queues_by_reason [mac80211]) from [<7f16e5ac>] (ath_complete_reset+0xdc/0x138 [ath9k])<br>[ 335.294187] [<7f16e5ac>] (ath_complete_reset [ath9k]) from [<7f16e9ac>] (ath_reset_internal+0x180/0x254 [ath9k])<br>[ 335.304389] [<7f16e9ac>] (ath_reset_internal [ath9k]) from [<7f16eaa0>] (ath_reset_work+0x20/0x40 [ath9k])<br>[ 335.314062] [<7f16eaa0>] (ath_reset_work [ath9k]) from [<80036b18>] (process_one_work+0x124/0x334)<br>[ 335.323023] [<80036b18>] (process_one_work) from [<80037d60>] (worker_thread+0x140/0x524)<br>[ 335.331207] [<80037d60>] (worker_thread) from [<8003c040>] (kthread+0xf4/0x108)<br>[ 335.338524] [<8003c040>] (kthread) from [<8000ed40>] (ret_from_fork+0x14/0x34)<br>[ 335.345750] INFO: rcu_preempt detected stalls on CPUs/tasks:<br><br><br><br></div><div>This I m facing this issue with both hostadp version (0.7.3 and 2.4).<br><br></div><div><br></div>Thanks,<br></div>Amit Gupta<br><div><div><div><br><br><br></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jul 28, 2015 at 11:45 PM, Amit Gupta <span dir="ltr"><<a href="mailto:amitkgupta09@gmail.com" target="_blank">amitkgupta09@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi All,<div>Thanks for your quick response.</div><div><br></div><div>Valdis,</div><div>I tried with strace stuff, but I faced CPU halt issue. Actually along with those error logs..sometime I m facing CPU HALT issue after executing #hostapd -B /etc/hostapd.conf.</div><div><br></div><div><br></div><div><br></div><div>Pranay,</div><div>I tried your way to debug kernel for finding the root cause of error log '<span style="font-size:12.8000001907349px">Failed to update rate sets in kernel module'.</span></div><div><span style="font-size:12.8000001907349px">But prior to that I started to debugging for </span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span class=""><span style="font-size:12.8000001907349px">IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready</span><br style="font-size:12.8000001907349px"></span><span class=""><span style="font-size:12.8000001907349px">IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready</span><span style="font-size:12.8000001907349px"><br></span></span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">those two logs.</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">Till now I found that before execution of __dev_open(</span><span style="font-size:12.8000001907349px">net/core/dev,c)</span><span style="font-size:12.8000001907349px"> function for 'wlan0' network interface, 'netif_carrier_off'(net/sched/sch_generic.c) function get called for 'wlan0',</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;color:rgb(120,120,120);overflow:auto;font-weight:bold">void <a href="http://lxr.free-electrons.com/ident?v=4.0;i=netif_carrier_off" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">netif_carrier_off</a>(struct <a href="http://lxr.free-electrons.com/ident?v=4.0;i=net_device" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">net_device</a> *<a href="http://lxr.free-electrons.com/ident?v=4.0;i=dev" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">dev</a>)
{
if (!<a href="http://lxr.free-electrons.com/ident?v=4.0;i=test_and_set_bit" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">test_and_set_bit</a>(__LINK_STATE_NOCARRIER, &<a href="http://lxr.free-electrons.com/ident?v=4.0;i=dev" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">dev</a>-><a href="http://lxr.free-electrons.com/ident?v=4.0;i=state" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">state</a>)) {
if (<a href="http://lxr.free-electrons.com/ident?v=4.0;i=dev" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">dev</a>-><a href="http://lxr.free-electrons.com/ident?v=4.0;i=reg_state" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">reg_state</a> == NETREG_UNINITIALIZED)
return;
<a href="http://lxr.free-electrons.com/ident?v=4.0;i=atomic_inc" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">atomic_inc</a>(&<a href="http://lxr.free-electrons.com/ident?v=4.0;i=dev" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">dev</a>->carrier_changes);
<a href="http://lxr.free-electrons.com/ident?v=4.0;i=linkwatch_fire_event" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">linkwatch_fire_event</a>(<a href="http://lxr.free-electrons.com/ident?v=4.0;i=dev" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">dev</a>);
}
}</pre></div><div><span style="font-size:12.8000001907349px"> which result into condition checking passed in '</span><a href="http://lxr.free-electrons.com/ident?v=4.0;i=addrconf_qdisc_ok" style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;font-weight:bold;text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">addrconf_qdisc_ok</a>'<font color="#787878" face="Monaco, Courier New, Courier, monospace"><span style="font-size:12.3500003814697px"><b> function.</b></span></font></div><div><font color="#787878" face="Monaco, Courier New, Courier, monospace"><span style="font-size:12.3500003814697px"><b><br></b></span></font></div><div><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;color:rgb(120,120,120);overflow:auto;font-weight:bold">In 'net/ipv6/addrconf.c' file</pre><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;color:rgb(120,120,120);overflow:auto;font-weight:bold">if (<a href="http://lxr.free-electrons.com/ident?v=4.0;i=event" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">event</a> == <a href="http://lxr.free-electrons.com/ident?v=4.0;i=NETDEV_UP" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">NETDEV_UP</a>) {
if (!<a href="http://lxr.free-electrons.com/ident?v=4.0;i=addrconf_qdisc_ok" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">addrconf_qdisc_ok</a>(<a href="http://lxr.free-electrons.com/ident?v=4.0;i=dev" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">dev</a>)) {
<b><i>/* device is not ready yet. */</i></b> <a href="http://lxr.free-electrons.com/ident?v=4.0;i=pr_info" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">pr_info</a>(<i>"ADDRCONF(NETDEV_UP): %s: link is not ready\n"</i>,
<a href="http://lxr.free-electrons.com/ident?v=4.0;i=dev" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">dev</a>-><a href="http://lxr.free-electrons.com/ident?v=4.0;i=name" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">name</a>);
break;
}</pre><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;color:rgb(120,120,120);overflow:auto;font-weight:bold">in same file: </pre><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;color:rgb(120,120,120);overflow:auto;font-weight:bold"><span style="font-size:0.95em">static inline </span><a href="http://lxr.free-electrons.com/ident?v=4.0;i=bool" style="font-size:0.95em;text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">bool</a><span style="font-size:0.95em"> </span><a href="http://lxr.free-electrons.com/ident?v=4.0;i=addrconf_qdisc_ok" style="font-size:0.95em;text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">addrconf_qdisc_ok</a><span style="font-size:0.95em">(const struct </span><a href="http://lxr.free-electrons.com/ident?v=4.0;i=net_device" style="font-size:0.95em;text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">net_device</a><span style="font-size:0.95em"> *</span><a href="http://lxr.free-electrons.com/ident?v=4.0;i=dev" style="font-size:0.95em;text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">dev</a>) <span style="font-size:0.95em">{</span></pre><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;color:rgb(120,120,120);overflow:auto;font-weight:bold"><span style="font-size:0.95em">return !</span><a href="http://lxr.free-electrons.com/ident?v=4.0;i=qdisc_tx_is_noop" style="font-size:0.95em;text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">qdisc_tx_is_noop</a><span style="font-size:0.95em">(</span><a href="http://lxr.free-electrons.com/ident?v=4.0;i=dev" style="font-size:0.95em;text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">dev</a><span style="font-size:0.95em">);</span></pre><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;color:rgb(120,120,120);overflow:auto;font-weight:bold"><span style="font-size:0.95em">}</span></pre><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;color:rgb(120,120,120);overflow:auto;font-weight:bold"><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;overflow:auto">In include/net/sch_generic.h file</pre><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;overflow:auto"><span style="font-size:0.95em"> static inline </span><a href="http://lxr.free-electrons.com/ident?v=4.0;i=bool" style="font-size:0.95em;text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">bool</a><span style="font-size:0.95em"> </span><a href="http://lxr.free-electrons.com/ident?v=4.0;i=qdisc_tx_is_noop" style="font-size:0.95em;text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">qdisc_tx_is_noop</a><span style="font-size:0.95em">(const struct </span><a href="http://lxr.free-electrons.com/ident?v=4.0;i=net_device" style="font-size:0.95em;text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">net_device</a><span style="font-size:0.95em"> *</span><a href="http://lxr.free-electrons.com/ident?v=4.0;i=dev" style="font-size:0.95em;text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">dev</a><span style="font-size:0.95em">)</span></pre><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;overflow:auto"><span style="font-size:0.95em"> {</span></pre><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;overflow:auto"><span style="font-size:0.95em"> unsigned int </span><a href="http://lxr.free-electrons.com/ident?v=4.0;i=i" style="font-size:0.95em;text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">i</a><span style="font-size:0.95em">;</span></pre></pre><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;color:rgb(120,120,120);overflow:auto;font-weight:bold"><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;overflow:auto">
for (<a href="http://lxr.free-electrons.com/ident?v=4.0;i=i" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">i</a> = 0; <a href="http://lxr.free-electrons.com/ident?v=4.0;i=i" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">i</a> < <a href="http://lxr.free-electrons.com/ident?v=4.0;i=dev" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">dev</a>->num_tx_queues; <a href="http://lxr.free-electrons.com/ident?v=4.0;i=i" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">i</a>++) {
struct <a href="http://lxr.free-electrons.com/ident?v=4.0;i=netdev_queue" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">netdev_queue</a> *txq = <a href="http://lxr.free-electrons.com/ident?v=4.0;i=netdev_get_tx_queue" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">netdev_get_tx_queue</a>(<a href="http://lxr.free-electrons.com/ident?v=4.0;i=dev" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">dev</a>, <a href="http://lxr.free-electrons.com/ident?v=4.0;i=i" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">i</a>);
if (<a href="http://lxr.free-electrons.com/ident?v=4.0;i=rcu_access_pointer" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">rcu_access_pointer</a>(txq->qdisc) != &<a href="http://lxr.free-electrons.com/ident?v=4.0;i=noop_qdisc" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">noop_qdisc</a>)
return <a href="http://lxr.free-electrons.com/ident?v=4.0;i=false" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">false</a>;
}
return <a href="http://lxr.free-electrons.com/ident?v=4.0;i=true" style="text-decoration:none;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:rgb(153,153,153);color:black" target="_blank">true</a>;
}</pre></pre><pre style="font-family:Monaco,'Courier New',Courier,monospace;font-size:0.95em;padding:1em;color:rgb(120,120,120);overflow:auto;font-weight:bold"><br></pre></div><div>I am still looking to find the cause execution of net_carrier_off function before execution of __dev_open function for 'wlan0' network interface. This behavior I am not observing with my other wired and pseudo network interfaces. </div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">'Failed to update rate sets in kernel module'</span><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">This error log may come because of previous issue as wlan0 device is not active till this point.</span></div><div><br></div><div><br></div><div>One more thing I tried, I cross compiled 'hostapd' version 2.4 for my target board and executed that on target board.I did not get above mentioned error log and till now no CPU halt issue.</div><div><br></div><div>Previously i was working with hostapd 0.7.3.</div><div><br></div><div><br></div><div>So is it like, hostapd old version is not compatible with new linux kernel version as hostapd 0.7.3 version is running with no issue at linux 3.4.</div><div><br></div><div>Any advice and suggestions will be appreciable/helpful.</div><div><br></div><div>Thanks,</div><div>Amit Gupta</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Fri, Jul 24, 2015 at 11:06 AM, Pranay Srivastava <span dir="ltr"><<a href="mailto:pranjas@gmail.com" target="_blank">pranjas@gmail.com</a>></span> wrote:<br></span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Amit<div><div class="h5"><br>
<span><br>
On Thu, Jul 23, 2015 at 11:09 PM, <<a href="mailto:Valdis.Kletnieks@vt.edu" target="_blank">Valdis.Kletnieks@vt.edu</a>> wrote:<br>
> On Thu, 23 Jul 2015 12:31:18 +0530, Amit Gupta said:<br>
><br>
>> Configuration file: /etc/hostapd.conf<br>
>> [ 199.672712] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready<br>
>> Failed to update rate sets in kernel module<br>
>> [ 199.687566] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready<br>
>> Using interface wlan0 with hwaddr 00:0e:8e:38:29:e6 and ssid 'test_wifi_2'<br>
>><br>
>><br>
>> Eventhough I am getting this error log from user space application(hostapd)<br>
>> code, my wifi device is working fine. But I want to remove this error log<br>
>> and want to know the root cause this error log.<br>
><br>
> My gut reaction is that hostapd is issuing ioctl() calls in the wrong order<br>
> and/or failing to allow a long enough delay or wait for a specific event.<br>
><br>
> I'd recommend running something like:<br>
><br>
> # strace -f hostapd -B /etc/hostapd.conf > /tmp/strace.out 2>&1<br>
<br>
</span>The driver gets notified of this change via netlink socks. I also face<br>
the same issue and so far I found this<br>
<br>
Hostapd calls<br>
--->i802_set_rate_sets<br>
<br>
which then creates a nlmsg which is handled in driver as<br>
<br>
--driver--<br>
-->nl80211_set_bss<br>
--->rdev_change_bss<br>
<br>
now change_bss is defined as<br>
<br>
ieee80211_change_bss (net/mac80211/cfg.c) unless ofcourse your driver<br>
provides something else.<br>
<br>
There seems to be no error from here.<br>
<br>
So maybe it's the send_recv call in hostapd in function<br>
i802_set_rate_sets? I'll look into it more.<br>
<span><br>
><br>
> and go look for what failing call was made just before the write call<br>
> that output 'Failed to update'. Hopefully, tracing the flow of that<br>
> call will reveal a kernel code path that output the 'link is not ready'<br>
> message. Then you'll know what 'if (somecondition)' landed you in that<br>
> situation.<br>
><br>
> Knowing that, it will become a lot easier to figure out what happened.<br>
><br>
</span>> _______________________________________________<br>
> Kernelnewbies mailing list<br></div></div>
> <a href="mailto:Kernelnewbies@kernelnewbies.org" target="_blank">Kernelnewbies@kernelnewbies.org</a><br>
> <a href="http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies" rel="noreferrer" target="_blank">http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies</a><br>
><br>
<span><font color="#888888"><br>
<br><span class="HOEnZb"><font color="#888888">
<br>
--<br>
---P.K.S<br>
</font></span></font></span></blockquote></div><br></div>
</blockquote></div><br></div>