da9063 - Reboot issue : koops rcu_note_context_switch
Hello, I got a kernel warning on the reboot of my embedded board, and sometimes I got stuck in the reboot. I work on the IMX6Q with a PMIC da9063 and the last kernel release. It happens in the da9063_wdt_restart function at the regmap_write call in the CONTROL_F register. (drivers/watchdog/da9063_wdt.c) If I access to other register in that function, I don't have this issue. I don't see the link between the rcu as noted in the koops and the write in a specific register of the pmic. Could you help me to go further in the debug? [ 32.850293] ------------[ cut here ]------------ [ 32.854929] WARNING: CPU: 0 PID: 724 at kernel/rcu/tree_plugin.h:337 rcu_note_context_switch+0xac/0x554 [ 32.864322] Modules linked in: [ 32.867386] CPU: 0 PID: 724 Comm: reboot Not tainted 5.0.4-yocto-standard+ #49 [ 32.874609] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree) [ 32.881155] [<80111164>] (unwind_backtrace) from [<8010c81c>] (show_stack+0x10/0x14) [ 32.888911] [<8010c81c>] (show_stack) from [<808bde64>] (dump_stack+0x88/0x9c) [ 32.896147] [<808bde64>] (dump_stack) from [<8012077c>] (__warn+0xdc/0xf4) [ 32.903031] [<8012077c>] (__warn) from [<801208ac>] (warn_slowpath_null+0x40/0x48) [ 32.910607] [<801208ac>] (warn_slowpath_null) from [<8017596c>] (rcu_note_context_switch+0xac/0x554) [ 32.919751] [<8017596c>] (rcu_note_context_switch) from [<808d78f8>] (__schedule+0x70/0x584) [ 32.928196] [<808d78f8>] (__schedule) from [<808d7e60>] (schedule+0x54/0xb0) [ 32.935256] [<808d7e60>] (schedule) from [<808dbc5c>] (schedule_hrtimeout_range_clock+0xd8/0x15c) [ 32.944137] [<808dbc5c>] (schedule_hrtimeout_range_clock) from [<808dbcf8>] (schedule_hrtimeout_range+0x18/0x20) [ 32.954319] [<808dbcf8>] (schedule_hrtimeout_range) from [<808db630>] (usleep_range+0x6c/0x90) [ 32.962941] [<808db630>] (usleep_range) from [<80681ac4>] (i2c_imx_start+0xa0/0x13c) [ 32.970692] [<80681ac4>] (i2c_imx_start) from [<80682874>] (i2c_imx_xfer+0x68/0xe94) [ 32.978447] [<80682874>] (i2c_imx_xfer) from [<8067c4c4>] (__i2c_transfer+0x168/0x2cc) [ 32.986372] [<8067c4c4>] (__i2c_transfer) from [<8067c68c>] (i2c_transfer+0x64/0xe4) [ 32.994128] [<8067c68c>] (i2c_transfer) from [<804dee9c>] (regmap_i2c_read+0x58/0x88) [ 33.001968] [<804dee9c>] (regmap_i2c_read) from [<804daee4>] (_regmap_raw_read+0xac/0x11c) [ 33.010239] [<804daee4>] (_regmap_raw_read) from [<804daf8c>] (_regmap_bus_read+0x38/0x60) [ 33.018511] [<804daf8c>] (_regmap_bus_read) from [<804da0a8>] (_regmap_read+0x60/0xb8) [ 33.026436] [<804da0a8>] (_regmap_read) from [<804da508>] (_regmap_update_bits+0xb0/0xec) [ 33.034620] [<804da508>] (_regmap_update_bits) from [<804da5e0>] (_regmap_select_page+0x9c/0xf8) [ 33.043412] [<804da5e0>] (_regmap_select_page) from [<804dab18>] (_regmap_raw_write_impl+0x4dc/0x69c) [ 33.052637] [<804dab18>] (_regmap_raw_write_impl) from [<804db5ec>] (regmap_write+0x3c/0x5c) [ 33.061087] [<804db5ec>] (regmap_write) from [<806be9ec>] (da9063_wdt_restart+0x54/0xa0) [ 33.069185] [<806be9ec>] (da9063_wdt_restart) from [<806bd288>] (watchdog_restart_notifier+0x18/0x2c) [ 33.078416] [<806bd288>] (watchdog_restart_notifier) from [<8013f148>] (notifier_call_chain+0x48/0x84) [ 33.087731] [<8013f148>] (notifier_call_chain) from [<8013f3d0>] (atomic_notifier_call_chain+0x34/0x48) [ 33.097138] [<8013f3d0>] (atomic_notifier_call_chain) from [<8010b12c>] (machine_restart+0x78/0x7c) [ 33.106194] [<8010b12c>] (machine_restart) from [<80140aac>] (sys_reboot+0xd8/0x1d0) [ 33.113945] [<80140aac>] (sys_reboot) from [<80101000>] (ret_fast_syscall+0x0/0x54) [ 33.121604] Exception stack(0xed1e1fa8 to 0xed1e1ff0) [ 33.126663] 1fa0: 00012230 7e895db4 fee1dead 28121969 01234567 00000000 [ 33.134847] 1fc0: 00012230 7e895db4 00000003 00000058 000225f8 00000000 00000000 00000000 [ 33.143027] 1fe0: 76f26ea0 7e895c14 00010fd8 76f26ec0 [ 33.148080] ---[ end trace 1e06dce5f82e2a7a ]--- -- Regards Kory
Hi Kory Maincent. On Tue, 26 Mar 2019 at 13:21, Köry Maincent <kory.maincent@navocap.com> wrote:
Hello,
I got a kernel warning on the reboot of my embedded board, and sometimes I got stuck in the reboot. I work on the IMX6Q with a PMIC da9063 and the last kernel release.
It happens in the *da9063_wdt_restart* function at the *regmap_write* call in the CONTROL_F register. (drivers/watchdog/da9063_wdt.c)
From trace we are able figure out the flow,
reboot | | invokes system call(sys_reboot) | | watchdog_restart)notifier | | da9063_wdt_restart | | It sending I2c rear/write commands. Is this expected behaviour ?? If I access to other register in that function, I don't have this issue. The reason could be ,you are accessing other registers directly not through I2C.
I don't see the link between the rcu as noted in the koops and the write in a specific register of the pmic.
Could you help me to go further in the debug?
[ 32.850293] ------------[ cut here ]------------
[ 32.854929] WARNING: CPU: 0 PID: 724 at kernel/rcu/tree_plugin.h:337 rcu_note_context_switch+0xac/0x554
[ 32.864322] Modules linked in:
[ 32.867386] CPU: 0 PID: 724 Comm: reboot Not tainted 5.0.4-yocto-standard+ #49
[ 32.874609] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
[ 32.881155] [<80111164>] (unwind_backtrace) from [<8010c81c>] (show_stack+0x10/0x14)
[ 32.888911] [<8010c81c>] (show_stack) from [<808bde64>] (dump_stack+0x88/0x9c)
[ 32.896147] [<808bde64>] (dump_stack) from [<8012077c>] (__warn+0xdc/0xf4)
[ 32.903031] [<8012077c>] (__warn) from [<801208ac>] (warn_slowpath_null+0x40/0x48)
[ 32.910607] [<801208ac>] (warn_slowpath_null) from [<8017596c>] (rcu_note_context_switch+0xac/0x554)
[ 32.919751] [<8017596c>] (rcu_note_context_switch) from [<808d78f8>] (__schedule+0x70/0x584)
[ 32.928196] [<808d78f8>] (__schedule) from [<808d7e60>] (schedule+0x54/0xb0)
[ 32.935256] [<808d7e60>] (schedule) from [<808dbc5c>] (schedule_hrtimeout_range_clock+0xd8/0x15c)
[ 32.944137] [<808dbc5c>] (schedule_hrtimeout_range_clock) from [<808dbcf8>] (schedule_hrtimeout_range+0x18/0x20)
[ 32.954319] [<808dbcf8>] (schedule_hrtimeout_range) from [<808db630>] (usleep_range+0x6c/0x90)
[ 32.962941] [<808db630>] (usleep_range) from [<80681ac4>] (i2c_imx_start+0xa0/0x13c)
[ 32.970692] [<80681ac4>] (i2c_imx_start) from [<80682874>] (i2c_imx_xfer+0x68/0xe94)
[ 32.978447] [<80682874>] (i2c_imx_xfer) from [<8067c4c4>] (__i2c_transfer+0x168/0x2cc)
[ 32.986372] [<8067c4c4>] (__i2c_transfer) from [<8067c68c>] (i2c_transfer+0x64/0xe4)
[ 32.994128] [<8067c68c>] (i2c_transfer) from [<804dee9c>] (regmap_i2c_read+0x58/0x88)
[ 33.001968] [<804dee9c>] (regmap_i2c_read) from [<804daee4>] (_regmap_raw_read+0xac/0x11c)
[ 33.010239] [<804daee4>] (_regmap_raw_read) from [<804daf8c>] (_regmap_bus_read+0x38/0x60)
[ 33.018511] [<804daf8c>] (_regmap_bus_read) from [<804da0a8>] (_regmap_read+0x60/0xb8)
[ 33.026436] [<804da0a8>] (_regmap_read) from [<804da508>] (_regmap_update_bits+0xb0/0xec)
[ 33.034620] [<804da508>] (_regmap_update_bits) from [<804da5e0>] (_regmap_select_page+0x9c/0xf8)
[ 33.043412] [<804da5e0>] (_regmap_select_page) from [<804dab18>] (_regmap_raw_write_impl+0x4dc/0x69c)
[ 33.052637] [<804dab18>] (_regmap_raw_write_impl) from [<804db5ec>] (regmap_write+0x3c/0x5c)
[ 33.061087] [<804db5ec>] (regmap_write) from [<806be9ec>] (da9063_wdt_restart+0x54/0xa0)
[ 33.069185] [<806be9ec>] (da9063_wdt_restart) from [<806bd288>] (watchdog_restart_notifier+0x18/0x2c)
[ 33.078416] [<806bd288>] (watchdog_restart_notifier) from [<8013f148>] (notifier_call_chain+0x48/0x84)
[ 33.087731] [<8013f148>] (notifier_call_chain) from [<8013f3d0>] (atomic_notifier_call_chain+0x34/0x48)
[ 33.097138] [<8013f3d0>] (atomic_notifier_call_chain) from [<8010b12c>] (machine_restart+0x78/0x7c)
[ 33.106194] [<8010b12c>] (machine_restart) from [<80140aac>] (sys_reboot+0xd8/0x1d0)
[ 33.113945] [<80140aac>] (sys_reboot) from [<80101000>] (ret_fast_syscall+0x0/0x54)
[ 33.121604] Exception stack(0xed1e1fa8 to 0xed1e1ff0)
[ 33.126663] 1fa0: 00012230 7e895db4 fee1dead 28121969 01234567 00000000
[ 33.134847] 1fc0: 00012230 7e895db4 00000003 00000058 000225f8 00000000 00000000 00000000
[ 33.143027] 1fe0: 76f26ea0 7e895c14 00010fd8 76f26ec0
[ 33.148080] ---[ end trace 1e06dce5f82e2a7a ]---
-- Regards Kory _______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Thanks, Hariprasad k
Hi Hariprasad, On 28/03/2019 19:24, hariprasad kelam wrote: Hi Kory Maincent. On Tue, 26 Mar 2019 at 13:21, Köry Maincent <kory.maincent@navocap.com<mailto:kory.maincent@navocap.com>> wrote: Hello, I got a kernel warning on the reboot of my embedded board, and sometimes I got stuck in the reboot. I work on the IMX6Q with a PMIC da9063 and the last kernel release. It happens in the da9063_wdt_restart function at the regmap_write call in the CONTROL_F register. (drivers/watchdog/da9063_wdt.c) From trace we are able figure out the flow, reboot | | invokes system call(sys_reboot) | | watchdog_restart)notifier | | da9063_wdt_restart | | It sending I2c rear/write commands. Is this expected behaviour ?? Yes it is. If I access to other register in that function, I don't have this issue. The reason could be ,you are accessing other registers directly not through I2C. I was wrong on this point, because of bad order debug printing in the drivers. I manage to avoid the issue with a hotfix : adding rcu_read_unlock(); before I2C writing in the da9063_wdt_restart function. There is no more koops warning neither freezing reboot. I don't think it is a clean patch and I don't understand the relation to preemption. It may happens because of an RCU critical section not closed? Regards, Kory
More readable with quote character sorry for the duplicate. Hi Hariprasad,
Hi Kory Maincent.
On Tue, 26 Mar 2019 at 13:21, Köry Maincent <kory.maincent@navocap.com <mailto:kory.maincent@navocap.com>> wrote:
Hello,
I got a kernel warning on the reboot of my embedded board, and sometimes I got stuck in the reboot. I work on the IMX6Q with a PMIC da9063 and the last kernel release.
It happens in the /da9063_wdt_restart/ function at the /regmap_write/ call in the CONTROL_F register. (drivers/watchdog/da9063_wdt.c)
From trace we are able figure out the flow,
reboot | | invokes system call(sys_reboot) | | watchdog_restart)notifier | | da9063_wdt_restart | | It sending I2c rear/write commands.
Is this expected behaviour ??
Yes it is.
If I access to other register in that function, I don't have this issue. The reason could be ,you are accessing other registers directly not through I2C. I was wrong on this point, because of bad order debug printing in the drivers.
I manage to avoid the issue with a hotfix : adding rcu_read_unlock(); before I2C writing in the da9063_wdt_restart function. There is no more koops warning neither freezing reboot. I don't think it is a clean patch and I don't understand the relation to preemption. It may happens because of an RCU critical section not closed? Regards Kory.
participants (2)
-
hariprasad kelam -
Köry Maincent