"inconsistent lock state"

Jan. 27, 2011

      Hello,

I'm using spinlocks (spin_lock_irqsave() / spin_unlock_irqrestore()) to 
protect shared resources between softirq, hardirq and non-irq kernel 
contexts. When I configure PROVE_LOCKING I get

  =================================
[ INFO: inconsistent lock state ]
2.6.27.54 #148
---------------------------------
inconsistent {softirq-on-W} -> {in-softirq-W} usage.
khubd/85 [HC0[0]:SC1[1]:HE0:SE0] takes:
  (&priv->lock){-+..}, at: [<90152430>] schedule_ptds+0x54/0x564
{softirq-on-W} state was registered at:
   [<9003856e>] trace_hardirqs_on_caller+0x92/0xc4
   [<900385a8>] trace_hardirqs_on+0x8/0xc
   [<90058fc2>] __slab_alloc+0xee/0x3e4
   [<90059f9c>] kmem_cache_alloc+0x30/0x70

etc...

I don't understand why. Assuming that "{softirq-on-W} -> {in-softirq-W}" 
means that the problem is that the lock was taken in "softirq on" (irqs 
enabled) state, and at some later time in softirq context, then what is 
the problem? spin_lock_irqsave() disables irqs, right? So after the lock 
is taken in non-irq context, softirq cannot interrupt?

There are code paths like this:

{
	spin_lock_irqsave(...);
	...
	spin_unlock()
	...
	spin_lock()
	...
	spin_unlock_irqrestore(...);
}

in both non-irq and softirq contexts. Could this be a problem? If I 
understand correctly, spin_lock() and spin_unlock() does not touch irq 
settings, so the usage above should be fine, right?

Thanks,
Arvid Brodin
Enea Services Stockholm AB