Kernel schedules kernel tasks on isolated cpus, SCHED_FIFO prevents kernel tasks from running

[Andrei Hurynovich]

Thank you Valdis.

Yes, I'm basically getting what I want - the RT proc never ever gives up 
to the system. There are a plenty of cores left to run non-rt tasks on 
the machine.


The question is why this old 2.6 kernel decide that it needs per-cpu 
events and kblockd tasks.

Maybe someone can give a hint in what subsystem's documentation can I 
find anything about workqueue tasks.

/Documentation/kernel-per-CPU-kthreads.txt is great, but described 
controls appeared only in 3.10.x :)


On 06/28/2017 01:04 PM, valdis.kletnieks at vt.edu wrote:
> On Wed, 28 Jun 2017 08:39:07 -0500, Andrei Hurynovich said:
>> We set sysctl kernel.sched_rt_runtime_us = -1 so realtime threads are
>> NEVER interrupted.
>> According to /proc/sched_debug, it seems that kernel still schedules
>> some SCHED_OTHER(e.g. non-realtime) kernel tasks to isolated cpus - for
>> example cpu 18 get tasks events/18 and kblockd/18 that are stuck in
>> runnable(but not running state), so those kernel processes never got a
>> single time slice because our realtime process hogs 100% of cpu.
> This is what happens when you have a priority inversion - when you tell the system
> to give 100% to a process, you shouldn't be surprised when other tasks don't
> get any service.
>
>> The question is: Is it possible to never schedule kernel tasks on
>> selected cpus?
> Only if the userspace process on that CPU never makes system calls - which is
> very unlikely if the process has actual real-time requirements.
>
> Also, if your "real-time" process is taking 100% of the CPU, you have a disaster
> waiting to happen.  You have zero headroom for dealing with unexpected events.
> Thought experiment:  What happens if your real-world part of the system has
> an unexpected error, that requires 1% of a CPU for error recovery?  You are
> forced to either ignore the error or miss a real-time deadline.
>
> You might want to think about dividing up your process into 2 parts - one that
> handles the *actual* real-time work and only uses (for example) 20-30% of a
> CPU, and the parts that don't have actual real-time constraints that can then
> run with the rest of the available CPU, but allow other threads (such as kernel)
> to execute as well.
>

-- 
Thanks,
Andrei Hurynovich

Charlesworth Research LLC.
http://www.charlesworthresearch.com/