Kernel schedules kernel tasks on isolated cpus, SCHED_FIFO prevents kernel tasks from running

[valdis.kletnieks at vt.edu]

On Wed, 28 Jun 2017 08:39:07 -0500, Andrei Hurynovich said:
> We set sysctl kernel.sched_rt_runtime_us = -1 so realtime threads are
> NEVER interrupted.

> According to /proc/sched_debug, it seems that kernel still schedules
> some SCHED_OTHER(e.g. non-realtime) kernel tasks to isolated cpus - for
> example cpu 18 get tasks events/18 and kblockd/18 that are stuck in
> runnable(but not running state), so those kernel processes never got a
> single time slice because our realtime process hogs 100% of cpu.

This is what happens when you have a priority inversion - when you tell the system
to give 100% to a process, you shouldn't be surprised when other tasks don't
get any service.

> The question is: Is it possible to never schedule kernel tasks on
> selected cpus?

Only if the userspace process on that CPU never makes system calls - which is
very unlikely if the process has actual real-time requirements.

Also, if your "real-time" process is taking 100% of the CPU, you have a disaster
waiting to happen.  You have zero headroom for dealing with unexpected events.
Thought experiment:  What happens if your real-world part of the system has
an unexpected error, that requires 1% of a CPU for error recovery?  You are
forced to either ignore the error or miss a real-time deadline.

You might want to think about dividing up your process into 2 parts - one that
handles the *actual* real-time work and only uses (for example) 20-30% of a
CPU, and the parts that don't have actual real-time constraints that can then
run with the rest of the available CPU, but allow other threads (such as kernel)
to execute as well.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 486 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20170628/6c639e8d/attachment.bin