Linux kernel cgroup v2 CFS - CPU throttled_usec accounting?
Hi community, My question: In Linux kernel cgroup v2’s CFS scheduler, how is cpu.max throttled_usec accounted when a cgroup with multiple threads gets throttled during a single quota period? Specifically, is throttled_usec tracked as the total wall-clock time that the cgroup was throttled as a whole, or is it a sum of the throttled times of all individual threads? Kernel Version: "5.14.0-284.11.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 9 11:41:53 PDT 2023 x86_64 x86_64 x86_64 GNU/Linux" Distro: Oracle Linux 9.x Regards, AlexL
On Sat, 11 Jan 2025 at 13:29, Alexandru Lazarev <alexandru.lazarev@gmail.com> wrote:
In Linux kernel cgroup v2’s CFS scheduler, how is cpu.max throttled_usec accounted when a cgroup with multiple threads gets throttled during a single quota period? Specifically, is throttled_usec tracked as the total wall-clock time that the cgroup was throttled as a whole, or is it a sum of the throttled times of all individual threads?
Kernel Version: "5.14.0-284.11.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 9 11:41:53 PDT 2023 x86_64 x86_64 x86_64 GNU/Linux" Distro: Oracle Linux 9.x
Hi When the cgroup is throttled, all tasks (threads) within the cgroup are simultaneously prevented from running. The `throttled_time` counter starts incrementing when the cgroup as a whole enters the throttled state and stops when the cgroup is no longer throttled. Quote: "throttled_time: The total time duration (in nanoseconds) for which entities of the group have been throttled." https://docs.kernel.org/scheduler/sched-bwc.html#:~:text=throttled_time The source: void unthrottle_cfs_rq(struct cfs_rq *cfs_rq) { ... cfs_b->throttled_time += rq_clock(rq) - cfs_rq->throttled_clock; ... static int cpu_extra_stat_show(struct seq_file *sf, struct cgroup_subsys_state *css) { struct task_group *tg = css_tg(css); struct cfs_bandwidth *cfs_b = &tg->cfs_bandwidth; ... throttled_usec = cfs_b->throttled_time; Costa Thanks
Hi Costa, Thx U very much for your response. On Sat, Jan 11, 2025 at 7:03 PM Costa Shulyupin <costa.shul@redhat.com> wrote:
<alexandru.lazarev@gmail.com> wrote:
In Linux kernel cgroup v2’s CFS scheduler, how is cpu.max throttled_usec accounted when a cgroup with multiple threads gets throttled during a single quota period? Specifically, is throttled_usec tracked as the total wall-clock time that the cgroup was throttled as a whole, or is it a sum of the throttled times of all individual threads?
Kernel Version: "5.14.0-284.11.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 9 11:41:53 PDT 2023 x86_64 x86_64 x86_64 GNU/Linux" Distro: Oracle Linux 9.x
Hi
When the cgroup is throttled, all tasks (threads) within the cgroup are simultaneously prevented from running. The `throttled_time` counter starts incrementing when the cgroup as a whole enters the throttled state and stops when the cgroup is no longer throttled.
Quote:
"throttled_time: The total time duration (in nanoseconds) for which entities of the group have been throttled."
https://docs.kernel.org/scheduler/sched-bwc.html#:~:text=throttled_time
Wanted experts' confirmations :), because IMHO I got confused by this "The total time duration (in nanoseconds) for which entities". Someone (those 'smart' AI chats) interpreted them as cumulative time of all throttled threads/processes within a cgroup, and also due to missing description of "throttled_usec" in cgroup v2 in https://docs.kernel.org/admin-guide/cgroup-v2.html#cgroup-v2-cpu:~:text=nr_t...
The source:
void unthrottle_cfs_rq(struct cfs_rq *cfs_rq) { ... cfs_b->throttled_time += rq_clock(rq) - cfs_rq->throttled_clock; ...
static int cpu_extra_stat_show(struct seq_file *sf, struct cgroup_subsys_state *css) { struct task_group *tg = css_tg(css); struct cfs_bandwidth *cfs_b = &tg->cfs_bandwidth; ... throttled_usec = cfs_b->throttled_time;
Costa Thanks
Regards, AlexL
participants (2)
-
Alexandru Lazarev -
Costa Shulyupin