CPU Load Balancer / Scheduler / CFS
I'm seeing some bizarre differences between two servers (one running 2.6.32 and the other 2.6.18) with how processes are spread across CPUs. I created a simple test client: https://gist.github.com/2306761 which forks 4 times to create processes which just consume cpu. On 2.6.18, the processes are distributed across CPUs as expected (all 4 processes are consuming 100%). But on 2.6.32, the distribution varies. Sometimes it's (100%, 33%, 33%, 33%), other times it's (100%, 100%, 50%, 50%), etc.. while all the other CPUs sit completely idle.. Both servers have many cores (more than 4). Both servers each have 2 physical processors, and are hyper-threaded. I've done many other similar tests (tried running the "stress" program with different parameters and I also got bizarre results. It basically looks like the CPU Load Balancer never moves a process to a different CPU under any circumstances). I haven't set anything regarding cpusets / etc. so it should all be default behavior. Also, if I manually move the processes to other cpus (using taskset) then I can get them to all run at 100%. What could be going on here?
Hi Robert... On Fri, Apr 6, 2012 at 00:24, Robert W <robertwalters83@gmail.com> wrote:
But on 2.6.32, the distribution varies. Sometimes it's (100%, 33%, 33%, 33%), other times it's (100%, 100%, 50%, 50%), etc.. while all the other CPUs sit completely idle..
Both servers have many cores (more than 4). Both servers each have 2 physical processors, and are hyper-threaded.
Ahem, that HT might be the cause. To elaborate more. Linux process scheduler tend to avoid the HT sibling, because they are not "real" processor. Or if not possible to avoid at all, then put at least load as possible. Also, Linux kernel tries to "stick" a process into certain core within certain time interval. This is done to make sure the L1/L2 cache is utilized as long as possible. Processor relocation will trigger cache invalidation and that costs a lot. CMIIW.... -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
participants (2)
-
Mulyadi Santosa -
Robert W