<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Aug 18, 2020 at 9:32 PM Muni Sekhar <<a href="mailto:munisekharrms@gmail.com">munisekharrms@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi all,<br>
<br>
I’ve two identical Linux systems with only kernel differences.<br>
While doing kernel profiling with perf, I got the below mentioned<br>
metrics for Scheduler benchmarks.<br>
<br>
1st system (older kernel version compared to the other system) benchmark result:<br>
<br>
$ perf bench sched messaging -g 64<br>
# Running 'sched/messaging' benchmark:<br>
# 20 sender and receiver processes per group<br>
# 64 groups == 2560 processes run<br>
<br>
Total time: 2.936 [sec]<br>
<br>
<br>
2nd system benchmark result:<br>
<br>
$ perf bench sched messaging -g 64<br>
# Running 'sched/messaging' benchmark:<br>
# 20 sender and receiver processes per group<br>
# 64 groups == 2560 processes run<br>
<br>
Total time: 10.074 [sec]<br>
<br>
<br>
So as per scheduler benchmark results, clearly a huge difference<br>
between two systems.<br>
Can anyone suggest to me how to dive deeper to know the root cause for<br>
it. Also are there any tunable kernel parameters related to this one?<br>
<br>
<br>
<br>
-- <br>
Thanks,<br>
Sekhar<br>
<br>
_______________________________________________<br>
Kernelnewbies mailing list<br>
<a href="mailto:Kernelnewbies@kernelnewbies.org" target="_blank">Kernelnewbies@kernelnewbies.org</a><br>
<a href="https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies" rel="noreferrer" target="_blank">https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies</a><br>
</blockquote></div><br clear="all"><div>Aside from very far different kernel version, I think you need to use perf record and visualized it with heat map too (check Brendan Gregg blog)</div><div><br></div><div>On the lower level side, check cache miss on L1/L2/L3 rate on both configuration. That might reveal something<br></div><div><br></div>-- <br><div dir="ltr" class="gmail_signature">regards,<br><br>Mulyadi Santosa<br>Freelance Linux trainer and consultant<br><br>blog: <a href="http://the-hydra.blogspot.com" target="_blank">the-hydra.blogspot.com</a><br>training: <a href="http://mulyaditraining.blogspot.com" target="_blank">mulyaditraining.blogspot.com</a></div></div><div id="DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2"><br>
<table style="border-top:1px solid #d3d4de">
<tr>
<td style="width:55px;padding-top:13px"><a href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail" target="_blank"><img src="https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repeat-v1.gif" alt="" width="46" height="29" style="width: 46px; height: 29px;"></a></td>
<td style="width:470px;padding-top:12px;color:#41424e;font-size:13px;font-family:Arial,Helvetica,sans-serif;line-height:18px">Virus-free. <a href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail" target="_blank" style="color:#4453ea">www.avast.com</a>
</td>
</tr>
</table><a href="#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2" width="1" height="1"></a></div>