interpret the result of perf stat
Hello, I start to use perf to profile my program, but I'm not quite sure if I understand some events. First, perf can profile bus cycles. Is it the bus cycles of FSB? Second, what is task clock? It seems to me that task clock is the ticks used by a process. It should include the ticks used in the kernel mode and the user space mode, as long as it is used in the context of the process. I suppose it doesn't count the cycles used by the interrupts such as page faults and softirq, even though the irqs are caused by the process. Is it correct? Thanks, Da
On Sun, Oct 2, 2011 at 5:07 AM, Zheng Da <zhengda1936@gmail.com> wrote:
Hello,
I start to use perf to profile my program, but I'm not quite sure if I understand some events.
Firstly perf is using hardware to read these performance information. So for intel CPU, depending on the different model, u can find the relevant information in the reference manual from Intel here: http://www.intel.com/Assets/en_US/PDF/manual/248966.pdf In particular, the various model of Intel CPU is listed in page xvii, and along with it are the various performance information u can extract out - the last item is the BUS information (B.6.10).
First, perf can profile bus cycles. Is it the bus cycles of FSB? Second, what is task clock? It seems to me that task clock is the ticks used by a process. It should include the ticks used in the kernel mode and the user space mode, as long as it is used in the context of the process. I suppose it doesn't count the cycles used by the interrupts such as page faults and softirq, even though the irqs are caused by the process. Is it correct?
Thanks, Da
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Regards, Peter Teoh
On Tue, Oct 4, 2011 at 9:53 AM, Peter Teoh <htmldeveloper@gmail.com> wrote:
On Sun, Oct 2, 2011 at 5:07 AM, Zheng Da <zhengda1936@gmail.com> wrote:
Hello,
I start to use perf to profile my program, but I'm not quite sure if I understand some events.
Firstly perf is using hardware to read these performance information. So for intel CPU, depending on the different model, u can find the relevant information in the reference manual from Intel here:
http://www.intel.com/Assets/en_US/PDF/manual/248966.pdf
In particular, the various model of Intel CPU is listed in page xvii, and along with it are the various performance information u can extract out - the last item is the BUS information (B.6.10).
First, perf can profile bus cycles. Is it the bus cycles of FSB? Second, what is task clock? It seems to me that task clock is the
task clock, from reading the implementation in kernel/events/core.c, i suspect is a software event, not hardware. Ie, it counts upon each task switch. To quote from core.c: /* * Software event: task time clock */ static void task_clock_event_update(struct perf_event *event, u64 now) { More details here: http://lwn.net/Articles/312720/
ticks used by a process. It should include the ticks used in the kernel mode and the user space mode, as long as it is used in the context of the process. I suppose it doesn't count the cycles used by the interrupts such as page faults and softirq, even though the irqs are caused by the process. Is it correct?
on the contrary, i think it should include, as to exclude these cycles does entail stopping the hrtimer.....read into core.c further.
Thanks, Da
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Regards, Peter Teoh
-- Regards, Peter Teoh
Thank you, Peter. I didn't realize Intel has explained all performance counters in its manual, and I was desperately looking for more detailed explanation on the Internet. On Mon, Oct 3, 2011 at 11:19 PM, Peter Teoh <htmldeveloper@gmail.com> wrote:
On Tue, Oct 4, 2011 at 9:53 AM, Peter Teoh <htmldeveloper@gmail.com> wrote:
On Sun, Oct 2, 2011 at 5:07 AM, Zheng Da <zhengda1936@gmail.com> wrote:
Hello,
I start to use perf to profile my program, but I'm not quite sure if I understand some events.
Firstly perf is using hardware to read these performance information. So for intel CPU, depending on the different model, u can find the relevant information in the reference manual from Intel here:
http://www.intel.com/Assets/en_US/PDF/manual/248966.pdf
In particular, the various model of Intel CPU is listed in page xvii, and along with it are the various performance information u can extract out - the last item is the BUS information (B.6.10).
First, perf can profile bus cycles. Is it the bus cycles of FSB? Second, what is task clock? It seems to me that task clock is the
task clock, from reading the implementation in kernel/events/core.c, i suspect is a software event, not hardware. Ie, it counts upon each task switch. To quote from core.c: /* * Software event: task time clock */ static void task_clock_event_update(struct perf_event *event, u64 now) {
More details here: http://lwn.net/Articles/312720/
ticks used by a process. It should include the ticks used in the kernel mode and the user space mode, as long as it is used in the context of the process. I suppose it doesn't count the cycles used by the interrupts such as page faults and softirq, even though the irqs are caused by the process. Is it correct?
on the contrary, i think it should include, as to exclude these cycles does entail stopping the hrtimer.....read into core.c further.
I'm disappointed that the information isn't documented somewhere. The developers shouldn't expect users to read the code to understand how to use the tool :( BTW, is there any wiki where people can share what they found in the kernel? Thanks, Da
On Thu, Oct 6, 2011 at 10:39 PM, Zheng Da <zhengda1936@gmail.com> wrote:
Thank you, Peter. I didn't realize Intel has explained all performance counters in its manual, and I was desperately looking for more detailed explanation on the Internet.
On Mon, Oct 3, 2011 at 11:19 PM, Peter Teoh <htmldeveloper@gmail.com> wrote:
On Tue, Oct 4, 2011 at 9:53 AM, Peter Teoh <htmldeveloper@gmail.com>
On Sun, Oct 2, 2011 at 5:07 AM, Zheng Da <zhengda1936@gmail.com> wrote:
Hello,
I start to use perf to profile my program, but I'm not quite sure if I understand some events.
Firstly perf is using hardware to read these performance information.
So
for intel CPU, depending on the different model, u can find the relevant information in the reference manual from Intel here:
http://www.intel.com/Assets/en_US/PDF/manual/248966.pdf
In particular, the various model of Intel CPU is listed in page xvii, and along with it are the various performance information u can extract out
wrote: -
the last item is the BUS information (B.6.10).
First, perf can profile bus cycles. Is it the bus cycles of FSB? Second, what is task clock? It seems to me that task clock is the
task clock, from reading the implementation in kernel/events/core.c, i suspect is a software event, not hardware. Ie, it counts upon each task switch. To quote from core.c: /* * Software event: task time clock */ static void task_clock_event_update(struct perf_event *event, u64 now) {
More details here: http://lwn.net/Articles/312720/
ticks used by a process. It should include the ticks used in the kernel mode and the user space mode, as long as it is used in the context of the process. I suppose it doesn't count the cycles used by the interrupts such as page faults and softirq, even though the irqs are caused by the process. Is it correct?
on the contrary, i think it should include, as to exclude these cycles does entail stopping the hrtimer.....read into core.c further. I'm disappointed that the information isn't documented somewhere. The developers shouldn't expect users to read the code to understand how to use the tool :(
i suspect it is documented in many places, at here is one good one: http://developer.apple.com/library/IOS/#documentation/DeveloperTools/Concept... and from the side, looking at Appendix A, u can see a table of all the PerfCounters for ARM11, and then another section will list out for Intel, and then PPC etc.....
BTW, is there any wiki where people can share what they found in the kernel?
Thanks, Da
-- Regards, Peter Teoh
Hey Peter, If you are talking about the documentation of performance counters, we can find very complete reference http://software.intel.com/sites/products/documentation/hpc/amplifierxe/en-us... Thanks, Da On Sat, Oct 8, 2011 at 11:18 AM, Peter Teoh <htmldeveloper@gmail.com> wrote:
On Thu, Oct 6, 2011 at 10:39 PM, Zheng Da <zhengda1936@gmail.com> wrote:
Thank you, Peter. I didn't realize Intel has explained all performance counters in its manual, and I was desperately looking for more detailed explanation on the Internet.
On Mon, Oct 3, 2011 at 11:19 PM, Peter Teoh <htmldeveloper@gmail.com> wrote:
On Tue, Oct 4, 2011 at 9:53 AM, Peter Teoh <htmldeveloper@gmail.com> wrote:
On Sun, Oct 2, 2011 at 5:07 AM, Zheng Da <zhengda1936@gmail.com> wrote:
Hello,
I start to use perf to profile my program, but I'm not quite sure if I understand some events.
Firstly perf is using hardware to read these performance information. So for intel CPU, depending on the different model, u can find the relevant information in the reference manual from Intel here:
http://www.intel.com/Assets/en_US/PDF/manual/248966.pdf
In particular, the various model of Intel CPU is listed in page xvii, and along with it are the various performance information u can extract out - the last item is the BUS information (B.6.10).
First, perf can profile bus cycles. Is it the bus cycles of FSB? Second, what is task clock? It seems to me that task clock is the
task clock, from reading the implementation in kernel/events/core.c, i suspect is a software event, not hardware. Ie, it counts upon each task switch. To quote from core.c: /* * Software event: task time clock */ static void task_clock_event_update(struct perf_event *event, u64 now) {
More details here: http://lwn.net/Articles/312720/
ticks used by a process. It should include the ticks used in the kernel mode and the user space mode, as long as it is used in the context of the process. I suppose it doesn't count the cycles used by the interrupts such as page faults and softirq, even though the irqs are caused by the process. Is it correct?
on the contrary, i think it should include, as to exclude these cycles does entail stopping the hrtimer.....read into core.c further.
I'm disappointed that the information isn't documented somewhere. The developers shouldn't expect users to read the code to understand how to use the tool :(
i suspect it is documented in many places, at here is one good one: http://developer.apple.com/library/IOS/#documentation/DeveloperTools/Concept... and from the side, looking at Appendix A, u can see a table of all the PerfCounters for ARM11, and then another section will list out for Intel, and then PPC etc.....
BTW, is there any wiki where people can share what they found in the kernel?
Thanks, Da
-- Regards, Peter Teoh
This is from LinuxKongress 2010: The New Linux 'perf' Tools - Vger http://vger.kernel.org/~acme/perf/lk2010-perf-paper.pdf http://www.linux-kongress.org/2010/slides/lk2010-perf-acme.pdf Enjoy.... On Sun, Oct 9, 2011 at 2:42 AM, Zheng Da <zhengda1936@gmail.com> wrote:
Hey Peter,
If you are talking about the documentation of performance counters, we can find very complete reference http://software.intel.com/sites/products/documentation/hpc/amplifierxe/en-us...
Thanks, Da
On Sat, Oct 8, 2011 at 11:18 AM, Peter Teoh <htmldeveloper@gmail.com> wrote:
On Thu, Oct 6, 2011 at 10:39 PM, Zheng Da <zhengda1936@gmail.com> wrote:
Thank you, Peter. I didn't realize Intel has explained all performance counters in its manual, and I was desperately looking for more detailed explanation on the Internet.
On Mon, Oct 3, 2011 at 11:19 PM, Peter Teoh <htmldeveloper@gmail.com> wrote:
On Tue, Oct 4, 2011 at 9:53 AM, Peter Teoh <htmldeveloper@gmail.com> wrote:
On Sun, Oct 2, 2011 at 5:07 AM, Zheng Da <zhengda1936@gmail.com> wrote:
Hello,
I start to use perf to profile my program, but I'm not quite sure if I understand some events.
Firstly perf is using hardware to read these performance information. So for intel CPU, depending on the different model, u can find the relevant information in the reference manual from Intel here:
http://www.intel.com/Assets/en_US/PDF/manual/248966.pdf
In particular, the various model of Intel CPU is listed in page xvii, and along with it are the various performance information u can extract out - the last item is the BUS information (B.6.10).
First, perf can profile bus cycles. Is it the bus cycles of FSB? Second, what is task clock? It seems to me that task clock is the
task clock, from reading the implementation in kernel/events/core.c, i suspect is a software event, not hardware. Ie, it counts upon each task switch. To quote from core.c: /* * Software event: task time clock */ static void task_clock_event_update(struct perf_event *event, u64 now) {
More details here: http://lwn.net/Articles/312720/
ticks used by a process. It should include the ticks used in the kernel mode and the user space mode, as long as it is used in the context of the process. I suppose it doesn't count the cycles used by the interrupts such as page faults and softirq, even though the irqs are caused by the process. Is it correct?
on the contrary, i think it should include, as to exclude these cycles does entail stopping the hrtimer.....read into core.c further.
I'm disappointed that the information isn't documented somewhere. The developers shouldn't expect users to read the code to understand how to use the tool :(
i suspect it is documented in many places, at here is one good one: http://developer.apple.com/library/IOS/#documentation/DeveloperTools/Concept... and from the side, looking at Appendix A, u can see a table of all the PerfCounters for ARM11, and then another section will list out for Intel, and then PPC etc.....
BTW, is there any wiki where people can share what they found in the kernel?
Thanks, Da
-- Regards, Peter Teoh
-- Regards, Peter Teoh
The link below gives a very good user guide of perf. http://code.google.com/p/kernel/wiki/PerfUserGuide#Sampling_with_perf_record Da On Sun, Oct 9, 2011 at 10:11 PM, Peter Teoh <htmldeveloper@gmail.com> wrote:
This is from LinuxKongress 2010:
The New Linux 'perf' Tools - Vger
http://vger.kernel.org/~acme/perf/lk2010-perf-paper.pdf
http://www.linux-kongress.org/2010/slides/lk2010-perf-acme.pdf
Enjoy....
On Sun, Oct 9, 2011 at 2:42 AM, Zheng Da <zhengda1936@gmail.com> wrote:
Hey Peter,
If you are talking about the documentation of performance counters, we can find very complete reference http://software.intel.com/sites/products/documentation/hpc/amplifierxe/en-us...
Thanks, Da
On Sat, Oct 8, 2011 at 11:18 AM, Peter Teoh <htmldeveloper@gmail.com> wrote:
On Thu, Oct 6, 2011 at 10:39 PM, Zheng Da <zhengda1936@gmail.com> wrote:
Thank you, Peter. I didn't realize Intel has explained all performance counters in its manual, and I was desperately looking for more detailed explanation on the Internet.
On Mon, Oct 3, 2011 at 11:19 PM, Peter Teoh <htmldeveloper@gmail.com> wrote:
On Tue, Oct 4, 2011 at 9:53 AM, Peter Teoh <htmldeveloper@gmail.com> wrote:
On Sun, Oct 2, 2011 at 5:07 AM, Zheng Da <zhengda1936@gmail.com> wrote:
Hello,
I start to use perf to profile my program, but I'm not quite sure if I understand some events.
Firstly perf is using hardware to read these performance information. So for intel CPU, depending on the different model, u can find the relevant information in the reference manual from Intel here:
http://www.intel.com/Assets/en_US/PDF/manual/248966.pdf
In particular, the various model of Intel CPU is listed in page xvii, and along with it are the various performance information u can extract out - the last item is the BUS information (B.6.10).
First, perf can profile bus cycles. Is it the bus cycles of FSB? Second, what is task clock? It seems to me that task clock is the
task clock, from reading the implementation in kernel/events/core.c, i suspect is a software event, not hardware. Ie, it counts upon each task switch. To quote from core.c: /* * Software event: task time clock */ static void task_clock_event_update(struct perf_event *event, u64 now) {
More details here: http://lwn.net/Articles/312720/
ticks used by a process. It should include the ticks used in the kernel mode and the user space mode, as long as it is used in the context of the process. I suppose it doesn't count the cycles used by the interrupts such as page faults and softirq, even though the irqs are caused by the process. Is it correct?
on the contrary, i think it should include, as to exclude these cycles does entail stopping the hrtimer.....read into core.c further.
I'm disappointed that the information isn't documented somewhere. The developers shouldn't expect users to read the code to understand how to use the tool :(
i suspect it is documented in many places, at here is one good one: http://developer.apple.com/library/IOS/#documentation/DeveloperTools/Concept... and from the side, looking at Appendix A, u can see a table of all the PerfCounters for ARM11, and then another section will list out for Intel, and then PPC etc.....
BTW, is there any wiki where people can share what they found in the kernel?
Thanks, Da
-- Regards, Peter Teoh
-- Regards, Peter Teoh
Hi, I have a SH4 board: $ uname -a Linux LINUX7109 2.6.23.17_stm23_A18B-HMP_7109-STSDK #1 PREEMPT Fri Aug 6 16:08:19 ART 2010 sh4 unknown and suppose I have eaten pretty much all the memory, and have only 9 MB left (aprox.) $ free total used free shared buffers cached Mem: 48072 42276 5796 0 172 3264 -/+ buffers/cache: 38840 9232 Swap: 0 0 0 Now, when I try to launch a single thread with default stack size (8 MB) the pthread_create() call fails with ENOMEM. If I strace my test code, I can see that the function that is failing is mmap: old_mmap(NULL, 8388608, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) However, when I set the stack size to a lower value using ulimit -s: ulimit -s 7500 I can now launch 10 threads. Each thread does not allocate anything, so it is only consuming the minimum overhead (aprox. 8 kb per thread, right?). So, my question is: Knowing that mmap doesnt actually consume the memory (due to overcommit=0, right?): Why is pthread_create (or mmap) failing when memory available is above the thread stack size ? Where in linux source is the decision made to permit or not the mmap call ? Thanks! Ezequiel.
Hi Ezequiel :) 2011/10/11 Ezequiel García <elezegarcia@yahoo.com.ar>:
Hi,
I have a SH4 board:
Alright, I use x86 for the basis of my (guessing) answer, so things might vary... :)
and suppose I have eaten pretty much all the memory, and have only 9 MB left (aprox.)
$ free total used free shared buffers cached Mem: 48072 42276 5796 0 172 3264 -/+ buffers/cache: 38840 9232 Swap: 0 0 0
so, excluding page cache aside, you had 9232 kilobytes = 9 MB free RAM.
ulimit -s 7500 I can now launch 10 threads. Each thread does not allocate anything, so it is only consuming the minimum overhead (aprox. 8 kb per thread, right?).
8MB failed, roughly 7.5 MB success....0.5 difference...
So, my question is:
Knowing that mmap doesnt actually consume the memory (due to overcommit=0, right?):
I guess this has somewhat a relation with OOM (Out of Memory detection). The thing is, even though mmap() doesn't allocate anything initially, but still you commit a memory area. And kernel, in this case, will promise up to certain degree that it will satisfy your request at its best. Initially, you requested 8 MB. Knowing that you had 9 MB free RAM, I think you got ENOMEM since that would just leave 1 MB and quite likely that is under safe minimum free RAM allowed. I roughly recall that somewhere around 2-5% of RAM must be kept unallocated from user need to cope with urgent situation or something like that. It was shown that, when you use 7.5 MB, that slight 0.5 MB made you didn't hit the mark...so you're safe now and kernel allow that. Sorry I couldn't point you to exact kernel source code line. lxr.linux.no is your best friend.... -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
participants (4)
-
Ezequiel García -
Mulyadi Santosa -
Peter Teoh -
Zheng Da