clock_gettime function doesn't scale to real time.. and changing CNTFRQ_EL0 doesn't make any change..(arm64)
Hello linux experts and newbies, I have ported linux on our arm64 fpga board. Both 5.10.0 and 5.15.xx works ok with minimal config. I have run a simple application and timed the processing time using clock_gettime function. It felt like it took almost 2.3 seconds but the program say it took only 0.36 seconds. Here is how I did it in the application. Int main() { struct timespec start, stop; float exec_time_sec, exec_time_nsec; //check start time if (clock_gettime(CLOCK_REALTIME, &start) == -1 ) { perror ("clock_gettime"); exit(EXIT_FAILURE); } Do something... (calculate fibonacci value for 1 ~ 30) //check end time if (clock_gettime(CLOCK_REALTIME, &stop) == -1 ) { perror ("clock_gettime"); exit(EXIT_FAILURE); } //Normalize to mili second exec_time_sec = (float)(stop.tv_sec - start.tv_sec); exec_time_nsec = (float)((double)(stop.tv_nsec - start.tv_nsec)/(double)BILLION); printf("Execution time : %f sec\n", exec_time_sec + exec_time_nsec); return 0; } I used u-boot program for loading linux kernel and the u-boot program sets the CNTFRQ_EL0 register with 5000000. (which is 5MHz, I heard the system clock runs at 5MHz in the board). The description of the register in armv8 arch manual says : This register is provided so that software can discover the frequency of the system counter. It must be programmed with this value as part of system initialization. The value of the register is not interpreted by hardware. I tried setting the CNTFRQ_EL0 with 20Mhz, expecting the execution to be displayed 4 times shorter but it is the same! I couldn't find how linux uses clock_gettime. How can I solve this problem? Any advice will be deeply appreciated. Thank you! Chan Kim
On Fri, Nov 04, 2022 at 06:04:15PM +0900, Chan Kim wrote:
Hello linux experts and newbies,
I have ported linux on our arm64 fpga board. Both 5.10.0 and 5.15.xx works ok with minimal config. I have run a simple application and timed the processing time using clock_gettime function. It felt like it took almost 2.3 seconds but the program say it took only 0.36 seconds. Here is how I did it in the application.
Hello, while I'm by no means an authority on kernel code and the board you are developing for I did spot some problems with your provided example code.
Int main() {
struct timespec start, stop; float exec_time_sec, exec_time_nsec;
//check start time if (clock_gettime(CLOCK_REALTIME, &start) == -1 ) { perror ("clock_gettime"); exit(EXIT_FAILURE); }
CLOCK_REALTIME is defined as "wall time" in `man clock_gettime` so it should not be impacted by the CPU frequency.
Do something... (calculate fibonacci value for 1 ~ 30)
//check end time
if (clock_gettime(CLOCK_REALTIME, &stop) == -1 ) { perror ("clock_gettime"); exit(EXIT_FAILURE); }
//Normalize to mili second exec_time_sec = (float)(stop.tv_sec - start.tv_sec);
You are not converting seconds to milliseconds here but in the printf call below you are asuming it's the same time unit.
exec_time_nsec = (float)((double)(stop.tv_nsec - start.tv_nsec)/(double)BILLION);
1 millisecond is 0.000001 nanoseconds. So you want MILLION here. Also, it's a lot of casting.
printf("Execution time : %f sec\n", exec_time_sec + exec_time_nsec);
return 0; }
I used u-boot program for loading linux kernel and the u-boot program sets the CNTFRQ_EL0 register with 5000000. (which is 5MHz, I heard the system clock runs at 5MHz in the board). The description of the register in armv8 arch manual says :
This register is provided so that software can discover the frequency of the system counter. It must be programmed with this value as part of system initialization. The value of the register is not interpreted by hardware.
I tried setting the CNTFRQ_EL0 with 20Mhz, expecting the execution to be displayed 4 times shorter but it is the same! I couldn't find how linux uses clock_gettime. How can I solve this problem? Any advice will be deeply appreciated.
Thank you!
Chan Kim
I would suggest that you have a read through `man clock_gettime`. After fixing your code. In particular you might be interested in the actual CPU time clocks. That said. I'm not sure these are the system calls you should be looking at. Br, Linus Probert
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
On Fri, 04 Nov 2022 14:34:15 +0530, Chan Kim wrote:
Hello linux experts and newbies,
I have ported linux on our arm64 fpga board. Both 5.10.0 and 5.15.xx works ok with minimal config.
I have run a simple application and timed the processing time using clock_gettime function.
It felt like it took almost 2.3 seconds but the program say it took only 0.36 seconds.
Try a simple command line loop to see if the problem is with system or your program: while $(sleep 1); do echo "Hi"; done; Also, while of no concern here, note that kernel also has a real-time config (CONFIG_PREEMPT_RT).
Here is how I did it in the application.
Int main() { struct timespec start, stop; float exec_time_sec, exec_time_nsec;
//check start time if (clock_gettime(CLOCK_REALTIME, &start) == -1 ) { perror ("clock_gettime"); exit(EXIT_FAILURE); }
Do something... (calculate fibonacci value for 1 ~ 30)
//check end time if (clock_gettime(CLOCK_REALTIME, &stop) == -1 ) { perror ("clock_gettime"); exit(EXIT_FAILURE); }
//Normalize to mili second exec_time_sec = (float)(stop.tv_sec - start.tv_sec); exec_time_nsec = (float)((double)(stop.tv_nsec - start.tv_nsec)/(double)BILLION); printf("Execution time : %f sec\n", exec_time_sec + exec_time_nsec);
return 0;
}
Adding to what Linus said in the other reply, CLOCK_REALTIME is not guaranteed to be monotonic. For calculation of intervals, you should instead use CLOCK_MONOTONIC or CLOCK_MONOTONIC_RAW. Also, try just having Fibonacci calculation in main, and then use the `time` command to achieve what you want.
I used u-boot program for loading linux kernel and the u-boot program sets the CNTFRQ_EL0 register with 5000000.
(which is 5MHz, I heard the system clock runs at 5MHz in the board).
The description of the register in armv8 arch manual says :
This register is provided so that software can discover the frequency of the system counter. It must be programmed with this value as part of system initialization. The value of the register is not interpreted by hardware.
I tried setting the CNTFRQ_EL0 with 20Mhz, expecting the execution to be displayed 4 times shorter but it is the same!
Because as the description says, this register is not interpreted by the hardware. That means this won't affect hardware in any way, and thus won't increase frequency. That's why your execution time remains the same. The value stored here is for use by software to know the clock frequency of the machine, so it can do its time related calculations appropriately. What you did is equivalent of trying to hoodwink the system!
I couldn't find how linux uses clock_gettime.
If you mean how clock_gettime function is defined, see it here: https://elixir.bootlin.com/linux/latest/source/kernel/time/posix-clock.c#L25... Otherwise: man clock_gettime
How can I solve this problem? Any advice will be deeply appreciated.
Try making the changes and let us know. This is new for me too!
Thank you! Chan Kim
Please use plain text email instead of HTML, that's the kernel mailing list etiquette. It is also superior once you get the hang of it! :) Thanks, Siddh
Hi,Siddh and Linus, I tried using 'time' command to measure the time and my program output is the same in commercial intel machine (ubuntu 20.04). ... fib(041) = 165580141 fib(042) = 267914296 fibonacci finishing... Execution time : 3.801840 sec real 0m3.806s user 0m3.802s sys 0m0.005s So it looks like the application program doesn't have problem. In the program exec_time_nsec is actually in unit of second. Because tv_nsec is in unit of nsec, I'm converting it to second by dividing it with BILLION. And yes, the CNTFRQ_EL0 is for software use. Hardware just runs with clock and it know only the number of clocks and doesn't know 1000000 clock period physically represents what seconds. CNTFRQ_EL0 is letting the software able to convert the number of clocks to physical time. I'm at home now and can't do experiment with the board now. I'll try following the pc_clock_gettime function as you showed to see how it works. Thanks for the advices. Chan Kim
-----Original Message----- From: Siddh Raman Pant <code@siddh.me> Sent: Saturday, November 5, 2022 4:51 AM To: Chan Kim <ckim@etri.re.kr> Cc: Kernel Newbies <kernelnewbies@kernelnewbies.org> Subject: Re: clock_gettime function doesn't scale to real time.. and changing CNTFRQ_EL0 doesn't make any change..(arm64)
On Fri, 04 Nov 2022 14:34:15 +0530, Chan Kim wrote:
Hello linux experts and newbies,
I have ported linux on our arm64 fpga board. Both 5.10.0 and 5.15.xx works ok with minimal config.
I have run a simple application and timed the processing time using clock_gettime function.
It felt like it took almost 2.3 seconds but the program say it took only 0.36 seconds.
Try a simple command line loop to see if the problem is with system or your program: while $(sleep 1); do echo "Hi"; done;
Also, while of no concern here, note that kernel also has a real-time config (CONFIG_PREEMPT_RT).
Here is how I did it in the application.
Int main() { struct timespec start, stop; float exec_time_sec, exec_time_nsec;
//check start time if (clock_gettime(CLOCK_REALTIME, &start) == -1 ) { perror ("clock_gettime"); exit(EXIT_FAILURE); }
Do something... (calculate fibonacci value for 1 ~ 30)
//check end time if (clock_gettime(CLOCK_REALTIME, &stop) == -1 ) { perror ("clock_gettime"); exit(EXIT_FAILURE); }
//Normalize to mili second exec_time_sec = (float)(stop.tv_sec - start.tv_sec); exec_time_nsec = (float)((double)(stop.tv_nsec - start.tv_nsec)/(double)BILLION); printf("Execution time : %f sec\n", exec_time_sec + exec_time_nsec);
return 0;
}
Adding to what Linus said in the other reply, CLOCK_REALTIME is not guaranteed to be monotonic. For calculation of intervals, you should instead use CLOCK_MONOTONIC or CLOCK_MONOTONIC_RAW.
Also, try just having Fibonacci calculation in main, and then use the `time` command to achieve what you want.
I used u-boot program for loading linux kernel and the u-boot program sets the CNTFRQ_EL0 register with 5000000.
(which is 5MHz, I heard the system clock runs at 5MHz in the board).
The description of the register in armv8 arch manual says :
This register is provided so that software can discover the frequency of the system counter. It must be programmed with this value as part of system initialization. The value of the register is not interpreted by hardware.
I tried setting the CNTFRQ_EL0 with 20Mhz, expecting the execution to be displayed 4 times shorter but it is the same!
Because as the description says, this register is not interpreted by the hardware. That means this won't affect hardware in any way, and thus won't increase frequency. That's why your execution time remains the same.
The value stored here is for use by software to know the clock frequency of the machine, so it can do its time related calculations appropriately.
What you did is equivalent of trying to hoodwink the system!
I couldn't find how linux uses clock_gettime.
If you mean how clock_gettime function is defined, see it here: https://elixir.bootlin.com/linux/latest/source/kernel/time/posix- clock.c#L259
Otherwise: man clock_gettime
How can I solve this problem? Any advice will be deeply appreciated.
Try making the changes and let us know. This is new for me too!
Thank you! Chan Kim
Please use plain text email instead of HTML, that's the kernel mailing list etiquette. It is also superior once you get the hang of it! :)
Thanks, Siddh
Hello all, I fixed this problem and now the time measurement and commands like sleep works just fine. Two points I fixed : - I had 'clock-frequency' property set with wrong frequency in my timer node in the device tree so I removed it. The document says when the boot loader sets CNTFRQ register correctly, we don't have to provide 'clock-frequency' property value. - The correct frequency of the system counter (arm464) was 10MHz in our board. Previously I set CNTFRQ register with 5MHz but I fixed it to 10MHz. One more thing to note. The system counter has both system register view and memory mapped register view. Previously I said even if I set cntfrq_el0 register with some values (using system register, "msr cntfrq_el0, COUNTER_FREQUENCY") it did not change anything. It was because I set the same register with old value (5MHz) using the memory mapped access later (like with "writel(COUNTER_FREQUENCY, 0x4c018020);"). Hope this helps someone later. Thank you! Chan Kim
-----Original Message----- From: Chan Kim <ckim@etri.re.kr> Sent: Saturday, November 5, 2022 8:46 PM To: 'Siddh Raman Pant' <code@siddh.me> Cc: 'Kernel Newbies' <kernelnewbies@kernelnewbies.org> Subject: RE: clock_gettime function doesn't scale to real time.. and changing CNTFRQ_EL0 doesn't make any change..(arm64)
Hi,Siddh and Linus,
I tried using 'time' command to measure the time and my program output is the same in commercial intel machine (ubuntu 20.04). ... fib(041) = 165580141 fib(042) = 267914296 fibonacci finishing... Execution time : 3.801840 sec
real 0m3.806s user 0m3.802s sys 0m0.005s
So it looks like the application program doesn't have problem. In the program exec_time_nsec is actually in unit of second. Because tv_nsec is in unit of nsec, I'm converting it to second by dividing it with BILLION. And yes, the CNTFRQ_EL0 is for software use. Hardware just runs with clock and it know only the number of clocks and doesn't know 1000000 clock period physically represents what seconds. CNTFRQ_EL0 is letting the software able to convert the number of clocks to physical time. I'm at home now and can't do experiment with the board now. I'll try following the pc_clock_gettime function as you showed to see how it works.
Thanks for the advices. Chan Kim
-----Original Message----- From: Siddh Raman Pant <code@siddh.me> Sent: Saturday, November 5, 2022 4:51 AM To: Chan Kim <ckim@etri.re.kr> Cc: Kernel Newbies <kernelnewbies@kernelnewbies.org> Subject: Re: clock_gettime function doesn't scale to real time.. and changing CNTFRQ_EL0 doesn't make any change..(arm64)
On Fri, 04 Nov 2022 14:34:15 +0530, Chan Kim wrote:
Hello linux experts and newbies,
I have ported linux on our arm64 fpga board. Both 5.10.0 and 5.15.xx works ok with minimal config.
I have run a simple application and timed the processing time using clock_gettime function.
It felt like it took almost 2.3 seconds but the program say it took only 0.36 seconds.
Try a simple command line loop to see if the problem is with system or your program: while $(sleep 1); do echo "Hi"; done;
Also, while of no concern here, note that kernel also has a real-time config (CONFIG_PREEMPT_RT).
Here is how I did it in the application.
Int main() { struct timespec start, stop; float exec_time_sec, exec_time_nsec;
//check start time if (clock_gettime(CLOCK_REALTIME, &start) == -1 ) { perror ("clock_gettime"); exit(EXIT_FAILURE); }
Do something... (calculate fibonacci value for 1 ~ 30)
//check end time if (clock_gettime(CLOCK_REALTIME, &stop) == -1 ) { perror ("clock_gettime"); exit(EXIT_FAILURE); }
//Normalize to mili second exec_time_sec = (float)(stop.tv_sec - start.tv_sec); exec_time_nsec = (float)((double)(stop.tv_nsec - start.tv_nsec)/(double)BILLION); printf("Execution time : %f sec\n", exec_time_sec + exec_time_nsec);
return 0;
}
Adding to what Linus said in the other reply, CLOCK_REALTIME is not guaranteed to be monotonic. For calculation of intervals, you should instead use CLOCK_MONOTONIC or CLOCK_MONOTONIC_RAW.
Also, try just having Fibonacci calculation in main, and then use the `time` command to achieve what you want.
I used u-boot program for loading linux kernel and the u-boot program sets the CNTFRQ_EL0 register with 5000000.
(which is 5MHz, I heard the system clock runs at 5MHz in the board).
The description of the register in armv8 arch manual says :
This register is provided so that software can discover the frequency of the system counter. It must be programmed with this value as part of system initialization. The value of the register is not interpreted by hardware.
I tried setting the CNTFRQ_EL0 with 20Mhz, expecting the execution to be displayed 4 times shorter but it is the same!
Because as the description says, this register is not interpreted by the hardware. That means this won't affect hardware in any way, and thus won't increase frequency. That's why your execution time remains the same.
The value stored here is for use by software to know the clock frequency of the machine, so it can do its time related calculations appropriately.
What you did is equivalent of trying to hoodwink the system!
I couldn't find how linux uses clock_gettime.
If you mean how clock_gettime function is defined, see it here: https://elixir.bootlin.com/linux/latest/source/kernel/time/posix- clock.c#L259
Otherwise: man clock_gettime
How can I solve this problem? Any advice will be deeply appreciated.
Try making the changes and let us know. This is new for me too!
Thank you! Chan Kim
Please use plain text email instead of HTML, that's the kernel mailing list etiquette. It is also superior once you get the hang of it! :)
Thanks, Siddh
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
On Tue, 08 Nov 2022 10:45:55 +0530 Chan Kim wrote:
Hello all,
I fixed this problem and now the time measurement and commands like sleep works just fine.
Two points I fixed : - I had 'clock-frequency' property set with wrong frequency in my timer node in the device tree so I removed it. The document says when the boot loader sets CNTFRQ register correctly, we don't have to provide 'clock-frequency' property value. - The correct frequency of the system counter (arm464) was 10MHz in our board. Previously I set CNTFRQ register with 5MHz but I fixed it to 10MHz.
One more thing to note. The system counter has both system register view and memory mapped register view. Previously I said even if I set cntfrq_el0 register with some values (using system register, "msr cntfrq_el0, COUNTER_FREQUENCY") it did not change anything. It was because I set the same register with old value (5MHz) using the memory mapped access later (like with "writel(COUNTER_FREQUENCY, 0x4c018020);").
Hope this helps someone later. Thank you!
Chan Kim
Glad to know. Thanks for sharing the updates and the cause behind it! Thanks, Siddh
There is a typo : Setting cntfrq_el0 is done by " msr cntfrq_el0, x0 " in the code.
-----Original Message----- From: Chan Kim <ckim@etri.re.kr> Sent: Tuesday, November 8, 2022 2:16 PM To: 'kernelnewbies@kernelnewbies.org' <kernelnewbies@kernelnewbies.org> Cc: 'Siddh Raman Pant' <code@siddh.me>; 'Linus Probert' <linus.probert@gmail.com> Subject: RE: clock_gettime function doesn't scale to real time.. and changing CNTFRQ_EL0 doesn't make any change..(arm64)
Hello all,
I fixed this problem and now the time measurement and commands like sleep works just fine.
Two points I fixed : - I had 'clock-frequency' property set with wrong frequency in my timer node in the device tree so I removed it. The document says when the boot loader sets CNTFRQ register correctly, we don't have to provide 'clock-frequency' property value. - The correct frequency of the system counter (arm464) was 10MHz in our board. Previously I set CNTFRQ register with 5MHz but I fixed it to 10MHz.
One more thing to note. The system counter has both system register view and memory mapped register view. Previously I said even if I set cntfrq_el0 register with some values (using system register, "msr cntfrq_el0, COUNTER_FREQUENCY") it did not change anything. It was because I set the same register with old value (5MHz) using the memory mapped access later (like with "writel(COUNTER_FREQUENCY, 0x4c018020);").
Hope this helps someone later. Thank you!
Chan Kim
-----Original Message----- From: Chan Kim <ckim@etri.re.kr> Sent: Saturday, November 5, 2022 8:46 PM To: 'Siddh Raman Pant' <code@siddh.me> Cc: 'Kernel Newbies' <kernelnewbies@kernelnewbies.org> Subject: RE: clock_gettime function doesn't scale to real time.. and changing CNTFRQ_EL0 doesn't make any change..(arm64)
Hi,Siddh and Linus,
I tried using 'time' command to measure the time and my program output is the same in commercial intel machine (ubuntu 20.04). ... fib(041) = 165580141 fib(042) = 267914296 fibonacci finishing... Execution time : 3.801840 sec
real 0m3.806s user 0m3.802s sys 0m0.005s
So it looks like the application program doesn't have problem. In the program exec_time_nsec is actually in unit of second. Because tv_nsec is in unit of nsec, I'm converting it to second by dividing it with BILLION. And yes, the CNTFRQ_EL0 is for software use. Hardware just runs with clock and it know only the number of clocks and doesn't know 1000000 clock period physically represents what seconds. CNTFRQ_EL0 is letting the software able to convert the number of clocks to physical time. I'm at home now and can't do experiment with the board now. I'll try following the pc_clock_gettime function as you showed to see how it works.
Thanks for the advices. Chan Kim
-----Original Message----- From: Siddh Raman Pant <code@siddh.me> Sent: Saturday, November 5, 2022 4:51 AM To: Chan Kim <ckim@etri.re.kr> Cc: Kernel Newbies <kernelnewbies@kernelnewbies.org> Subject: Re: clock_gettime function doesn't scale to real time.. and changing CNTFRQ_EL0 doesn't make any change..(arm64)
On Fri, 04 Nov 2022 14:34:15 +0530, Chan Kim wrote:
Hello linux experts and newbies,
I have ported linux on our arm64 fpga board. Both 5.10.0 and 5.15.xx works ok with minimal config.
I have run a simple application and timed the processing time using clock_gettime function.
It felt like it took almost 2.3 seconds but the program say it took only 0.36 seconds.
Try a simple command line loop to see if the problem is with system or your program: while $(sleep 1); do echo "Hi"; done;
Also, while of no concern here, note that kernel also has a real-time config (CONFIG_PREEMPT_RT).
Here is how I did it in the application.
Int main() { struct timespec start, stop; float exec_time_sec, exec_time_nsec;
//check start time if (clock_gettime(CLOCK_REALTIME, &start) == -1 ) { perror ("clock_gettime"); exit(EXIT_FAILURE); }
Do something... (calculate fibonacci value for 1 ~ 30)
//check end time if (clock_gettime(CLOCK_REALTIME, &stop) == -1 ) { perror ("clock_gettime"); exit(EXIT_FAILURE); }
//Normalize to mili second exec_time_sec = (float)(stop.tv_sec - start.tv_sec); exec_time_nsec = (float)((double)(stop.tv_nsec - start.tv_nsec)/(double)BILLION); printf("Execution time : %f sec\n", exec_time_sec + exec_time_nsec);
return 0;
}
Adding to what Linus said in the other reply, CLOCK_REALTIME is not guaranteed to be monotonic. For calculation of intervals, you should instead use CLOCK_MONOTONIC or CLOCK_MONOTONIC_RAW.
Also, try just having Fibonacci calculation in main, and then use the `time` command to achieve what you want.
I used u-boot program for loading linux kernel and the u-boot program sets the CNTFRQ_EL0 register with 5000000.
(which is 5MHz, I heard the system clock runs at 5MHz in the board).
The description of the register in armv8 arch manual says :
This register is provided so that software can discover the frequency of the system counter. It must be programmed with this value as part of system initialization. The value of the register is not interpreted by hardware.
I tried setting the CNTFRQ_EL0 with 20Mhz, expecting the execution to be displayed 4 times shorter but it is the same!
Because as the description says, this register is not interpreted by the hardware. That means this won't affect hardware in any way, and thus won't increase frequency. That's why your execution time remains the same.
The value stored here is for use by software to know the clock frequency of the machine, so it can do its time related calculations appropriately.
What you did is equivalent of trying to hoodwink the system!
I couldn't find how linux uses clock_gettime.
If you mean how clock_gettime function is defined, see it here: https://elixir.bootlin.com/linux/latest/source/kernel/time/posix- clock.c#L259
Otherwise: man clock_gettime
How can I solve this problem? Any advice will be deeply appreciated.
Try making the changes and let us know. This is new for me too!
Thank you! Chan Kim
Please use plain text email instead of HTML, that's the kernel mailing list etiquette. It is also superior once you get the hang of it! :)
Thanks, Siddh
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
participants (3)
-
Chan Kim -
Linus Probert -
Siddh Raman Pant