<div dir="ltr">
Hi, I have been experimenting with the FIFO scheduler and tick rate in order<br>
to reduce execution speed for tasks; I need to run legacy code that runs on old<br>
slow hardware at approximately the same speed on new faster hardware while<br>
preserving "realtime-like" properties of the code. The general idea<br>
was to add a delay<br>
during each task, tick to mimic the execution speed of the slower system.<br>
<br>
At the end of this mail are my modifications in the kernel, where I for the<br>
FIFO scheduler introduces a delay of 500 microseconds for each scheduling<br>
tick, that is<br>
my intention is to use up about 500 microseconds of the scheduling tick for the<br>
task. Assuming a tick length of 1000 microseconds we effectively reduce the<br>
execution speed to about 1 - 500/1000 or about 50%.<br><div><br>
</div><div>I have made a small test program to verify that my approach works as expected</div>
which I run in userspace (delay_test.c):<br>
<br>
#include <stdio.h><br>#include <unistd.h><br>#include <time.h><br>#include <sched.h><br>#include <stdint.h><br>#include <sys/time.h><br><br>static inline void timespec_diff(struct timespec *a, struct timespec *b,<br> struct timespec *result) {<br> result->tv_sec = a->tv_sec - b->tv_sec;<br> result->tv_nsec = a->tv_nsec - b->tv_nsec;<br> if (result->tv_nsec < 0) {<br> --result->tv_sec;<br> result->tv_nsec += 1000000000L;<br> }<br>}<br><br>void main(int argc, char *args[])<br>{<br> struct sched_param param;<br> param.sched_priority = 99; <br> sched_setscheduler(0,SCHED_FIFO,¶m);<br> struct timespec start,now,diff;<br> uint64_t sleep_delay = 100L;<br> clock_gettime(CLOCK_MONOTONIC, &start);<br><br> printf("Start is %lu.%lu\n",start.tv_sec,start.tv_nsec);<br> for(;;)<br> {<br> usleep(sleep_delay);<br> clock_gettime(CLOCK_MONOTONIC, &now);<br> timespec_diff(&now,&start,&diff);<br> printf("Time is %lus %luns\n",(uint64_t)diff.tv_sec,(uint64_t)diff.tv_nsec);<br><br> }<br>}
<br>
<br>
<br>
I compile the above program like this (assuming a 2GHz CPU core that is<br>
evacuated and assigned a dedicated Cgroup) $ gcc ....<br>
and create it like this with Cgroup v1<br>
<div>sudo cgcreate -t $USER:$GRP -a $USER:$GRP -g cpuset,memory,freezer:tes</div><div>echo "1" | sudo tee /sys/fs/cgroup/cpuset/test/cpuset.cpus > /dev/null</div><div>echo "0" | sudo tee /sys/fs/cgroup/cpuset/test/cpuset.mems > /dev/null<br>echo "1" | sudo tee /sys/fs/cgroup/cpuset/test/cpuset.cpu_exclusive > /dev/null</div>
<br>
and run it with<br>
sudo cgexec -g memory,freezer,cpuset:test ./a.out
<br>
<br>
The expected output is something like<br>
<div>
Start is 1835.474539419
</div>
Time is 0s 200000ns<br>Time is 0s 300000ns<br><div>Time is 0s 500000ns</div><div>
<div>Time is 0s 1000000ns
<-- Here scheduler delay kicks i.
</div>
</div>
Time is 0s 1100000ns
<br>
Time is 0s 1200000ns
<br>
...<br>
<br><div>
and so on. Instead, I get this</div><div>
Start is 1835.474539419<br>Time is 0s 175382ns<br>Time is 0s 283956ns<br>Time is 0s 391289ns<br>Time is 0s 501217ns
<-- Around here scheduler delay should have kicked in.<br>Time is 0s 610443ns <- Instead roughly the same delay is added.<br>Time is 0s 721963ns<br>Time is 0s 835387ns<br>Time is 0s 948290ns <br></div><div>
Time is 0s 1061414ns <br></div><div><br></div>Things I have tried so far: Turning off SMT, making sure that I am using<br>CONFIG_HZ_PERIODIC , Isolating tasks using cgroups and isolcpus, making sure<br>that the system is running at the correct tick rate and changing the frequency<br>in combination with the Governor. Is it something I am missing in regards to<br>doing this slowdown? Any answers or hints would be appreciated. <br>
<div><br></div><div>My own delay method I have added to the kernel (register_time.c):</div><div><br></div>
static uint64_t rdtsc2(void){<br>
uint32_t hi, lo;<br>
__asm__ __volatile__ ("xorl %%eax,%%eax\ncpuid" ::: "%rax",<br>
"%rbx", "%rcx", "%rdx");<br>
__asm__ __volatile__ ("rdtsc" : "=a"(lo), "=d"(hi));<br>
return (((uint64_t)hi)<<32) | ((uint64_t)lo) ; }<br>
<br><div>uint64_t get_khz(void){<br> uint64_t aperf,mperf,khz;<br> unsigned long flags;<br><br> local_irq_save(flags);<br> rdmsrl(MSR_IA32_APERF,aperf);<br> rdmsrl(MSR_IA32_MPERF,mperf);<br> local_irq_restore(flags);<br><br> khz = div64_u64((cpu_khz * aperf),mperf);<br><br> return khz;<br>}</div><div><br></div><div>
uint64_t nanos_to_ticks(uint64_t nanos){<br>
return (get_khz() * 1000L * nanos)/NSEC_PER_SEC;<br>
}
</div><div> /* <br></div><div> *Sleeps for the specified amount of ticks.</div> * Ticks to sleep can be calculated with nanos_to_tick().<br><div> * Going from ticks to nanoseconds is done using ticks_to_nanos().</div><div> */<br></div>
uint64_t tick_sleep(uint64_t ticks_to_sleep) {<br>
uint64_t now;<br>
uint64_t sleep_until;<br>
uint64_t delta_ticks;<br>
<br>
preempt_disable();<br>
now = rdtsc2();<br>
sleep_until = now + ticks_to_sleep;<br>
while (now < sleep_until)<br>
{<br>
now = rdtsc2();<br>
}<br>
delta_ticks = now - sleep_until;<br>
preempt_enable();<br>
return delta_ticks;<br>
}<br>
<br>
The part where I try to add the delay (rt.c):<br>
<br>
static void task_tick_rt(struct rq *rq, struct task_struct *p, int queued) {<br>
struct sched_rt_entity *rt_se = &p->rt;<br>
<br>
update_curr_rt(rq);<br>
update_rt_rq_load_avg(rq_clock_pelt(rq), rq, 1);<br>
<br>
watchdog(rq, p);<br>
<br>
/*<br>
* RR tasks need a special form of timeslice management.<br><div>
* FIFO tasks have no timeslices.</div><div> *
CONFIG_PWM_FRAMEWORK is used to activate the special FIFO<br></div>
*/<br>
#ifdef CONFIG_PWM_FRAMEWORK<br>
if (p->policy == SCHED_FIFO)<br>
{<br>
tick_sleep(nanos_to_ticks(500000));<br>
return;<br>
}<br>
#else<br>
if (p->policy != SCHED_RR)<br>
{<br>
return;<br>
}<br>
#endif<br>
<br>
if (--p->rt.time_slice)<br>
return;<br>
<br>
p->rt.time_slice = sched_rr_timeslice;<br>
<br>
/*<br>
* Requeue to the end of the queue if we (and all of our ancestors) are not<br>
* the only element on the queue<br>
*/<br>
for_each_sched_rt_entity(rt_se)<br>
{<br>
if (rt_se->run_list.prev != rt_se->run_list.next)<br>
{<br>
requeue_task_rt(rq, p, 0);<br>
resched_curr(rq);<br>
return;<br>
}<br>
}<br>
}<br>
<br>
<br>
Best Regards<br>
Jacob M
</div>