On Mon, Feb 15, 2016 at 6:06 PM, Nitin Varyani <varyani.nitin1@gmail.com> wrote:
Hi I have studied LInux kernel CFS scheduling algorithm - the vruntime, weights, nice value, etc. I am able to understand the code. Actually the task given to me is really very huge. I am told to design a distributed process scheduling algorithm. A very simple implementation of it will be sufficient for me. Current distributed OS are patch work over the linux kernels, that is, they are responsible for load balancing through process migration but the scheduling is taken care by the single machine linux kernels. My task is to make the scheduling algorithm itself as distributed. That is a scheduler makes a decision whether to migrate a task or to keep the task in the current system. I need some design aspects of how to achieve it. Another thing which I want to know is that whether this job is possible for a kernel newbie like me.
On Sat, Feb 13, 2016 at 3:12 PM, Nitin Varyani <varyani.nitin1@gmail.com> wrote:
thanks
On Sat, Feb 13, 2016 at 2:19 PM, Henrik Austad <henrik@austad.us> wrote:
On Sat, Feb 13, 2016 at 11:42:57AM +0530, Nitin Varyani wrote:
Hello,
Hi Nitin,
I want to understand the flow of code of process scheduler of linux kernel. What I have understood is that The task marks itself as sleeping, puts itself on a wait queue, removes itself from the red-black tree of runnable, and calls schedule() to select a new process to execute.
for Waking back up The task is set as runnable, removed from the wait queue, and added back to the red-black tree.
Can I get the details of which function does what? in sched/core.c and
in
sched/fair.c I am concerned only with fair scheduler. There are so many functions in these two files that I am totally confused.
Then core.c and fair.c is the best bet.
You could also pick up a copy of Linux kernel development (By Love), it gives a nice introduction to the overall flow of .. well mostly everything. :)
In kernel/sched/sched.h you have a struct called 'struct sched_class" which is a set of function-points. This is used by the core machinery to call into scheduling-class specific code. At the bottom of fair.c, you see said struct being populated.
Also, if you want to see what really happens, try enabling function-tracing, but limit it to sched-functions only (and sched-events, those are also useful to see what triggers things)
mount -t debugfs nodev /sys/kernel/debug cd /sys/kernel/debug/tracing echo 0 > tracing_on echo function > current_tracer echo "sched*" > set_ftrace_filter echo 1 > events/sched/enable echo 1 > tracing_on ... wait for a few secs echo 0 > tracing_on
cat trace > /tmp/trace.txt
Now, look at trace.txt and correlate it to the scheduler code :)
Good luck!
-- Henrik Austad
On Mon, Feb 15, 2016 at 7:37 PM, Nitin Varyani <varyani.nitin1@gmail.com> wrote:
On Mon, Feb 15, 2016 at 6:06 PM, Nitin Varyani <varyani.nitin1@gmail.com> wrote:
Hi I have studied LInux kernel CFS scheduling algorithm - the vruntime, weights, nice value, etc. I am able to understand the code. Actually the task given to me is really very huge. I am told to design a distributed process scheduling algorithm. A very simple implementation of it will be sufficient for me. Current distributed OS are patch work over the linux kernels, that is, they are responsible for load balancing through process migration but the scheduling is taken care by the single machine linux kernels. My task is to make the scheduling algorithm itself as distributed. That is a scheduler makes a decision whether to migrate a task or to keep the task in the current system. I need some design aspects of how to achieve it. Another thing which I want to know is that whether this job is possible for a kernel newbie like me.
On Sat, Feb 13, 2016 at 3:12 PM, Nitin Varyani <varyani.nitin1@gmail.com> wrote:
thanks
On Sat, Feb 13, 2016 at 2:19 PM, Henrik Austad <henrik@austad.us> wrote:
On Sat, Feb 13, 2016 at 11:42:57AM +0530, Nitin Varyani wrote:
Hello,
Hi Nitin,
I want to understand the flow of code of process scheduler of linux kernel. What I have understood is that The task marks itself as sleeping, puts itself on a wait queue, removes itself from the red-black tree of runnable, and calls schedule() to select a new process to execute.
for Waking back up The task is set as runnable, removed from the wait queue, and added back to the red-black tree.
Can I get the details of which function does what? in sched/core.c
and in
sched/fair.c I am concerned only with fair scheduler. There are so many functions in these two files that I am totally confused.
Then core.c and fair.c is the best bet.
You could also pick up a copy of Linux kernel development (By Love), it gives a nice introduction to the overall flow of .. well mostly everything. :)
In kernel/sched/sched.h you have a struct called 'struct sched_class" which is a set of function-points. This is used by the core machinery to call into scheduling-class specific code. At the bottom of fair.c, you see said struct being populated.
Also, if you want to see what really happens, try enabling function-tracing, but limit it to sched-functions only (and sched-events, those are also useful to see what triggers things)
mount -t debugfs nodev /sys/kernel/debug cd /sys/kernel/debug/tracing echo 0 > tracing_on echo function > current_tracer echo "sched*" > set_ftrace_filter echo 1 > events/sched/enable echo 1 > tracing_on ... wait for a few secs echo 0 > tracing_on
cat trace > /tmp/trace.txt
Now, look at trace.txt and correlate it to the scheduler code :)
Good luck!
-- Henrik Austad
Please don't top post :) Use bottom post . Sounds like what you're going to do is highly similar to openMosix. check their source code. Please note that openmosix is patch against 2.4.x linux kernel. When they're about to made it compatible to 2.6.x, the project stalls. See Linux IPMI project and see if you can help them out -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com This email has been sent from a virus-free computer protected by Avast. www.avast.com <https://www.avast.com/sig-email> <#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
It is similar to openMosix but still quite different. Open Mosix is built on the top of existing linux kernels. The scheduling is taken care by the existing linux kernels. Open Mosix is responsible for workload distribution. This project is first of its kind. On Wed, Feb 17, 2016 at 11:40 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com
wrote:
On Mon, Feb 15, 2016 at 7:37 PM, Nitin Varyani <varyani.nitin1@gmail.com> wrote:
On Mon, Feb 15, 2016 at 6:06 PM, Nitin Varyani <varyani.nitin1@gmail.com> wrote:
Hi I have studied LInux kernel CFS scheduling algorithm - the vruntime, weights, nice value, etc. I am able to understand the code. Actually the task given to me is really very huge. I am told to design a distributed process scheduling algorithm. A very simple implementation of it will be sufficient for me. Current distributed OS are patch work over the linux kernels, that is, they are responsible for load balancing through process migration but the scheduling is taken care by the single machine linux kernels. My task is to make the scheduling algorithm itself as distributed. That is a scheduler makes a decision whether to migrate a task or to keep the task in the current system. I need some design aspects of how to achieve it. Another thing which I want to know is that whether this job is possible for a kernel newbie like me.
On Sat, Feb 13, 2016 at 3:12 PM, Nitin Varyani <varyani.nitin1@gmail.com
wrote:
thanks
On Sat, Feb 13, 2016 at 2:19 PM, Henrik Austad <henrik@austad.us> wrote:
On Sat, Feb 13, 2016 at 11:42:57AM +0530, Nitin Varyani wrote:
Hello,
Hi Nitin,
I want to understand the flow of code of process scheduler
of
linux kernel. What I have understood is that The task marks itself as sleeping, puts itself on a wait queue, removes itself from the red-black tree of runnable, and calls schedule() to select a new process to execute.
for Waking back up The task is set as runnable, removed from the wait queue, and added back to the red-black tree.
Can I get the details of which function does what? in sched/core.c and in sched/fair.c I am concerned only with fair scheduler. There are so many functions in these two files that I am totally confused.
Then core.c and fair.c is the best bet.
You could also pick up a copy of Linux kernel development (By Love), it gives a nice introduction to the overall flow of .. well mostly everything. :)
In kernel/sched/sched.h you have a struct called 'struct sched_class" which is a set of function-points. This is used by the core machinery to call into scheduling-class specific code. At the bottom of fair.c, you see said struct being populated.
Also, if you want to see what really happens, try enabling function-tracing, but limit it to sched-functions only (and sched-events, those are also useful to see what triggers things)
mount -t debugfs nodev /sys/kernel/debug cd /sys/kernel/debug/tracing echo 0 > tracing_on echo function > current_tracer echo "sched*" > set_ftrace_filter echo 1 > events/sched/enable echo 1 > tracing_on ... wait for a few secs echo 0 > tracing_on
cat trace > /tmp/trace.txt
Now, look at trace.txt and correlate it to the scheduler code :)
Good luck!
-- Henrik Austad
Please don't top post :) Use bottom post .
Sounds like what you're going to do is highly similar to openMosix. check their source code.
Please note that openmosix is patch against 2.4.x linux kernel. When they're about to made it compatible to 2.6.x, the project stalls. See Linux IPMI project and see if you can help them out
-- regards,
Mulyadi Santosa Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
This email has been sent from a virus-free computer protected by Avast. www.avast.com <https://www.avast.com/sig-email> <#1657133857_DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
On Wed, Feb 17, 2016 at 12:48:33PM +0530, Nitin Varyani wrote:
It is similar to openMosix but still quite different. Open Mosix is built on the top of existing linux kernels. The scheduling is taken care by the existing linux kernels. Open Mosix is responsible for workload distribution. This project is first of its kind.
No, it's not, it's been tried before, I suggest doing a bit more research before making statements like that. good luck, greg k-h
Check here: https://www.kernel.org/doc/Documentation/scheduler/sched-design-CFS.txt and http://www.ibm.com/developerworks/library/l-completely-fair-scheduler/ Regards, Boyan Vladinov On 17.02.2016 18:23, Greg KH wrote:
On Wed, Feb 17, 2016 at 12:48:33PM +0530, Nitin Varyani wrote:
It is similar to openMosix but still quite different. Open Mosix is built on the top of existing linux kernels. The scheduling is taken care by the existing linux kernels. Open Mosix is responsible for workload distribution. This project is first of its kind. No, it's not, it's been tried before, I suggest doing a bit more research before making statements like that.
good luck,
greg k-h
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Check here: https://www.kernel.org/doc/Documentation/scheduler/sched-design-CFS.txt and http://www.ibm.com/developerworks/library/l-completely-fair-scheduler/ Regards, Boyan Vladinov On 17.02.2016 18:23, Greg KH wrote:
On Wed, Feb 17, 2016 at 12:48:33PM +0530, Nitin Varyani wrote:
It is similar to openMosix but still quite different. Open Mosix is built on the top of existing linux kernels. The scheduling is taken care by the existing linux kernels. Open Mosix is responsible for workload distribution. This project is first of its kind. No, it's not, it's been tried before, I suggest doing a bit more research before making statements like that.
good luck,
greg k-h
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
participants (4)
-
Boyan Vladinov -
Greg KH -
Mulyadi Santosa -
Nitin Varyani