Diploma project with the Linux kernel
Hello, I am a Computer Science student and for my last year I need to make and present a 'diploma project' at the end of June. So far I want to make a kernel module, whose description is in the following paragraph. I feel comfortable with C and my OS knowledge is maybe slightly better than my OS course. My question is: Would it possible to pull this off? I have no experience with the kernel and I want to get into kernel development, so this would be a perfect opportunity for that. My only issue is that this may be too complex for my experience. My idea: Something along the lines of checkpoint-restart as a kernel module. I want to ultimately be able to migrate a running process to a different machine (assuming same at least some basic similarity). I know of BLCR <http://crd.lbl.gov/departments/computer-science/CLaSS/research/BLCR/> and I am planning on using it as a guide, although I am unsure about working on it directly. As far as I know, the grading process does not require this to be 100% complete, so I am aiming at transferring at least all the memory, restoring file descriptors and maybe child processes/threads. I will really appreciate any feedback! Cheers, Boyan
On 10/4/18 2:44 PM, Boian Karatotev wrote:
Hello,
I am a Computer Science student and for my last year I need to make and present a 'diploma project' at the end of June. So far I want to make a kernel module, whose description is in the following paragraph. I feel comfortable with C and my OS knowledge is maybe slightly better than my OS course. My question is: Would it possible to pull this off? I have no experience with the kernel and I want to get into kernel development, so this would be a perfect opportunity for that. My only issue is that this may be too complex for my experience.
My idea: Something along the lines of checkpoint-restart as a kernel module. I want to ultimately be able to migrate a running process to a different machine (assuming same at least some basic similarity). I know of BLCR <http://crd.lbl.gov/departments/computer-science/CLaSS/research/BLCR/> and I am planning on using it as a guide, although I am unsure about working on it directly. As far as I know, the grading process does not require this to be 100% complete, so I am aiming at transferring at least all the memory, restoring file descriptors and maybe child processes/threads.
I will really appreciate any feedback! Cheers, Boyan
Interesting. I wouldn't mind helping. Ruben
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 http://www.nylxs.com - Leadership Development in Free Software http://www.brooklyn-living.com Being so tracked is for FARM ANIMALS and and extermination camps, but incompatible with living as a free human being. -RI Safir 2013
On Thu, 04 Oct 2018 21:44:14 +0300, Boian Karatotev said:
I am a Computer Science student and for my last year I need to make and present a 'diploma project' at the end of June. So far I want to make a kernel module, whose description is in the following paragraph. I feel comfortable with C and my OS knowledge is maybe slightly better than my OS course. My question is: Would it possible to pull this off? I have no experience with the kernel and I want to get into kernel development, so this would be a perfect opportunity for that. My only issue is that this may be too complex for my experience.
My idea: Something along the lines of checkpoint-restart as a kernel module. I want to ultimately be able to migrate a running process to a different machine (assuming same at least some basic similarity). I know of BLCR <http://crd.lbl.gov/departments/computer-science/CLaSS/research/BLCR/> and I am planning on using it as a guide, although I am unsure about working on it directly. As far as I know, the grading process does not require this to be 100% complete, so I am aiming at transferring at least all the memory, restoring file descriptors and maybe child processes/threads.
You mean you want to re-invent the current checkpoint-restart code that's been in the kernel since v3.10 back in June 2013? (see kernel/kcmp.c for the gory details). Note that migrating a running process to a different machine is a *lot* trickier, especially if it has things like open files or network connections. "Assume at least some basic similarity" isn't anywhere *near* good enough - if the process has /home/fred/wombats/my_terabyte_database open, you're going to need to have it at the same place in the filesystem and data synced across to the new target (particularly fun if the process scribbles some more on the file while you're busy migrating it, or if it hasn't done an fsync). Similarly, if it has a TCP connection open to someplace else, you're going to have to figure out what to do with the IP 4-tuple and sequence numbers to avoid breaking the connection. And if it's HPC software using MPI configured to do RDMA over Infiniband, that's even uglier.... In fact, migrating an entire virtual machine is easier than migrating one process, because you don't have to worry about recovering the process state, that's all in kernel memory that you migrate with the VM. Move the VM, take down the IP on the old hypervisor, set up the IP on the new one, toss out a gratuitous ARP packet so other machines on the subnet notice, and you're ready to go... There's a *reason* why VMWare gets away with charging lots of money for their enterprise-class software that supports migrating a live VM across hypervisors. It's a lot harder to do than you think.
This might be of interest to you: https://www.criu.org On Thu, Oct 4, 2018 at 8:23 PM <valdis.kletnieks@vt.edu> wrote:
On Thu, 04 Oct 2018 21:44:14 +0300, Boian Karatotev said:
I am a Computer Science student and for my last year I need to make and present a 'diploma project' at the end of June. So far I want to make a kernel module, whose description is in the following paragraph. I feel comfortable with C and my OS knowledge is maybe slightly better than my OS course. My question is: Would it possible to pull this off? I have no experience with the kernel and I want to get into kernel development, so this would be a perfect opportunity for that. My only issue is that this may be too complex for my experience.
My idea: Something along the lines of checkpoint-restart as a kernel module. I want to ultimately be able to migrate a running process to a different machine (assuming same at least some basic similarity). I know of BLCR < http://crd.lbl.gov/departments/computer-science/CLaSS/research/BLCR/> and I am planning on using it as a guide, although I am unsure about working on it directly. As far as I know, the grading process does not require this to be 100% complete, so I am aiming at transferring at least all the memory, restoring file descriptors and maybe child processes/threads.
You mean you want to re-invent the current checkpoint-restart code that's been in the kernel since v3.10 back in June 2013? (see kernel/kcmp.c for the gory details).
Note that migrating a running process to a different machine is a *lot* trickier, especially if it has things like open files or network connections. "Assume at least some basic similarity" isn't anywhere *near* good enough - if the process has /home/fred/wombats/my_terabyte_database open, you're going to need to have it at the same place in the filesystem and data synced across to the new target (particularly fun if the process scribbles some more on the file while you're busy migrating it, or if it hasn't done an fsync). Similarly, if it has a TCP connection open to someplace else, you're going to have to figure out what to do with the IP 4-tuple and sequence numbers to avoid breaking the connection. And if it's HPC software using MPI configured to do RDMA over Infiniband, that's even uglier....
In fact, migrating an entire virtual machine is easier than migrating one process, because you don't have to worry about recovering the process state, that's all in kernel memory that you migrate with the VM. Move the VM, take down the IP on the old hypervisor, set up the IP on the new one, toss out a gratuitous ARP packet so other machines on the subnet notice, and you're ready to go...
There's a *reason* why VMWare gets away with charging lots of money for their enterprise-class software that supports migrating a live VM across hypervisors. It's a lot harder to do than you think.
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
On Fri, 05 Oct 2018 02:58:23 +0100, Mike Krinkin said:
This might be of interest to you: https://www.criu.org
That's got two problems - first, it's userspace. And second, it's fairly mature software, which means it's not suitable for a student project by itself, and all the low-hanging fruit for improvements has probably already been done (meaning that further extensions will be technically challenging...) A better source for project ideas is to do a literature search and find proof-of-concept projects that *didn't* turn into mature software, and need work to turn them into actual running code....
On Fri, Oct 5, 2018 at 3:09 AM <valdis.kletnieks@vt.edu> wrote:
On Fri, 05 Oct 2018 02:58:23 +0100, Mike Krinkin said:
This might be of interest to you: https://www.criu.org
That's got two problems - first, it's userspace.
Well, it's called userspace, but it does require a significant kernel support.
And second, it's fairly mature software, which means it's not suitable for a student project by itself, and all the low-hanging fruit for improvements has probably already been done (meaning that further extensions will be technically challenging...)
I guess, that someone from the CRIU team might be a better person to tell whether it's the case. It doesn't take a lot of effort to ask whether they have tasks that might become a student project, especially considering that the team doing CRIU did in the past at least collaborate with univeristies in Russia where students actually worked on CRIU.
A better source for project ideas is to do a literature search and find proof-of-concept projects that *didn't* turn into mature software, and need work to turn them into actual running code....
You mean you want to re-invent the current checkpoint-restart code that's been in the kernel since v3.10 back in June 2013? (see kernel/kcmp.c for the gory details).
Too bad I hadn't found this out earlier. I had hopes this wasn't really done in kernels, so I had hopes to at least try as a lurning opportunity as an official project. I guess not It doesn't take a lot of effort to ask whether they have tasks that might
become a student project,
I'll do that, thanks! I guess this is pointless doing, so I'll look around elsewhere. Does anyone know anything else that might be suitable for a school project?
On 10/5/18, Boian Karatotev <boian4o1@gmail.com> wrote:
You mean you want to re-invent the current checkpoint-restart code that's been in the kernel since v3.10 back in June 2013? (see kernel/kcmp.c for the gory details).
Too bad I hadn't found this out earlier. I had hopes this wasn't really done in kernels, so I had hopes to at least try as a lurning opportunity as an official project. I guess not
It doesn't take a lot of effort to ask whether they have tasks that might
become a student project,
I'll do that, thanks!
I guess this is pointless doing, so I'll look around elsewhere. Does anyone know anything else that might be suitable for a school project?
Hi.. The first thing that always comes to mind for me is to track down bug reports. For the kind of project you're talking about here... Maybe track down *wishlist* bugs where users are asking for features that don't exist yet? Always feels like there's half a bazillion bug reports spread across all the various distributions on the Internet so starting with a keyword topic helps. If nothing friendly sounding pops up, maybe what you do find will still spawn an idea that's attractive to your talents. Good luck! :) Cindy :) -- Cindy-Sue Causey Talking Rock, Pickens County, Georgia, USA * Woof!! ~My Dog *
participants (5)
-
Boian Karatotev -
Cindy-Sue Causey -
Mike Krinkin -
Ruben Safir -
valdis.kletnieks＠vt.edu