<div dir="ltr">Well, my bad. I was wrong there. Your example was correct. Even in a multi-kernel like Barrelfish, each core manages only its own share of memory. RAM is explicitly partitioned between cores running on each kernel and any data structures that need to be shared across cores have to be replicated in each of the memory partitions and consistency maintained through IPC.<div>
<br></div><div style>~Gaurav</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Feb 8, 2013 at 10:29 PM, Gaurav Jain <span dir="ltr"><<a href="mailto:gjainroorkee@gmail.com" target="_blank">gjainroorkee@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I guess you misinterpreted me... A multi-kernel like Barrelfish would have separate kernel images running on different cores, but each of the core/kernel would be able to address the <i>entire</i> physical memory (including any memory-mapped devices). Hence, the database would still be addressable/reachable from all cores. Traditional locking mechanisms to protect the database (like in a single-kernel Linux) would be good here, too. OR, like in Barrelfish, we could have explicit inter-core communication messages to implement synchronization/consistency.<span class="HOEnZb"><font color="#888888"><div>
<br></div><div>~ Gaurav</div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Feb 8, 2013 at 9:34 PM, Jeff Haran <span dir="ltr"><<a href="mailto:Jeff.Haran@citrix.com" target="_blank">Jeff.Haran@citrix.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Well, for instance what we do. We maintain a big database of cell phone user IP addresses and other info in a kernel module running on a 24 core X86-64 system. All 24 cores can and do access that data. If each core had access to only a single partition of memory, then that DB would have to be spread across the various partitions and if the one core that could access a specific cell phone user’s info was busy while others were idle, access to that user’s would be delayed. The application is quite performance sensitive so having the ability for all cores to operate on all data makes for better performance.<u></u><u></u></span></font></p>
<p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></font></p><p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Jeff<u></u><u></u></span></font></p>
<p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></font></p><div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal"><b><font face="Tahoma"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";font-weight:bold">From:</span></font></b><font face="Tahoma"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Gaurav Jain [mailto:<a href="mailto:gjainroorkee@gmail.com" target="_blank">gjainroorkee@gmail.com</a>] <br>
<b><span style="font-weight:bold">Sent:</span></b> Friday, February 08, 2013 11:54 AM</span></font></p><div><div><font face="Tahoma"><br><b><span style="font-weight:bold">To:</span></b> Jeff Haran<br><b><span style="font-weight:bold">Subject:</span></b> Re: Kernel code interrupted by Timer<u></u><u></u></font></div>
</div><p></p></div></div><div><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">"</span></font><font size="3" color="#1f497d" face="Calibri"><span style="font-size:11.5pt;font-family:"Calibri","sans-serif";color:#1f497d">you can solve problems that can’t be easily solved if all the data is partitioned by CPU core."</span></font><u></u><u></u></p>
<div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p></div><div><p class="MsoNormal"><font size="3" color="#1f497d" face="Calibri"><span style="font-size:11.5pt;font-family:"Calibri","sans-serif";color:#1f497d">Can you please give an example or two of the aforementioned problems? I can think of efficient scheduling - a single kernel knowing what cores/processors are idle would help make better decisions without the overhead of message passing in separate kernels on separate cores.</span></font><u></u><u></u></p>
</div><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p></div><div><p class="MsoNormal"><font size="3" color="#1f497d" face="Calibri"><span style="font-size:11.5pt;font-family:"Calibri","sans-serif";color:#1f497d">~Gaurav</span></font><u></u><u></u></p>
</div></div><div><p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">On Fri, Feb 8, 2013 at 8:16 PM, Jeff Haran <<a href="mailto:Jeff.Haran@citrix.com" target="_blank">Jeff.Haran@citrix.com</a>> wrote:<u></u><u></u></span></font></p>
<div><div><p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I did not write Linux, so just guessing here. I don’t think the issue is so much kernel images as shared access to kernel data. The synchronization primitives in the kernel (spin locks, read-write locks, RCU, etc) are there to protect data from being corrupted during concurrent access, not code. I am not familiar with barrelfish but I think most modern OSes, for instance the various BSD derivatives, work like this. I have no recent experience with it, but I’d bet Windows does the same thing. If you can run multiple cores on the same data safely by providing proper locking, you can solve problems that can’t be easily solved if all the data is partitioned by CPU core.</span></font><u></u><u></u></p>
<p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></font><u></u><u></u></p><p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Getting the locking right though can be a challenge, which is of course good for software engineer job security. 8^)</span></font><u></u><u></u></p>
<p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></font><u></u><u></u></p><p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Jeff</span></font><u></u><u></u></p>
<p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></font><u></u><u></u></p><div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal"><b><font face="Tahoma"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";font-weight:bold">From:</span></font></b><font face="Tahoma"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Gaurav Jain [mailto:<a href="mailto:gjainroorkee@gmail.com" target="_blank">gjainroorkee@gmail.com</a>] <br>
<b><span style="font-weight:bold">Sent:</span></b> Friday, February 08, 2013 11:03 AM<br><b><span style="font-weight:bold">To:</span></b> Jeff Haran<br><b><span style="font-weight:bold">Subject:</span></b> Re: Kernel code interrupted by Timer</span></font><u></u><u></u></p>
</div></div><div><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">"</span></font><font size="3" color="#1f497d" face="Calibri"><span style="font-size:11.5pt;font-family:"Calibri","sans-serif";color:#1f497d">On multi-core systems this should be done regardless of whether kernel pre-emption is enabled or not"</span></font><u></u><u></u></p>
<div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p></div><div><p class="MsoNormal"><font size="3" color="#1f497d" face="Calibri"><span style="font-size:11.5pt;font-family:"Calibri","sans-serif";color:#1f497d">-- Do you know why such a design decision was made - That a single kernel image should manage all the cores! I have worked on Barrelfish (</span></font><a href="http://www.barrelfish.org/" target="_blank">http://www.barrelfish.org/</a>)<font size="3" color="#1f497d" face="Calibri"><span style="font-size:11.5pt;font-family:"Calibri","sans-serif";color:#1f497d"> a bit, and it boots a separate kernel on each of the cores. To me that appears to be the more intuitive approach. What advantages do we get by having a single kernel image on all cores? Is the convenience of being in the same address space (the single kernel's) at all cores at all times, the only/major reason? </span></font><u></u><u></u></p>
</div><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p></div><div><p class="MsoNormal"><font size="3" color="#1f497d" face="Calibri"><span style="font-size:11.5pt;font-family:"Calibri","sans-serif";color:#1f497d">~Gaurav</span></font><u></u><u></u></p>
</div></div><div><p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">On Fri, Feb 8, 2013 at 7:51 PM, Jeff Haran <<a href="mailto:Jeff.Haran@citrix.com" target="_blank">Jeff.Haran@citrix.com</a>> wrote:<u></u><u></u></span></font></p>
<div><div><p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I think you will find that this is a matter of kernel configuration. If kernel pre-emption is enabled at build time, then following the interrupt another process could get scheduled and it wouldn’t matter what interrupt went off. Typically some sort of explicit locking or RCU is used to protect critical sections in cases like this. On multi-core systems this should be done regardless of whether kernel pre-emption is enabled or not.</span></font><u></u><u></u></p>
<p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></font><u></u><u></u></p><p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Jeff Haran</span></font><u></u><u></u></p>
<p class="MsoNormal"><font color="#1f497d" face="Calibri"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></font><u></u><u></u></p><div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal"><b><font face="Tahoma"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";font-weight:bold">From:</span></font></b><font face="Tahoma"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:kernelnewbies-bounces@kernelnewbies.org" target="_blank">kernelnewbies-bounces@kernelnewbies.org</a> [mailto:<a href="mailto:kernelnewbies-bounces@kernelnewbies.org" target="_blank">kernelnewbies-bounces@kernelnewbies.org</a>] <b><span style="font-weight:bold">On Behalf Of </span></b>Gaurav Jain<br>
<b><span style="font-weight:bold">Sent:</span></b> Friday, February 08, 2013 9:09 AM<br><b><span style="font-weight:bold">To:</span></b> Kernel Newbies<br><b><span style="font-weight:bold">Subject:</span></b> Kernel code interrupted by Timer</span></font><u></u><u></u></p>
</div></div><div><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">What happens if the kernel executing in some process context (let's say executing a time-consuming syscall) gets interrupted by the Timer - which is apparently allowed in 2.6 onwards kernels.<u></u><u></u></span></font></p>
<div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p></div><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">My understanding is that once the interrupt handler is done executing, we should switch back to where the kernel code was executing. Specifically, the interrupt handler for the Timer interrupt should not schedule some other task since that might leave kernel data in an inconsistent state - kernel didn't finish doing whatever it was doing when interrupted. <u></u><u></u></span></font></p>
</div><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p></div><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">So, does the Timer interrupt handler include such a policy for the above case? <u></u><u></u></span></font></p>
</div><div><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p></div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">-- <u></u><u></u></span></font></p>
<div><p class="MsoNormal"><font size="3" face="Arial"><span style="font-size:12.0pt;font-family:"Arial","sans-serif"">Gaurav Jain</span></font><u></u><u></u></p><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div><div><p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p></div></div></div></div></div></div></div></div></div></div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><br><br clear="all"><u></u><u></u></span></font></p><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">-- <u></u><u></u></span></font></p><div><p class="MsoNormal"><font size="3" face="Arial"><span style="font-size:12.0pt;font-family:"Arial","sans-serif"">Gaurav Jain</span></font><u></u><u></u></p>
<div><p class="MsoNormal"><font size="3" face="Arial"><span style="font-size:12.0pt;font-family:"Arial","sans-serif"">Associate Software Engineer</span></font><u></u><u></u></p></div><div><p class="MsoNormal">
<font size="3" face="Arial"><span style="font-size:12.0pt;font-family:"Arial","sans-serif"">VxVM Escalations Team, SAMG</span></font><u></u><u></u></p></div><div><p class="MsoNormal"><font size="3" face="Arial"><span style="font-size:12.0pt;font-family:"Arial","sans-serif"">Symantec Software India Pvt. Ltd.</span></font><u></u><u></u></p>
</div><div><p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p></div></div></div></div></div></div></div></div></div><p class="MsoNormal">
<font size="3" face="Times New Roman"><span style="font-size:12.0pt"><br><br clear="all"><u></u><u></u></span></font></p><div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p>
</div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">-- <u></u><u></u></span></font></p><div><p class="MsoNormal"><font size="3" face="Arial"><span style="font-size:12.0pt;font-family:"Arial","sans-serif"">Gaurav Jain</span></font><u></u><u></u></p>
<div><p class="MsoNormal"><font size="3" face="Arial"><span style="font-size:12.0pt;font-family:"Arial","sans-serif"">Associate Software Engineer</span></font><u></u><u></u></p></div><div><p class="MsoNormal">
<font size="3" face="Arial"><span style="font-size:12.0pt;font-family:"Arial","sans-serif"">VxVM Escalations Team, SAMG</span></font><u></u><u></u></p></div><div><p class="MsoNormal"><font size="3" face="Arial"><span style="font-size:12.0pt;font-family:"Arial","sans-serif"">Symantec Software India Pvt. Ltd.</span></font><u></u><u></u></p>
</div><div><p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p></div></div></div></div></div></div></div></div></blockquote>
</div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><font face="arial, helvetica, sans-serif">Gaurav Jain</font><div><font face="arial, helvetica, sans-serif">Associate Software Engineer</font></div><div><font face="arial, helvetica, sans-serif">VxVM Escalations Team, SAMG<br>
</font></div><div><font face="arial, helvetica, sans-serif">Symantec Software India Pvt. Ltd.</font></div><div><font face="arial, helvetica, sans-serif"><br><br></font></div></div>
</div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><font face="arial, helvetica, sans-serif">Gaurav Jain</font><div><font face="arial, helvetica, sans-serif">Associate Software Engineer</font></div>
<div><font face="arial, helvetica, sans-serif">VxVM Escalations Team, SAMG<br></font></div><div><font face="arial, helvetica, sans-serif">Symantec Software India Pvt. Ltd.</font></div><div><font face="arial, helvetica, sans-serif"><br>
<br></font></div></div>
</div></div>