How do _you_ read the linux source?
The following question gets asked a lot, "I know C, but reading the kernel source is hard, what should I do?" and the common response is "ctags." It's a lot like asking "how can I build a house?" and receiving the response "screwdriver." There is obviously more to it then learning C and installing ctags. As a newbie myself, I recently had to overcome this problem, Here's what I did: First, I asked myself 'who understands the linux kernel?'. Linus was the obvious answer, so I hunted down anything that he wrote. I found that he had an autobiography, so I picked up a copy and read it. In it, I learned about the foundational 'open, close, read, write, fork, exec' system calls. Second, I decided to try to write my own kernel. In writing it, I learned how central the filesystem was to unix, I learned the significance of interrupts, I wrote a few drivers, and I ran into (and sometimes got past) a few nasty deadlocks. Third, I subscribed to the mailing lists and begun to read through them. I got caught up on the way things are currently done, and I found and read the way linux approached the problems that I had already become familar with. Ok, lets take that and break it down; First; Find if someone has done this before. If they have, find out how they did it. Second; If you can, try praticing it or solving said problem yourself. Third; Now that you have the foundations, figure out how its currently being done. Now for the important part. It's an algorithmn. See: if(someone has done this before) figure out how they did it; if(we can try) try to solve it ourselves; figure out how it's currently being done; And that's how I usually solve a problem. _However_, I didn't come up with that algorithm myself. I had a programmer friend who was very skilled. He always seemed to do what others thought to be impossible. One day I asked him how he did it, and he said "well, I look if some one has done it before, then I try it, then I look at how people do it now." I imitated this, and it lead me to become a much better programmer. My point is this, I learned how to do things by learning the thought process of someone who could. I'm not saying that that example is the best way, or even a good one. I'm saying that learning the way others think and approach problems is key to success. The problem a lot of newbies are having is in 'separating the trunk from the leaves.' So my question is this: Experienced kernel developers, how do _you_ read source code? How do you separate the trunk from the leaves? What do you do when you read code you're not familiar with? How do you learn? What's your algorithm? I mean obviously it might change on the context, or you might have more then one way. But still, any peek into the way you've learned to approach problems is incredibly valuable. Anyways, thanks for your time.
On 2015-04-19 09:57 PM, r00nk@simplecpu.com wrote:
The following question gets asked a lot, "I know C, but reading the kernel source is hard, what should I do?" and the common response is "ctags." It's a lot like asking "how can I build a house?" and receiving the response "screwdriver." There is obviously more to it then learning C and installing ctags. As a newbie myself, I recently had to overcome this problem, Here's what I did:
First, I asked myself 'who understands the linux kernel?'. Linus was the obvious answer, so I hunted down anything that he wrote. I found that he had an autobiography, so I picked up a copy and read it. In it, I learned about the foundational 'open, close, read, write, fork, exec' system calls. Second, I decided to try to write my own kernel. In writing it, I learned how central the filesystem was to unix, I learned the significance of interrupts, I wrote a few drivers, and I ran into (and sometimes got past) a few nasty deadlocks. Third, I subscribed to the mailing lists and begun to read through them. I got caught up on the way things are currently done, and I found and read the way linux approached the problems that I had already become familar with.
Ok, lets take that and break it down;
First; Find if someone has done this before. If they have, find out how they did it. Second; If you can, try praticing it or solving said problem yourself. Third; Now that you have the foundations, figure out how its currently being done.
Now for the important part. It's an algorithmn. See:
if(someone has done this before) figure out how they did it; if(we can try) try to solve it ourselves; figure out how it's currently being done;
And that's how I usually solve a problem. _However_, I didn't come up with that algorithm myself. I had a programmer friend who was very skilled. He always seemed to do what others thought to be impossible. One day I asked him how he did it, and he said "well, I look if some one has done it before, then I try it, then I look at how people do it now." I imitated this, and it lead me to become a much better programmer. My point is this, I learned how to do things by learning the thought process of someone who could. I'm not saying that that example is the best way, or even a good one. I'm saying that learning the way others think and approach problems is key to success.
The problem a lot of newbies are having is in 'separating the trunk from the leaves.' So my question is this: Experienced kernel developers, how do _you_ read source code? How do you separate the trunk from the leaves? What do you do when you read code you're not familiar with? How do you learn? What's your algorithm?
I mean obviously it might change on the context, or you might have more then one way. But still, any peek into the way you've learned to approach problems is incredibly valuable.
Anyways, thanks for your time.
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
There were a few things I did when starting to learn the kernel 1. Read Robert Love's Linux Kernel Development, I don't care how much you think you known about the kernel read and trace the actual kernel code with this book!! 2.Read a book on device drivers and the Linux networking stack, I read Linux networking internals for this 3. Build Linux from Scratch, this helps you understand how the system gets build for a distro and how things get connected to the kernel/tool chain either directly or through the c library/ other libraries 4. Read Linux Virtual Memory Management as this helps with understanding the key areas of the virtual memory and memory management subsystems(somewhat outdated in terms of reporting on newer kernel releases but the concepts are explained well) 5.Read The Linux Programming Interface to get used to the system calls/APIs for user space the kernel supports and how it is done(Chapter 5 of Linux Kernel Development also helps with this) 5. Read the Kernel wikis and documentation in the kernel tree related to areas your interested in 6. Find a area or a few your interested in and play around with them(working on this as of late) 7. Learn the best way for you to debug/trace bugs in the kernel, I learned the hard way this only comes with experience. This is not user space where bugs are much easier to trace,basically understand your system calls,threading theory,how to use gdb and memory analysis toolkit,maybe tool chain issues if working with embedded systems. The kernel is very complex in debugging everything from deadlock to registers being written from being wrong on exact hardware can happen. Further more due to the kernel being only a layer above hardware we very easily get bugs that can only be reduplicated on exact hardware.(working on this too) 8. This a add on to 7,learn assembly for tracing bugs as this is very helpful for reading register trace backs of oops/kernel panics if necessary for debugging.(another thing on my kernel learning list) Hope this Helps, Nick
2015-04-20 5:11 GMT+02:00 nick <xerofoify@gmail.com>:
There were a few things I did when starting to learn the kernel 1. Read Robert Love's Linux Kernel Development, I don't care how much you think you known about the kernel read and trace the actual kernel code with this book!! 2.Read a book on device drivers and the Linux networking stack, I read Linux networking internals for this
Thank you Nick, I'm just getting started with linux kernel development and have been looking around for books both more general and specific to networking. One thing in common several books I've found have is that they are based on the 2.6 version of the kernel (or older). Some parts have changed but are entire chapters in the above mentioned books too old to make sense when working on version linux kernel 4.1 and beyond?
On 2015-04-20 01:45 AM, Christoffer Holmstedt wrote:
2015-04-20 5:11 GMT+02:00 nick <xerofoify@gmail.com>:
There were a few things I did when starting to learn the kernel 1. Read Robert Love's Linux Kernel Development, I don't care how much you think you known about the kernel read and trace the actual kernel code with this book!! 2.Read a book on device drivers and the Linux networking stack, I read Linux networking internals for this
Thank you Nick, I'm just getting started with linux kernel development and have been looking around for books both more general and specific to networking. One thing in common several books I've found have is that they are based on the 2.6 version of the kernel (or older). Some parts have changed but are entire chapters in the above mentioned books too old to make sense when working on version linux kernel 4.1 and beyond?
What you need to do is look at the source code and see what has changed this writing of the book. This is what I do when reading kernel programming books or documentation. Further more around way is to read the logs of the networking mailing lists for the kernel to see what has changed recently. Nick
On Mon, 20 Apr 2015 07:45:37 +0200, Christoffer Holmstedt said:
Thank you Nick, I'm just getting started with linux kernel development and have been looking around for books both more general and specific to networking. One thing in common several books I've found have is that they are based on the 2.6 version of the kernel (or older). Some parts have changed but are entire chapters in the above mentioned books too old to make sense when working on version linux kernel 4.1 and beyond?
The details have changed. The basic concepts haven't. In fact, the *basic* concepts change so slowly that reading Bach's book on the SYSV kernel or McKusic & Liefler on the BSD kernel will teach you a lot. In fact, I'll go out on a limb and say that reading how *other* systems have solved the same issues is a Good Thing. Knowing just "Linux solved it this way" is one thing, but knowing "Linux solved it this way, but BSD took a different approach because X, and IBM's VM/SP did Y instead to avoid the issue entirely" is a whole different level of understanding.
On Sun, 19 Apr 2015, nick wrote:
On 2015-04-19 09:57 PM, r00nk@simplecpu.com wrote:
The following question gets asked a lot, "I know C, but reading the kernel source is hard, what should I do?" and the common response is "ctags." It's a lot like asking "how can I build a house?" and receiving the response "screwdriver." There is obviously more to it then learning C and installing ctags. As a newbie myself, I recently had to overcome this problem, Here's what I did:
... snip ...
The problem a lot of newbies are having is in 'separating the trunk from the leaves.' So my question is this: Experienced kernel developers, how do _you_ read source code? How do you separate the trunk from the leaves? What do you do when you read code you're not familiar with? How do you learn? What's your algorithm?
*sigh* ... this is the wrong question, in the same way that asking, "how do i start writing kernel code?" is the wrong question. someone once made the brilliant analogy that asking how to start contributing to the kernel is akin to asking, "i want to write a book ... what should i write about?" if you don't already know what interests you, no one else can help you start writing. in the same way, asking "how do i read kernel code?" is akin to walking into a library and asking, "i want to read something ... where do i start?" if you don't already know what part of the kernel interests you, then you should be asking yourself why you should be reading kernel code in the first place. it seems that there is a regular supply of posters whose opening question is something like, "i'm interested in the kernel ... how do i get started?" i submit that it's not the mandate of this list to answer that question. rather, i submit that it's the mandate of this list to answer questions you have *after* you get started and have shown that you've invested the time to dig in on your own. unless you've demonstrated the interest and ability to do at least a *little* homework and research before asking questions, you're probably not going to get much help here, and you might ask yourself why you're interested in the first place. rday -- ======================================================================== Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday ========================================================================
On Sun, 19 Apr 2015, nick wrote:
On 2015-04-19 09:57 PM, r00nk@simplecpu.com wrote:
The problem a lot of newbies are having is in 'separating the trunk from the leaves.' So my question is this: Experienced kernel developers, how do _you_ read source code? How do you separate the trunk from the leaves? What do you do when you read code you're not familiar with? How do you learn? What's your algorithm?
*sigh* ... this is the wrong question, in the same way that asking, "how do i start writing kernel code?" is the wrong question...
How to start isn't what I'm trying to ask. Like, if this were mountain climbing, I'm not asking how to start climbing. I've gotten to the point where I've developed my own technique to climb, now I'm asking others how they climb, so I can learn/improve my own technique based on theirs.
it seems that there is a regular supply of posters whose opening question is something like, "i'm interested in the kernel ... how do i get started?" i submit that it's not the mandate of this list to answer that question. rather, i submit that it's the mandate of this list to answer questions you have *after* you get started and have shown that you've invested the time to dig in on your own.
Right, I'm not asking how to get started, I'm asking how you (not I) go about digging, to see if your method is better then mine. I have already done a fair amount of digging myself (knock on wood). I think that meta-cognition is a important part of any skill set. I'm trying to learn the way skilled kernel developers think, because learning how skilled people think generally leads into valuable insights for that said skill. In my (admittedly small) personal experience, trying to get an idea of how kernel developers think is difficult. There are a few places that sort of do it (Documentation/CodingStyle, Documentation/*, lkml), but those are a bit more geared to management then skill. I figured rather then trying to parse those and come up with a model that probably isn't true, I'd just directly ask. I dunno. Maybe its not an appropriate question for this mailing list.
On Mon, Apr 20, 2015 at 05:46:57PM -0700, r00nk@simplecpu.com wrote:
I figured rather then trying to parse those and come up with a model that probably isn't true, I'd just directly ask. I dunno. Maybe its not an appropriate question for this mailing list.
I appreciate the question and responses.
Robert P. J. Day wrote:
On Sun, 19 Apr 2015, nick wrote:
On 2015-04-19 09:57 PM, r00nk@simplecpu.com wrote:
The following question gets asked a lot, "I know C, but reading the kernel source is hard, what should I do?" and the common response is "ctags." It's a lot like asking "how can I build a house?" and receiving the response "screwdriver." There is obviously more to it then learning C and installing ctags. As a newbie myself, I recently had to overcome this problem, Here's what I did: ... snip ...
The problem a lot of newbies are having is in 'separating the trunk from the leaves.' So my question is this: Experienced kernel developers, how do _you_ read source code? How do you separate the trunk from the leaves? What do you do when you read code you're not familiar with? How do you learn? What's your algorithm? *sigh* ... this is the wrong question, in the same way that asking, "how do i start writing kernel code?" is the wrong question. someone once made the brilliant analogy that asking how to start contributing to the kernel is akin to asking, "i want to write a book ... what should i write about?" if you don't already know what interests you, no one else can help you start writing.
<snip> Maybe it's the wrong question, but it's sure stimulating a lot of good (as in informative) answers. Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
On Sun, Apr 19, 2015 at 06:57:49PM -0700, r00nk@simplecpu.com wrote:
The problem a lot of newbies are having is in 'separating the trunk from the leaves.' So my question is this: Experienced kernel developers, how do _you_ read source code? How do you separate the trunk from the leaves? What do you do when you read code you're not familiar with? How do you learn? What's your algorithm?
I print out the source code, using a huge font as to take up lots of space, bind it all together, and relax in a bathtub full of warm bubbles and drink wine while reading the code and scribbling comments in the margins with a colored soap shard. After falling asleep due to the wine and warm bath, wake up in a few hours freezing cold, surrounded by soggy pieces of paper with all of the ink washed off. But my brain has absorbed it all and I can then resume coding from where I left off. </sarcasm> I use vgrep[1], that's it, no ctags needed. Reality isn't romantic, there is no magic solution to hard tasks. greg k-h [1] https://github.com/vrothberg/vgrep
On Sun, Apr 19, 2015 at 06:57:49PM -0700, r00nk@simplecpu.com wrote:
The problem a lot of newbies are having is in 'separating the trunk from the leaves.' So my question is this: Experienced kernel developers, how do _you_ read source code? How do you separate the trunk from the leaves? What do you do when you read code you're not familiar with? How do you learn? What's your algorithm?
Maybe it could help to firstly focus on data structures/types rather than functions; and I would discourage to read code like a book, I mean from left to right and from top to bottom. And, take a subsystem/part (even if it's very small) of interest and just focus on it. For instance, I guess there is plenty of documentation on how linux boots up: read it, and search through the source where what you have read is done. compile your own kernel, if you haven't done it yet! HTH
On Tue, Apr 21, 2015 at 02:16:49AM +0200, Milton Krutt wrote:
On Sun, Apr 19, 2015 at 06:57:49PM -0700, r00nk@simplecpu.com wrote:
The problem a lot of newbies are having is in 'separating the trunk from the leaves.' So my question is this: Experienced kernel developers, how do _you_ read source code? How do you separate the trunk from the leaves? What do you do when you read code you're not familiar with? How do you learn? What's your algorithm?
Maybe it could help to firstly focus on data structures/types rather than functions; and I would discourage to read code like a book, I mean from left to right and from top to bottom. And, take a subsystem/part (even if it's very small) of interest and just focus on it. For instance, I guess there is plenty of documentation on how linux boots up: read it, and search through the source where what you have read is done.
Booting is so messed up that it might not be the place to start. Asking how to read the code is a non-sensible question. Its like asking how do you speak French. The answer is, I speak it. You don't know how to speak it? Well staring at incomrehesible code aint helping you any. It requires one to interact with it, and that requires background. So the question is, I just do and if I can't understand it I roll up my sleeves and research, ask, discuss, and work with it. There is no forrest... there is no trees.
compile your own kernel, if you haven't done it yet!
HTH
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 http://www.nylxs.com - Leadership Development in Free Software http://www2.mrbrklyn.com/resources - Unpublished Archive http://www.coinhangout.com - coins! http://www.brooklyn-living.com Being so tracked is for FARM ANIMALS and and extermination camps, but incompatible with living as a free human being. -RI Safir 2013
-----Original Message----- From: kernelnewbies-bounces@kernelnewbies.org [mailto:kernelnewbies- bounces@kernelnewbies.org] On Behalf Of r00nk@simplecpu.com Sent: Sunday, April 19, 2015 6:58 PM To: kernelnewbies@kernelnewbies.org Subject: How do _you_ read the linux source?
The following question gets asked a lot, "I know C, but reading the kernel source is hard, what should I do?" and the common response is "ctags." It's a lot like asking "how can I build a house?" and receiving the response "screwdriver."
I'll put in another plug for Coverity's source code browser here. https://scan.coverity.com/ To use it you need to sign-up for an account, but that is free and it's much better for browsing large projects than ctags, cscope, grep, etc., IMHO. Jeff Haran
On 04/20/2015 12:32 PM, Jeff Haran wrote:
I'll put in another plug for Coverity's source code browser here.
To use it you need to sign-up for an account, but that is free and it's much better for browsing large projects than ctags, cscope, grep, etc., IMHO.
if you want to learn nothing and sit behind a proprietary software wall to access your free software.
participants (11)
-
Christoffer Holmstedt -
Greg KH -
Jeff Haran -
John de la Garza -
Miles Fidelman -
Milton Krutt -
nick -
r00nk＠simplecpu.com -
Robert P. J. Day -
Ruben Safir -
Valdis.Kletnieks＠vt.edu