Assorted newb questions

[Jaret Flores]

Hello All,
I'm new to the mailing list and have some introductory questions.
After starting the Eudyptula challenge, I began to read 'Linux Kernel
Development - 2nd ed.' and 'Linux Device Drivers - 3rd ed.'.  I think
I am getting a feel for the data structures and a general picture of
how the kernel works, but I have some questions that might help me
connect some of the dots.  I'll try to explain what I think is going
on and perhaps you all can tell me what is really going on.  Please
feel free to provide references rather than re-explaining everything
if the references already do a good job!  Here I go...sorry in advance
for the length!

==== Kernel Questions ====

When a user-space process is `clone()`d, it is provided a `struct
task_struct`, call it `my_task`, from a slab cache that already has
`struct task_struct`s allocated but free to be assigned.  If
`CLONE_VM` is is passed to `clone()`, then `my_task` inherits the
parent task's address space (which means `my_task` is a thread),
otherwise it is provided a `struct mm_struct` from another slab cache
and the object is referenced in `my_task->mm`.

1) Assuming `my_task` does not inherit its parent's address space,
what does `my_task`s address space have mapped initially?  E.g. what
entries are in `pgd`, `pmd` and `pte`?  Does `my_task->mm` inherit the
kernel's page tables?  In particular, does `my_task` "see" 4gb of
memory where the first 896mb (ZONE_NORMAL) are reserved for the kernel
plus some extra VMA's in ZONE_HIGHMEM containing data specific to the
process?

2) This is slightly connected to question (1).  Suppose that `my_task`
needs to open a file that may or may not be in main memory.  Given
`/path/to/some_file.txt`, the VFS traverses the dentries `/`, `path`,
`to` and `some_file.txt` to check that the path is valid by
sequentially doing something along these lines (of course, something
like this would only happen if the dentry `some_file.txt` were not
already in the dentry cache):

e.g., for `/`:
i) Somehow obtain the dentry structure for `/`.
ii) Look up the inode via `dentry->d_inode` and use this to determine
the address of `/`'s data (on disk?...at least initially)
iii) The actual data associated to `/` contains a list of the files it
contains.  Therefore, this data must be mapped into main
memory...first check if the data is in a page in the `pagecache`,
otherwise it must be read from disk first.
iv) Check that data contains an entry for `path` (the next dentry).
v) If `path` is an entry in `/`s data, repeat at (i) for `path`.

Repeat these steps to determine of the path is invalid or until the
data for the file `some_file.txt` is memory mapped.  Firstly, does
this "algorithm" reflect how the file data is found/moved to main
memory?  Second, even if (i) only happens once at boot, how does the
kernel initially obtain a dentry for the root directory/file?...or for
any file?  Are these created on-the-fly or are they stored on disk?
The reason I say that this is connected to (1) is that the data needed
to run `my_task`, e.g. the code itself, must be loaded into main
memory before being mapped into the process's address space when the
process is created.  I assumed when `my_task` is created, the code is
mapped into memory using the same sequence of steps...after all, the
process itself is started by looking up and executing a file.

3)  This has to do with how data is actually read from disk.  A disk
can only read data in sector sized chunks and the kernel only reads
memory in block sized chunks.  Therefore, the block size must be a
multiple of the sector size (which is fixed by the hardware) and there
is an additional limitation that a block must be smaller than a page.
Suppose sectors are 512b and blocks are 1kb (which is below the upper
bound for both 32/64bit x86 processors).  Now, suppose `some_file.txt`
is a file that is, say, 1.2kb in size and is not currently in main
memory.  Once the location of `some_file.txt`s data is found on disk
(via 2 above), it must be read into main memory in block sized chunks.

i) Can the hard disk only start reading at sector boundaries?  i.e.
the chunks 0-1kb, .5kb-1.5kb, etc...  Or perhaps only at block
boundaries (as enforced by the kernel)?  0-1kb, 1-2kb, 2-3kb, etc...
Or can it start reading at any byte on the disk as long as it reads a
contiguous block sized chunk?  i.e. (some number of bytes into
disk)-(some number of bytes into disk + 1kb)
ii)  Regardless of where the disk can start reading data, the size of
`some_file.txt`s data requires that at least 2kb must be read from
disk.  Depending on the offset of the data inside a sector/block and
where the disk can start reading, `some_file.txt`s data will lie
inside of a 2kb block surrounded by .8kb of (possibly) random data
(possibly on both sides).  This data would then be moved to a page in
memory and the address of the page as well as the offset of
`some_file.txt`s data inside the page would then be available to the
process for file operations.  Is that .8kb of space, now in main
memory, wasted?  Or can a future block of data read from disk write
over the "dead space" surrounding `some_file.txt`'s data?

==== Other ====

1)  It seems like most of the material (outside of the kernel itself)
corresponds to version 2.6.  Why is this?  Has nothing "significant"
changed with the core structure of the kernel since then?  Or is it
just that no one has decided to write a book about a newer version?

2) In some header files I see comments that look like:

/**
  * struct some_struct - description
  * @property1: description
  * ...
  * @propertyn: description
  */

This style looks very similar to http://usejsdoc.org/ which is used to
build html documentation from code comments.  Does the kernel use a
tool like this?  i.e. is there html documentation generated from the
code comments?

====

Thanks in advance for any info you can provide!

-Jaret