Best way to configure Linux kernel for a machine

[Simon]

On Wed, Jan 16, 2013 at 7:17 AM, Shraddha Kamat <sh2008ka at gmail.com> wrote:
> I normally do the kernel configuration on my machine like this -
>
> * copy the distro configuration file  to the kernel dir
> * make menuconfig (answer Y's/N's/M's) Normally keep return key pressed
> for default answers
> * then do the actual kernel compilation
>
>
> Now, I know that this is not a clean way to do the kernel compilation
> (although it has worked for me for thousands of times that I have
> compiled and successfully booted up with the kernel - without any issues
> - whatsoever !)
>
> But this time , I am bent upon coming up with a configuration
> specifically targeted to my machine. What is the best way to do this ?

Hi Shraddha,
   your story is almost identical to mine.  Although, I successfully
moved to the next chapter and I can surely share what I learned and
how I worked.  I do not believe this is the best way, after all I'm
still just a noob.  ;)

  Oh and... I hope the contents of my email can be a sufficient excuse
for its length!  I'm giving my experience, my method and the reasons
behind my choices and moves.  I just hope reading this wont cause a
buffer overflow somewhere in your frontal cortex! ;)

  I started very very long ago with Redhat (before year 2000), then I
moved to slackware and stayed there for a longer while.  All this time
I was using the defaults provided by the distros.  Then, perhaps
around 2008, I tried Slax and became very avid for fine tuning and
customizing everything.  During my research I found the Gentoo
distribution, switched over, loved it and migrated all my computers
over to it.  In the process I copied the ".config" file from my
previous stable install (I think it came from slackware) to all my
machines (at the time, they were all intel, single core).  Soon after,
I added a quad-core, then an old AMD athlon, and later (and last) I
bought a AMD 64bit dual-core laptop.  I used my 32bit distribution and
the same old 32bit kernel on that 64bit machine, until I decided this
whole situation was no longer tenable.  I needed 64bit, I needed
better that 386, I needed SMP, I wanted to get rid of all those device
drivers I will never ever use...  and more...

If I read your post right, this is where you stand now.  So here's what I did:

  The first thing was to start doing the same old way, but now using
git to get kernel updates.  Once I got used to it, the next step was
to setup a new branch in my git repository on each machine and
configured it to update that branch from one of my computer that I
dedicated as the "authoritative" repository.  Then I noticed it
doesn't quite work to take the 32bit config I had and simply switch it
to a 64bit architecture using menuconfig (really, don't even try,
there's lots of options to change, none of which are evident to the
newbie).  So, on each machine I did a "make i386_defconfig" on the
32bit systems and "make x86_64_defconfig" on the 64bit one.  I made a
commit of those .config files on each machine as the starting point of
my new configuration and I started compiling.  Then I rebooted and
gave it a try.

  Of course it didn't work and couldn't even boot up to init!  So on
each machine I checked what the immediate problem was and applied the
most minimal correction to my .config (using menuconfig) and making a
new commit after the change.  Some of the changes could be shared
among some of my machines so I pulled (aka merged) those commits onto
the other systems' config.  Each commit or small group of commits
would represent one feature I needed to add on a machine.  The commit
messages were like "Adding support for USB 1 and 2", then the next
would be "Fixing support for USB block devices (usbkeys)", etc...
This process was long and difficult but I got excellent help on
irc.freenode.net (in ##linux and #gentoo, thanks guys!).  The reason
it was difficult was my lack of knowledge of the complex
interdependencies of various things I needed.  But fortunately, the
needs that I had were very precise, the errors I got were very clear,
therefore my questions were very well stated, and the answers came
easilly and clearly (oh and pretty quickly too, except on Friday and
Saturday nights, hehe).  Within about week, it was done.  Everything
was supported on each machine (and nothing more), everything was very
stable (the video card was the only thing that kept giving me
trouble).

  This led to a totally different configuration on each machine, with
a single way of handling them all in the exact same way (with git).
The kernel bzImage shrunk to around 50% what it was before (I dropped
support for so many unused network cards and other stuff)!  I also
decided to make all these "features" built into the kernel (as opposed
to modules).  And left the very minimum as module (actually only 1 or
2 modules I seldom used and were not needed at boot time).  The result
was a kernel that no longer required an initrd to boot, which made the
whole process even simpler.

Keeping it up-to-date:

  You didn't ask, but I'll spare you writing that question too!  I'm a
lazy guy and I don't want to take an hour to update all my computers,
so I chose to "lock" my kernel into a specific major version and only
update when a new minor version comes out.  I am currently running
3.2.x and I am quite happy.

  To update a minor version, all I need to do is (well, after updating
the git repository), "make oldconfig" or "make silentoldconfig",
there's no questions asked using either one (or I can't recall having
had any).  That's because there should be no new features that you may
want or not.  It's mostly bug fixes, performance fixes, etc.  So, now
that I'm on kernel 3.2.x, every update makes it more and more stable.
Ok, after that "make oldconfig", I do a "make prepare", then I check
the output of "make kernelrelease" to see if has the "-dirty" suffix.
If it does, it means that I made some change to my kernel config
without commiting that change.  Sometimes it's ok, there's something I
want to try, but sooner or later, I'll commit the change and get a
clean kernelrelease.  In all cases (except those tries), I want a
clean commit before I compile.  This is true for minor and major
version upgrades.

  Then, I do launch "make -jX" where X is double the number of cores I
have or when I'm really lazy, I put double the maximum number of cores
one of my PC has.  I read many tips and tricks for this value.  The
most common one was to use the number of cores you have plus one.
However, if I do that, I can see that CPU utilisation isn't Always
100% and on my machine the bottleneck really is the CPU.  My guess is
that this value cannot be optimal over such a long build and would
benefit from varying over the build, depending on how much it reads
and writes, etc... but that's a dream in truecolor, oh and it's still
just a noob's guess!  Anyway, the value is quite irrelevant in my
case, since I usually start that command before leaving for work or
before going to sleep.

  At the end, I issue a "make install", which calls a script in
/root/bin/installkernel (the build commands are issued as a one line
cmd as root).  This script calls "make firmware_install" and "make
modules_install" it copies the resulting bzImage to /boot, renames it
by appending the kernelrelease to the filename, and modify a symlink
to point to it and another symlink to point to the previous kernel.  I
use grub for booting and in grub.conf, I load the symlinks (called
bzImage-Latest and -Previous, I sometimes manually create -Stable and
more rarely -Experimental).  Grub works like a charm with these
symlinks.  Oh and the last thing my ~/installkernel does is to remind
me to reboot!  ;)

  Sometimes I get bored or I'm in need of some adventure and I do a
major version upgrade.  I say "adventure", but in reality it's quite
simple.  The process is similar to what I do above, except that I
always move from one major version to the very next, without skipping
any.  I skipped some in the past and this leads to _a_lot_ of
questions asked with "make oldconfig" and I had some trouble in the
past.  If I want to upgrade to the latest, I still do the move to the
next version, got through the whole process, but before compiling, I
move again to the next version, again and again.  I've never had any
trouble that way.  It take a bit more time, but I do that once every
3-6 months anyway!  When doing the "make oldconfig" I get asked if I
want some new features.  My idea is that my hardware hasn't changed,
neither have my needs, and everything is stable and working fine, so
why would I need anything new?  I do the default answer for all
questions asked.  I then run "make prepare && make kernelrelease",
verify the output and then I make my commit before compiling.

  In case of trouble with the new kernel, I simply reboot into my
"bzImage-Previous" and I drop the very last update.  My approach with
these problems is to wait for yet another (usually minor) update to
arrive and upgrade to it.  As far as I can remember, since I got my
first "stable kernels", the only problems I encountered had to do with
the video card.  I tried using prioprietary drivers and all kinds of
things to get it working top shape.  In my experience, this process is
a different adventure with every graphic card out there and I still
don't understand all I did to get it to work.  The list of commits I
made is long and not as clean as for every other features;  lots of
trial and errors.  I finally got it though!  ;)

  As you can see, the whole process is very standardised and very
"script-friendly".  Indeed, all I said above concerning the
maintenance is written in one script that I run before work/sleep on
every machines (it also takes care of updating the git repositories)
and all I ever need to do after is a reboot.  I haven't seen any
issues since I worked that way.  I would gladly publish that script,
but I won't.  My reasons are that it's written in PHP instead of
something more independent like shell scripts, and I modified and
over-modified it over time that it has become very very dirty.
Another reason, is I have gotten more experience since I first wrote
it and I'm thinking on re-writing it from scratch.

  Oh, one last thing.  You noticed I never ever wrote anything about
"make clean" or "make mrproper".  That's because I never use them.
Instead, I use git to do these operations.  In git, I created a branch
I called "_clean" with only one commit that contains nothing.
Checking out that branch removes everything from the working
directory.  After doing this, I checkout the kernel release that I
want to build, then I pull the latest version of my config.  I do this
for every update I do.  Once the whole update is finished and I'm
happy, I usually reclaim some disk space by checking out _clean, then
the version and my config and I only run "make prepare && make
modules_prepare" and that's it!

  I'm really sorry for the length of this email.  I should never have
studied litterature in college! ;P

  I sincerely hope it helped, or at least gave you some good ideas to
invent your own methodology.  In any case, I clearly recommend using a
versionning software to keep track of your changes to the .config
file.  That's how I finally started learning.

Good luck and have fun!
  Simon