I am actually looking at some changes that litter the kernel with short code snippets and thus according to papers i have read can result in CPU hits of around 48% when applied is userspace. I am curious how you would best measure the impact of similar modifications (since obviously one isn't always in the kernel code when executing a process). My interest is in testing different approaches of making pointer runtime checks.

My theory would be that perhaps writing some test code that exercises different extremes (calling the syscall api continually to do a range of tasks) to see how much slower the code would be between the safer version and the old version. This however might not reflect real world performance for I/O (sockets) such as web servers that spend less time in the kernel. I have seen a paper that benchmarked kvm against xen server but I haven't seen any kernel space papers measuring degradations in overall system performance when adding safety checks(perhaps redundant sometimes) into the kernel.



On Tue, Oct 16, 2018 at 1:08 AM <valdis.kletnieks@vt.edu> wrote:
On Mon, 15 Oct 2018 23:42:03 +0800, Carter Cheng said:

> I was wondering what are some good ways to assess the performance impact of
> kernel modifications. Are there some papers in the literature where this is
> done? Does one need to differentiate between CPU bound and different types
> of I/O bound processes etc?

That is *so* totally dependent on exactly what the modification is, that
there's no right answer here.

The things you will want to measure for a new TCP flow control module (to
measure the difference between, say, cubic and new_reno and fq_codel and
your new module) will be *totally* different from changes to an LSM, which again
will be different from an overhaul of a disk I/O scheduler.

And then, the environment matters as well.  The performance metrics that I care
about on my laptop (which is used as a desktop replacement) are "can I do a
kernel build and my desktop environment still work well" type things.  But the
numbers I care about on the machines I maintain across the hall in the data
center are different - those are disk storage, backup, and archive - so I'm
willing to burn a lot of CPU in both kernel and userspace if it gets me more
IOPs and throughput - important when you have 500+ million files in a single
petabyte-plus file system.  Meanwhile, the guys a few cubicles down are doing
HPC, which means they want as little kernel CPU usage as possible because that
gets in the way of user computations.

And sometimes, it doesn't matter in the slightest what the performance impact is,
because the change is required for correctness - running incorrect code faster is
still running incorrect code.  See the recent Spectre patches for an example.