Determining patch impact on a specific config

[Greg KH]

On Wed, Aug 17, 2016 at 02:01:28PM +0000, Nicholas Mc Guire wrote:
> On Wed, Aug 17, 2016 at 03:52:16PM +0200, Greg KH wrote:
> > On Wed, Aug 17, 2016 at 03:25:44PM +0200, Greg KH wrote:
> > > On Wed, Aug 17, 2016 at 12:39:39PM +0000, Nicholas Mc Guire wrote:
> > > > 
> > > > Hi !
> > > > 
> > > >  For a given patch I would like to find out if it impacts a
> > > >  given configuration or not. Now of course one could compile the
> > > >  kernel for the configuration prior to the patch, then apply the
> > > >  patch and recompile to find out if there is an impact but I would
> > > >  be looking for some smarter solution. Checking files only 
> > > >  unfortunately will not do it, due to ifdefs and friends so make
> > > >  would detect a change and recompile even if the affeted code 
> > > >  area is actualy dropped by the preprocessor.
> > > > 
> > > >  What Im trying to do is find out is, how many of the e.g. stable
> > > >  fixes of 4.4-4.4.14 would have impacted a given configuration - the
> > > >  whole exercise is intended for some statistical analysis of bugs
> > > >  in linux-stable.
> > 
> > Also, are you going to be analyizing the bugs in the stable trees, or
> > the ones we just happen to fix?
> > 
> > Note, that's not always the same thing :)
> >
> what we have been looking at first is the stable fixes
> for which the bug-commit is known via Fixes: patch. That only
> a first approximation but correlates very good with the
> overall stable fix rates. And from the regression analysis
> of the stable fix rates over versions one then can exstimate the
> residual bugs if one knows the distribution of the bug 
> survival times - which one again can estimate based on the
> bug-fixes that have Fixes: tags. 

That is all relying on the Fixes: tags, which are not used evenly across
the kernel at all.  Heck, there are still major subsystems that NEVER
mark a single patch for the stable trees, let alone adding Fixes: tags.
Same thing goes for most cpu architectures.

So be careful about what you are trying to measure, it might just be not
what you are assuming it is...

Also note that LWN.net already published an article based on the fixes:
tags and tracking that in stable releases.

> I dont know yet how robust these models will be at the end
> but from what we have until now I do think we can come up
> with quite sound predictions for the residual faults in the
> kernel.

Based on what I know about how stable patches are picked and applied, I
think you will find it is totally incorrect.  But hey, what do I know?
:)

> Some early results where presented at ALS in Japan on July 14th
> but this still needs quite a bit of work.

Have a pointer to that presentation?

thanks,

greg k-h