On Wed, Aug 17, 2016 at 04:17:19PM +0200, Greg KH wrote:
On Wed, Aug 17, 2016 at 02:01:28PM +0000, Nicholas Mc Guire wrote:
On Wed, Aug 17, 2016 at 03:52:16PM +0200, Greg KH wrote:
On Wed, Aug 17, 2016 at 03:25:44PM +0200, Greg KH wrote:
On Wed, Aug 17, 2016 at 12:39:39PM +0000, Nicholas Mc Guire wrote:
Hi !
For a given patch I would like to find out if it impacts a given configuration or not. Now of course one could compile the kernel for the configuration prior to the patch, then apply the patch and recompile to find out if there is an impact but I would be looking for some smarter solution. Checking files only unfortunately will not do it, due to ifdefs and friends so make would detect a change and recompile even if the affeted code area is actualy dropped by the preprocessor.
What Im trying to do is find out is, how many of the e.g. stable fixes of 4.4-4.4.14 would have impacted a given configuration - the whole exercise is intended for some statistical analysis of bugs in linux-stable.
Also, are you going to be analyizing the bugs in the stable trees, or the ones we just happen to fix?
Note, that's not always the same thing :)
what we have been looking at first is the stable fixes for which the bug-commit is known via Fixes: patch. That only a first approximation but correlates very good with the overall stable fix rates. And from the regression analysis of the stable fix rates over versions one then can exstimate the residual bugs if one knows the distribution of the bug survival times - which one again can estimate based on the bug-fixes that have Fixes: tags.
That is all relying on the Fixes: tags, which are not used evenly across the kernel at all. Heck, there are still major subsystems that NEVER mark a single patch for the stable trees, let alone adding Fixes: tags. Same thing goes for most cpu architectures.
Well for the config we studied it was not that bad 4.4 - 4.4.13 stable bug-fix commits total with % with fix Fixes: Fixes commits tag tag in 1643 589 subsys kernel 3.89% 4.75% 43.7% mm 1.82% 2.17% 53.3% block 0.36% 0.84% 83.3%! fs 8.76% 4.92%* 20.1%* net 9.31% 12.56% 48.3% drivers 47.96% 49.23% 36.8% include 6.87% 19.18% 28.3%* arch/x86 4.50% 12.56% 33.7% (Note that the precentages here do not add up to 100% because we just picked out x86 and did not include all subsystems e.g. lib is missing). So fs is significantly below and include a bit - block is hard to say simply because it was only 6 stable fixes of which 5 had Fixes: tags so that sample is too small. Correlating overall stable-fixes distribution over sublevels with stabel-fixes with Fixes: tag gives me an R^2 of 0.76 so that does show that for any trending using Fixes: tags is resonable. As noted we are looking at statistic properties to come up with expected values nothing more.
So be careful about what you are trying to measure, it might just be not what you are assuming it is...
A R^2 of 0.76 does indicate that the commits with Fixes: tags in 4.4 series is quite well representing the overall stable fixes.
Also note that LWN.net already published an article based on the fixes: tags and tracking that in stable releases.
ok will go dig for that - I did not stumble across that yet - actually did check lwn.net for Fixes tag related infos and found some patches noted - specifically Doc patches.
I dont know yet how robust these models will be at the end but from what we have until now I do think we can come up with quite sound predictions for the residual faults in the kernel.
Based on what I know about how stable patches are picked and applied, I think you will find it is totally incorrect. But hey, what do I know? :)
Well if I look at the overall stable fixes developlment - not just those with Fixes: tags I get very clear trends if we look at at stable fixes over sublevels (linear model using gamma-distribution) ver intercept slope p-value DoF AIC 3.2 4.2233783 0.0059133 < 2-16 79 2714.8 3.4 3.9778258 -0.0005657 0.164 * 110 4488 3.10 4.3841885 -0.0085419 < 2-16 98 2147.1 3.12 4.7146752 -0.0014718 0.0413 58 1696.9 3.14 4.6159638 -0.0131122 < 2-16 70 2124.8 3.18 4.671178 -0.006517 7.34-5 34 1881.2 4.1 4.649701 -0.004211 0.09 25 1231.8 4.4 5.049331 -0.039307 7.69-11 12 571.48 So while the confidence levels of some (notable 3.4) is not that exciting the overall trend does look resonably establshied that the slop is turning negative - indicating that the number of stable-fixes of sublevels systematically decreases with sub-lvels, which does indicate a stable development process.
Some early results where presented at ALS in Japan on July 14th but this still needs quite a bit of work.
Have a pointer to that presentation?
They probably are somewher on the ALS site - but I just dropped them to our web-server at http://www.opentech.at/Statistics.pdf and http://www.opentech.at/TechSummary.pdf This is quite a rough summary - so if anyone wants the actual data or R commands used - let me know - no issue with sharing this and having people tell me that Im totally wrong :) thx! hofrat