On Thu, Jul 24, 2014 at 12:44 PM, Lucas Tanure <tanure@linux.com> wrote:

Could start cleaning a driver ? http://kernelnewbies.org/OPWfirstpatch#head-4abafc61af197fb6e3d6cda623b00bdd...

The job here is about how to send patchs and not doing a awesome code.

-- Lucas Tanure +55 (19) 988176559

On Thu, Jul 24, 2014 at 1:38 PM, Nick Krause <xerofoify@gmail.com> wrote:

...

Is there any work for a kernel newbie that you guys known of? Cheers Nick

_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

I sent in some patches to Greg on the linux-next tree for staging. Cheers Nick

Hi Yi, No. You can't. But this page http://kernelnewbies.org/OPWfirstpatch is a awesome start point for anyone, including myself. Do a clean up driver as a start point. After that try to find some spot in linux to help/maintain, but be sure that understand a lot of that point before sending anything to kernel main list. Thanks! -- Lucas Tanure +55 (19) 988176559 On Sat, Jul 26, 2014 at 6:13 PM, Yi Li <lovelylich@gmail.com> wrote:

于 2014/7/25 0:44, Lucas Tanure 写道:

Could start cleaning a driver ? http://kernelnewbies.org/OPWfirstpatch#head-4abafc61af197fb6e3d6cda623b00bdd...

The job here is about how to send patchs and not doing a awesome code.

Hi Lucas, Because you have mentioned OPW, which is short for "Outstreach Program for Women". but I am a *boy*, could I do some works on this program ?

regards, Yi

-- Lucas Tanure +55 (19) 988176559

On Thu, Jul 24, 2014 at 1:38 PM, Nick Krause <xerofoify@gmail.com> wrote:

...

Is there any work for a kernel newbie that you guys known of? Cheers Nick

_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

On Thu, Jul 24, 2014 at 12:51 PM, Andev <debiandev@gmail.com> wrote:

On Thu, Jul 24, 2014 at 12:38 PM, Nick Krause <xerofoify@gmail.com> wrote:

...

Is there any work for a kernel newbie that you guys known of? Cheers Nick

Your first task will be reading and _understanding_ the following:

Greg Kroah-Hartman, "How to piss off a kernel subsystem maintainer". <http://www.kroah.com/log/linux/maintainer.html> <http://www.kroah.com/log/linux/maintainer-02.html> <http://www.kroah.com/log/linux/maintainer-03.html> <http://www.kroah.com/log/linux/maintainer-04.html> <http://www.kroah.com/log/linux/maintainer-05.html>

from Documentation/SubmittingPatches.

-- Andev

Yes, I didn't build test and that is a huge mistake. Cheers Nick

How to make environment like (build and test) for working on kernel patch, what are the resources should be available for that. thanks. On Fri, Jul 25, 2014 at 7:53 AM, Nick Krause <xerofoify@gmail.com> wrote:

On Thu, Jul 24, 2014 at 1:10 PM, Nick Krause <xerofoify@gmail.com> wrote:

...

On Thu, Jul 24, 2014 at 12:51 PM, Andev <debiandev@gmail.com> wrote:

...

On Thu, Jul 24, 2014 at 12:38 PM, Nick Krause <xerofoify@gmail.com> wrote:

...

Is there any work for a kernel newbie that you guys known of? Cheers Nick

Your first task will be reading and _understanding_ the following:

Greg Kroah-Hartman, "How to piss off a kernel subsystem maintainer". <http://www.kroah.com/log/linux/maintainer.html> <http://www.kroah.com/log/linux/maintainer-02.html> <http://www.kroah.com/log/linux/maintainer-03.html> <http://www.kroah.com/log/linux/maintainer-04.html> <http://www.kroah.com/log/linux/maintainer-05.html>

from Documentation/SubmittingPatches.

-- Andev

Yes, I didn't build test and that is a huge mistake. Cheers Nick

Andev, I having been doing build tests and checkpatch in staging for the last month. It doesn't seem like it's worth my time as so my other people are doing it. I want an interesting project one that is challenging and rewarding :). Nick

_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

On Fri, 25 Jul 2014, Lucas Tanure wrote:

Hi,  Watch : http://www.youtube.com/watch?v=LLBrBBImJt4

Read :  http://kernelnewbies.org/OPWfirstpatch http://kernelnewbies.org/KernelBuild  http://lwn.net/Articles/571980/

Goal : Clone, build and run linux-next. 

After that you can look for how to clean up a staging driver.

if i may (since i'm sure i antagonized my share of kernel gurus once upon a time as i was getting my feet wet), start with cleaning up the docs. read up on a subsystem, peruse the code, check the comments, see if there's accompanying explanation in the Documentation/ directory, and either correct obvious mistakes or improve what's there. the obvious advantage to this is that you can't break stuff by working on documentation. and everyone wants better documentation. rday -- ======================================================================== Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday ========================================================================

On Fri, 25 Jul 2014, Nick Krause wrote:

I have already watched this and I have no idea on how to write the docs as my knowledge of kernel subsystems is rather limited.

um ... i think we've identified your problem, then. rday -- ======================================================================== Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday ========================================================================

On Thu, 24 Jul 2014 22:23:00 -0400, Nick Krause said:

I having been doing build tests and checkpatch in staging for the last month. It doesn't seem like it's worth my time as so my other people are doing it. I want an interesting project one that is challenging and rewarding :).

OK. I'm gonna be blunt. That's about as dumb as people who post "What should my next blog post be about?", and for exactly the same reasons. If you aren't *internally* motivated by a specific goal/interest, we're not going to be able to help. Personally, I'm motivated to do testing and debugging because I have a quarter acre of Linux boxes across the hall that need stable kernels, and more users over on campus. And every single bug I catch in linux-next or Fedora Rawhide and file a *good* bug report on is one less bug that one of my users can trip over and file a poor bug report about. ;) Other people are motivated by the fact their employer is paying them to write a Foobar 2890 driver. Some people find filesystems intellectually interesting, and others worry far too much about security. And so on. But if nothing like that is jumping out at you, maybe you should go look around and see if there's something in userspace that *does* jump out at you.

On Jul 25 2014, Nick Krause wrote:

...

But if nothing like that is jumping out at you, maybe you should go look around and see if there's something in userspace that *does* jump out at you.

I am interested in file systems and will be working on brtfs and ext4. Cheers Nick

If you want an annoying problem, explain and/or fix directory performance on ext4. I've got a server where an ls of a directory took 5 seconds, according to "time", even though it only has 295 entries at present. It's a rather large directory, 4 times the size of a normal directory in the opinion of ls -ld, and filefrag reports it has 7 extents. I'm also getting anecdotal reports of rename() taking unreasonable amounts of time, probably with at least one of the directories involved being in a similar state. (They would have been created and populated by the same software.) Probably the directory had a large number of files and/or subdirectories some time in the past, and this is "expected behaviour". Expected or not, it seems to me that it's about time that linux handles directories with something closer to the efficiency with which it handles files ;-) -- Arlie (Arlie Stephens arlie@worldash.org)

On Fri, 25 Jul 2014 15:23:42 -0700, Arlie Stephens said:

If you want an annoying problem, explain and/or fix directory performance on ext4. I've got a server where an ls of a directory took 5 seconds, according to "time", even though it only has 295 entries at present.

I don't suppose you could get a trace of where that ls is spending its time with the kernel's trace facilities, or even just getting a stack trace of where that ls is in the kernel? I'll go out on a limb and ask if a *second* ls of the same directory runs quickly because it's now cache-hot. If so, I'd start looking at whether there's large amounts of *other* disk activity going on, and the reads of the directory are getting hung in the I/O queue behind other disk read/writes. Also, are you doing an 'ls' (which just requires reading the name/inode# pairs), or an 'ls -l' whihc in addition requires a stat() call to read in the inode itself)? That makes a lot of difference. Cache-cold on my laptop, and a *huge* Mail/linux-kernel directory (yes, it really *is* an 11M directory, it's got a half-million entries in it): [~] echo 3 >| /proc/sys/vm/drop_caches [~] cd Mail [~/Mail] time ls linux-kernel/ | wc -l 478401 real 0m2.387s user 0m0.500s sys 0m0.433s [~/Mail] ls -ld linux-kernel/ drwxr-xr-x. 2 valdis valdis 11005952 Jul 25 19:30 linux-kernel/ [~/Mail] time ls -l linux-kernel/ | wc -l 478402 real 0m32.915s user 0m2.483s sys 0m20.787s

On Jul 25 2014, Valdis.Kletnieks@vt.edu wrote:

On Fri, 25 Jul 2014 15:23:42 -0700, Arlie Stephens said:

...

If you want an annoying problem, explain and/or fix directory performance on ext4. I've got a server where an ls of a directory took 5 seconds, according to "time", even though it only has 295 entries at present.

I don't suppose you could get a trace of where that ls is spending its time with the kernel's trace facilities, or even just getting a stack trace of where that ls is in the kernel?

These are all very good questions. To my amazement, I found that no one had yet fixed the problem by deleting and recreating the directory, and I do have sudo access. This time it was only 4 seconds... real 0m3.992s user 0m0.005s sys 0m0.052s

I'll go out on a limb and ask if a *second* ls of the same directory runs quickly because it's now cache-hot. If so, I'd start looking at whether there's large amounts of *other* disk activity going on, and the reads of the directory are getting hung in the I/O queue behind other disk read/writes.

Sure enough, the cache saved me on a second read - real 0m0.010s user 0m0.000s sys 0m0.010s

Also, are you doing an 'ls' (which just requires reading the name/inode# pairs), or an 'ls -l' whihc in addition requires a stat() call to read in the inode itself)? That makes a lot of difference. Cache-cold on my laptop, and a *huge* Mail/linux-kernel directory (yes, it really *is* an 11M directory, it's got a half-million entries in it):

I was doing a vanilla ls. So was the original reporter, unless he has some really strange aliases. I'm afraid I'll be rather unpopular if I drop the caches on the system in question, creating a burst of poor performance, so my best bet is probably to see what I can do with ftrace on Monday, or perhaps partway through the weekend. There is normally a fair amount of disk activity going on - much of it writes. So I can expect cached blocks to age out in a reasonable time.

[~] echo 3 >| /proc/sys/vm/drop_caches [~] cd Mail [~/Mail] time ls linux-kernel/ | wc -l 478401

real 0m2.387s user 0m0.500s sys 0m0.433s [~/Mail] ls -ld linux-kernel/ drwxr-xr-x. 2 valdis valdis 11005952 Jul 25 19:30 linux-kernel/

Compared to your directory, mine is microscopic $ ls -ld xxxx drwxr-xr-x 2 yyy yyy 36864 Jul 25 12:19 xxxx

[~/Mail] time ls -l linux-kernel/ | wc -l 478402

real 0m32.915s user 0m2.483s sys 0m20.787s

-- Arlie (Arlie Stephens arlie@worldash.org)

On Fri, Jul 25, 2014 at 9:22 PM, Nick Krause <xerofoify@gmail.com> wrote:

On Fri, Jul 25, 2014 at 9:08 PM, Arlie Stephens <arlie@worldash.org> wrote:

...

On Jul 25 2014, Valdis.Kletnieks@vt.edu wrote:

...

On Fri, 25 Jul 2014 15:23:42 -0700, Arlie Stephens said:

...

If you want an annoying problem, explain and/or fix directory performance on ext4. I've got a server where an ls of a directory took 5 seconds, according to "time", even though it only has 295 entries at present.

I don't suppose you could get a trace of where that ls is spending its time with the kernel's trace facilities, or even just getting a stack trace of where that ls is in the kernel?

These are all very good questions.

To my amazement, I found that no one had yet fixed the problem by deleting and recreating the directory, and I do have sudo access. This time it was only 4 seconds... real 0m3.992s user 0m0.005s sys 0m0.052s

...

I'll go out on a limb and ask if a *second* ls of the same directory runs quickly because it's now cache-hot. If so, I'd start looking at whether there's large amounts of *other* disk activity going on, and the reads of the directory are getting hung in the I/O queue behind other disk read/writes.

Sure enough, the cache saved me on a second read - real 0m0.010s user 0m0.000s sys 0m0.010s

...

Also, are you doing an 'ls' (which just requires reading the name/inode# pairs), or an 'ls -l' whihc in addition requires a stat() call to read in the inode itself)? That makes a lot of difference. Cache-cold on my laptop, and a *huge* Mail/linux-kernel directory (yes, it really *is* an 11M directory, it's got a half-million entries in it):

I was doing a vanilla ls. So was the original reporter, unless he has some really strange aliases.

I'm afraid I'll be rather unpopular if I drop the caches on the system in question, creating a burst of poor performance, so my best bet is probably to see what I can do with ftrace on Monday, or perhaps partway through the weekend.

There is normally a fair amount of disk activity going on - much of it writes. So I can expect cached blocks to age out in a reasonable time.

...

[~] echo 3 >| /proc/sys/vm/drop_caches [~] cd Mail [~/Mail] time ls linux-kernel/ | wc -l 478401

real 0m2.387s user 0m0.500s sys 0m0.433s [~/Mail] ls -ld linux-kernel/ drwxr-xr-x. 2 valdis valdis 11005952 Jul 25 19:30 linux-kernel/

Compared to your directory, mine is microscopic

$ ls -ld xxxx drwxr-xr-x 2 yyy yyy 36864 Jul 25 12:19 xxxx

...

[~/Mail] time ls -l linux-kernel/ | wc -l 478402

real 0m32.915s user 0m2.483s sys 0m20.787s

-- Arlie

(Arlie Stephens arlie@worldash.org)

Arlie, Whenever you get around to it is fine. Just send me a log. Cheers Nick

Arlie, just a friendly reminder can you try to send me the log this week. Regards Nick

On Wed, 30 Jul 2014 10:38:13 -0700, Arlie Stephens said:

On the good side, Vladis' observations of his mail directory have been a great help.

And remember, that's on a single laptop-class hard drive, no fancy raid or anything. (Though it *is* a hybrid, with 32G of flash cache on the front end). You throw some *real* hardware at it, it of course would go even faster.

Hi Nick, [Context - directory ls taking 4-15 seconds; directory large, with long filenames, but nowhere near as huge as Valdis' mail directory.] I've now discovered a really bizarre pattern, and I'm inclined to stop blaming the file system until some clarity develops. If I ever get it to the point where I can produce a high quality bug report - with or without patch - I will do so - but what I have now is anything but clear and high quality. On Jul 30 2014, Nick Krause wrote:

On Wed, Jul 30, 2014 at 3:48 PM, <Valdis.Kletnieks@vt.edu> wrote:

...

On Wed, 30 Jul 2014 10:38:13 -0700, Arlie Stephens said:

...

On the good side, Vladis' observations of his mail directory have been a great help.

And remember, that's on a single laptop-class hard drive, no fancy raid or anything. (Though it *is* a hybrid, with 32G of flash cache on the front end).

You throw some *real* hardware at it, it of course would go even faster.

Just send me the logs and anything else you think may help me. Please note cc the ext4 mailing list as this will also let the other ext4 developers and maintainers known about your problem. Cheers Nick

I'm now in a state of complete bafflement. It turns out we have a whole collection of misbehaving directories, making this testable without waiting for caches to clear. I have a couple of strace's of fast ls's, and a function ftrace that captured about half of a 7 second ls. (The latter is huge, and probably not suitable for posting.) I also have a really bizarre observation, the kind that makes you wonder whether you are actually dreaming. It appears that the misbehaviour is strongly influenced by the choice of "time" function. The problem only occurs when using the shell built-in. /usr/bin/time always produces a fast response. Stranger still - flat out impossible, I'd have said before seeing it - a "fast" ls, run with /usr/bin/time can be followed *immediately* by a slow "ls", run with bash' time. It's as if the first one doesn't warm the cache, which is completely absurd - except I've been able to make this happen 5 times in a row, first with strace and then without. # with /usr/bin/time the ls is fast $ time -p ls bad_dir ... real 0.21 user 0.00 sys 0.00 # with the builtin time, right *after* the strace run, the time can be # horrible. $ time -p ls bad_dir ... real 5.60 user 0.00 sys 0.17 # run it again, and the directory is in cache as expected. $ time -p ls bad_dir ... real 0.11 user 0.00 sys 0.02 This is not an artefact of one or other time reporting incorrectly - I'm noticing a long pause before output occurs, but only on the middle test of the three. I can't imagine any sane way for this to be happening, short of coincidence or user error - and I've now seen this sequence 5 times in a row, on 5 different directories created and populated by the same app. (Three times with strace, twice without.) -- Arlie (Arlie Stephens arlie@worldash.org)

On Thu, Jul 31, 2014 at 7:41 PM, Henry Hallam <henry@pericynthion.org> wrote:

Try redirecting the ls output to /dev/null or a file, thus disabling its color highlighting and thus removing a bunch of syscalls. See if it's now the same no matter what choice of 'time'.

On Thu, Jul 31, 2014 at 4:36 PM, Arlie Stephens <arlie@worldash.org> wrote:

...

Hi Nick,

[Context - directory ls taking 4-15 seconds; directory large, with long filenames, but nowhere near as huge as Valdis' mail directory.]

I've now discovered a really bizarre pattern, and I'm inclined to stop blaming the file system until some clarity develops. If I ever get it to the point where I can produce a high quality bug report - with or without patch - I will do so - but what I have now is anything but clear and high quality.

On Jul 30 2014, Nick Krause wrote:

...

On Wed, Jul 30, 2014 at 3:48 PM, <Valdis.Kletnieks@vt.edu> wrote:

...

On Wed, 30 Jul 2014 10:38:13 -0700, Arlie Stephens said:

...

On the good side, Vladis' observations of his mail directory have been a great help.

And remember, that's on a single laptop-class hard drive, no fancy raid or anything. (Though it *is* a hybrid, with 32G of flash cache on the front end).

You throw some *real* hardware at it, it of course would go even faster.

Just send me the logs and anything else you think may help me. Please note cc the ext4 mailing list as this will also let the other ext4 developers and maintainers known about your problem. Cheers Nick

I'm now in a state of complete bafflement.

It turns out we have a whole collection of misbehaving directories, making this testable without waiting for caches to clear.

I have a couple of strace's of fast ls's, and a function ftrace that captured about half of a 7 second ls. (The latter is huge, and probably not suitable for posting.)

I also have a really bizarre observation, the kind that makes you wonder whether you are actually dreaming. It appears that the misbehaviour is strongly influenced by the choice of "time" function. The problem only occurs when using the shell built-in. /usr/bin/time always produces a fast response.

Stranger still - flat out impossible, I'd have said before seeing it - a "fast" ls, run with /usr/bin/time can be followed *immediately* by a slow "ls", run with bash' time. It's as if the first one doesn't warm the cache, which is completely absurd - except I've been able to make this happen 5 times in a row, first with strace and then without.

# with /usr/bin/time the ls is fast $ time -p ls bad_dir ... real 0.21 user 0.00 sys 0.00

# with the builtin time, right *after* the strace run, the time can be # horrible. $ time -p ls bad_dir ... real 5.60 user 0.00 sys 0.17

# run it again, and the directory is in cache as expected. $ time -p ls bad_dir ... real 0.11 user 0.00 sys 0.02

This is not an artefact of one or other time reporting incorrectly - I'm noticing a long pause before output occurs, but only on the middle test of the three.

I can't imagine any sane way for this to be happening, short of coincidence or user error - and I've now seen this sequence 5 times in a row, on 5 different directories created and populated by the same app. (Three times with strace, twice without.)

-- Arlie

(Arlie Stephens arlie@worldash.org)

_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

I agree with Hugo, seems right to send me the output in a file to read to see if this actually is a bug with ext4. Regards Nick