January 2018 - kernelnewbies@kernelnewbies.org

Kernelupdate
by Frank Smith Jan. 26, 2018

Jan. 26, 2018

3 2

Re: [perf] perf test and probe fails sometimes on 4.9
by Pintu Kumar Jan. 25, 2018

Jan. 25, 2018

Hi, ** Changed the subject now, since these issues are related to general perf commands. Following are the issues: 1) perf probe --add schedule - FAILED output: schedule is out of .text, skip it. Error: Failed to add events. what is the issue here? Sometimes it pass and sometimes it fails... Similar is the case of 'perf inject' as well. 2) perf test - 1 FAILURE 37.1: Test basic BPF filtering : FAILED! 37.2: Test BPF prologue generation : Skip 37.3: Test BPF relocation checker : Skip bpf: config program 'func=SyS_epoll_wait' symbol:SyS_epoll_wait file:(null) line:0 offset:0 return:0 lazy:(null) bpf: config 'func=SyS_epoll_wait' is ok Looking at the vmlinux_path (8 entries long) Using /boot/vmlinux for symbols Could not open debuginfo. Try to use symbols. SyS_epoll_wait is out of .text, skip it. bpf_probe: failed to convert perf probe eventsFailed to add events selected by BPF test child finished with -1 ---- end ---- Test BPF filter subtest 0: FAILED! Looks like both 1,2 are related. Since, CONFIG_DEBUG_INFO is not enabled, I manually copied the vmlinux to /boot/ folder. ------- Some more info: Kernel build dir is set to /lib/modules/4.9.20-sc-amd-x86-64/build set env: KBUILD_DIR=/lib/modules/4.9.20-sc-amd-x86-64/build unset env: KBUILD_OPTS include option is set to -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/5/include -I./arch/x86/include -I./arch/x86/include/generated/uapi -I./arch/x86/include/generated -I./include -I./arch/x86/include/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h set env: NR_CPUS=8 set env: LINUX_VERSION_CODE=0x40914 set env: CLANG_EXEC=/usr/bin/clang set env: CLANG_OPTIONS=-xc set env: KERNEL_INC_OPTIONS= -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/5/include -I./arch/x86/include -I./arch/x86/include/generated/uapi -I./arch/x86/include/generated -I./include -I./arch/x86/include/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h set env: WORKING_DIR=/lib/modules/4.9.20-sc-amd-x86-64/build If you have any clue about these failure please help me. Thanks, Pintu On Wed, Jan 24, 2018 at 8:23 PM, Pintu Kumar <pintu.ping(a)gmail.com> wrote: > Hi, > > Thanks for your help. > Yes it was a sub version issue. > > Earlier, while building the kernel I left the SUBLEVEL as blank. > > make -j8 bindeb-pkg SUBLEVEL=20 > > After passing the correct sublevel now the test is working. > > But still following are failing: > > 16: Try 'import perf' in python, checking link problems : FAILED! > 37.2: Test BPF prologue generation : FAILED! > > > This is the error I get: > > 16: Try 'import perf' in python, checking link problems : > --- start --- > test child forked, pid 7637 > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > ImportError: No module named perf > test child finished with -1 > ---- end ---- > Try 'import perf' in python, checking link problems: FAILED! > > > Looking at the vmlinux_path (8 entries long) > symsrc__init: cannot get elf header. > Failed to find the path for kernel: Invalid ELF file > bpf_probe: failed to convert perf probe eventsFailed to add events > selected by BPF > test child finished with -1 > ---- end ---- > Test BPF filter subtest 1: FAILED! > > > > Thanks, > Pintu > > > On Wed, Jan 24, 2018 at 6:39 AM, Wangnan (F) <wangnan0(a)huawei.com> wrote: >> >> >> On 2018/1/23 20:37, Pintu Kumar wrote: >>> >>> Hi All, >>> >>> I am verifying all perf tests on Ubuntu-16 x86-64 platform using the >>> kernel version 4.9.20. >>> I have installed several others packages including: clang, llvm >>> >>> But, when I run 'perf test' I get some FAILURE. >>> Specially, 'perf test LLVM' is failing. >>> >>> Please check the below error logs: >>> >>> ---------------------------------------------------- >>> # perf test LLVM >>> 35: Test LLVM searching and compiling : >>> 35.1: Basic BPF llvm compiling test : FAILED! >>> 35.2: Test kbuild searching : Skip >>> 35.3: Compile source for BPF prologue generation test : Skip >>> 35.4: Compile source for BPF relocation test : Skip >>> ---------------------------------------------------- >>> >>> When I run with -v I get this: >>> >>> ----------------------------------------------------- >>> # perf test -v LLVM >>> 35: Test LLVM searching and compiling : >>> 35.1: Basic BPF llvm compiling test : >>> --- start --- >>> test child forked, pid 3304 >>> Unablt to get kernel version from uname '4.9-xxxx-amd-x86-64' >>> WARNING: unable to get correct kernel building directory. >>> Hint: Set correct kbuild directory using 'kbuild-dir' option in [llvm] >>> section of ~/.perfconfig or set it to "" to suppress kbuild >>> detection. >>> >>> Unablt to get kernel version from uname '4.9-xxxx-amd-x86-64' >>> >>> >>> int _version SEC("version") = LINUX_VERSION_CODE; >>> ' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS >>> -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS >>> $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign >>> -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o >>> - >>> libbpf: 2129190-4a0 doesn't provide kernel version >>> Failed to parse test case 'Basic BPF llvm compiling test' >>> test child finished with -1 >>> ---- end ---- >>> Test LLVM searching and compiling subtest 0: FAILED! >>> ----------------------------------------------------- >>> >>> >>> Basic BPF LLVM compiling test is failing. >>> It reports that bpf could not able to read the kernel version from uname. >>> Is it because of this that 'perf test LLVM' is failing ? >>> >>> My uname says: >>> 4.9-xxxx-amd-x86-64 >> >> >> 'perf test LLVM' requires a well formed uname string (%d.%d.%d). Please see >> tools/perf/fetch_kernel_version . If your distro uses another method to >> report >> kernel version number (from your uname string, sublevel can't be >> determined), >> please help us improve that function like what we have done for ubuntu >> (checking /proc/version_signature). >> >> >> >>> My kernel version is: 4.9.20 (from Makefile) >>> >>> So, I dont think there is any problem with kernel version. >>> >>> If any body have come across this problem please let me know how to >>> resolve this issue. >>> >>> >>> >>> Thank You! >>> >>> Regards, >>> Pintu >> >> >>

1 0

Testing low level UART driver
by Muni Sekhar Jan. 25, 2018

Jan. 25, 2018

Hi All, Does Linux kernel tree has any standard utilities to test any low level UART driver? -- Thanks, Sekhar

2 5

Re: [tools/perf] perf test LLVM failure on 4.9
by Pintu Kumar Jan. 24, 2018

Jan. 24, 2018

Hi, Thanks for your help. Yes it was a sub version issue. Earlier, while building the kernel I left the SUBLEVEL as blank. make -j8 bindeb-pkg SUBLEVEL=20 After passing the correct sublevel now the test is working. But still following are failing: 16: Try 'import perf' in python, checking link problems : FAILED! 37.2: Test BPF prologue generation : FAILED! This is the error I get: 16: Try 'import perf' in python, checking link problems : --- start --- test child forked, pid 7637 Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: No module named perf test child finished with -1 ---- end ---- Try 'import perf' in python, checking link problems: FAILED! Looking at the vmlinux_path (8 entries long) symsrc__init: cannot get elf header. Failed to find the path for kernel: Invalid ELF file bpf_probe: failed to convert perf probe eventsFailed to add events selected by BPF test child finished with -1 ---- end ---- Test BPF filter subtest 1: FAILED! Thanks, Pintu On Wed, Jan 24, 2018 at 6:39 AM, Wangnan (F) <wangnan0(a)huawei.com> wrote: > > > On 2018/1/23 20:37, Pintu Kumar wrote: >> >> Hi All, >> >> I am verifying all perf tests on Ubuntu-16 x86-64 platform using the >> kernel version 4.9.20. >> I have installed several others packages including: clang, llvm >> >> But, when I run 'perf test' I get some FAILURE. >> Specially, 'perf test LLVM' is failing. >> >> Please check the below error logs: >> >> ---------------------------------------------------- >> # perf test LLVM >> 35: Test LLVM searching and compiling : >> 35.1: Basic BPF llvm compiling test : FAILED! >> 35.2: Test kbuild searching : Skip >> 35.3: Compile source for BPF prologue generation test : Skip >> 35.4: Compile source for BPF relocation test : Skip >> ---------------------------------------------------- >> >> When I run with -v I get this: >> >> ----------------------------------------------------- >> # perf test -v LLVM >> 35: Test LLVM searching and compiling : >> 35.1: Basic BPF llvm compiling test : >> --- start --- >> test child forked, pid 3304 >> Unablt to get kernel version from uname '4.9-xxxx-amd-x86-64' >> WARNING: unable to get correct kernel building directory. >> Hint: Set correct kbuild directory using 'kbuild-dir' option in [llvm] >> section of ~/.perfconfig or set it to "" to suppress kbuild >> detection. >> >> Unablt to get kernel version from uname '4.9-xxxx-amd-x86-64' >> >> >> int _version SEC("version") = LINUX_VERSION_CODE; >> ' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS >> -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS >> $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign >> -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o >> - >> libbpf: 2129190-4a0 doesn't provide kernel version >> Failed to parse test case 'Basic BPF llvm compiling test' >> test child finished with -1 >> ---- end ---- >> Test LLVM searching and compiling subtest 0: FAILED! >> ----------------------------------------------------- >> >> >> Basic BPF LLVM compiling test is failing. >> It reports that bpf could not able to read the kernel version from uname. >> Is it because of this that 'perf test LLVM' is failing ? >> >> My uname says: >> 4.9-xxxx-amd-x86-64 > > > 'perf test LLVM' requires a well formed uname string (%d.%d.%d). Please see > tools/perf/fetch_kernel_version . If your distro uses another method to > report > kernel version number (from your uname string, sublevel can't be > determined), > please help us improve that function like what we have done for ubuntu > (checking /proc/version_signature). > > > >> My kernel version is: 4.9.20 (from Makefile) >> >> So, I dont think there is any problem with kernel version. >> >> If any body have come across this problem please let me know how to >> resolve this issue. >> >> >> >> Thank You! >> >> Regards, >> Pintu > > >

1 0

How does the Deadline IO Scheduler work?
by Sreyan Chakravarty Jan. 24, 2018

Jan. 24, 2018

I am trying my best to understand how the deadline scheduler actually works for certain scenarios. But there is actually a lack of proper documentation for IO Schedulers in particular. Hence I would be really grateful if someone would help me clearing my doubts: My questions are as follows: 1) Since the deadline scheduler copies each request into two queues ie. the sorted by block number and the queue sorted by deadlines. So effectively there are two copies of the same IO request. So what happens if a request is serviced from the deadline queue. How is that request's corresponding version removed from the sorted queue? A queue is not a random access data structure, thus to remove the request from the sorted queue the kernel must traverse the head and delink the request if the queue is implemented via a linked list. Otherwise the kernel must be keeping track of the requests that have already executed. So how does the kernel handle this? 2) Imagine the device is busy serving a request for block 50 the next request is for block 56, then a request for block 30 comes in. What does the scheduler do? Does it keep go on to dispatch the request for block 56? Or does it pick block 30 since the scheduler performs sorting? If it picks block 56 that makes sense since it is desirable for the disk head to move in one direction rather than going back to 30. But then how is the sorting done? If it picks 30 then the sorting is taken care of but then the disk is doubling back and there is little benefit to the sorting step. So which one of the above will happen. Would be very helpful if someone helped me understand these doubts. Regards, Sreyan Chakravarty

1 0

[tools/perf] perf test LLVM failure on 4.9
by Pintu Kumar Jan. 24, 2018

Jan. 24, 2018

Hi All, I am verifying all perf tests on Ubuntu-16 x86-64 platform using the kernel version 4.9.20. I have installed several others packages including: clang, llvm But, when I run 'perf test' I get some FAILURE. Specially, 'perf test LLVM' is failing. Please check the below error logs: ---------------------------------------------------- # perf test LLVM 35: Test LLVM searching and compiling : 35.1: Basic BPF llvm compiling test : FAILED! 35.2: Test kbuild searching : Skip 35.3: Compile source for BPF prologue generation test : Skip 35.4: Compile source for BPF relocation test : Skip ---------------------------------------------------- When I run with -v I get this: ----------------------------------------------------- # perf test -v LLVM 35: Test LLVM searching and compiling : 35.1: Basic BPF llvm compiling test : --- start --- test child forked, pid 3304 Unablt to get kernel version from uname '4.9-xxxx-amd-x86-64' WARNING: unable to get correct kernel building directory. Hint: Set correct kbuild directory using 'kbuild-dir' option in [llvm] section of ~/.perfconfig or set it to "" to suppress kbuild detection. Unablt to get kernel version from uname '4.9-xxxx-amd-x86-64' int _version SEC("version") = LINUX_VERSION_CODE; ' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o - libbpf: 2129190-4a0 doesn't provide kernel version Failed to parse test case 'Basic BPF llvm compiling test' test child finished with -1 ---- end ---- Test LLVM searching and compiling subtest 0: FAILED! ----------------------------------------------------- Basic BPF LLVM compiling test is failing. It reports that bpf could not able to read the kernel version from uname. Is it because of this that 'perf test LLVM' is failing ? My uname says: 4.9-xxxx-amd-x86-64 My kernel version is: 4.9.20 (from Makefile) So, I dont think there is any problem with kernel version. If any body have come across this problem please let me know how to resolve this issue. Thank You! Regards, Pintu

4 3

Jbd2 async problem
by Rock Lee Jan. 23, 2018

Jan. 23, 2018

Hi: I am working on jbd2 with checksum v3, trying to learn something from the async feature. However, when I read the replay process in do_one_process(), I got a little confused. When async is on, commit blocks have the chance to reach the disk before metadata blocks(which are on BJ_Shadow). If just at this moment, power cutting occurs. Next mount time, we have to recovery journal. Since checksum v3 is been chosen, metadata could be copyed to filesystem one by one, if the metadata block has the right crc value. But in the scenario I described before, commit block may reached disk before some metadata blocks in a transaction. Current jbd2 code(linux 4.15) just skips those metadata blocks which have no correct crc, and still copies the metadata blocks with the right crc. I think in order to keep the consistency, the whole transaction should be discarded, as long as not all the metadata blocks have the right crc value in the transaction. But why jbd2 still copies the metadata block(with the right crc) to filesystem when bad crc metadata blocks exist ? I will appreciate it if you could help me. -- Cheers, Rock

2 2

migrate_pages internals: lazy copy
by Karaoui mohamed lamine Jan. 20, 2018

Jan. 20, 2018

Hi list, Does the "migrate_pages" syscall migrate memory pages in a *lazy* manner? if not, is there another syscall or patch that does it? Regards,

2 2

list of all subsystems (swap space)
by inventsekar Jan. 20, 2018

Jan. 20, 2018

Hi All, 1. so I tried to find out which subsystem the swap space comes under.. when I googled for list of subsystems Linux architecture, no luck. Between, I found out that kernel map ( http://www.makelinux.net/kernel_map/ ), which gives me kind of subsystems/modules. Is there any documentation/links for list of subsystems on Linux?!?! 2. it looks like swap space itself a separate subsystem/module. I thought to subscribe to some lkml dl for swap, but I am not finding any DL at all. Please suggest. Best regards Sekar On Jan 15, 2018 11:52 PM, "inventsekar" <inventsekar(a)gmail.com> wrote: Hi All, I have been reading about swap space and got these questions... 1. Btrfs was the only one fs which got this below issue or any other fs also? 2. The other way around - is there any FS which is best suitable (best performance) for swap file? 2. Is this issue still there or the wiki isn't updated? *https://wiki.archlinux.org/index.php/swap <https://wiki.archlinux.org/index.php/swap>* *Warning:* Btrfs <https://wiki.archlinux.org/index.php/Btrfs> does not support swap files. Failure to heed this warning may result in file system corruption. While a swap file may be used on Btrfs when mounted through a loop device, this will result in severely degraded swap performance. Any suggestions, links, documentations please. Best regards Sekar

2 2

About File systems magic numbers
by inventsekar Jan. 18, 2018

Jan. 18, 2018

Hi All, i have been reading Unix filesystem - evolution, design and implementation by Steve D. Pate and it provides a simple file system source code for learning and playing with. On the header file, it defines the fs magic number as #define UX_MAGIC 0x58494e55 1. which means, will it assign this magic number to ALL files created on this sample fs? That can't be so. Otherwise, why we need to define a magic number? 2. I assume, magic number will be assigned to files by the tools which we use to create the files(like vi, cat, etc). Is this correct? 3. It seems, windows files also have the magic numbers. let's say I copy some files from windows to Linux. We should have the common magic numbers between different Operating systems, so there won't be confusions. I mean, is there any standards common for all os's files magic numbers?!? Best regards Sekar

4 4