August 2020 - kernelnewbies@kernelnewbies.org

Re: Scheduler benchmarks
by Muni Sekhar April 29, 2021

April 29, 2021

On Tue, Aug 18, 2020 at 11:45 PM peter enderborg <peter.enderborg(a)sony.com> wrote: > > On 8/18/20 7:53 PM, Muni Sekhar wrote: > > On Tue, Aug 18, 2020 at 11:06 PM Greg KH <greg(a)kroah.com> wrote: > >> On Tue, Aug 18, 2020 at 11:01:35PM +0530, Muni Sekhar wrote: > >>> On Tue, Aug 18, 2020 at 10:44 PM Greg KH <greg(a)kroah.com> wrote: > >>>> On Tue, Aug 18, 2020 at 10:24:13PM +0530, Muni Sekhar wrote: > >>>>> On Tue, Aug 18, 2020 at 8:06 PM Greg KH <greg(a)kroah.com> wrote: > >>>>>> On Tue, Aug 18, 2020 at 08:00:11PM +0530, Muni Sekhar wrote: > >>>>>>> Hi all, > >>>>>>> > >>>>>>> I’ve two identical Linux systems with only kernel differences. > >>>>>> What are the differences in the kernels? > >>>> You didn't answer this question, is this the same kernel source being > >>>> compared here? Same version? Same compiler? Everything identical? > >>> Both systems are having exactly the same hardware configuration. > >>> Compiler and kernel versions are different. One system has Ubuntu > >>> 16.04.4 LTS(4.4.0-66-generic kernel with gcc version 5.4.0) kernel and > >>> the other one has Ubuntu 18.04.4 LTS(4.15.0-91-generic kernel with gcc > >>> version 7.5.0). > >> Those are _very_ different kernel versions, with many years and tens of > >> thousands of different changes between them. > >> > >> Hopefully the newer kernel is faster, so just stick with that :) > > But unfortunately the newer kernel is very slow, that is the reason > > for starting this investigation :) > > Any type of help, and guidelines to dive deeper will be highly appreciated. > > On the 4.4 kernel you dont have > > +CONFIG_RETPOLINE=y > +CONFIG_INTEL_RDT=y Thanks! That is helpful. Yes, I see 4.4 kernel don't have the above two config options. What analysis can be done to narrow down the root cause? Any example of reference could be helpful to understand. > > And your base is very different two. > > Try to use mainline on both system and see. > > You can also use the same base kernel version from ubuntu and > > run your test. > > > >> greg k-h > > > > > -- Thanks, Sekhar

6 6

How to make /dev/ttyACM0 (and friends) exclusive?
by Jeffrey Walton Oct. 11, 2020

Oct. 11, 2020

Hi Everyone, I'm trying to trackdown a problem using my dialup modem. I have a program that opens the modem and watches caller id's. It flashes the hook when a telemarketer calls. It works well until... When another program opens the modem then my program starts reading intermittent responses intended for the other program. I cannot seem to open the device in exclusive mode. The current open is (I also tried with O_EXCL): int modem = open(device_path, O_RDWR | O_NOCTTY | O_SYNC); I'm not sure if it because udev configures the device incorrectly or the kernel driver is configured incorrectly. Or maybe it is impossible to do. Does anyone know how I can make the device exclusive? Thanks in advance.

6 12

Cannot boot kernel 2.6.35 or 2.6.24 with qemu 5.1
by Cao Nitro Oct. 3, 2020

Oct. 3, 2020

Hi all, As the title mentions, I cannot boot Linux kernel 2.6.35 or 2.6.24 with QEMU 5.1 on Arch Linux. QEMU hasn’t any output when running. Environment: OS: Arch Linux 5.8.1-arch1-1 GCC: 5.5.0, installed from AUR. QEMU: qemu-headless 5.1.0-1 Architecture: x86_64 The command I used is: qemu-system-x86_64 -kernel ./arch/x86/boot/bzImage -append "console=ttyS0" -nographic -m 512M -d int,exec,cpu,pcall,cpu_reset,guest_errors -D /tmp/qemu.log This is the config file of 2.6.24: https://termbin.com/w6a8 This is the config file of 2.6.35: https://termbin.com/x8sq The content of /tmp/qemu.log are all CPU Reset. The size of /tmp/qemu.log is nearly 400MiB… This is first 100 lines of the log: https://termbin.com/pv34 Thanks for your help first :)

2 1

Re: [Cocci] coccinelle: Convert comma to semicolons (was Re: [PATCH] checkpatch: Add test for comma use that should be semicolon)
by Joe Perches Sept. 24, 2020

Sept. 24, 2020

(forwarding on to kernel-janitors/mentees and kernelnewbies) Just fyi for anyone that cares: A janitorial task for someone might be to use Julia's coccinelle script below to convert the existing instances of commas that separate statements into semicolons. https://lore.kernel.org/lkml/alpine.DEB.2.22.394.2008201856110.2524@hadrien/ On Thu, 2020-08-20 at 19:03 +0200, Julia Lawall wrote: > > > I have a bunch of variations of this that are more complicated than I > > > would have expected. One shorter variant that I have is: > > > > > > @@ > > > expression e1,e2; > > > statement S; > > > @@ > > > > > > S > > > e1 > > > -, > > > +; > > > (<+... e2 ...+>); > > > > > > This will miss cases where the first statement is the comma thing. But I > > > think it is possible to improve this now. I will check. > > > > Hi Julia. > > > > Right, thanks, this adds a dependency on a statement > > before the expression. Any stragglers would be easily > > found using slightly different form. > > There are not very many of these in linux kernel. > > > > Another nicety would be to allow the s/,/;/ conversion to > > find both b and c in this sequence: > > a = 1; > > b = 2, > > c = 3, > > d = 4; > > without running the script multiple times. > > There are many dozen uses of this style in linux kernel. > > > > I tried variants of adding a comma after the e2 expression, > > but cocci seems to have parsing problems with: > > > > @@ > > expression e1; > > expression e2; > > @@ > > e1 > > - , > > + ; > > e2, > > This doesn't work because it's not a valid expression. > > The problem is solved by doing: > > e1 > - , > + ; > e2 > ... when any > > But that doesn't work in the current version of Coccinelle. I have fixed > the problem, though and it will work shortly. > > > I do appreciate that coccinelle adds braces for multiple > > expression comma use after an if. > > > > i.e.: > > if (foo) > > a = 1, b = 2; > > becomes > > if (foo) { > > a = 1; b = 2; > > } > > I wasn't sure what was wanted for such things. Should b = 2 now be on a > separate line? > > I took the strategy of avoiding the problem and leaving those cases as is. > That also solves the LIST_HEAD problem. But if it is wanted to change > these commas under ifs, then that is probably possible too. > > My current complete solution is as follows. The first rule avoids changing > commas in macros, where thebody of the macro is just an expression. The > second rule uses position variables to ensure that the two expression are > on different lines. > > @r@ > expression e1,e2; > statement S; > position p; > @@ > > e1 ,@S@p e2; > > @@ > expression e1,e2; > position p1; > position p2 : > script:ocaml(p1) { not((List.hd p1).line_end = (List.hd p2).line) }; > statement S; > position r.p; > @@ > > e1@p1 > -,@S@p > +; > e2@p2 > ... when any > > The generated patch is below. > > julia > > diff -u -p a/drivers/reset/hisilicon/reset-hi3660.c b/drivers/reset/hisilicon/reset-hi3660.c > --- a/drivers/reset/hisilicon/reset-hi3660.c > +++ b/drivers/reset/hisilicon/reset-hi3660.c > @@ -89,7 +89,7 @@ static int hi3660_reset_probe(struct pla > return PTR_ERR(rc->map); > } > > - rc->rst.ops = &hi3660_reset_ops, > + rc->rst.ops = &hi3660_reset_ops; > rc->rst.of_node = np; > rc->rst.of_reset_n_cells = 2; > rc->rst.of_xlate = hi3660_reset_xlate; The rest of the changes are in the link above...

4 4

Compiled Kernel stuck in booting
by Yashodhan Joshi Sept. 23, 2020

Sept. 23, 2020

Hello, I use a dual boot laptop with windows and Ubuntu,with grub2. I Followed the instruction in https://kernelnewbies.org/OutreachyfirstpatchSetup to clone the source code and compile the kernel, which was done without any errors. When I tried to install the kernel using 'sudo make modules_install install' it showed a warning when updating initramfs, that it will attempt to resume from UUID, which is my SWAP space, but other than that it showed no other errors. When I tried to boot into it from the grub menu after restarting, it showed message of loading the kernel and then 'setting up intitial ramdisk', after which nothing further happend. The default kernel that came with ubuntu is still working and boots up into desktop environment withing 2-3 minutes after the 'ramdisk' message, but even though I kept the system running for 20 minutes after the 'ramdisk' message for the compiled kernel, niether any further proccess seems to be happening nor any error messages are shown. Can someone help me figuring out what is wrong and how can I run the kernel? Thank You. Also can someone point me in right direction to send suggestions about updating the information on kernelnewbies website? The information about connecting mutt to Gmail is quite outdated, and I would like to help updating it.

3 2

How to acquire the occupancy time of CPUs within a certain duration, i.e. 10ms, 1ms, and etc?
by 孙世龙 sunshilong Sept. 5, 2020

Sept. 5, 2020

Hi, list How to acquire the occupancy time of CPUs within a certain duration, i.e. 10ms, 1ms, and etc? Could somebody please shed some light on this matter? I would be grateful to have some help with this question. Best Regards. Sunshilong

2 2

Relationship between Rx/Tx ring and Skbuff.
by Vasu M Sept. 4, 2020

Sept. 4, 2020

Hi, I would like to understand the journey of the packet in Linux kernel. There are many resources that explain this differently but as I have understood it on a high level, when a packet is received in a NIC, DMA moves the packets from the NIC frame buffer into the RX ring buffer in the driver. A hardware interrupt is then raised and the top half moves the packet to the RX ring buffer. Is it right to understand that the bottom half tasklets then drain the buffer and move the packets to sk_buff data structure? Are the RX/TX buffers different from sk_buff in terms of memory regions or the sk_buff simply points to the physical address of the packets residing in the RX ring and then does the manipulation of the pointers to send it up in the kernel IP stack? I know this is a broad topic but I would like to seek clarification from the experts on my understanding of packet movement inside the kernel. Or can you point me to a resource which can be a close explanation about the implementation on how the kernel handles in > 4.x ? Thanks, Vasu

4 4

Please give advise about my first patch attempt
by Thomas Schmitt Sept. 3, 2020

Sept. 3, 2020

Hi, i am preparing my first patch. May i ask for review and advise how to make it acceptable ? The patch shall fix an old regression of cdrom and sr drivers. Although drivers/cdrom has no mailing list entry in MAINTAINERS, i assume that such patches should go to linux-scsi(a)vger.kernel.org which is listed for drivers/scsi/sr*. Right ? So the change affects two different subsystems which have the same maintainer and have a very close relation. Do i need to split it into two, nevertheless ? The change is relative to git tag v5.8 of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git It is not locally committed yet, so i cannot run git format-patch for now. I plan to do if i get some approval here. I tested with three different optical drives at SATA and USB by dd if=/dev/sr0 bs=2048 count=1 of=/dev/null Unchanged yields ENOMEDIUM or EIO. Changed yields success. Please also tell me if my mailer (used with this mail) would cause problems. -------------------------------------------------------------------------- Affected files: drivers/cdrom/cdrom.c drivers/scsi/sr.c include/linux/cdrom.h Check passed: scripts/checkpatch.pl reports total: 0 errors, 0 warnings, 131 lines checked -------------------------------------------------------------------------- Intended commit message: -------------------------------------------------------------------------- Re-enable waiting for CD drive to complete medium decision after tray load If open("/dev/sr0", O_RDONLY); pulls in the tray of an optical drive, it immediately returns -1 with errno ENOMEDIUM or the first read(2) fails with EIO. Later, when the drive has stopped blinking, another open() yields success and read() works. This affects not only userland reading of the device file but also mounting the device. Commit 210ba1d1724f5c4ed87a2ab1a21ca861a915f734 of january 2008 switched sr_drive_status() away from indirectly using sr_do_ioctl() for sending TEST UNIT READY commands. sr_do_ioctl() has a wait-and-retry loop for the drive's intermediate sense reply: 2 04 01 Logical unit is in process of becoming ready Since then sr_drive_status() does not wait any more. By commit 96bcc722c47d07b6fd05c9d0cb3ab8ea5574c5b1 of july 2008 it now returns CDS_DRIVE_NOT_READY if above drive reply is received. But the function open_for_data() in drivers/cdrom/cdrom.c expects that its call of cdo->drive_status() waits until the drive has decided about the usability of the inserted medium. Any reply other than CDS_DISC_OK yields ENOMEDIUM. Even if the drive waits with its reply to START/STOP UNIT until it is ready (e.g. Pioneer BDR-S09), the first read(2) yields EIO, because the newly loaded medium gets not properly assessed by check_disk_change() before sr_block_open() returns. Fix the problem by a new function wait_for_medium_decision() which gets called by open_for_data() instead of directly calling cdo->drive_status(). Make sure that scsi/sr.c assesses the newly loaded medium without additional risk of deadlock. After loading the tray and due waiting for readiness, cdrom_open() now returns -EDEADLK to urge its caller to drop its mutex, to re-run check_disk_change(), and then to call cdrom_open() again. Only scsi/sr.c is for now aware of this. cdrom/gdrom.c, paride/pcd.c, and ide/ide-cd.c call cdrom_open() too, but i cannot test them. So their unchanged code will return -EDEADLK instead of retrying. Not worse or more deceiving than -ENOMEDIUM or -EIO. Signed-off-by: Thomas Schmitt <scdbackup(a)gmx.net> -------------------------------------------------------------------------- Code change: -------------------------------------------------------------------------- diff --git a/drivers/cdrom/cdrom.c b/drivers/cdrom/cdrom.c index d82b3b7658bd..9e113b0dfc82 100644 --- a/drivers/cdrom/cdrom.c +++ b/drivers/cdrom/cdrom.c @@ -286,6 +286,18 @@ #include <scsi/scsi_common.h> #include <scsi/scsi_request.h> +/* + * For the wait-and-retry loop after possibly having loaded the drive tray. + * 10 retries in 20 seconds are hardcoded in sr_do_ioctl() which was used + * up to 2008. + * But time spans up to 25 seconds were measured by libburn on + * drives connected via SATA or USB-SATA bridges. + * So 20 retries * 2000 ms = 40 seconds seems more appropriate. + */ +#define CD_OPEN_MEDIUM_RETRY_MAX 20 +#define CD_OPEN_MEDIUM_RETRY_MSLEEP 2000 +#include <linux/delay.h> + /* used to tell the module to turn on full debugging messages */ static bool debug; /* default compatibility mode */ @@ -1040,10 +1052,38 @@ static void cdrom_count_tracks(struct cdrom_device_info *cdi, tracktype *tracks) tracks->cdi, tracks->xa); } +static +int wait_for_medium_decision(struct cdrom_device_info *cdi) +{ + int ret, retry = 0; + const struct cdrom_device_ops *cdo = cdi->ops; + + /* Wait until the intermediate drive status CDS_DRIVE_NOT_READY ends */ + while (1) { + ret = cdo->drive_status(cdi, CDSL_CURRENT); + if (ret == CDS_DRIVE_NOT_READY && + retry++ < CD_OPEN_MEDIUM_RETRY_MAX) + msleep(CD_OPEN_MEDIUM_RETRY_MSLEEP); + else + break; + } + if (ret != CDS_DISC_OK) + return ret; + /* + * It is hard to test whether very recent readiness can cause race + * conditions with media change events. So wait a while to never + * undercut the average delay between actual readiness and detection + * which was tested without this additional msleep(). + */ + msleep(CD_OPEN_MEDIUM_RETRY_MSLEEP / 2); + + return CDS_DISC_OK; +} + static int open_for_data(struct cdrom_device_info *cdi) { - int ret; + int ret, tray_was_moved = 0; const struct cdrom_device_ops *cdo = cdi->ops; tracktype tracks; cd_dbg(CD_OPEN, "entering open_for_data\n"); @@ -1069,13 +1109,17 @@ int open_for_data(struct cdrom_device_info *cdi) ret=-ENOMEDIUM; goto clean_up_and_return; } + tray_was_moved = 1; } else { cd_dbg(CD_OPEN, "bummer. this drive can't close the tray.\n"); ret=-ENOMEDIUM; goto clean_up_and_return; } - /* Ok, the door should be closed now.. Check again */ - ret = cdo->drive_status(cdi, CDSL_CURRENT); + /* + * The door should be closed now, or in the process of + * closing and assessing the medium. + */ + ret = wait_for_medium_decision(cdi); if ((ret == CDS_NO_DISC) || (ret==CDS_TRAY_OPEN)) { cd_dbg(CD_OPEN, "bummer. the tray is still not closed.\n"); cd_dbg(CD_OPEN, "tray might not contain a medium\n"); @@ -1090,6 +1134,14 @@ int open_for_data(struct cdrom_device_info *cdi) ret = -ENOMEDIUM; goto clean_up_and_return; } + if (tray_was_moved) { + /* + * Bail out now to let callers like sr_block_open() + * unlock their mutex and run check_disk_change() + */ + ret = -EDEADLK; + goto clean_up_and_return; + } } cdrom_count_tracks(cdi, &tracks); if (tracks.error == CDS_NO_DISC) { diff --git a/drivers/scsi/sr.c b/drivers/scsi/sr.c index 0c4aa4665a2f..a26c33c44ab9 100644 --- a/drivers/scsi/sr.c +++ b/drivers/scsi/sr.c @@ -521,7 +521,7 @@ static int sr_block_open(struct block_device *bdev, fmode_t mode) { struct scsi_cd *cd; struct scsi_device *sdev; - int ret = -ENXIO; + int ret = -ENXIO, tried = 0; cd = scsi_cd_get(bdev->bd_disk); if (!cd) @@ -529,11 +529,16 @@ static int sr_block_open(struct block_device *bdev, fmode_t mode) sdev = cd->device; scsi_autopm_get_device(sdev); +retry: + tried++; check_disk_change(bdev); mutex_lock(&cd->lock); ret = cdrom_open(&cd->cdi, bdev, mode); mutex_unlock(&cd->lock); + if (ret == -EDEADLK && tried <= 1) + /* Tray was loaded. Assess medium by check_disk_change(). */ + goto retry; scsi_autopm_put_device(sdev); if (ret) diff --git a/include/linux/cdrom.h b/include/linux/cdrom.h index 8543fa59da72..68d8297e6208 100644 --- a/include/linux/cdrom.h +++ b/include/linux/cdrom.h @@ -100,8 +100,17 @@ int cdrom_read_tocentry(struct cdrom_device_info *cdi, struct cdrom_tocentry *entry); /* the general block_device operations structure: */ + +/* + * Peculiarity: + * cdrom_open() returns -EDEADLK after automatically loading the tray, because + * it expects to be called under various mutexes after the callers called + * check_disk_change() outside of these mutex acquirations. Callers which get + * -EDEADLK should for once re-run check_disk_change() and retry cdrom_open(). + */ extern int cdrom_open(struct cdrom_device_info *cdi, struct block_device *bdev, fmode_t mode); + extern void cdrom_release(struct cdrom_device_info *cdi, fmode_t mode); extern int cdrom_ioctl(struct cdrom_device_info *cdi, struct block_device *bdev, fmode_t mode, unsigned int cmd, unsigned long arg); -------------------------------------------------------------------------- Have a nice day :) Thomas

6 10

Re: cpu-freq: running the perf increases the data rate?
by Subhashini Rao Beerisetty Aug. 31, 2020

Aug. 31, 2020

On Fri, Aug 28, 2020 at 6:04 PM Artem Bityutskiy <dedekind1(a)gmail.com> wrote: > > On Thu, 2020-08-27 at 22:25 +0530, Subhashini Rao Beerisetty wrote: > > I have an application which finds the data rate over the PCIe > > interface. I’m getting the lesser data rate in one of my Linux X86 > > systems. > > Some more description, may be? Do you have a PCIe device reading one > RAM buffer and then writing to another RAM buffer? Or does it generate > dome data and writes them to a RAM buffer? Presumably it uses DMA? How > much is the CPU involved into the process? Are we talking about > transferring few kilobytes or gigabytes? Thanks a lot for your help and reply. Regarding hardware setup, Xilinx PCIe FPGA endpoint is connected to HOST CPU via PCIe bus. Xilinx PCIe FPGA endpoint has the DMA_REF block and it provides a mechanism to DMA transfer data at the maximum rate between host CPU memory and a FIFO in the DMA-REF block. The host software sets up some data in it’s memory, it then transfers the data to the DMA-REF’s FIFO and then reads it back into a different location in its host memory. This is repeated in a loop. There is a register in the DMA-REF block that gives an indication of transfer speed. > > > When I change the scaling_governor from "powersave" to "performance" > > mode for each CPU, then there is slight improvement in the PCIe data > > rate. > > Definitely this makes your CPU(s) run at max speed, but depending on > platform and settings, this may also affect C-states. Are the CPU(s) > generally idle while you measure, or busy (involved into the test)? You > could run 'turbostat' while measuring the bandwidth, to get some CPU > statistics (e.g., do C-states happen during the PCI test, how busy are > the CPUs). > > > Parallely I started profiling the workload with perf. Whenever I start > > running the profile command “perf stat -a -d -p <PID>” surprisingly > > the application resulted in excellent data rate over PCIe, but when I > > kill the perf command again PCIe data rate drops. I am really confused > > about this behavior.Any clues from this behaviour? > > Well, one possible reason that comes to mind - you get rid of C-states > when you rung perf, and this increases the PCI bandwidth. You can just > try disabling C-states (there are sysfs knobs) and check it out. > Turbostat could be useful to check for this (with and without perf, run > 'turbostat sleep 10' or something like this (measure for 10 seconds in > this example), do this while running your PCI test. Disabling the C-states improved the throughput a lot, thanks a lot for pointing this out. Could you please give some more explanation on how disabling C-states improved the throughput? As you suggested I collected and attached the turbostat log with and without perf while running the PCIe test. In my system, only 'performance\powersave' are listed in scaling_available_governors. Rest other governors "userspace\ondemand\schedutil" are not listed in available_goverors. What might be the reason for this? > > But I am really just guessing here, I do not know enough about your > test and the system (e.g., "a Linux x86" system can be so many things, > like Intel or AMD server or a mobile device)… It's an Intel Atom processor. > >

1 0

What mailer to use with Gmail?
by Leam Hall Aug. 28, 2020

Aug. 28, 2020

Setting aside the desire to cuss as gmail, I'm having difficult making Thunderbird not send base64 attachments. Now an upgrade as cost me the ability to revert. :( Suggestions for a mailer to use for patches, and other things that don't like base64? Thanks! Leam

3 2