- kernelnewbies@kernelnewbies.org - Kernelnewbies Mailing Lists

Which vger list is for linux-firmware?
by Jeffrey Walton March 5, 2022

March 5, 2022

Hi Everyone, Distros like Ubuntu provide a package called linux-firmware. The problem I am trying to solve is, the latest linux-firmware seems to have broken network stacks on my Ubuntu and Fedora machines. Just about all (maybe all) of my Linux boxes lost networking last night after they got the linux-firmware updates. The only machine I have with networking is a Windows machine. Which vger list do we use with questions about the package? Thanks in advance.

2 1

Exporting cpu instruction set to kernel .config file
by Guddla Rupesh March 4, 2022

March 4, 2022

I have seen the config from /proc/config.gz and found that processor family as generic x86 64. Actually my system consists of i3 10th gen comet lake 10100 which has integrated graphics and pci express 4.0 and also lots of new instructions like avx2, sse4 etc.,. The main reason of compiling source code of kernel is it detects all my hardware and provide support for opencl, video acceleration etc.,. But I can't find any option to select my processor especially comet lake or atleast skylake and instead there is an option called core2 in the section processor family. Finally I have disabled secure boot mode in bios and compiled source code successfully and now I am able to boot into the latest build kernel but there is no difference in performance between old and new kernels. I think that normal kernel shipped by the distro is generic and it doesn't have support for latest instructions like avx2, sse4 etc.,. To get support we must compile kernel with lots of tuning. I have exported the modules running on system and disable others by running the command make localmodconfig What I want is " is there any way to export my cpu instruction set to kernel .config file and compile kernel and install the modules.". If I can compile kernel source code then my Linux system will run at extreme speeds never before.

5 6

Does fixmap pages for fdt require 4MB memory?
by Chan Kim March 4, 2022

March 4, 2022

Hello all, I'm porting linux on an FPGA board which has only 8MB SRAM in place of DDR address range for now. (using linux-5.4.21) So this means, seen from software, the DDR memory exists only for 0x80000000 ~ 0x807fffff. I loaded kernel image (Image) at 0x80080000 and it takes space to 0x8061d008. For there is so scare space, I loaded the dtb at 0x807fd000. Now the kernel stops during setup_machine_fdt in setup_arch. I guess maybe it's because the kernel could not setup fixmap for fdt. (I recently learned the kernel uses fixmap for the fdt and it requires 4MB separate pages) I just started to look into early_fixmap_init function but can anyone tell me if this can be the real problem? Isn't it possible to use 8MB memory to run linux? (I'm doing this before the board is fully ready) Thank you! Chan Kim

1 0

Questions about data protection in the kernel
by Kai Meyer March 3, 2022

March 3, 2022

Full disclosure, I am a software engineer for Arcserve, responsible for maintaining our linux kernel driver component of our data protection software suites. I am here looking for help seeking answers to two specific questions. Not knowing who to ask or really how to communicate some of my questions, it seemed most appropriate to start my journey here with kernel newbies. Why does the kernel only provide disk snapshot capabilities via Device Mapper (and lvm)? I am aware of multiple companies that offer linux products that come with their own kernel modules to provide full featured snapshot capabilities. Some of these linux offerings are decades old and some are open source now. If they were to require device mapper to be setup previously, it would be difficult to insert themselves into many environments. All of the drivers seem to perform the same hook by replacing the fops->submit_bio (previously known as q->make_request_fn, and more on this in the next question). It seems to me there is at least some demand for data protection functionality outside of device mapper, and this hook could be so much cleaner if it was officially supported by the kernel. (I would be thrilled to learn that the answer is "because nobody has volunteered to write it.") In context of Multi-Queue Block IO (blk-mq), what is the future of the older Single Queue interface? I have puzzled together some of blk-mq's history simply by interacting with it as it has become necessary; reading code and git commit messages. What I have gleaned so far has me wondering if and when the future will be Multi-Queue only. I have read as many LWN articles I can find on the subject, but the future of IO queuing is still unclear to me. Thank you for your time and helping me to learn how to engage with the linux kernel community. Any feedback on how and where to ask questions would also be greatly appreciated. _____________________________________________________________________________________________________________ [cid:arcserve-email-logo_566d469b-c8dc-46eb-909b-300e3f3e47a1.jpg]<https://arcserve.com/> Kai Meyer | Sr. Software Engineer Office: 801.871.2765 | Mobile: | kai.meyer(a)arcserve.com arcserve.com<https://www.arcserve.com/> | Twitter<https://twitter.com/Arcserve> | LinkedIn<https://www.linkedin.com/company/arcserve/> | YouTube<https://www.youtube.com/user/arcserve> _____________________________________________________________________________________________________________ If you are not the intended recipient of this message or received it erroneously, please notify the sender and delete it, together with any attachments, and be advised that any dissemination or copying of this message is prohibited.

1 0

Re: Exporting cpu instruction set to kernel .config file
by FMDF March 3, 2022

March 3, 2022

On Thu, 3 Mar 2022, 20:01 Rogério Valentim Feitoza da Silva, < rogerio.silva3920(a)gmail.com> wrote: Rupesh has disabled Secure Boot because the kernel that Rupesh > compiled couldn't boot with Secure Boot enabled. > GRUB failed to load the kernel with the message "bad shim signature" > and the "boot" command failed with > "you need to load the kernel first" error (because the kernel failed to > load). > Ah, I understand. This is one more reason why he shouldn't change his .config file: he has no knowledge of how kernels are built and of how the boot process in modern PC works :) I mean that he shouldn't had disabled secure boot for his custom kernel. Developers / Administrators generate their own machine owner key and enroll them in UEFI. Then they build custom signed kernels and loadable modules, install and boot. Disabling secure boot just because one doesn't know how to enroll their keys and sign kernel objects is a very dangerous thing to do. It's easy and lots of tutorials are in the web. Regards, Fabio M. De Francesco > -Rogério Valentim >

1 0

Fwd: Exporting cpu instruction set to kernel .config file
by FMDF March 3, 2022

March 3, 2022

Sorry, I forgot to add kernelnewbies in the email below... ---------- Forwarded message --------- From: FMDF <fmdefrancesco(a)gmail.com> Date: Thu, 3 Mar 2022, 19:43 Subject: Re: Exporting cpu instruction set to kernel .config file To: Guddla Rupesh <rupeshforu3(a)gmail.com> On Thu, 3 Mar 2022, 16:21 Guddla Rupesh, <rupeshforu3(a)gmail.com> wrote: I think that all the kernels shipped by distros are not considering latest > processors instruction set. I think that even windows is doing the same. > > If opencl is detected properly any application can have improvement in > performance about 10 percent. > > I can't find any option to set opencl in the Linux kernel config file or > in any of the kernel source files. > > Atleast I am lucky to select option core 2 in the main .config file of > kernel. > > Finally my request is there any way to export instruct set of my cpu to > the kernel configuration and so after compilation my video encoding takes > less time than previous. > > If it's a difficult task leave it and if it's possible try to suggest how > to do so. > You have not read Torin's and my replies. I can assure you that Linux uses ALL the latest extensions of Intel and AMD instruction sets. And you didn't even read the code that Torin pointed out to you. Personally I have written documentation for various x86-64 newer subsystem. There is nothing of those ISA that Linux kernel and OS don't support. The fact that the configuration option is simply called CONFIG_CORE2 (or whatever) has no meaning at all. If you can, please attach GDB to executing code or read the Assembly output by GCC and see yourself what instructions Linux uses in the subsystems of your choice. Regards, Fabio M. De Francesco >

1 0

Re: Exporting cpu instruction set to kernel .config file
by FMDF March 3, 2022

March 3, 2022

On Thu, 3 Mar 2022, 19:43 FMDF, <fmdefrancesco(a)gmail.com> wrote: > On Thu, 3 Mar 2022, 16:21 Guddla Rupesh, <rupeshforu3(a)gmail.com> wrote: > > I think that all the kernels shipped by distros are not considering latest >> processors instruction set. I think that even windows is doing the same. >> >> If opencl is detected properly any application can have improvement in >> performance about 10 percent. >> >> I can't find any option to set opencl in the Linux kernel config file or >> in any of the kernel source files. >> >> Atleast I am lucky to select option core 2 in the main .config file of >> kernel. >> >> Finally my request is there any way to export instruct set of my cpu to >> the kernel configuration and so after compilation my video encoding takes >> less time than previous. >> >> If it's a difficult task leave it and if it's possible try to suggest how >> to do so. >> > > You have not read Torin's and my replies. I can assure you that Linux uses > ALL the latest extensions of Intel and AMD instruction sets. > > And you didn't even read the code that Torin pointed out to you. > > Personally I have written documentation for various x86-64 newer > subsystem. There is nothing of those ISA that Linux kernel and OS don't > support. The fact that the configuration option is simply called > CONFIG_CORE2 (or whatever) has no meaning at all. > > If you can, please attach GDB to executing code or read the Assembly > output by GCC and see yourself what instructions Linux uses in the > subsystems of your choice. > > Regards, > > Fabio M. De Francesco > >>

1 0

Re: How to compile source code of new kernel specifically for my pc hardware with uefi secure boot.
by Guddla Rupesh March 3, 2022

March 3, 2022

Pentium MMX was launched about 30 years ago and at that time video processing was new and there was no proper hardware just to play any video file I mean if you play any video file video playback stops sometimes and resume after some time. At present there is lot of development in video hardware and at the same time new technologies in both hardware and software has been evolved. Coming into hardware CPUs with integrated graphics came and coming into software many video and audio codecs have been developed like h265, vpx, av1, aac, opus etc.,.Of these aac and opus are audio codecs which produces lowest disk size at reasonable quality and h265, av1 are video codecs which provides lowest disk size at reasonable quality. Most of the time I download videos from youtube and other sources which are of large size. When you record a video in android smartphone it takes around 600 mb for 3 minutes 50 seconds and it uses h265 codec. It is difficult to store such files in my android smartphone. Mostly I use Linux and ffmpeg tool to compress these video files. Of h265 and av1 aom encoders. av1 aom encoder provide lowest disk size at lowest video bitrate. H265 encoding and decoding are supported in all hardware including pc and smartphone but unfortunately av1 files are not supported in none but definitely I think that it will become defacto standard in future. If h265 encoding takes 1 minute then aom av1 encoding takes about 10 minutes. If video acceleration is present the encoding takes about 7 minutes. Video acceleration come into action only if GPU is detected properly and the necessary drivers are installed. If the system can detect opecl version of GPU then av1 encoding takes about 5 minutes through ffmpeg. As MMX was trending about 30 years ago I think that latest processors have vector instructions for video transcoding or encoding like avx2 or avx 512, simd etc.,. These new instructions are not present in 15 years back processors. As 64 bit processor came into action around 20 years back nowadays processors are 10 times are greater in performance than old. I think that all the kernels shipped by distros are not considering latest processors instruction set. I think that even windows is doing the same. If opencl is detected properly any application can have improvement in performance about 10 percent. I can't find any option to set opencl in the Linux kernel config file or in any of the kernel source files. Atleast I am lucky to select option core 2 in the main .config file of kernel. Finally my request is there any way to export instruct set of my cpu to the kernel configuration and so after compilation my video encoding takes less time than previous. If it's a difficult task leave it and if it's possible try to suggest how to do so. On Thu, 3 Mar 2022, 2:15 am Rogério Valentim Feitoza da Silva, < rogerio.silva3920(a)gmail.com> wrote: > On Wednesday, 2 March 2022 at 13:56, Guddla Rupesh > <rupeshforu3(a)gmail.com> wrote: > > > > I have seen the config from /proc/config.gz and found that processor > family as generic x86 64. > > > > Actually my system consists of i3 10th gen comet lake 10100 which has > integrated graphics and pci express 4.0 and also lots of new instructions > like avx2, sse4 etc.,. > > > > The main reason of compiling source code of kernel is it detects all my > hardware and provide support for opencl, video acceleration etc.,. But I > can't find any option to select my processor especially comet lake or > atleast skylake and instead there is an option called core2 in the section > processor family. > > > > Finally I have disabled secure boot mode in bios and compiled source > code successfully and now I am able to boot into the latest build kernel > but there is no difference in performance between old and new kernels. > > > > I think that normal kernel shipped by the distro is generic and it > doesn't have support for latest instructions like avx2, sse4 etc.,. To get > support we must compile kernel with lots of tuning. > > > > I have exported the modules running on system and disable others by > running the command > > > > make localmodconfig > > > > What I want is " is there any way to export my cpu instruction set to > kernel .config file and compile kernel and install the modules.". > > > > If I can compile kernel source code then my Linux system will run at > extreme speeds never before. > > > > > > On Tue, 1 Mar 2022, 11:28 pm Rogério Valentim Feitoza da Silva, < > rogerio.silva3920(a)gmail.com> wrote: > >> > >> On Tuesday, 1 March 2022 at 00:08, Guddla Rupesh <rupeshforu3(a)gmail.com> > wrote: > >> > > >> > Hi I am Rupesh from India and I have pc i3 processor and h510 > motherboard It has uefi. I have installed open suse tumblewood and all the > packages have been updated. As the default kernel provided by open suse > tumblewood is not working properly I want to compile source code of new > kernel which is obtained from kernel.org and the kernel source code > present in /usr/src/linux*** but I can't. > >> > > >> > As the pc is uefi based I am getting lot of errors related to > signing. I have installed all latest packages related to gcc, make, ctags, > cscope, open ssh, open SSL, auto make, auto conf, cmake etc.,. > >> > > >> > I have created the config file from the existing configuration of > system using the following command > >> > > >> > make localmodconfig > >> > > >> > I have succeeded in compiling source code of new kernel using make > command but when I execute the command > >> > > >> > make install > >> > > >> > I am getting error as > >> > > >> > " certificate must have code signing extended key usage defined for > secure boot ". > >> > > >> > After some time vmlinux, initrd files are created but when I try to > boot the newly compiled kernel from grub I am getting errors as > >> > > >> > "bad shim signature" > >> > "you need to load the kernel first" > >> > > >> > I have tried a number of ways to compile successfully such as > disabling secure boot in yast boot loader, selecting load all modules by > verifying signature etc.,. > >> > > >> > Currently my .config file consists of the following lines containing > the word sig > >> > > >> > [CODE] > >> > CONFIG_SIGNALFD=y > >> > CONFIG_KEXEC_SIG=y > >> > CONFIG_KEXEC_SIG_FORCE=y > >> > CONFIG_KEXEC_BZIMAGE_VERIFY_SIG=y > >> > # CONFIG_STRICT_SIGALTSTACK_SIZE is not set > >> > CONFIG_ACPI_TINY_POWER_BUTTON_SIGNAL=38 > >> > CONFIG_OLD_SIGSUSPEND3=y > >> > CONFIG_COMPAT_OLD_SIGACTION=y > >> > CONFIG_DYNAMIC_SIGFRAME=y > >> > CONFIG_MODULE_SIG_FORMAT=y > >> > CONFIG_MODULE_SIG=y > >> > CONFIG_MODULE_SIG_FORCE=y > >> > CONFIG_MODULE_SIG_ALL=y > >> > # CONFIG_MODULE_SIG_SHA1 is not set > >> > # CONFIG_MODULE_SIG_SHA224 is not set > >> > CONFIG_MODULE_SIG_SHA256=y > >> > # CONFIG_MODULE_SIG_SHA384 is not set > >> > # CONFIG_MODULE_SIG_SHA512 is not set > >> > CONFIG_MODULE_SIG_HASH="sha256" > >> > CONFIG_TCP_MD5SIG=y > >> > CONFIG_CFG80211_REQUIRE_SIGNED_REGDB=y > >> > # DesignWare PCI Core Support > >> > # end of DesignWare PCI Core Support > >> > CONFIG_I2C_DESIGNWARE_CORE=y > >> > # CONFIG_I2C_DESIGNWARE_SLAVE is not set > >> > CONFIG_I2C_DESIGNWARE_PLATFORM=y > >> > CONFIG_I2C_DESIGNWARE_BAYTRAIL=y > >> > # CONFIG_I2C_DESIGNWARE_PCI is not set > >> > # CONFIG_SPI_DESIGNWARE is not set > >> > # CONFIG_SND_HDA_CODEC_SIGMATEL is not set > >> > # CONFIG_USB_ISIGHTFW is not set > >> > CONFIG_FS_VERITY_BUILTIN_SIGNATURES=y > >> > CONFIG_INTEGRITY_SIGNATURE=y > >> > # CONFIG_IMA_SIG_TEMPLATE is not set > >> > CONFIG_IMA_APPRAISE_MODSIG=y > >> > CONFIG_IMA_KEYRINGS_PERMIT_SIGNED_BY_BUILTIN_OR_SECONDARY=y > >> > CONFIG_SIGNED_PE_FILE_VERIFICATION=y > >> > # Certificates for signature checking > >> > CONFIG_MODULE_SIG_KEY="certs/signing_key.pem" > >> > CONFIG_MODULE_SIG_KEY_TYPE_RSA=y > >> > # CONFIG_MODULE_SIG_KEY_TYPE_ECDSA is not set > >> > # end of Certificates for signature checking > >> > CONFIG_CHECK_SIGNATURE=y > >> > CONFIG_SIGNATURE=y > >> > [/CODE] > >> > > >> > > >> > Kindly try to suggest how to compile the source code of kernel for > uefi system with automatic key singing and how to boot the compiled kernel > from grub2. > >> > > >> > > >> > Regards, > >> > Rupesh. > >> > _______________________________________________ > >> > Kernelnewbies mailing list > >> > Kernelnewbies(a)kernelnewbies.org > >> > https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies > >> The "bad shim signature" error is probably from GRUB's "kernel" > >> command. That might mean the "kernel" command has failed to load the > >> kernel, and therefore the "boot" command failed because there is no > >> kernel to transfer control to. > >> > >> -Rogério Valentim > > > > _______________________________________________ > > Kernelnewbies mailing list > > Kernelnewbies(a)kernelnewbies.org > > https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies > If your system is working well, then you don't need to compile a new > Linux kernel. The Linux kernel rarely or never does floating-point > calculations (which is what AVX is used for) and it also doesn't use > SSE for string processing (which is what SSE is used for). I have > looked at the implementation of strcmp() and strcpy() that the Linux > kernel uses, and it doesn't use SSE. Therefore, AVX/SSE won't make a > difference in Linux kernel performance. > > However, user-space applications that frequently process strings > and/or floating-point calculations use the system's C library, which > in most Linux distros is the GNU C library, which optimizes the > strcmp(), strcpy() and other string-processing functions using SSE and > math functions using AVX. But the Linux kernel configuration won't > affect AVX/SSE in userspace. > > Basically, if your system is working well, you don't need to compile > the Linux kernel to use new x86 instruction sets like SSE4 or AVX2. > > -Rogério Valentim >

1 0

Event Tracing in the Linux kernel
by David Kahurani March 3, 2022

March 3, 2022

Hello, I have compiled a pretty recent Linux kernel mostly basing the configuration on the older kernel. I am now testing out the event tracing system but seem to be having a problem. Having enabled all events with # echo *:* > /sys/kernel/debug/tracing/set_event As described here [1], I expect that dmesg shows me some traces but there aren't any. My dmesg logs are empty(just the same as with not having enabled the events). What could I be missing? Am I checking the wrong place for the trace? Are there no tracepoints getting hit? Any hits and pointers would be much appreciated. Regards, David

3 4

weird printk operation early at start_kernel..
by Chan Kim March 3, 2022

March 3, 2022

Hello all, I'm debugging linux boot on an FPGA board (with limited RAM size, only 8MB SRAM in place of DDR3 memory). I'm using linux 5.4.21 for now and because the printk doesn't work yet, I'm writing some info on memory to see how it's doing. After it goes into start_kernel, it calls smp_setup_processor_id(); and inside it, it calls pr_info which is just printk. ==== init/main.c uint64_t mydebug2[64] = {0,}; uint32_t myidx = 0; uint64_t mycnt = 0; // for left shift, 64bit asmlinkage __visible void __init start_kernel(void) { char *command_line; char *after_dashes; uint64_t el; set_task_stack_end_magic(&init_task); mydebug2[(myidx++)&63] = (mycnt++ << 32) + 0x1000; smp_setup_processor_id(); mydebug2[(myidx++)&63] = (mycnt++ << 32) + 0x2000; ==== arch/arm64/kernel/setup.c void __init smp_setup_processor_id(void) { u64 mpidr = read_cpuid_mpidr() & MPIDR_HWID_BITMASK; cpu_logical_map(0) = mpidr; /* * clear __my_cpu_offset on boot CPU to avoid hang caused by * using percpu variable early, for example, lockdep will * access percpu variable inside lock_release */ set_my_cpu_offset(0); mydebug2[(myidx++)&63] = (mycnt++ << 32) + 0x1100; pr_info("test message\n"); // just printk mydebug2[(myidx++)&63] = (mycnt++ << 32) + 0x1200; pr_info("Booting Linux on physical CPU 0x%010lx [0x%08x]\n", // printk (unsigned long)mpidr, read_cpuid_id()); } ==== kernel/printk/printk.c asmlinkage __visible int printk(const char *fmt, ...) { va_list args; int r; va_start(args, fmt); //why.. read_pkcontext(0x32); mydebug2[(myidx++)&63] = (mycnt++ << 32) + 0x1110; r = vprintk_func(fmt, args); mydebug2[(myidx++)&63] = (mycnt++ << 32) + 0x1120; va_end(args); return r; } ==== kernel/printk/printk_safe.c __printf(1, 0) int vprintk_func(const char *fmt, va_list args) { /* * Try to use the main logbuf even in NMI. But avoid calling console * drivers that might have their own locks. */ mydebug2[(myidx++)&63] = (mycnt++ << 32) + 0x1111; if ((this_cpu_read(printk_context) & PRINTK_NMI_DIRECT_CONTEXT_MASK) && raw_spin_trylock(&logbuf_lock)) { mydebug2[(myidx++)&63] = (mycnt++ << 32) + 0x1112; int len; len = vprintk_store(0, LOGLEVEL_DEFAULT, NULL, 0, fmt, args); raw_spin_unlock(&logbuf_lock); defer_console_output(); return len; } mydebug2[(myidx++)&63] = (mycnt++ << 32) + 0x1113; /* Use extra buffer in NMI when logbuf_lock is taken or in safe mode. */ if (this_cpu_read(printk_context) & PRINTK_NMI_CONTEXT_MASK) { mydebug2[(myidx++)&63] = (mycnt++ << 32) + 0x1114; return vprintk_nmi(fmt, args); } //write_pkcontext(); /* Use extra buffer to prevent a recursion deadlock in safe mode. */ mydebug2[(myidx++)&63] = (mycnt++ << 32) + 0x1115; if (this_cpu_read(printk_context) & PRINTK_SAFE_CONTEXT_MASK) { mydebug2[(myidx++)&63] = (mycnt++ << 32) + 0x1116; return vprintk_safe(fmt, args); } /* No obstacles. */ mydebug2[(myidx++)&63] = (mycnt++ << 32) + 0x1117; return vprintk_default(fmt, args); } When I examine mydebug2 data, it is like this. (the physical address of mydebug2 is 0x80622000 and I can read the value in memory using ocd, The LSB 32 bits are displayed first here. The memory is set non-cacheable for this test.) 0x80622000: 00001000 00000000 00001100 00000001 00001110 00000002 00001111 00000003 0x80622020: 00001113 00000004 00001115 00000005 00001117 00000006 00001110 00000007 0x80622040: 00001111 00000008 00001113 00000009 00001115 0000000a 00001116 0000000b 0x80622060: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 As you can see, in the vprintk_safe function, it goes into vprintk_default and then without 0x1120 being written, printk is called again and this time it goes into vprintk_nmi and seems to hang (waiting for console lock I remember..). Could anyone give me some light on what's going on here? (Actual print to serial port is supposed to come out much later inside setup_arch). Any comment will be very much appreciated. Thank you! Chan Kim

1 1