December 2022 - kernelnewbies@kernelnewbies.org

Source code organization
by Billie Alsup (balsup) April 23, 2024

April 23, 2024

I have a set of drivers that I would like to upstream. These are primarily MFD style drivers supporting Cisco-8000 FPGAs. The devices can be instantiated through multiple top level drivers, which provide the access mechanism for the MFD driver. For example, an FPGA can be accessed via PCI, or via i2c, or via SLPC, or via P2PM (a point-to-point interface). We currently build these drivers out of tree, under a single directory drivers/cisco. I note that existing drivers are spread out across the kernel tree, sometimes by bus (pci, i2c), sometimes by function (gpio, net/mdio, spi), and sometimes under the generic mfd. I would like to start the upstream review process for our drivers, but first want to get recommendations on the source code layout. Is it permissible to keep a top level directory such as drivers/cisco to organize our code? It is not only the source code that is affected, but also provides a central place for Kconfig entries. Our FPGAs have multiple logical blocks, each of which is handled by a different MFD driver, e.g. i2c controllers, gpio controllers, spi controllers, mdio controllers. There can be multiple instances of each block as well (so multiple MFD devices are instantiated for each driver). And of course, there can be multiple FPGAs in a system, each with different combinations of logical blocks. The drivers themselves are pretty specific to our FPGAs, thus it makes sense to use Kconfig to select a hardware platform to automatically select the set of MFD drivers (and top level bus drivers) that would apply. Would a source layout putting all the code under drivers/cisco be acceptable in this case, or do I need to move things around and spread out the Kconfig entries across directories? I note that a single drivers/cisco would simplify any related modifications to MAINTAINERS as well.

2 5

How is the size of init ram disk determined when initramfs is used?
by Chan Kim Jan. 2, 2023

Jan. 2, 2023

Hello all, I'm using initramfs.cpio.gz for initial file system image. I embed it in the kernel Image file. After linux boots, when I'm in the shell, I can create files. But I'm curious how much the file system can grow. I remember somewhere reading that the initramdisk is made and the initramfs archive is extracted in the ramdisk. Then how is the ramdisk size determined? During the build usig Kconfig variable, or does it dynamically grow? Thanks in advance. Chan Kim

4 4

Meaning of 'no-map' in reserved-memory, and accessing a memory region from two OSes using physical adderss
by Chan Kim Dec. 27, 2022

Dec. 27, 2022

Hello all, In the device tree, I have declared a reserved-memory node and a child node inside it. And I have put 'no-map' property to that reserved memory and a device node uses that reserved memory. But in the driver for the device, when I use the memremap'ed virtual address, the read/write runs ok. But using physical memory, it crashes. I thought with 'no-map' property, I could just use the physical address but it turned out I can't. The document on reserved-memory says this about the 'no-map' property. (in Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt) no-map (optional) - empty property - Indicates the operating system must not create a virtual mapping of the region as part of its standard mapping of system memory, nor permit speculative access to it under any circumstances other than under the control of the device driver using the region. I can't understand what 'the OS must not create a virtual mapping of the region as part of its standard mapping of system memory'. Any reserved-memory is excluded from the usage of OS, then, what does this 'no-map' mean? The memremap function just ran without error and I could use the region with returned virtual address. The thing I want to do is accessing the memory region with physical address, and I hope another OS which connects to our board using PCIe link (our side is endpoint) can also access the region with the PCIe mapped physical address (by itself declaring the region as reserved-memory). Is this kind things impossible? Thank you for reading Best regards, Chan Kim

1 0

2 modules from 1 source ?
by jim.cromie＠gmail.com Dec. 20, 2022

Dec. 20, 2022

Id like to build 2 modules (with different names) from a single source file, with 2nd being dependent on the 1st. Specifically, Ive got: lib/test_dynamic_debug.c I want A: lib/test_dynamic_debug.ko B: lib/test_dynamic_debug_submod.ko I expect that the code just needs an #ifdef #else #endif block to clearly put the dependor & dependee elements next to each other. My question is how to do this in the Makefile ? this does most of it --- a/lib/Makefile +++ b/lib/Makefile @@ -78,7 +78,8 @@ obj-$(CONFIG_TEST_SORT) += test_sort.o obj-$(CONFIG_TEST_USER_COPY) += test_user_copy.o obj-$(CONFIG_TEST_STATIC_KEYS) += test_static_keys.o obj-$(CONFIG_TEST_STATIC_KEYS) += test_static_key_base.o -obj-$(CONFIG_TEST_DYNAMIC_DEBUG) += test_dynamic_debug.o +obj-$(CONFIG_TEST_DYNAMIC_DEBUG) += test_dynamic_debug.o test_dynamic_debug_submod.o +CFLAGS_test_dynamic_debug_submod.o += -DTEST_DYNAMIC_DEBUG_SUBMOD obj-$(CONFIG_TEST_PRINTF) += test_printf.o but how do I tell it the 2nd target ? make[2]: *** No rule to make target 'lib/test_dynamic_debug_submod.o', needed by 'lib/modules.order'. Stop. make[1]: *** [/home/jimc/projects/lx/wk-test/scripts/Makefile.build:500: lib] Error 2 CFLAGS_test_dynamic_debug_submod.o += -\ DTEST_DYNAMIC_DEBUG_SUBMOD A.ko:

2 3

large global static arrays
by Ani Sinha Dec. 15, 2022

Dec. 15, 2022

Hi I’m curious to know whether large static arrays can be declared inside any module ( kernel source .c file). I am aware that kernel stack space can be limited and so within the function it’s not a good idea to declare large sized arrays. I’m just wondering if the same restriction applies to global declaration of arrays. Thanks Ani

3 5

unmap memory mapped with devm_ioremap_resource
by Adrian Fiergolski Dec. 10, 2022

Dec. 10, 2022

Hello, I am extending xilinx-hls driver (https://github.com/Xilinx/linux-xlnx/blob/master/drivers/media/platform/xil…) to support of_overlay. In order to reflect in the driver changes being applied by the overlay to the device tree, I need to unmap first the memory mapped with devm_ioremap_resource. How to do it properly? I think it's uncommon, as normally kernel (devm_* functions) manages those resources itself with devres, so I can't find inspiration in other drivers. Regards, Adrian

4 7

The method to alloc DMA ring buffer
by Tony He Dec. 8, 2022

Dec. 8, 2022

Hi all, This is my first question in the kernel newbies mailing list. Forgive me if there is incorrect behavior. I'm studying the NIC driver with the book <<Understanding the linux network internals>>. This book uses 3com 3c59x NIC(3c59x.c) as an example. The driver is very old and few people discuss it, but it's a good place to start because it's simpler than many other drivers with advanced offload features. In the interrupt handler boomerang_rx(), I see it pre-allocate one new skb and dma map before calling netif_rx. boomerang_rx() { ...... /* Pre-allocate the replacement skb. If it or its * mapping fails then recycle the buffer thats already * in place */ newskb = netdev_alloc_skb_ip_align(dev, PKT_BUF_SZ); if (!newskb) { dev->stats.rx_dropped++; goto clear_complete; } newdma = dma_map_single(vp->gendev, newskb->data, PKT_BUF_SZ, DMA_FROM_DEVICE); if (dma_mapping_error(vp->gendev, newdma)) { dev->stats.rx_dropped++; consume_skb(newskb); goto clear_complete; } /* Pass up the skbuff already on the Rx ring. */ skb = vp->rx_skbuff[entry]; vp->rx_skbuff[entry] = newskb; vp->rx_ring[entry].addr = cpu_to_le32(newdma); skb_put(skb, pkt_len); dma_unmap_single(vp->gendev, dma, PKT_BUF_SZ, DMA_FROM_DEVICE); vp->rx_nocopy++; ...... netif_rx(skb); ....... } However, I see intel e1000 driver optimizes this. Intel doesn't pre-allocate one at a time. e1000_clean_rx_irq() { ...... /* return some buffers to hardware, one at a time is too slow */ if (unlikely(cleaned_count >= E1000_RX_BUFFER_WRITE)) { adapter->alloc_rx_buf(adapter, rx_ring, cleaned_count); cleaned_count = 0; ...... } I just want to know why this is faster. For all scenarios or some scenarios? Can someone analyse it rigorously? My guess is this method reduces the delay when many packets come, right? Thanks in advance! Tony

2 2

unable to find the virtual to physical mapping
by Arun chandran Dec. 6, 2022

Dec. 6, 2022

Hi, I am trying to find the virtual to physical memory mapping of a process with kernel/linux-4.18.0-348.7.1.el8_5/tools/vm/page-types.c a) It gives the below output for a process with pid 19721 # ./page-types -p 19721 -l | head voffset offset len flags 400 108bf36 1 __RU_lA____M_______________________________ 600 135bf80 1 ___U_l_____Ma_b____________________________ 601 1258e11 1 ___U_l_____Ma_b____________________________ 6aa 110075a 1 ___U_l_____Ma_b____________________________ 7f335f72c 2741b88 4 __RU_lA____M_______________________________ 7f335f730 20546b1 3 __RU_lA____M_______________________________ 7f335f733 2741b3e 2 __RU_lA____M_______________________________ 7f335f735 2741a20 1e __RU_lA____M_______________________________ 7f335f753 2054690 1d __RU_lA____M_______________________________ Here the physical offsets are 28 bits. By adjusting for the 4k pagesize I can derive a meaningful 40 bit physical address. b) for another process it is giving # ./page-types -p 19694 -l | head voffset offset len flags 400 3f0366 1 __RU_lA____M_______________________________ 401 19a32b 1 __RU_lA____M_______________________________ 402 18d702 1 __RU_lA____M_______________________________ 403 3f3fe5 1 __RU_lA____M_______________________________ 404 1fbaa8 1 __RU_lA____M_______________________________ 405 3f3cea 1 __RU_lA____M_______________________________ 406 110455 1 __RU_lA____M_______________________________ 407 30a7a8 1 __RU_lA____M_______________________________ 408 2eaaee 1 __RU_lA____M_______________________________ In this, all the physical offsets are only 24 bits. Why is it like that? The flags are not showing it as a huge page also. RAM is mapped as # cat /proc/iomem | grep -i ram 00001000-0009ffff : System RAM 00100000-03ffffff : System RAM 04047000-75ceffff : System RAM 76000000-9fdbb017 : System RAM 9fdbb018-9fdcc457 : System RAM 9fdcc458-9fdcd017 : System RAM 9fdcd018-9fddb857 : System RAM 9fddb858-a21f1fff : System RAM a2245000-a2271fff : System RAM a2273000-a66c2fff : System RAM a99ff000-abffffff : System RAM 100000000-104fdfffff : System RAM 1050000000-204fefffff : System RAM 2050000000-304fefffff : System RAM 3050000000-404f2fffff : System RAM 4050000000-504fefffff : System RAM 5050000000-604fefffff : System RAM 6050000000-704fefffff : System RAM 7050000000-804fefffff : System RAM How do I find out to which NUMA node's ram the physical address "3f0366" belongs? --Arun

1 1

OBJTOOL Build error
by John Aron Dec. 1, 2022

Dec. 1, 2022

Hello - I have an idea of where to begin: our kernel code compiles and works on Red Hat, CentOS, and Fedora. In Ubuntu 20.04, I have an error. root@form:/home/john/thor-linux/Kernel/ubuntu20.04# make rmmod: ERROR: Module thor is not currently loaded make: [Makefile:7: all] Error 1 (ignored) make[1]: Entering directory '/usr/src/linux-headers-5.4.0-131-generic' CC [M] /home/john/thor-linux/Kernel/ubuntu22.04/thor.o /home/john/thor-linux/Kernel/ubuntu22.04/thor.o: warning: objtool: _Controller_process_response_map()+0x1b3: unreachable instruction Building modules, stage 2. MODPOST 1 modules CC [M] /home/john/thor-linux/Kernel/ubuntu22.04/thor.mod.o LD [M] /home/john/thor-linux/Kernel/ubuntu22.04/thor.ko make[1]: Leaving directory '/usr/src/linux-headers-5.4.0-131-generic' make[1]: Entering directory '/usr/src/linux-headers-5.4.0-131-generic' CLEAN /home/john/thor-linux/Kernel/ubuntu22.04/Module.symvers make[1]: Leaving directory '/usr/src/linux-headers-5.4.0-131-generic' #@sudo dmesg -C #@sudo insmod /usr/local/etc/thor.ko filename: /usr/local/etc/thor.ko version: 0.1 description: THOR KMOD author: Aronetics license: GPL srcversion: BC856FA85DB2FEFD38A1B2A depends: retpoline: Y name: thor vermagic: 5.4.0-131-generic SMP mod_unload modversions #@sudo dmesg root@form:/home/john/thor-linux/Kernel/ubuntu20.04# <mailto:root@form:/home/john/thor-linux/Kernel/ubuntu20.04#> Every 2.0s: tail -n30 /var/lib/dkms/thor/1.0.1/build/make.log DKMS make.log for thor-1.0.1 for kernel 5.4.0-131-generic (x86_64) Thu 24 Nov 2022 01:10:33 PM EST make: Entering directory '/usr/src/linux-headers-5.4.0-131-generic' CC [M] /var/lib/dkms/thor/1.0.1/build/thor.o /var/lib/dkms/thor/1.0.1/build/thor.o: warning: objtool: _Controller_process_response_map()+0x1b3: unreachable instruction Building modules, stage 2. MODPOST 1 modules CC [M] /var/lib/dkms/thor/1.0.1/build/thor.mod.o LD [M] /var/lib/dkms/thor/1.0.1/build/thor.ko make: Leaving directory '/usr/src/linux-headers-5.4.0-131-generic' Is this an error in objtool on Ubuntu within /usr/src/linux-headers-5.4.0-${26-130}/tools/objtool ? >From RHEL 8 [root@profile rhel8]# make make[1]: Entering directory '/usr/src/kernels/4.18.0-372.26.1.el8_6.x86_64' CC [M] /home/john/thor-linux/Kernel/rhel8/thor.o Building modules, stage 2. MODPOST 1 modules CC /home/john/thor-linux/Kernel/rhel8/thor.mod.o LD [M] /home/john/thor-linux/Kernel/rhel8/thor.ko make[1]: Leaving directory '/usr/src/kernels/4.18.0-372.26.1.el8_6.x86_64' make[1]: Entering directory '/usr/src/kernels/4.18.0-372.26.1.el8_6.x86_64' CLEAN /home/john/thor-linux/Kernel/rhel8/.tmp_versions CLEAN /home/john/thor-linux/Kernel/rhel8/Module.symvers make[1]: Leaving directory '/usr/src/kernels/4.18.0-372.26.1.el8_6.x86_64' #@sudo dmesg -C #@sudo insmod /usr/local/etc/thor.ko filename: /usr/local/etc/thor.ko version: 0.1 description: THOR KMOD author: Aronetics license: GPL rhelversion: 8.6 srcversion: FC77130B3C3461BF8DE20B6 depends: name: thor vermagic: 4.18.0-372.26.1.el8_6.x86_64 SMP mod_unload modversions #@sudo dmesg [root@profile rhel8]# John

4 5