- kernelnewbies@kernelnewbies.org - Kernelnewbies Mailing Lists

MADV_ZERO
by Robert Nagy Sept. 1, 2015

Sept. 1, 2015

I have recently switched over to Linux after encountering an issue that seems unsolvable in Windows with the hope of finding a solution in Linux. Basically what I need to achieve is IPC persisted to a huge file. I’m doing this currently by memory-mapping a huge backing file and always sequentially writing to the file in a circular fashion 24/7. However, this has some major throughput issues since overwriting pages will always cause a page-faults, even if entire pages are overwritten, which totally trashes disk performance. Basically I would need a flag for madvice (e.g. MADV_ZERO) with similar functionality to FALLOC_FL_ZERO_RANGE so that I would get much faster zero fill page faults instead. The closest I’ve come is to use MADV_REMOVE before overwriting the range, however, that is a suboptimal as it will from my understanding fragment the backing file and potentially degrade performance over time. What I’d like to be able to do is something like: int mm_fast_write(void* dst, void* src, size_t length) { if (dst & ~PAGE_MASK) return -EINVAL; if (src & ~PAGE_MASK) return -EINVAL; if (length & ~PAGE_MASK) return -EINVAL; madvice(ptr, len, MADV_ZERO); memcpy(ptr, src, len); madvice(ptr, len, MADV_DONTNEED); // Might not do anything without msync? return 0; } Is it possible to implement or emulate something like MADV_ZERO in user mode? Or should I look into modifying the kernel? I believe it could be implemented based on madvice_remove by simply replacing the FALLOC_FL_PUNCH_HOLE flag with FALLOC_FL_ZERO_RANGE?

1 0

Re: [PATCH v2 0/3] SysFS driver for QEMU fw_cfg device
by Gabriel L. Somlo Sept. 1, 2015

Sept. 1, 2015

Hi Christopher, On Wed, Aug 26, 2015 at 02:15:03PM -0400, Christopher Covington wrote: > On 08/19/2015 04:49 PM, Gabriel L. Somlo wrote: > > On Wed, Aug 19, 2015 at 11:42:02AM +0200, Ard Biesheuvel wrote: > >> On 19 August 2015 at 11:38, Ard Biesheuvel <ard.biesheuvel(a)linaro.org> wrote: > >>> From: "Gabriel L. Somlo" <somlo(a)cmu.edu> > >>>> Several different architectures supported by QEMU are set up with a > >>>> "firmware configuration" (fw_cfg) device, used to pass configuration > >>>> "blobs" into the guest by the host running QEMU. > >>>> > >>>> Historically, these config blobs were mostly of interest to the guest > >>>> BIOS, but since QEMU v2.4 it is possible to insert arbitrary blobs via > >>>> the command line, which makes them potentially interesting to userspace > >>>> (e.g. for passing early boot environment variables, etc.). > >>>> > >>> > >>> Does 'potentially interesting' mean you have a use case? Could you elaborate? > > > > My personal one would be something like: > > > > cat > guestinfo.txt << EOT > > KEY1="val1" > > KEY2="val2" > > ... > > EOT > > > > qemu-system-x86_64 ... -fw-cfg name="opt/guestinfo",file=./guestinfo.txt ... > > > > Then, from inside the guest: > > > > . /sys/firmware/qemu_fw_cfg/by_name/opt/guestinfo/raw > > > > do_something_with $KEY1 $KEY2 > > ... > > > > But I'm thinking this is only one of the many positive things one > > could do with the ability to access random host-supplied blobs from > > guest userspace :) > > I do this with kernel parameters: > > host: > qemu-system-aarch64 -append="KEY1=val1 KEY2=val2" > > guest: > KEY1=`sed -nr s/.*KEY1=([^ ]+).*/\1/ /proc/cmdline` > KEY2=`sed -nr s/.*KEY2=([^ ]+).*/\1/ /proc/cmdline` > > do_something_with $KEY1 $KEY2 > > In practice it's just script=hostfile, where hostfile is available to the > guest via a 9P passthrough filesystem mount. > > While quite architecture specific, I've also previously used an > "angel-cmdline" tool for similar purposes. Peter's recent semihosting patches > support such a tool for AArch64. (On AArch32 upstream QEMU disallows > semihosting from userspace.) > > Before I had 9P on all the simulators I regularly ran, I used a semihosting > based "angel-load" tool. Someone (maybe it was you) did suggest this during an early thread on the QEMU dev list. I had considered this, then thought about piggybacking on smbios (e.g. the type 40 "additional information" table), but then realized "wait, smbios is currently being inserted into the guest via fw_cfg, so maybe direct fw_cfg blob transfer *is* the most asynchronous and out-of-band way I can do this... :) True, writing a fw_cfg driver is still Linux-specific (leaves out Windows, which is something I still care about), but dumping a fw_cfg blob using 'cat' or 'cp' on linux has a certain appeal :) Yeah, the immediate use case I have personally can be worked around with kernel command line args, in a slightly less out-of-band, but still quite serviceable way. I'm thinking a fw_cfg driver is just a flexible way to make other use cases possible in a (IMHO) neat and clean way... Plus, it got me to learn about kobjects, now studying DTs on ARM, so it's fun ;) Thanks, --Gabriel

1 0

line wrap error
by punit vara Sept. 1, 2015

Sept. 1, 2015

Hi I am new to kernel development .I want to submit patch but I checked my patch it showed ERROR: patch seems to be corrupt (line wrapped?) how to remove this error ? can any one help.

2 1

How can I shrink the Linux page cache from within kernel space?
by David Sorber Aug. 29, 2015

Aug. 29, 2015

I posted this question on stackoverflow yesterday but it occurs to me that this is a far better place to ask: I'm working on a system that involves some custom hardware and a custom Linux device driver I wrote for the hardware. The system occasionally needs to move large amounts of data very rapidly and therefore my driver dynamically (i.e. when needed) allocates large (1 GB) DMA buffers which are used and then freed when they are no longer needed. To allocate such large buffers I actually allocate a bunch of smaller buffers (256 X 4MB) using dma_alloc_coherent and then map them contiguously into user space using remap_pfn_range. This works very well most of the time. During testing, after the system has been running test cases for a long time, I sometimes see DMA allocation failures where one of the dma_alloc_coherent calls in my driver fails which causes my application layer software to crash. I was finally able to track down this problem and I discovered that when I see DMA allocation failures the Linux kernel page cache is very full. For example, on the last failure that I captured the page cache filled 27 GB of the 32 GB of RAM on my system. I suspected that the page cache "fullness" was causing dma_alloc_coherent calls to fail. To test this theory I manually emptied the page cache using: # echo 1 > /proc/sys/vm/drop_caches This dropped the size of the cache from 27 GB to 94 MB and I was able to allocate 20+ 1 GB DMA buffers with no issues. Clearly the page cache is a beneficial thing so I would prefer not to have to completely empty it every time I run out of space when allocating DMA buffers. My questions is this: how can I dynamically shrink the page cache in kernel space such that if a call to dma_alloc_coherent fails I can recover just enough space so that I can retry the call and have it succeed? My system is x86_64 based running a 3.16.x Linux kernel. I have found some vague references that suggest what I'm attempting may be possible, for example "These objects are automatically reclaimed by the kernel when memory is needed elsewhere on the system." (from: https://www.kernel.org/doc/Documentation/sysctl/vm.txt) But I have not yet found any specifics that indicate how the memory is reclaimed. Any assistance with this would be greatly appreciated! -David

3 2

Another idea for a patch
by Umair Khan Aug. 29, 2015

Aug. 29, 2015

Hi KNB, I finally realized that the idea of writing a kernel patch to make brightness persistent across reboot was bad. I thought of another thing which could be done. In my laptop, there is a key called "Fn". Also, on some of the function keys, there are small symbols for a particular operation. Those keys coupled with the Fn key perform that particular operation. Now, Fn coupled with F6 key should disable the trackpad. But, it doesn't do so. It does disable trackpad in Windows though. Do you think this could be a suitable patch ? If yes, from where should I start ? On a separate note, I'm 1/3rd way across ldd3. Umair Delhi, India

1 0

Process for adding a new source(driver) to linux kernel tree
by Raul Piper Aug. 29, 2015

Aug. 29, 2015

HI, I wanted to know what is the process for adding a new source(say driver) to the kernel tree.I am not asking for how to add a driver ( like modifying make file, Kconfig, adding driver to staging or my_driver folder) but the process for adding it to the source tree Like : - which branch to add -Any format of the files or directory - whom to mail(maintainer,developer,tester,etc) - Do I have to upload any test results - Do I have to build it for any minimum number of platforms - Do I need to send mail to the maintainer of that subsystem or fraemwork? - Any tool to run the coding conventions checks,bugs checks,Memory leak checks,etc Kindly let me know. Thanks and Rgds, Rp

2 2

Identify Beacon Frame
by Gunjan Mehta Aug. 28, 2015

Aug. 28, 2015

Hi All, I am new to kernel development. I am writing a kernel module that captures packet at the netfilter pre - routing hook. I only want to capture the BEACON FRAME. I have few queries: 1>When the beacon frame comes to receiving machine do the sk_buff gets allocated? 2> How can i identify the beacon frame at the pre routing hook? 3> Beacon frame consist of a string for authentication of client with the server, can i use sk_buff pointers to get access to that string, if sk_buff is allocated? Regards Gunjan

4 7

[PATCH 0/3] SysFS driver for QEMU firmware config device (fw_cfg)
by Gabriel L. Somlo Aug. 27, 2015

Aug. 27, 2015

From: "Gabriel Somlo" <somlo(a)cmu.edu> This patch set makes QEMU fw_cfg blobs available for viewing (read-only) via SysFS. Several different architectures supported by QEMU are set up with a "firmware configuration" (fw_cfg) device, used to pass configuration "blobs" into the guest by the host running QEMU. Historically, these config blobs were mostly of interest to the guest BIOS, but since QEMU v2.4 it is possible to insert arbitrary blobs via the command line, which makes them potentially interesting to userspace (e.g. for passing early boot environment variables, etc.). In addition to cc-ing the people and lists indicated by get-maintainer.pl, I've added a few extra lists suggested by Matt Fleming on the qemu-devel list, as well as the qemu-devel list itself. Also cc-ing kernelnewbies, as this is my very first kenel contribution, so please go easy on me for whatever silly n00b mistakes I might have still missed, in spite of trying hard to do all my homework properly... :) The series consists of three patches: 1/3 - probes for the qemu fw_cfg device in locations known to work on the supported architectures, in decreasing order of "likelihood". While it *may* be possible to detect the presence of fw_cfg via acpi or dtb (on x86 and arm, respectively), there's no way I know of attempting that on sun4 and ppc/mac, so I've stuck with simply probing (the fw_cfg_modes[] structure and fw_cfg_io_probe() function) in fw_cfg.c. I could use some advice on how else that could be done more elegantly, if needed. Upon successfully detecting a present fw_cfg device, we set up /sys/firmware/fw_cfg/by_select entries for each blob available on the fw_cfg device. 2/3 - export kset_find_obj() (in lib/kobject.c) for use with modules, which will come in handy in patch #3, below. 3/3 - add a "user friendly" way of listing fw_cfg blobs by name rather than by unique selector key. Since fw_cfg blob names are traditionally set up to look like path names, it would be nice to mirror that fact when displaying them under /sys/firmware/fw_cfg in a "by_name" subdirectory. I'm using ksets to reflect subdirectories matching "dirname" tokens separated by '/' within each fw_cfg blob "filename", and symlinks into the "by_select" subdirectory for each "basename". Since the fw_cfg device doesn't enforce that blob names have well-behaved and non-conflicting names, it is possible (though unlikely) that there will be blobs named: "etc/foo/bar" and "etc/foo" where "foo" will try to be both a subdirectory AND a symlink under /sys/firmware/fw_cfg/by_name/etc/. In such an event, the latter fw_cfg blob will simply be skipped from having an entry created under the user-friendly "by_name" subdirectory, remaining listed only as an entry under the "by_select" subdirectory. I have tested this on x86_64 and armv7hl+lpae kernels. Builds and installs OK as both a module and linked directly into the kernel. TIA for the feedback and advice, --Gabriel Gabriel Somlo (3): firmware: introduce sysfs driver for QEMU's fw_cfg device kobject: export kset_find_obj() to be used from modules firmware: fw_cfg: create directory hierarchy for fw_cfg file names Documentation/ABI/testing/sysfs-firmware-fw_cfg | 211 +++++++++ drivers/firmware/Kconfig | 10 + drivers/firmware/Makefile | 1 + drivers/firmware/fw_cfg.c | 542 ++++++++++++++++++++++++ lib/kobject.c | 1 + 5 files changed, 765 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-firmware-fw_cfg create mode 100644 drivers/firmware/fw_cfg.c -- 2.4.3

2 6

SLUB vs SLAB allocator
by Navin Parakkal Aug. 27, 2015

Aug. 27, 2015

Hi, I found that in many worst case scenarios like fragmention of allocator , slub performs well than slab. I also noticed that Centos /Ubuntu etc switched to SLUB but SLES still uses SLAB in the default image. Any particular reason where SLAB is the choice ? I ask this since i find SLUB to be the default choice since 2.6.23 as per https://en.wikipedia.org/wiki/SLUB_(software) Regards, Navin

1 0

Planning to write a patch
by Umair Khan Aug. 26, 2015

Aug. 26, 2015

Hello everyone, I've been thinking of writing a patch a lot lately. But with my level of knowledge I cannot do something groundbreaking. One thing which came to my mind is to write a patch related to the driver which controls the brightness of the display. What happens now is I lower down the brightness, and when I restart the laptop, it's back to the highest amount. I'm using Ubuntu 14.04 LTS on top of Linux Kernel 4.2 rc3+. I was thinking of writing the current brightness to sysfs and read that value when the driver is started during the boot. So, my question is that is it already implemented in the driver and just that Ubuntu resets it on every reboot from userspace ? And if it is not implemented, is it worth implementing ? Thanks Umair Delhi, India

4 7