August 2015 - kernelnewbies@kernelnewbies.org

Comprehensive Linux API manual by subsystem Required
by Sasha Mckinsey Aug. 21, 2015

Aug. 21, 2015

Hello, I know this is not a question which belongs in the Newbies list but there is a strong chance that someone out here might know about such a resource. I am developing a user space program which interacts with "file system" and "cryptography" so I looking for a comprehensive on the lines of "MSDN in Windows" which talks about the API by subsystem. Kindly provide the URL if you have such a manual/ resource in mind which is available on the net. Thanks!

3 2

samples driver is not compiling and building at present dir ".ko" file
by ravi ranjan Mishra Aug. 21, 2015

Aug. 21, 2015

Hi, Just for learning purpose , i need to compile the sample directory in mainline kernel . can anybody tell to how build samples directory through make. Thanks, Ravi

2 1

Storage Training and Development Project
by Robert Clove Aug. 21, 2015

Aug. 21, 2015

Hi, I am looking to learn storage technology/domain (NVMe included) and want to do some development project on the same. Can some one provide me with good institutes in India that provide the same or in US that provide online training . Regards

2 1

Re: [PATCH v2 0/3] SysFS driver for QEMU fw_cfg device
by Ard Biesheuvel Aug. 21, 2015

Aug. 21, 2015

(missed some cc's) On 19 August 2015 at 11:38, Ard Biesheuvel <ard.biesheuvel(a)linaro.org> wrote: > From: "Gabriel L. Somlo" <somlo(a)cmu.edu> > > Hi Gabriel, > >> Several different architectures supported by QEMU are set up with a >> "firmware configuration" (fw_cfg) device, used to pass configuration >> "blobs" into the guest by the host running QEMU. >> >> Historically, these config blobs were mostly of interest to the guest >> BIOS, but since QEMU v2.4 it is possible to insert arbitrary blobs via >> the command line, which makes them potentially interesting to userspace >> (e.g. for passing early boot environment variables, etc.). >> > > Does 'potentially interesting' mean you have a use case? Could you elaborate? > >> In addition to cc-ing the people and lists indicated by get-maintainer.pl, >> I've added a few extra lists suggested by Matt Fleming on the qemu-devel >> list, as well as the qemu-devel list itself. >> >> Also cc-ing kernelnewbies, as this is my very first kenel contribution, >> so please go easy on me for whatever silly n00b mistakes I might have still >> missed, in spite of trying hard to do all my homework properly... :) >> >> The series consists of three patches: >> >> 1/3 - probes for the qemu fw_cfg device in locations known to work on >> the supported architectures, in decreasing order of "likelihood". >> >> While it *may* be possible to detect the presence of fw_cfg via >> acpi or dtb (on x86 and arm, respectively), there's no way I know >> of attempting that on sun4 and ppc/mac, so I've stuck with simply >> probing (the fw_cfg_modes[] structure and fw_cfg_io_probe() function) >> in fw_cfg.c. I could use some advice on how else that could be >> done more elegantly, if needed. >> > > Sorry, but this is really out of the question, at least on ARM, but surely on > other architectures as well. You can't just go around and probe random memory > addresses. Perhaps QEMU tolerates it, but on anything that resembles a real > system, this will immediately blow up. Also, what happens if the QEMU memory > map changes? Add more probes addresses? > > It is not /that/ difficult to simply wire it up to the DT and ACPI > infrastructures, there are plenty of examples in the kernel tree how to > accomplish that. As a bonus, it removes all the arch specific knowledge > from your code, which means that if QEMU grows support for another DT or > ACPI based architecture, it will just work. > > I am not sure how relevant sun4 and ppc/mac are for what you are trying to > accomplish, but perhaps it would be best to focus on x86 and ARM for now > and do it correctly. If the probing is actually needed, you can always add > it later. > > -- > Ard. >

3 4

Placement Allocation within the kernel
by Kenneth Adam Miller Aug. 20, 2015

Aug. 20, 2015

Suppose I want to do something analogous to C++'s new in userspace. But instead, I want an entirely new page table to be constructed at the location of my choosing. In addition, I want a specific region for that page table to manage, and this requires that this region no longer be available to the rest of kernel code in order that it cannot be served for allocation elsewhere. To illustrate what I want to do, I think it would be ok to imagine that I allocate a very large contiguous region of memory within which to do this. Once that region of memory is allocated, it can't be served elsewhere, so that serves my purpose. Then what I want to do is have an equivalent of kmalloc and free that not only takes size and a pointer respectively, but also the table which to reference in order to do the allocation from. To begin with, before I do work allocation with this table, I need a function that will construct such a table correctly at the location of my choosing within the large block that I mentioned that I've allocated before. The end result is what I need, and from which I've derived the above requirements: I can have the default memory management facilities within the kernel reused to suit my needs. Kmalloc and kfree will coalesce the table behind the scenes for me, which is what I want to avoid. Is the above possible? I'm pretty sure that it is.

2 1

msleep() in interrupt handler?
by Woody Wu Aug. 20, 2015

Aug. 20, 2015

Hi List, What will happen if call msleep() in an interrupt handler? I knew the msleep() will cause the calling process to sleep, but in the context of interruption, there is no process, so I am wondering what is getting 'sleep' in this situation. -woody -- Sent from Gmail Mobile

4 13

Fwd: is it possible to liimit the softirq re entrance times
by vichy Aug. 20, 2015

Aug. 20, 2015

after I tracing softirq flow in ARM platform, I have some colusion like below: 1. softirq is running at the end of any irq context, when irq is eabled. 2. as long as any Hard irq happen, softirq can be interrupt and reentrant My question are: 1. in __do_sofiq, there is a jiffies setting to make sure the whole softirq will not took to long. But in my case is there any many hard irq happen like below trace_softirq_entry(vec_nr); ------a h->action(h); trace_softirq_exit(vec_nr); ---------b from lttng, I see there 100 interrupt happen from entry to exit. is there kernel config or module parameter to let me down grade the hard irq count from 100 --> 80 or 90 and event can controlled me. I don't mean I control the count of hard irq, but I found this softirq is interrupt by 80 times. then I will not enable_irq until I finished the job. if no, isn't it possible to make starvation in above a and b? really appreciate ur kind help,

2 1

Fwd: 14.31% send_full_color_rect.isra.12
by Darshan Ghumare Aug. 20, 2015

Aug. 20, 2015

Hi, I ran following command, ./x86_64-softmmu/qemu-system-x86_64 -k en-us --show-cursor --enable-kvm -smp 2 -m 1024 -hda /var/opt/kvm/img/ubuntu14.04.img -drive if=none,id=drive1,cache=none,format=raw,aio=native,file=cvbd:// 127.0.0.1:6380/win7.img?block_size=4096 -device virtio-blk,drive=drive1 -vnc :0 perf out shoed me 14.31% send_full_color_rect.isra.12. Following is the output of 14.31% qemu-system-x86 libz.so.1.2.8 [.] 0x0000000000002ae7 ◆ - 0x7f6660d67ae7 ▒ + 0x7f6660d68968 ▒ + deflate ▒ - tight_compress_data ▒ - 14.31% send_full_color_rect.isra.12 ▒ send_sub_rect ▒ - tight_send_framebuffer_update ▒ 14.23% vnc_worker_thread_loop ▒ - 0.08% tight_send_framebuffer_update ▒ - 0.05% tight_send_framebuffer_update ▒ - tight_send_framebuffer_update ▒ 0.03% vnc_worker_thread_loop ▒ - 0.01% tight_send_framebuffer_update ▒ vnc_worker_thread_loop ▒ 0.03% vnc_worker_thread_loop ▒ - 0.00% send_sub_rect ▒ tight_send_framebuffer_update ▒ vnc_worker_thread_loop ▒ + 4.68% qemu-system-x86 libz.so.1.2.8 [.] 0x0000000000002ac0 ▒ + 3.65% qemu-system-x86 libz.so.1.2.8 [.] 0x0000000000002ae1 ▒ + 3.56% qemu-system-x86 libz.so.1.2.8 [.] 0x0000000000002ad0 ▒ + 2.71% qemu-system-x86 libz.so.1.2.8 [.] 0x0000000000002ac7 ▒ + 2.31% qemu-system-x86 qemu-system-x86_64 [.] vnc_refresh_server_surface ▒ - 1.90% qemu-system-x86 [kernel.kallsyms] [k] vcpu_enter_guest Following are the commands that I ran, root@Machine3:~# perf record --call-graph dwarf -p 4101 ^C[ perf record: Woken up 2141 times to write data ] [ perf record: Captured and wrote 536.328 MB perf.data (~23432515 samples) ] root@Machine3:~# perf report -g graph --no-children root@Machine3:~# uname -a Linux Machine3 3.16.0-46-generic #62~14.04.1-Ubuntu SMP Tue Aug 11 16:27:16 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux root@Machine3:~# egrep -c '(vmx|svm)' /proc/cpuinfo 4 root@Machine3:~# kvm-ok INFO: /dev/kvm exists KVM acceleration can be used Any reason why its taking so much time. Thanks, Darshan

1 0

[Off-topic] SCM using git
by amit mehta Aug. 19, 2015

Aug. 19, 2015

Hello All, This query is not about Linux kernel, but is rather generic query on development framework with git. Since, Linux Kernel project is significantly large, with astonishing number of people involved and large number of branches, I'm assuming that people have faced similar situation and your advice and guidance would be of great help to me. I'm currently pursuing Masters program and also working in a small tech Startup in the IOT domain. I'm involved in Firmware Development on ARM based SOCs and our team comprise three Engineers. We use git for SCM. Since we are in a very early phase and have very little turnaround time, from the requirement to final integration, we do not have a proper review process and hence everyone is allowed to commit in the master branch, or create, delete a branch, or a tag at will. We also do not have much insight into, how long a feature based branch would last. In last couple of months, I've seen some stale branches on which the development stopped quite early and some very active branch as well. Without much insight and probably due to not so well defined process, It is already becoming difficult to properly maintain the code consistency, filtering etc across several branches. For example, I create a feature remote branch (say origin/featureX) on which I continue my development and soon I realize that part (There is no way to pick partial commit) or a full commit that I recently made for the featureX should also go into another branches such as master and say featureY branch as well. But since, my colleague has been working on master, I get conflicts when I try to cherry-pick (At this moment, I do not want to rebase or merge two branches, I only need few of the commits from featureX branch to go into master and featureY). I resolve the conflicts and life goes on, but problem happens again later in future, if the two branches divert too much and a further need for cherry-picking or rebasing or merging is required. The point is that I also believe that it is a very good idea to maintain separate branches, based on feature or some other requirements, but it is also a n00bs nightmare when the conflicts are very high. There are also incidents when somebody forgot to add a particular commit to another branch and realized only later, when he was faced with debugging a issue and had to go through the git history, to figure out the issue. It would be nice, If for example a particular commit can be automatically propagated to other branches in this case. Please pitch in your suggestions, rule of thumb, tools and your way of countering such issue with less pain. Thanks, Amit

3 4

Safety in Kernel Development
by Kenneth Adam Miller Aug. 18, 2015

Aug. 18, 2015

Ok- so I know that C is the defacto standard for kernel development. What I'm not saying is that we should all move away from it or that it should be adopted internally. What I am saying is related to security concerns in developing a kernel driver. What may come of it may generally allow for better quality, but that's a separate topic. So kernel programming is very hard. It has both a high bar to entrance and even just getting code to compile and run is not really any guarantee at all that you've done a good job of authoring a kernel driver. I don't really believe that things like Klee really find all errors, but I think that a defense in depth approach would be good. So, when I can get my kernel object to compile, I know that I can test that it runs, but I would also like to have the confidence to know that it won't leak kernel memory or other resources and for that matter will not deference an invalid pointer. Things like Rust allow for better type safety help. In userland programs, SFI is good as a passive backup to type safety but I don't think that SFI is applicable to kernel land because the execution boundaries are not set up under a specific virtual memory scheme. CFI would also be good, but I don't know of any compiler implementation that I can use off the shelf in a kernel programming environment. I guess the best option IMHO is some way to codify the restrictions and semantics of operation somehow into an expressive language that can be checked at compile time. So, in the case of re-entrancy, I'd like an error at compilation time that could just prevent the entrance of bad code. In our case, we'd rather have some good code than a lot of bad code. Does anybody have any recommendations? Or shared interest?

8 17