March 2016 - kernelnewbies@kernelnewbies.org - Kernelnewbies Mailing Lists

PR Value in the top command
by Vishwas Srivastava March 22, 2016

March 22, 2016

Hi All, i was trying to understand the the meaning of the PR field in the "top" command. PR field is the kernel notion of importance of a task. It is loosely coupled with the "nice" value, which is a user-space concept. Roughly they are related with.. PR=20+nice (-20<= nice <=19) which means, PR can never go below 0. As an experiment to understand this even more, i created an user-app with real-time scheduling policy (SCHED_FIFO) and ran this application. while the application was running i triggerred "top" and what i see that the PR value for my application displays a -ve value. what does this mean. If at all this is correct, what are the valid range of PR which i should expect?

2 1

Re: copy offload (copy_file_range) and sendfile()
by Manoj Nayak March 22, 2016

March 22, 2016

1. sendfile() and splice uses temporary buffer in terms of pipe. do_splice_direct(in.file, &pos, out.file, &out_pos, count, fl) -> splice_direct_to_actor(struct file *in, struct splice_desc *sd, splice_direct_actor *actor) http://lxr.free-electrons.com/source/fs/splice.c#L602 default_file_splice_read() allocates a page, read data for file1 from disk to the page, then write the page to a pipe. default_file_splice_write() reads from the pipe, writes to a page, then write page to the file2. So this is not a zero-copy in kernel. This can be a zero-copy from userspace point of view as we are not doing copy to userspace. but still a copy is involved a we are doing write to temporary buffer, for example: pipe. 2. if copy_file_range() is defined for a filesystem operation then splice is not used. otherwise copy_file_range() uses splice method of temporary buffer in terms of a pipe. http://lxr.free-electrons.com/source/fs/read_write.c#L1412 ret = file_out->f_op->copy_file_range(file_in, pos_in, file_out, 1413 pos_out, len, flags); 1414 if (ret == -EOPNOTSUPP) 1415 ret = do_splice_direct(file_in, &pos_in, file_out, &pos_out, 1416 len > MAX_RW_COUNT ? MAX_RW_COUNT : len, 0); 1417 copy_file_range() does the following things that is much better method than splice. COPY_FR_COPY means to copy the data normally, accelerating the work at the filesystem level if possible. COPY_FR_REFLINK asks for the destination file to refer to the existing copy of the data without actually copying it. Some filesystems (Btrfs, for example) are able to share references to file blocks in this way. COPY_FR_DEDUP is like COPY_FR_REFLINK, but it only succeeds if the destination range already contains the same data as the source. The end result is files that look the same as before, but which are now sharing the data on-disk. It is thus a way of removing blocks of duplicated data within the filesystem. The COPY_FR_COPY operation will, in the absence of filesystem-level acceleration, copy the data directly through the kernel page cache; it is essentially a splice() operation. Copying through the page cache in this way is clearly more efficient than doing the copy in user space, since it avoids the need to copy the data out of the kernel and back in again. If possible, of course, copying with COPY_FR_REFLINK will be the most efficient approach. copy_file_range() does not do the copy. It does a clone of a range of blocks of a file. 2921 const struct file_operations btrfs_file_operations = { 2922 ... 2935 .copy_file_range = btrfs_copy_file_range, 2936 .clone_file_range = btrfs_clone_file_range, 2937 .dedupe_file_range = btrfs_dedupe_file_range, 2938 }; 3902 ssize_t btrfs_copy_file_range(struct file *file_in, loff_t pos_in, 3903 struct file *file_out, loff_t pos_out, 3904 size_t len, unsigned int flags) 3905 { 3906 ssize_t ret; 3907 3908 ret = btrfs_clone_files(file_out, file_in, pos_in, len, pos_out); 3909 if (ret == 0) 3910 ret = len; 3911 return ret; 3912 } Regards Manoj Nayak

1 0

Re: Attach my own pid
by Nitin Varyani March 22, 2016

March 22, 2016

struct task_struct { volatile long state; void *stack; ... *pid_t pid;* ... } You mean to say that just mapping the *pid_t pid* will do the job. Does the linux kernel not store pid somewhere else while forking a child? On Mon, Mar 21, 2016 at 4:18 PM, Pranay Srivastava <pranjas(a)gmail.com> wrote: > Nitin, > > > On Mon, Mar 21, 2016 at 4:03 PM, Nitin Varyani <varyani.nitin1(a)gmail.com> > wrote: > > .....Continued.... That is, if kernel at Computer 2 finds that pid of a > > process requesting a system call is 1500, the request is forwarded to > slave > > daemon which in turn contacts with the master daemon. Master daemon > requests > > the kernel for the system call and sends the result back to slave daemon. > > I don't think doing this by pid is better. It might suit you currently > but in the long run? > If you are able to send the whole context, why not map that pid to > your context internally instead of relying > on pid which is also visible outside your context. > > > > > On Mon, Mar 21, 2016 at 4:01 PM, Nitin Varyani <varyani.nitin1(a)gmail.com > > > > wrote: > >> > >> I am trying to create a distributed pid space. > >> > >> 0 to 2000 Computer 1 > >> 2001 to 4000 Computer 2 > >> 4001 to 6000 Computer 3 > >> > > your pid 2000 shouldn't have to be same pid 2000 on another node. You > just need the context right? > > >> and so on... > >> > >> I am running a master user-level process at Computer 1 which sends a > >> process context like code, data, registers, PC, etc as well as "pid" to > >> slave processes running at other computers. The responsibility of the > slave > >> process is to fork a new process on order of master process and attach > "pid" > >> given by the master to the new process it has forked. Any system call on > >> slave nodes will have an initial check of " Whether the process belongs > to > >> local node or to the master node?". That is, if kernel at Computer 2 > pid of > >> the process is 1500 > >> > >> > >> > >> On Mon, Mar 21, 2016 at 12:23 PM, <Valdis.Kletnieks(a)vt.edu> wrote: > >>> > >>> On Mon, 21 Mar 2016 10:33:44 +0530, Nitin Varyani said: > >>> > >>> > Sub-task 1: Until now, parent process cannot control the pid of the > >>> > forked > >>> > child. A pid gets assigned as a sequential number by the kernel at > the > >>> > time > >>> > the process is forked . I want to modify kernel in such a way that > >>> > parent > >>> > process can control the pid of the forked child. > >>> > >>> What does controlling the pid gain you? To what purpose? > >>> > >>> > Sub-task 2: On Linux, you can find the maximum PID value for your > >>> > system > >>> > with the following command: > >>> > > >>> > $ cat /proc/sys/kernel/pid_max > >>> > > >>> > Suppose pid_max=2000 for a system. I want that the parent process > >>> > should be > >>> > able to assign a pid which is greater that 2000 to the forked child. > >>> > >>> Again, why would you want to do that? > >>> > >>> Anyhow... > >>> > >>> echo 3000 > /proc/sys/kernel/pid_max > >>> fork a process that gets a pid over 2000. > >>> > >>> Done. > >>> > >>> Note that on 32 bit systems, using a pid_max of over 32768 will cause > >>> various things in /proc to blow up. > >>> > >>> I suspect that you need to think harder about what problem you're > >>> actually > >>> trying to solve here - what will you do with a controlled child PID? > Why > >>> does > >>> it even matter? > >> > >> > > > > > > _______________________________________________ > > Kernelnewbies mailing list > > Kernelnewbies(a)kernelnewbies.org > > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies > > > > > > -- > ---P.K.S >

2 1

"kdumpbase" dracut module
by Ronit Halder March 22, 2016

March 22, 2016

I want to know what is the job of "kdumpbase" dracut module.

4 4

Get sk PID from netfilter target
by Sargun Dhillon March 21, 2016

March 21, 2016

I'm attempting to set the mark of a packet based on the PID it was originated from in the kernel. I came up with the following code as I was trying to work through things, and I had setup the rule on the OUTPUT chain: static unsigned int static unsigned int pidmark_tg(struct sk_buff *skb, const struct xt_action_param *par) { struct file *filp; if (skb->sk == NULL || skb->sk->sk_socket == NULL) { skb->mark = 0; printk("Socket not local, not setting mark?\n"); return XT_CONTINUE; } filp = skb->sk->sk_socket->file; if (filp == NULL) { printk("Filp null. :|\n"); return XT_CONTINUE; } read_lock(&filp->f_owner.lock); const struct cred *cred = filp->f_cred; printk("Uid: %d\n", from_kuid_munged(&init_user_ns, cred->fsuid)); struct task_struct *task; printk("Pid: %d\n", pid_nr(filp->f_owner.pid)); task = pid_task(filp->f_owner.pid, filp->f_owner.pid_type); printk("Task: %x\n", task); read_unlock(&filp->f_owner.lock); return XT_CONTINUE; } Unfortunately, looking at the log, pid always is set to 0, and pid_task always return null. Is there any way I can fetch the pid that created the skb from a netfilter target?

1 0

Linux Kernel Group Scheduling Feature of CFS
by SUNITA March 21, 2016

March 21, 2016

Hello, Completely Fair Scheduler uses group Scheduling. That is every process gets an equal share of CPU time no matter how many threads are there. How to measure that CFS uses Group Scheduling Feature. Which performance tools like time, vmstat will help to prove this. Regards, Suniita

2 2

Querry Regarding Memory Alignment
by Vishwas Srivastava March 21, 2016

March 21, 2016

Hi All, I have 2 questions regarding memory access/alignment 1: can aligned/unaligned virtual address could translate to unaligned/aligned physical address ? or aligned virtual --- > aligned phy and unaligned virtual --- > unaligned phy ?? 2: Since the cpu access the memory in word size, for a large buffer, (say size N bytes where N > wordsize) isn't it sufficient for buffer to be properly aligned if its address % WORD_SIZE == 0? rather than address % N == 0 check?? Since cpu, anyways will not access more than word size at a time so any address which is divisible by wordsize is by default properly aligned from the cpu access point of view? Please share your inputs.

1 0

Can interrupt be missed between interrupt disable/enable window?
by Vishwas Srivastava March 21, 2016

March 21, 2016

Kernel code heavily uses the spinlock primitives spin_lock_irqsave/restore plus local interrupt disabling/ enabling, all across the code. Is there a possibility that the interrupts might get missed in this small window disable interrupts ............. ............. <<<<<<<<<<<---------------------- interrupts is trigerred here enable interrupts specially when the irq affinity has been set to the same core on which the above mentioned code (disabling / enabling the irq's) runs? How the linux deals with this kind of scenario?

3 2

add_disk() make my driver's initialisation, thus kernel stall.
by 张云 March 21, 2016

March 21, 2016

Hi, My memory disk block driver was compiled successfully. But when I insmod my module, initialisation stall at add_disk in setup_dev function. What’s wrong with my gendisk setup code? Thanks. static void setup_dev(struct blkplay_dev *dev, int which) { // setup disk size memset(dev, 0, sizeof(struct blkplay_dev)); dev->size = nsectors * hardsect_size; dev->data = vmalloc(dev->size); if (dev->data == NULL) { printk(KERN_NOTICE "vmalloc failure.\n"); return; } // init request queue spin_lock_init(&dev->lock); dev->queue = blk_init_queue(blkplay_request, &dev->lock); if (dev->queue == NULL) { printk(KERN_NOTICE "init queue failure.\n"); goto out_vfree; } //blk_queue_logical_block_size(dev->queue, hardsect_size); dev->queue->queuedata = dev; dev->gd = alloc_disk(BLKPLAY_MINORS); if (!dev->gd) { printk(KERN_NOTICE "alloc_disk failure!\n"); goto out_vfree; } dev->gd->major = major; dev->gd->first_minor = which * BLKPLAY_MINORS; dev->gd->fops = &blkplay_ops; dev->gd->queue = dev->queue; dev->gd->private_data = dev; snprintf(dev->gd->disk_name, 32, "blkplay%c", which + 'a'); set_capacity(dev->gd, nsectors * (hardsect_size/KERNEL_SECTOR_SIZE)); printk(KERN_ALERT "5\n”); // ********************************************* // initialisation stall at the statement below. // ********************************************* add_disk(dev->gd); printk(KERN_ALERT "6\n"); return; out_vfree: if (dev->data) vfree(dev->data); } The whole module code: #include <linux/module.h> #include <linux/init.h> #include <linux/blkdev.h> #include <linux/genhd.h> #include <linux/bio.h> #include <linux/kernel.h> #include <linux/slab.h> #include <linux/vmalloc.h> #include "debug.h" #define BLKPLAY_MINORS 16 #define KERNEL_SECTOR_SIZE 512 #define DEVICE_NUMBER 1 int major = 0; int nsectors = 1024; int hardsect_size = 512; static const char *module_name = "blkplay"; struct blkplay_dev { int size; uint8_t *data; spinlock_t lock; struct request_queue *queue; struct gendisk *gd; }; struct blkplay_dev * devices; int blkplay_open(struct block_device *dev, fmode_t mode) { return 0; } void blkplay_release(struct gendisk *disk, fmode_t mode) { } /* * Handle an I/O request. */ static void blkplay_transfer(struct blkplay_dev *dev, unsigned long sector, unsigned long nsect, char *buffer, int write) { unsigned long offset = sector*KERNEL_SECTOR_SIZE; unsigned long nbytes = nsect*KERNEL_SECTOR_SIZE; if ((offset + nbytes) > dev->size) { printk (KERN_NOTICE "Beyond-end write (%ld %ld)\n", offset, nbytes); return; } if (write) memcpy(dev->data + offset, buffer, nbytes); else memcpy(buffer, dev->data + offset, nbytes); } /* * Transfer a single BIO. */ static int vmem_disk_xfer_bio(struct blkplay_dev *dev, struct bio *bio) { struct bio_vec bvec; struct bvec_iter iter; sector_t sector = bio->bi_iter.bi_sector; bio_for_each_segment(bvec, bio, iter) { char *buffer = __bio_kmap_atomic(bio, iter); blkplay_transfer(dev, sector, bio_cur_bytes(bio) >> 9, buffer, bio_data_dir(bio) == WRITE); sector += bio_cur_bytes(bio) >> 9; __bio_kunmap_atomic(buffer); } return 0; } void blkplay_request(struct request_queue *q) { struct request *req; struct bio *bio; while (!blk_queue_stopped(q) && (req = blk_peek_request(q)) != NULL) { struct blkplay_dev *dev = req->rq_disk->private_data; blk_start_request(req); if (req->cmd_type != REQ_TYPE_FS) { printk (KERN_NOTICE "Skip non-fs request\n"); blk_end_request_all(req, -EIO); continue; } __rq_for_each_bio(bio, req) vmem_disk_xfer_bio(dev, bio); blk_end_request_all(req, 0); } } const struct block_device_operations blkplay_ops = { //.owner = THIS_MODULE, //.open = blkplay_open, //.release = blkplay_release, }; static void release_dev(struct blkplay_dev *dev) { del_gendisk(dev->gd); put_disk(dev->gd); blk_cleanup_queue(dev->queue); vfree(dev->data); } static void setup_dev(struct blkplay_dev *dev, int which) { // setup disk size memset(dev, 0, sizeof(struct blkplay_dev)); dev->size = nsectors * hardsect_size; dev->data = vmalloc(dev->size); if (dev->data == NULL) { printk(KERN_NOTICE "vmalloc failure.\n"); return; } // init request queue spin_lock_init(&dev->lock); dev->queue = blk_init_queue(blkplay_request, &dev->lock); if (dev->queue == NULL) { printk(KERN_NOTICE "init queue failure.\n"); goto out_vfree; } //blk_queue_logical_block_size(dev->queue, hardsect_size); dev->queue->queuedata = dev; dev->gd = alloc_disk(BLKPLAY_MINORS); if (!dev->gd) { printk(KERN_NOTICE "alloc_disk failure!\n"); goto out_vfree; } dev->gd->major = major; dev->gd->first_minor = which * BLKPLAY_MINORS; dev->gd->fops = &blkplay_ops; dev->gd->queue = dev->queue; dev->gd->private_data = dev; snprintf(dev->gd->disk_name, 32, "blkplay%c", which + 'a'); set_capacity(dev->gd, nsectors * (hardsect_size/KERNEL_SECTOR_SIZE)); printk(KERN_ALERT "5\n"); add_disk(dev->gd); printk(KERN_ALERT "6\n"); return; out_vfree: if (dev->data) vfree(dev->data); } static int __init blkplay_init(void) { int i; int ret; major = register_blkdev(major, module_name); if (0 >= major ) { goto reg_blkplay_fail; } else { printk(KERN_ALERT "Allocate major number %d\n", major); } PDEBUG("Init success!\n"); devices = kmalloc(DEVICE_NUMBER * sizeof(struct blkplay_dev), GFP_KERNEL); if (devices == NULL) { printk(KERN_ALERT "Allocate memory for devices failure.\n"); } for (i = 0; i < DEVICE_NUMBER; i++) { printk(KERN_ALERT "Set up device for %d\n", i); setup_dev(devices + i, i); } return 0; reg_blkplay_fail: PDEBUG("Init Error!\n"); return ret; } static void __exit blkplay_exit(void) { int i; for (i = 0; i < DEVICE_NUMBER; i++) { release_dev(&devices[i]); } unregister_blkdev(major, module_name); kfree(devices); PDEBUG("Exit.\n"); } module_init(blkplay_init); module_exit(blkplay_exit); MODULE_LICENSE("Dual GPL/BSD");

2 4

kernel_thread() causes segfault
by Shashank Khasare March 21, 2016

March 21, 2016

Hi, I want to write a new syscall in which caller process would create kernel thread which shares the process address space, file descriptor table, parent pid etc. The new kernel thread would be clone of current thread but it would never execute any userspace code. The kernel_thread() function with following arguments would be ideal to achieve this task: kernel_thread(some_function, some_args, CLONE_FS | CLONE_FILES | CLONE_PARENT) In latest kernels (v3.1x), this function causes segmentation fault in the user process. However same code works perfectly in older kernels (v2.6). According to this link <https://groups.google.com/forum/#%21searchin/linux.kernel/kernel_thread%28%…>, and this code <http://lxr.free-electrons.com/source/arch/x86/kernel/process_64.c?v=3.16#L1…> (call chain: kernel_thread -> do_fork -> copy_process -> copy_thread), it looks like only the kernel thread can spawn another kernel thread. (I tried to set PF_KTHREAD flag in current->flags before calling kernel_thread function, but the system crashed.) Is there any clean way of creating kernel thread that shares process address space, file descriptor table, parent pid etc? Thanks, Shashank Khasare

1 0