January 2017 - kernelnewbies@kernelnewbies.org

linux-mm mailing list
by Johannes Thoma Jan. 6, 2017

Jan. 6, 2017

Hi Kernel hackers, I have a proposal for an improvement of the CMA allocator (just about 5 lines of code to make it more robust). I would like to send it to the linux-mm mailing list (linux-mm(a)kvack.org according to MAINTAINERS), but it seems to be inactive. What is the best way to reach the maintainer of mm/cma.c ? Thanks a lot, - Johannes

3 3

Re: [PATCH] mm: cma: Retry from the beginning of cma area if all blocks are busy
by Johannes Thoma Jan. 6, 2017

Jan. 6, 2017

Hi Vladis, Thanks for your quick reply. Am 06.01.17 um 17:54 schrieb Valdis.Kletnieks(a)vt.edu: > On Fri, 06 Jan 2017 17:16:11 +0100, johannes(a)johannesthoma.com said: >> From: Johannes Thoma <johannes(a)johannesthoma.com> > >> This patch introduces a little extra loop that causes cma_alloc to >> rescan from the beginning when all checked memory areas are busy. > >> for (;;) { >> mutex_lock(&cma->lock); >> +scan_again: >> bitmap_no = bitmap_find_next_zero_area_off(cma->bitmap, >> bitmap_maxno, start, bitmap_count, mask, >> offset); >> if (bitmap_no >= bitmap_maxno) { > > It worries me that a few lines above, we have > > if (bitmap_count > bitmap_maxno) > return NULL; > > In this case, is >= correct rather than > ? > That might be the case, but would be an extra patch. >> + * Restart from the beginning if all areas were busy. >> + * This handles false failures when count is close >> + * to overall CMA size and the checked areas were >> + * busy temporarily. >> + */ >> + if (start != 0 && retries--) { > > Do we have any guarantee, or even good reason to believe, that this > will eventually make forward progress, or can the goto hang everything? > I'm not thrilled by a potentially unbounded loop inside a mutex_lock(), > especially when you holding the mutex may be preventing something else > from changing things so forward progress can be made.... > Good point. That is what the retries variable is good for. To make this more obvious, I should write: if (start != 0) { retries--; if (retries > 0) { start = 0; goto scan_again; } } (with int retries = 10; at the beginning of the function). Agreed if (start != 0 && retries--) { isn't good coding style. That should restrict it to a maximum of 10 iterations. This could be lowered if the size requested is smaller. Best, - Johannes

1 0

[PATCH] mm: cma: Retry from the beginning of cma area if all blocks are busy
by johannes＠johannesthoma.com Jan. 6, 2017

Jan. 6, 2017

From: Johannes Thoma <johannes(a)johannesthoma.com> When the size of the requested block is close to the total size of the contigous memory, cma_alloc fails early when no more free memory blocks are found by bitmap_find_next_zero_area_off and the memory areas tested are temporarily busy. This patch introduces a little extra loop that causes cma_alloc to rescan from the beginning when all checked memory areas are busy. We ran into this problem when a GPU driver tried to allocate cma memory on a busy system, but failed because the memory area was busy. Signed-off-by: Johannes Thoma <johannes(a)johannesthoma.com> --- mm/cma.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/cma.c b/mm/cma.c index c960459..35d68c8 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -368,6 +368,7 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align) unsigned long pfn = -1; unsigned long start = 0; unsigned long bitmap_maxno, bitmap_no, bitmap_count; + int retries = 10; struct page *page = NULL; int ret; @@ -390,10 +391,21 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align) for (;;) { mutex_lock(&cma->lock); +scan_again: bitmap_no = bitmap_find_next_zero_area_off(cma->bitmap, bitmap_maxno, start, bitmap_count, mask, offset); if (bitmap_no >= bitmap_maxno) { + /* + * Restart from the beginning if all areas were busy. + * This handles false failures when count is close + * to overall CMA size and the checked areas were + * busy temporarily. + */ + if (start != 0 && retries--) { + start = 0; + goto scan_again; + } mutex_unlock(&cma->lock); break; } -- 2.8.0-rc4

1 0

adding 32 bit compatibility layer in custom kernel module
by Pradeepa Kumar Jan. 4, 2017

Jan. 4, 2017

Hi experts down votefavorite <http://stackoverflow.com/questions/41455943/adding-32-bit-compatibility-lay…> in my 64 bit kernel, I have a custom kernel module providing new protocol and providing socket system calls. it works fine when 64 bit app runs. i am seeing issues when 32 bit app runs (for exp recvmsg() call does not work if msg has cmsghdrs as struct size of cmsghdr is different in 32 bit and 64 bit). This is because my custom kernel module does not have 32 bit compatibility layer ( but linux kernel has this in compat.c etc). - is it simple to add compatibility layer to my custom kernel module - how do i do this (any links ?) Thanks

3 5

[MMC_SPI] CMD18 error error -84 transferring data
by Amit Mahadik Jan. 4, 2017

Jan. 4, 2017

Hello, I am using ARM (CortexA9)development board which has SPI controller running at 5MHz. I want to interface MMC card on SPI bus. I have made changes to my board.c file to include the mmc_spi driver and mentioned the Chip select, mode (MODE 0) and frequency of operation (started with 40kHz). The initial card detection goes well. However, there are CRC errors when the block layer tries to read the partition table. I have attached log for reference. The card gets detected on Linux/Windows PC. The card is formatted with DOS partition (single partition). Linus kernel Version 4.4.16 Please help me regarding this issue. Regards, Amit.

1 0

Re: Fwd: Need some help regarding write_begin, write_end and writepage(s).
by Pranay Srivastava Jan. 4, 2017

Jan. 4, 2017

Hi Valdis, On Wed, Jan 4, 2017 at 12:58 AM, <Valdis.Kletnieks(a)vt.edu> wrote: > On Thu, 29 Dec 2016 12:35:52 +0530, Pranay Srivastava said: >> Hello, >> >> I'm trying to understand the write_begin, write_end and writepages / >> writepage call >> sequence by having a small file system coded myself. >> >> I'm not using block_write_begin / block_write_end also not using >> mpage_writepages >> as well. >> >> The issue however is that apart from write_begin and write_end, writepages never >> gets called. > > writepages name implies that it's the API for writing multiple blocks in one call. > Does your filesystem understand how to issue multi-block requests? Did you It's not about submitting batch request here. The issue I'm facing is that even after calling set_page_dirty [ with aops->set_page_dirty = __set_page_dirty_nobuffers] and doing __mark_inode_dirty(inode, I_DIRTY_PAGES | I_DIRTY) I was hoping that with an iput_final call, write_inode_now would be called which would trigger my writepage. But that doesn't happen. >From my understanding of iput_final, the writeout won't happen unless the file system is being unmounted[Correct?] [ A check of MS_ACTIVE]. But that would work only for the root inode,as when the file is closed [ say only 1 process has opened that file] so iput_final on that file's inode won't call write_inode_now since file system is mounted [Correct?] So now in order to trigger writepage, I have to make a call to write_inode_now with sync parameter as 0. So what I see now is 1. A write of say 1 MB is to be done. 2. generic_perform_write is called and it calls write_begin and write_end for each page. 3. My write_end has actually become synchronous write instead of a buffered one so each page actually is written out at write_end [ due to a call to write_inode_now in write_end] I was kind of hoping for somehow to batch these writes[ 1 page at a time] but on iput_final. Oh 1 more thing, in generic_write_iter, there's a call to fsync and this call never comes to my file system! I'm pretty sure I'm missing some dirty flags somewhere but it's not fallen in place currently. Thanks for your reply though. I really appreciate it. > *test* with userspace calls that *should* generate multi-block requests? A large write buffer > PAGE_SIZE ought to do it right?

1 0

Re: Doubt in Eudyptula Challenge
by Deeksha varshney Jan. 4, 2017

Jan. 4, 2017

sir, i have cleared my doubt thanks for the help regards deeksha varshney On 4 Jan 2017 00:56, <Valdis.Kletnieks(a)vt.edu> wrote: > On Mon, 02 Jan 2017 15:52:36 +0530, deeksha varshney said: > > Hello sir, > > I am not able to understand what to write in the subject line along with > > the ID while submitting the task 01.Please help > > > > I am getting the following reply from Little > > > > I can not understand your Subject: line, please read the introductory > > email and fix the subject to match exactly as it says in order for me to > > be able to parse it properly. > > So did you read the introductory email? Did it say what to use as a > subject? > Did you actually *do* so? > > Did you also read the part where it says do it *on your own*? >

1 0

Re: adding 32 bit compatibility layer in custom kernel module
by Pradeepa Kumar Jan. 4, 2017

Jan. 4, 2017

Thanks Valdis. Yes , custom kernel module has struct proto_ops pointing to the routines in the module. my kernel module routines do not cal any of linux provided socket* calls. it gets data from different app and provide that data back to app calling socket* APIs. i understand linux kernel calls 32 bit version of syscall handler from sys call table . https://kernel.org/doc/html/latest/process/adding-syscalls.html but how do i do this in custom kernel module. how to differentiate 32 bit and 64 app in my kernel module. do i need all stuff from compat.h and compat.c (if i use same files will that work) ? do i need to change some files in std kernel files too like syscall_32.tbl file , and recompile kernel ? It would be great if there is any example / any web links for this to understad Thanks On Wed, Jan 4, 2017 at 10:10 AM, <Valdis.Kletnieks(a)vt.edu> wrote: > On Wed, 04 Jan 2017 09:34:58 +0530, Pradeepa Kumar said: > > > I have a custom kernel module providing new protocol and providing > socket system calls. > > In general, a loadable module can provide a new protocol, but can't add new > syscalls. However, adding support for socket(), connect(), recvmsg() and > so on for a new protocol *can* be done from a module, as long as the > protocol provides a suitable struct proto_ops pointing to the routines > in the module. > > > i am seeing issues when 32 bit app runs (for exp recvmsg() call does not > > work if msg has cmsghdrs as struct size of cmsghdr is different in 32 bit > > and 64 bit). > > That's a good reason to double-check your code to ensure that you don't > make that same mistake. > > > This is because my custom kernel module does not have 32 bit > compatibility > > layer ( but linux kernel has this in compat.c etc). > > Hold that thought. > > > is it simple to add compatibility layer to my custom kernel module > > Given your previous sentence, you should be able to figure that out. > > > how do i do this (any links ?) > > /* > * linux/kernel/compat.c > * > * Kernel compatibililty routines for e.g. 32 bit syscall support > * on 64 bit kernels. > * > * Copyright (C) 2002-2003 Stephen Rothwell, IBM Corporation > > May already have the functions you need. If not, use the code as a guide > to writing the stuff you're still lacking. > > Note that in particular, recvmsg() combatability is already done for you > via > this chunk of code in net/compat.c: > > COMPAT_SYSCALL_DEFINE3(recvmsg, int, fd, struct compat_msghdr __user *, > msg, unsigned int, flags) > { > return __sys_recvmsg(fd, (struct user_msghdr __user *)msg, flags | > MSG_CMSG_COMPAT); > } > > Go look and see what the MSG_CMSG_COMPAT flag does if you want the gory > details. > >

1 0

Inject custom code or data into running process
by Sayutin Dmitry Jan. 3, 2017

Jan. 3, 2017

Hello, how one should inject code or data into allready running process? There is no need to start code execution at this point, but it should appear in it's virtual memory. Moreover, i want this data to persist across execve's or clone's (probably can be implemented as hook on appropriate kernel methodes) (If you want to know motivation for this -- I want to implement some new idea on sandboxing). Thanks in advance, Sayutin Dmitry <cdkrot(a)yandex.ru>

2 3

Re: Inject custom code or data into running process
by Sayutin Dmitry Jan. 3, 2017

Jan. 3, 2017

Yes, I understand points you provide. > but a royal pain to sandbox malicious code My idea is to get some assistance from kernel on it (possible with source patch or kernel module), but I would like to implement POC [proof-of-concept] myself, before showing it to the community. Let me return back to the original question (injection of code/data) LD_PRELOAD is quite a briliant way, but will not work on statically-linked code. However it may be enough for POC. 03.01.2017, 22:40, "valdis.kletnieks(a)vt.edu" <valdis.kletnieks(a)vt.edu>: > On Tue, 03 Jan 2017 22:24:11 +0300, Sayutin Dmitry said: > >> (If you want to know motivation for this -- I want to implement some new idea on sandboxing). > > There's pretty much nothing you can do inside the process to do sandboxing > against code that doesn't want to be sandboxed. In other words, it's > easy to sandbox possibly buggy code, but a royal pain to sandbox malicious > code. > > Hint: You can lead a horse to code, but you can't force it to call it. > > For instance, using LD_PRELOAD is a good way to front-end calls to glibc > code - but it doesn't do squat against malware that issues its own syscalls > inline to avoid your front end. Sayutin Dmitry <cdkrot(a)yandex.com>

1 0