I am testing my driver on much faster host processor and facing following issues: My host is too powerful and it can fill up device buffer queue very fast. I get best performance when I do busy wait, but this is not desirable and is bad design. I need to sleep and wake up quickly and predictability. Indication from device that queue has space, is coming in form of memory write (device writes to a memory location of i86 processor). I tried using wait_event_interruptible_timeout, I am depending on 2nd parameter of the function but it wake up is too slow, even tried using value of 1. Any suggestions ?
On Mon, May 02, 2011 at 11:32:32AM -0700, Abu Rasheda wrote:
I am testing my driver on much faster host processor and facing following issues:
My host is too powerful and it can fill up device buffer queue very fast.
What kind of device is this?
I get best performance when I do busy wait, but this is not desirable and is bad design.
I need to sleep and wake up quickly and predictability. Indication from device that queue has space, is coming in form of memory write (device writes to a memory location of i86 processor).
I tried using wait_event_interruptible_timeout, I am depending on 2nd parameter of the function but it wake up is too slow, even tried using value of 1.
Why not try increasing the buffer in your driver to handle any amount of data needed? thanks, greg k-h
On Mon, May 2, 2011 at 12:28 PM, Greg KH <greg@kroah.com> wrote:
On Mon, May 02, 2011 at 11:32:32AM -0700, Abu Rasheda wrote:
I am testing my driver on much faster host processor and facing following issues:
My host is too powerful and it can fill up device buffer queue very fast.
What kind of device is this?
its a networking device
I get best performance when I do busy wait, but this is not desirable and is bad design.
I need to sleep and wake up quickly and predictability. Indication from device that queue has space, is coming in form of memory write (device writes to a memory location of i86 processor).
I tried using wait_event_interruptible_timeout, I am depending on 2nd parameter of the function but it wake up is too slow, even tried using value of 1.
Why not try increasing the buffer in your driver to handle any amount of data needed?
Problem come from the fact that remote end can be slow, so device / driver cannot do much. If I increase buffer size, than running on (even) faster host processor will bring back same issue. I am looking for solution which will scale. I check for available buffer size if space is low, I sleep and retry. This is causing lot of CPU to be wasted. How can I optimize this ?
thanks,
greg k-h
On Mon, May 02, 2011 at 02:02:34PM -0700, Abu Rasheda wrote:
On Mon, May 2, 2011 at 12:28 PM, Greg KH <greg@kroah.com> wrote:
On Mon, May 02, 2011 at 11:32:32AM -0700, Abu Rasheda wrote:
I am testing my driver on much faster host processor and facing following issues:
My host is too powerful and it can fill up device buffer queue very fast.
What kind of device is this?
its a networking device
It sounds like a broken device, you need to be able to handle large data streams, right? Are you running Linux in this device and that is the issue you are having?
I get best performance when I do busy wait, but this is not desirable and is bad design.
I need to sleep and wake up quickly and predictability. Indication from device that queue has space, is coming in form of memory write (device writes to a memory location of i86 processor).
I tried using wait_event_interruptible_timeout, I am depending on 2nd parameter of the function but it wake up is too slow, even tried using value of 1.
Why not try increasing the buffer in your driver to handle any amount of data needed?
Problem come from the fact that remote end can be slow, so device / driver cannot do much. If I increase buffer size, than running on (even) faster host processor will bring back same issue. I am looking for solution which will scale.
I check for available buffer size if space is low, I sleep and retry. This is causing lot of CPU to be wasted.
How can I optimize this ?
Use NAPI for your network driver? good luck, greg k-h
I am testing my driver on much faster host processor and facing following issues:
My host is too powerful and it can fill up device buffer queue very fast.
What kind of device is this?
its a networking device
It sounds like a broken device, you need to be able to handle large data streams, right? Are you running Linux in this device and that is the issue you are having?
I am not running Linux on the device. As a said, if remote host does not send syn for TCP packets, my device queue will continue to grow. My question is simple, what would be correct way of handling things in this situation in software or if device is broken how would device behave differently ? I am here to learn.
Greetings,After reading Linux Kernel in a Nutshell (by the way, great book), I now have my own kernel configured for my hardware needs. However, I have some really strange behavior. On tty1, the same tty that launches X11, I have to press twice each keyboard key. Anyone has any idea what's going on? I am using a customized Debian Squeeze + OpenBox. Thanks in advance, and thanks Greg for writing the book. Ezequiel.
[Forgot to cc list] 2011/5/2 Ezequiel García <elezegarcia@yahoo.com.ar>
Greetings, After reading Linux Kernel in a Nutshell (by the way, great book), I now have my own kernel configured for my hardware needs. However, I have some really strange behavior. On tty1, the same tty that launches X11, I have to press twice each keyboard key. Anyone has any idea what's going on? I am using a customized Debian Squeeze + OpenBox. Thanks in advance, and thanks Greg for writing the book. Ezequiel.
Hi Ezequiel I'm new on this list as well but I don't think anyone will be able to help you with the limited info that you have provided. When you say you have a kernel configured for your hardware, did you: A) Download the kernel from http://www.kernel.org/ ? In that case you need to tell us which version you're using. B) Get the kernel from git a git tree http://git.kernel.org/? We still need to know exactly which version C) Recompile your debian kernel? (2.6.32?) If this bug is present with only a change in kernel config, it would be interesting to see the difference between the working and broken config. I hope that helps, Pico
Hi Abu... On Tue, May 3, 2011 at 01:32, Abu Rasheda <rcpilot2010@gmail.com> wrote:
I am testing my driver on much faster host processor and facing following issues:
My host is too powerful and it can fill up device buffer queue very fast.
I get best performance when I do busy wait, but this is not desirable and is bad design.
For me, busy waiting could be the best for your case... in fact, NAPI sometimes do that too....in most cases. Actually, devices usually fills the in-RAM buffer with the help of interrupt(s). So whenever in-device buffer has something new, it interrupts...then soft irq follows up by copying them to RAM buffe (circular buffer usually). To make it fast, DMA or other CPU less operation kicks in here. So I think that;s the key...by relying on the interrupts...then you will know when your buffer is about to be full. Other than that, I would still say, busy waiting could be your best bet. PS: here's crazy idea, if you can accept it. Put a guard page, right at the tail of your buffer,. Make it unwritable even by kernel mode. Right when you hit it, page fault should kicks in. Then you know buffer is full. This is inspired by the stack guard..usual method in user space to reduce buffer overflow risk. -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
On Tue, May 3, 2011 at 2:32 AM, Abu Rasheda <rcpilot2010@gmail.com> wrote:
I am testing my driver on much faster host processor and facing following issues:
My host is too powerful and it can fill up device buffer queue very fast.
I get best performance when I do busy wait, but this is not desirable and is bad design.
I need to sleep and wake up quickly and predictability. Indication from device that queue has space, is coming in form of memory write (device writes to a memory location of i86 processor).
I tried using wait_event_interruptible_timeout, I am depending on 2nd parameter of the function but it wake up is too slow, even tried using value of 1.
Any suggestions ?
Many of us are just newbies in this area, and therefore, get it working is much more important than to optimize it - u can safely said that the hardcore kernel developer has already optimize many of these problems away, and so if they cannot do it, there must be a reason...try to probe more first perhaps. High speed networking device has many special hardware features: IP/UDP/TCP checksum offload etc. http://www.fenrus.org/how-to-not-write-a-device-driver-paper.pdf http://www.sun.com/products/networking/infiniband/ibhcaPCI-E/docs/datasheet.... Read this about NAPI: http://www.linuxfoundation.org/collaborate/workgroups/networking/napi <http://www.linuxfoundation.org/collaborate/workgroups/networking/napi>Eg, "Interrupt mitigation" - whereby interrupt mechanism is disabled (in particular my desktop PC's r8196.c is using this feature) and polling takes over instead - but u have to implemented complicated mechanism to reinject the interrupt if necessary (read r8196.c). And if u really ready....this is a good writeup: http://datatag.web.cern.ch/datatag/howto/tcp.html Other possible suggestion/features: Jumbo frames: http://www.cyberciti.biz/faq/rhel-centos-debian-ubuntu-jumbo-frames-configur... PCI posting (pdf paper above and r8196.c). Disabling TCP software checksum (and use the hardware instead): http://www.linuxquestions.org/questions/linux-enterprise-47/how-to-disable-t... And for loads of other ideas (eg, TCP bypass): * http://ttthebear.blogspot.com/2008/07/linux-kernel-bypass-and-performance.ht... * Generally a lot of these ideas can be found in the kernel source codes - just search and copy the implementation.....the highest network data transfer is achieved in infiniband-based Mellanox card (in some China supercomputer), and this involved the use of GPU technology etc.... https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/2009-May/039639.html -- Regards, Peter Teoh
participants (6)
-
Abu Rasheda -
Ezequiel García -
Greg KH -
Mulyadi Santosa -
Peter Teoh -
Pico Geyer