The method to alloc DMA ring buffer
Hi all, This is my first question in the kernel newbies mailing list. Forgive me if there is incorrect behavior. I'm studying the NIC driver with the book <<Understanding the linux network internals>>. This book uses 3com 3c59x NIC(3c59x.c) as an example. The driver is very old and few people discuss it, but it's a good place to start because it's simpler than many other drivers with advanced offload features. In the interrupt handler boomerang_rx(), I see it pre-allocate one new skb and dma map before calling netif_rx. boomerang_rx() { ...... /* Pre-allocate the replacement skb. If it or its * mapping fails then recycle the buffer thats already * in place */ newskb = netdev_alloc_skb_ip_align(dev, PKT_BUF_SZ); if (!newskb) { dev->stats.rx_dropped++; goto clear_complete; } newdma = dma_map_single(vp->gendev, newskb->data, PKT_BUF_SZ, DMA_FROM_DEVICE); if (dma_mapping_error(vp->gendev, newdma)) { dev->stats.rx_dropped++; consume_skb(newskb); goto clear_complete; } /* Pass up the skbuff already on the Rx ring. */ skb = vp->rx_skbuff[entry]; vp->rx_skbuff[entry] = newskb; vp->rx_ring[entry].addr = cpu_to_le32(newdma); skb_put(skb, pkt_len); dma_unmap_single(vp->gendev, dma, PKT_BUF_SZ, DMA_FROM_DEVICE); vp->rx_nocopy++; ...... netif_rx(skb); ....... } However, I see intel e1000 driver optimizes this. Intel doesn't pre-allocate one at a time. e1000_clean_rx_irq() { ...... /* return some buffers to hardware, one at a time is too slow */ if (unlikely(cleaned_count >= E1000_RX_BUFFER_WRITE)) { adapter->alloc_rx_buf(adapter, rx_ring, cleaned_count); cleaned_count = 0; ...... } I just want to know why this is faster. For all scenarios or some scenarios? Can someone analyse it rigorously? My guess is this method reduces the delay when many packets come, right? Thanks in advance! Tony
On Tue, Dec 6, 2022 at 8:17 PM Tony He <huangya90@gmail.com> wrote: ...
I'm studying the NIC driver with the book <<Understanding the linux network internals>>. This book uses 3com 3c59x NIC(3c59x.c) as an example. The driver is very old and few people discuss it, but it's a good place to start because it's simpler than many other drivers with advanced offload features.
In the interrupt handler boomerang_rx(), I see it pre-allocate one new
...
goto clear_complete;
Oh m a goto. ...
However, I see intel e1000 driver optimizes this. Intel doesn't pre-allocate one at a time. .... I just want to know why this is faster. For all scenarios or some scenarios? Can someone analyse it rigorously?
The first step is to examine the hardware data pages. Most optimizations like this are shaped by hardware That said, buffer allocation in the kernel is slow(ish). Any time there is sufficient ram to preallocate, do so in moderation. ... Interrupt latency. It helps to have the hardware address a list of commands in a queue without hardware intervention. Networking: a buffer window sent to the client allows that client to send data without waiting on a response and is unlikely to need to resend unless the network has loss. The networking buffer window strategy is a giant topic. Interrupt latency: that is another book and it depends on the hardware. Again any hardware that can do useful work without interrupt servicing the better. Simple serial IO hardware drivers can be easier to research and analyze. Again start with the hardware data sheets for UARTS and hardware and software flow control over serial data links. Define: rigorously? Is this a homework assignment?
Thanks! Gained a lot. Tom Mitchell <niftylinkern@niftyegg.com> 于2022年12月8日周四 03:55写道：
On Tue, Dec 6, 2022 at 8:17 PM Tony He <huangya90@gmail.com> wrote: ...
I'm studying the NIC driver with the book <<Understanding the linux network internals>>. This book uses 3com 3c59x NIC(3c59x.c) as an example. The driver is very old and few people discuss it, but it's a good place to start because it's simpler than many other drivers with advanced offload features.
In the interrupt handler boomerang_rx(), I see it pre-allocate one new
...
goto clear_complete;
Oh m a goto. ...
However, I see intel e1000 driver optimizes this. Intel doesn't pre-allocate one at a time. .... I just want to know why this is faster. For all scenarios or some scenarios? Can someone analyse it rigorously?
The first step is to examine the hardware data pages. Most optimizations like this are shaped by hardware
That said, buffer allocation in the kernel is slow(ish). Any time there is sufficient ram to preallocate, do so in moderation. ... Interrupt latency. It helps to have the hardware address a list of commands in a queue without hardware intervention. Networking: a buffer window sent to the client allows that client to send data without waiting on a response and is unlikely to need to resend unless the network has loss.
The networking buffer window strategy is a giant topic.
Interrupt latency: that is another book and it depends on the hardware. Again any hardware that can do useful work without interrupt servicing the better.
Simple serial IO hardware drivers can be easier to research and analyze. Again start with the hardware data sheets for UARTS and hardware and software flow control over serial data links.
Define: rigorously? Is this a homework assignment?
No, this is NOT a homework assignment. Have worked a few years but my most areas were related to application instead of kernel and driver.
participants (2)
-
Tom Mitchell -
Tony He