relationship between cpu_affinity and Packet RX Processing

[Rami Rosen]

Hi, Kumar,

1) Regarding rx queues: what does
"ls /sys/class/net/eth0/queues"
show ?

2) can you try setting affinity for working only with CPU3
and see what happens after some traffic is sent by
cat /proc/interrupts?

regards,
Rami Rosen
http://ramirose.wix.com/ramirosen


On Tue, Mar 26, 2013 at 8:31 PM, Kumar amit mehta <gmate.amit at gmail.com> wrote:
> Hi All,
>
> I was reading some stuff on interrupts and irq lines today and I thought I'll
> expermient with the network rx path. To start with, I've a Virtual Machine
> running 3.8 linux kernel. My machine has 4 CPU cores, network (eth) interface
> is driven by pcnet_32 AMD driver and is tied to IRQ line #19. I started some
> network traffic and I notice that out of those 4 CPUs, only one of them is being
> used and despite changing the CPU affinity, I still don't see the other cores
> being used for this network traffic. So based on this behavior(please see the
> logs below), I've these following queries:
> i) Does it mean that this network card do not have multiple Rx Queues ?
> ii) I think all the modern NICs must be implementing multiple Rx Queues and
> hence Can someone please point me to the simplest of such implemenation in any
> of the in-tree drivers ?
> iii) I'm just doing a simple 'ping' to google with of big size packets, As I do
> not have a peer to use packetgen/netperf/iperf utilities.
>
> ref: Comments in double quotes.
> <logs>
> $ uname -r
> 3.8.0-rc6-next-20130208
>
> $ cat /proc/cpuinfo |grep processor
> processor       : 0
> processor       : 1
> processor       : 2
> processor       : 3
> "Total 4 cpu cores"
>
> $ cat /proc/interrupts|egrep 'eth0|CPU'
>         CPU0       CPU1       CPU2       CPU3
> 19:     5103         74         33          5   IO-APIC-fasteoi         eth0
> "IRQ Line #19 for the network device"
>
> $ lspci|grep -i ethernet
> 02:01.0 Ethernet controller: Advanced Micro Devices [AMD] 79c970 [PCnet32 LANCE]
> (rev 10)
>
> "AMD NIC"
> $ lspci -s 02:01.0 -vvv|grep module
>         Kernel modules: pcnet32
>
> $ lsmod|grep pcnet32
> pcnet32                40671  0
> "driver"
>
> # whoami
> root
>
> # cat /proc/irq/19/smp_affinity
> 03
>
> # cat /proc/irq/19/affinity_hint
> 00
>
> "I think smp_affinity is a bit map, therefore for all the four cores to be
> utilized, all 4 bits should be set to 1, which leads to 15(0xf), hence Chaning
> the cpu affinity"
> # echo 15 > /proc/irq/19/smp_affinity
> # cat /proc/irq/19/smp_affinity
> 15
>
> "started network traffic here and monitoring it"
> # cat /proc/interrupts|grep eth0
> 19:       5452         78         33          5   IO-APIC-fasteoi   eth0
> # cat /proc/interrupts|grep eth0
> 19:       5488         78         35          5   IO-APIC-fasteoi   eth0
> # cat /proc/interrupts|grep eth0
> 19:       5492         78         35          5   IO-APIC-fasteoi   eth0
> # cat /proc/interrupts|grep eth0
> 19:       5500         78         35          5   IO-APIC-fasteoi   eth0
> .................................
> .........after some time.........
> # cat /proc/interrupts|grep eth0
> 19:       6035         78         42          5   IO-APIC-fasteoi   eth0
>
> "Most of the packets are still getting routed through CPU0 *Only*."
> <logs>
>
> -Amit
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies