Packet drop while using BPF filter
Hi Guys, I am working on user level code which uses BPF filter. Solution has two threads and BPF filter for incoming packets to raw socket dedicated to each thread. Intent is to divide ingress traffic between two thread based on whether dest IP is even or odd. BPF filters used are: Even IP filter :-> tcpdump -i interface 'ether dst <dest mac> && ip[19] & 0x01 = 0' Odd IP filter :-> tcpdump -i interface 'ether dst <dest mac> && ip[19] & 0x01 = 1' There is a packet loss for odd IP thread even when CPU is available. Reason being packet drop due recv buffer full. Same amount of traffic is well handled by even IP thread(user code being same in both), scaling CPU to full 100%. In odd IP thread, If filter is changed to "tcpdump -i interface 'ether dst <dest mac> && ip proto 17' ", all goes fine. Also CPU usage at kernel drops from 50% to 4 % for a particular amount of traffic. Would appreciate any I/ps for reducing load at kernel for packet filtering(Odd/Even IP distribution) Cheers Mukesh
Hi Mukesh.. On Wed, Dec 21, 2011 at 01:38, Mukesh Yadav <mukesh.fkd@gmail.com> wrote:
BPF filters used are: Even IP filter :-> tcpdump -i interface 'ether dst <dest mac> && ip[19] & 0x01 = 0' Odd IP filter :-> tcpdump -i interface 'ether dst <dest mac> && ip[19] & 0x01 = 1'
There is a packet loss for odd IP thread even when CPU is available. Reason being packet drop due recv buffer full. Same amount of traffic is well handled by even IP thread(user code being same in both), scaling CPU to full 100%.
In odd IP thread, If filter is changed to "tcpdump -i interface 'ether dst <dest mac> && ip proto 17' ", all goes fine. Also CPU usage at kernel drops from 50% to 4 % for a particular amount of traffic.
I have zero experience with BPF, so this is just my theory. In kernel level, (or maybe between kernel and user level), by putting your odd and even filter, kernel work too hard to separate the packets into different "queue" or "channel"...or something like that. While in the mean time, incoming packets are bursting .... This makes buffers filled up fast. How about this... just filter anything that match your target MAC....fed this up to the user level application and let your application do the odd and even IP filtering? -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
Thanks Mulyadi, Yeah that a final alternative. Actually I was looking for some optimized way of getting packet filtered at kernel level using BPF filter. On 21 December 2011 06:22, Mulyadi Santosa <mulyadi.santosa@gmail.com>wrote:
Hi Mukesh..
On Wed, Dec 21, 2011 at 01:38, Mukesh Yadav <mukesh.fkd@gmail.com> wrote:
BPF filters used are: Even IP filter :-> tcpdump -i interface 'ether dst <dest mac> && ip[19] & 0x01 = 0' Odd IP filter :-> tcpdump -i interface 'ether dst <dest mac> && ip[19] & 0x01 = 1'
There is a packet loss for odd IP thread even when CPU is available. Reason being packet drop due recv buffer full. Same amount of traffic is well handled by even IP thread(user code being same in both), scaling CPU to full 100%.
In odd IP thread, If filter is changed to "tcpdump -i interface 'ether dst <dest mac> && ip proto 17' ", all goes fine. Also CPU usage at kernel drops from 50% to 4 % for a particular amount of traffic.
I have zero experience with BPF, so this is just my theory.
In kernel level, (or maybe between kernel and user level), by putting your odd and even filter, kernel work too hard to separate the packets into different "queue" or "channel"...or something like that. While in the mean time, incoming packets are bursting .... This makes buffers filled up fast.
How about this... just filter anything that match your target MAC....fed this up to the user level application and let your application do the odd and even IP filtering?
-- regards,
Mulyadi Santosa Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
-- Cheers Mukesh Yadav Senior Software Engineer Plot6 Sector 18 Aricent Gurgaon Mob: 9953024169
Hi Mukesh On Wed, Dec 21, 2011 at 12:25, Mukesh Yadav <mukesh.fkd@gmail.com> wrote:
Thanks Mulyadi, Yeah that a final alternative. Actually I was looking for some optimized way of getting packet filtered at kernel level using BPF filter.
You're welcome. Logically, your code should have no problem, but well it seems that the implementation in BPF is not so good somewhere. -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
Hi, Have you tried enabling -- /proc/sys/net/core/bpf_jit_enable -- provides JIT compiled BPF on 64 bit linux. * On Wed, Dec 21, 2011 at 12:08:04AM +0530, Mukesh Yadav <mukesh.fkd@gmail.com> wrote:
Hi Guys,
I am working on user level code which uses BPF filter. Solution has two threads and BPF filter for incoming packets to raw socket dedicated to each thread. Intent is to divide ingress traffic between two thread based on whether dest IP is even or odd.
BPF filters used are: Even IP filter :-> tcpdump -i interface 'ether dst <dest mac> && ip[19] & 0x01 = 0' Odd IP filter :-> tcpdump -i interface 'ether dst <dest mac> && ip[19] & 0x01 = 1'
There is a packet loss for odd IP thread even when CPU is available. Reason being packet drop due recv buffer full. Same amount of traffic is well handled by even IP thread(user code being same in both), scaling CPU to full 100%.
In odd IP thread, If filter is changed to "tcpdump -i interface 'ether dst <dest mac> && ip proto 17' ", all goes fine. Also CPU usage at kernel drops from 50% to 4 % for a particular amount of traffic.
Would appreciate any I/ps for reducing load at kernel for packet filtering(Odd/Even IP distribution)
Cheers Mukesh
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-------------------------- Raghavendra Prabhu GPG Id : 0xD72BE977 Fingerprint: B93F EBCB 8E05 7039 CD3C A4B8 A616 DCA1 D72B E977 www: wnohang.net
Hi Raghavendra, " /proc/sys/net/core/bpf_jit_**enable " is not present on machine. Seems kernel need to be configured for these. Btw CPU scaling issue is resolved. Packet drop was because all interrupt handling was happening on core where packets were getting drop. Tuning system for interrupt handling using smp_affinity resolved the issue. Thanks Mukesh On 22 December 2011 19:18, Raghavendra D Prabhu <raghu.prabhu13@gmail.com>wrote:
Hi,
Have you tried enabling -- /proc/sys/net/core/bpf_jit_**enable -- provides JIT compiled BPF on 64 bit linux.
* On Wed, Dec 21, 2011 at 12:08:04AM +0530, Mukesh Yadav < mukesh.fkd@gmail.com> wrote:
Hi Guys,
I am working on user level code which uses BPF filter. Solution has two threads and BPF filter for incoming packets to raw socket dedicated to each thread. Intent is to divide ingress traffic between two thread based on whether dest IP is even or odd.
BPF filters used are: Even IP filter :-> tcpdump -i interface 'ether dst <dest mac> && ip[19] & 0x01 = 0' Odd IP filter :-> tcpdump -i interface 'ether dst <dest mac> && ip[19] & 0x01 = 1'
There is a packet loss for odd IP thread even when CPU is available. Reason being packet drop due recv buffer full. Same amount of traffic is well handled by even IP thread(user code being same in both), scaling CPU to full 100%.
In odd IP thread, If filter is changed to "tcpdump -i interface 'ether dst <dest mac> && ip proto 17' ", all goes fine. Also CPU usage at kernel drops from 50% to 4 % for a particular amount of traffic.
Would appreciate any I/ps for reducing load at kernel for packet filtering(Odd/Even IP distribution)
Cheers Mukesh
______________________________**_________________
Kernelnewbies mailing list Kernelnewbies@kernelnewbies.**org <Kernelnewbies@kernelnewbies.org> http://lists.kernelnewbies.**org/mailman/listinfo/**kernelnewbies<http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies>
-------------------------- Raghavendra Prabhu GPG Id : 0xD72BE977 Fingerprint: B93F EBCB 8E05 7039 CD3C A4B8 A616 DCA1 D72B E977 www: wnohang.net
-- Cheers Mukesh Yadav Senior Software Engineer Plot6 Sector 18 Aricent Gurgaon Mob: 9953024169
* On Sat, Dec 24, 2011 at 09:57:50PM +0530, Mukesh Yadav <mukesh.fkd@gmail.com> wrote:
Hi Raghavendra,
" /proc/sys/net/core/bpf_jit_**enable " is not present on machine. Seems kernel need to be configured for these. Yes, it is a fairly new option -- CONFIG_BPF_JIT=y
Btw CPU scaling issue is resolved. Packet drop was because all interrupt handling was happening on core where packets were getting drop. Tuning system for interrupt handling using smp_affinity resolved the issue. That is generally how it is done. The core which handles packets initially handles the interrupts too. Do you mean all non-network interrupts were happening on same core ? Then give irqbalance a shot.
Thanks Mukesh
On 22 December 2011 19:18, Raghavendra D Prabhu <raghu.prabhu13@gmail.com>wrote:
Hi,
Have you tried enabling -- /proc/sys/net/core/bpf_jit_**enable -- provides JIT compiled BPF on 64 bit linux.
* On Wed, Dec 21, 2011 at 12:08:04AM +0530, Mukesh Yadav < mukesh.fkd@gmail.com> wrote:
Hi Guys,
I am working on user level code which uses BPF filter. Solution has two threads and BPF filter for incoming packets to raw socket dedicated to each thread. Intent is to divide ingress traffic between two thread based on whether dest IP is even or odd.
BPF filters used are: Even IP filter :-> tcpdump -i interface 'ether dst <dest mac> && ip[19] & 0x01 = 0' Odd IP filter :-> tcpdump -i interface 'ether dst <dest mac> && ip[19] & 0x01 = 1'
There is a packet loss for odd IP thread even when CPU is available. Reason being packet drop due recv buffer full. Same amount of traffic is well handled by even IP thread(user code being same in both), scaling CPU to full 100%.
In odd IP thread, If filter is changed to "tcpdump -i interface 'ether dst <dest mac> && ip proto 17' ", all goes fine. Also CPU usage at kernel drops from 50% to 4 % for a particular amount of traffic.
Would appreciate any I/ps for reducing load at kernel for packet filtering(Odd/Even IP distribution)
Cheers Mukesh
______________________________**_________________
Kernelnewbies mailing list Kernelnewbies@kernelnewbies.**org <Kernelnewbies@kernelnewbies.org> http://lists.kernelnewbies.**org/mailman/listinfo/**kernelnewbies<http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies>
-------------------------- Raghavendra Prabhu GPG Id : 0xD72BE977 Fingerprint: B93F EBCB 8E05 7039 CD3C A4B8 A616 DCA1 D72B E977 www: wnohang.net
participants (3)
-
Mukesh Yadav -
Mulyadi Santosa -
Raghavendra D Prabhu