<div dir="ltr">hi all:<div> I fix it. I make a error with:</div><div>###############################</div><div><div style="font-family:arial,sans-serif;font-size:14px"> skb_reset_tail_pointer(<span style="background-color:rgb(255,0,0)">skb</span>);</div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>send_skb->end = <span style="background-color:rgb(255,0,0)">skb</span>->tail + len + NUM;</div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>kmemcheck_annotate_bitfield(<span style="background-color:rgb(255,0,0)">skb</span>, flags1);</div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>kmemcheck_annotate_bitfield(<span style="background-color:rgb(255,0,0)">skb</span>, flags2);</div><div style="font-family:arial,sans-serif;font-size:14px"><br></div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>//send_skb->ip_summed = CHECKSUM_PARTIAL;<span style="white-space:pre-wrap"> </span></div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span></div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>struct skb_shared_info *shinfo;</div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>shinfo = skb_shinfo(<span style="background-color:rgb(255,0,0)">skb</span>);</div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>atomic_set(&shinfo->dataref, 2);</div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>shinfo->nr_frags = 0;</div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>shinfo->gso_size = 0;</div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>shinfo->gso_segs = 0;</div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>shinfo->gso_type = 0;</div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>shinfo->ip6_frag_id = 0;</div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>shinfo->tx_flags.flags = 0;</div><div style="font-family:arial,sans-serif;font-size:14px"><span style="white-space:pre-wrap"> </span>skb_frag_list_init(<span style="background-color:rgb(255,0,0)">skb</span>);</div></div><div>###############################</div><div><br></div><div>I should use send_skb instead of skb.</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">2014-09-25 17:58 GMT+08:00 lx <span dir="ltr"><<a href="mailto:lxlenovostar@gmail.com" target="_blank">lxlenovostar@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">hi all:<div> I have a mmap memory area, so I don't want to use alloc_skb(). I use this way:</div><div>1. use kmem_cache_alloc_node() for sk_buff struct.</div><div>2. the mmap memory area of "mmap_buf" for data area of packet.</div><div><br></div><div><br></div><div>the codes is:</div><div><br></div><div>alloc data memory area:</div><div>###############################################################</div><div><div>mmap_buf = vmalloc(mmap_size);</div><div><span style="white-space:pre-wrap"> </span>printk("vmalloc mmap_buf=%p mmap_size=%ld\n", (void *)mmap_buf, mmap_size);</div><div><span style="white-space:pre-wrap"> </span>if (!mmap_buf ) {</div><div><span style="white-space:pre-wrap"> </span>printk("vmalloc failed!\n");</div><div><span style="white-space:pre-wrap"> </span>return -1;</div><div><span style="white-space:pre-wrap"> </span>}</div><div><span style="white-space:pre-wrap"> </span>for (i = 0; i < mmap_size; i += PAGE_SIZE) {</div><div><span style="white-space:pre-wrap"> </span>SetPageReserved(vmalloc_to_page(mmap_buf + i));</div><div><span style="white-space:pre-wrap"> </span>}</div></div><div>###############################################################</div><div>I will mmap the "mmap_buf" area, in this way:</div><div>###############################################################</div><div><div>while (size > 0) {</div><div><span style="white-space:pre-wrap"> </span>pfn = vmalloc_to_pfn(ptmp);</div><div><span style="white-space:pre-wrap"> </span>if ((ret = remap_pfn_range(vma, start, pfn, PAGE_SIZE, PAGE_SHARED)) < 0) {</div><div><span style="white-space:pre-wrap"> </span>return ret;</div><div><span style="white-space:pre-wrap"> </span>}</div><div><span style="white-space:pre-wrap"> </span>start += PAGE_SIZE;</div><div><span style="white-space:pre-wrap"> </span>ptmp += PAGE_SIZE;</div><div><span style="white-space:pre-wrap"> </span>size -= PAGE_SIZE;</div><div><span style="white-space:pre-wrap"> </span>}</div></div><div>###############################################################</div><div><br></div><div>construct packet</div><div>################################################################</div><div><div>/*</div><div><span style="white-space:pre-wrap"> </span>* send packet </div><div><span style="white-space:pre-wrap"> </span>*/</div><div><span style="white-space:pre-wrap"> </span>spin_lock(&lock);</div><div><span style="white-space:pre-wrap"> </span>int eth_len, udph_len, iph_len, len;</div><div><span style="white-space:pre-wrap"> </span>eth_len = sizeof(struct ethhdr);</div><div><span style="white-space:pre-wrap"> </span>iph_len = sizeof(struct iphdr);</div><div><span style="white-space:pre-wrap"> </span>udph_len = sizeof(struct udphdr);</div><div><span style="white-space:pre-wrap"> </span>len<span style="white-space:pre-wrap"> </span>= eth_len + iph_len + udph_len;</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>/*</div><div><span style="white-space:pre-wrap"> </span> * build a new sk_buff</div><div><span style="white-space:pre-wrap"> </span> */</div><div><span style="white-space:pre-wrap"> </span>struct sk_buff *send_skb = kmem_cache_alloc_node(skbuff_head_cache, GFP_ATOMIC & ~__GFP_DMA, NUMA_NO_NODE);</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>if (!send_skb) {</div><div><span style="white-space:pre-wrap"> </span>return NF_DROP;</div><div><span style="white-space:pre-wrap"> </span>}</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>memset(send_skb, 0, offsetof(struct sk_buff, tail));</div><div><span style="white-space:pre-wrap"> </span>atomic_set(&send_skb->users, 2);</div><div><span style="white-space:pre-wrap"> </span>send_skb->head = mmap_buf + 1024;</div><div><span style="white-space:pre-wrap"> </span>send_skb->data = mmap_buf + 1024;</div><div><span style="white-space:pre-wrap"> </span>skb_reset_tail_pointer(skb);</div><div><span style="white-space:pre-wrap"> </span>send_skb->end = skb->tail + len + NUM;</div><div><span style="white-space:pre-wrap"> </span>kmemcheck_annotate_bitfield(skb, flags1);</div><div><span style="white-space:pre-wrap"> </span>kmemcheck_annotate_bitfield(skb, flags2);</div><div><br></div><div><span style="white-space:pre-wrap"> </span>//send_skb->ip_summed = CHECKSUM_PARTIAL;<span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>struct skb_shared_info *shinfo;</div><div><span style="white-space:pre-wrap"> </span>shinfo = skb_shinfo(skb);</div><div><span style="white-space:pre-wrap"> </span>atomic_set(&shinfo->dataref, 2);</div><div><span style="white-space:pre-wrap"> </span>shinfo->nr_frags = 0;</div><div><span style="white-space:pre-wrap"> </span>shinfo->gso_size = 0;</div><div><span style="white-space:pre-wrap"> </span>shinfo->gso_segs = 0;</div><div><span style="white-space:pre-wrap"> </span>shinfo->gso_type = 0;</div><div><span style="white-space:pre-wrap"> </span>shinfo->ip6_frag_id = 0;</div><div><span style="white-space:pre-wrap"> </span>shinfo->tx_flags.flags = 0;</div><div><span style="white-space:pre-wrap"> </span>skb_frag_list_init(skb);</div><div><span style="white-space:pre-wrap"> </span>memset(&shinfo->hwtstamps, 0, sizeof(shinfo->hwtstamps));</div><div><br></div><div><span style="white-space:pre-wrap"> </span>printk("mmap_buf + 1024 is %p\n", mmap_buf + 1024);<span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>skb_reserve(send_skb, len);</div><div><span style="white-space:pre-wrap"> </span>printk("data %p, len is %d\n", send_skb->data, len);<span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>skb_push(send_skb, sizeof(struct udphdr));</div><div><span style="white-space:pre-wrap"> </span>printk("udp data %p\n", send_skb->data);<span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>skb_reset_transport_header(send_skb);</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>/*</div><div> <span style="white-space:pre-wrap"> </span> * just instead of copy datas.</div><div> <span style="white-space:pre-wrap"> </span> */<span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>//send_skb->tail = send_skb->end;</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>udph = udp_hdr(send_skb);</div><div><span style="white-space:pre-wrap"> </span>udph->source = dport;</div><div><span style="white-space:pre-wrap"> </span>udph->dest = htons(ntohs(dport) + 1);</div><div><span style="white-space:pre-wrap"> </span>udph->len = htons(udph_len);</div><div><span style="white-space:pre-wrap"> </span>udph->check = 0;</div><div><span style="white-space:pre-wrap"> </span>udph->check = csum_tcpudp_magic(daddr, in_aton(dest_addr), udph_len, IPPROTO_UDP, csum_partial(udph, udph_len, 0));</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>if (udph->check == 0)</div><div><span style="white-space:pre-wrap"> </span>udph->check = CSUM_MANGLED_0;</div><div><br></div><div><span style="white-space:pre-wrap"> </span>skb_push(send_skb, sizeof(struct iphdr));</div><div><span style="white-space:pre-wrap"> </span>printk("ip data %p\n", send_skb->data);<span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>skb_reset_network_header(send_skb);</div><div><span style="white-space:pre-wrap"> </span>send_iph = ip_hdr(send_skb);</div><div><br></div><div><span style="white-space:pre-wrap"> </span>// iph->version = 4; iph->ihl = 5; </div><div><span style="white-space:pre-wrap"> </span>put_unaligned(0x45, (unsigned char *)send_iph);</div><div><span style="white-space:pre-wrap"> </span>send_iph->tos = 0;</div><div><span style="white-space:pre-wrap"> </span>put_unaligned(htons(iph_len) + htons(udph_len), &(send_iph->tot_len));</div><div><span style="white-space:pre-wrap"> </span>//send_iph->id = htons(atomic_inc_return(&ip_ident));</div><div><span style="white-space:pre-wrap"> </span>send_iph->id = 0;</div><div><span style="white-space:pre-wrap"> </span>send_iph->frag_off = 0;</div><div><span style="white-space:pre-wrap"> </span>send_iph->ttl = 64;</div><div><span style="white-space:pre-wrap"> </span>send_iph->protocol = IPPROTO_UDP;</div><div><span style="white-space:pre-wrap"> </span>send_iph->check = 0;</div><div><span style="white-space:pre-wrap"> </span>put_unaligned(daddr, &(send_iph->saddr));</div><div><span style="white-space:pre-wrap"> </span>put_unaligned(in_aton(dest_addr), &(send_iph->daddr));</div><div><span style="white-space:pre-wrap"> </span>send_iph->check = ip_fast_csum((unsigned char *)send_iph, send_iph->ihl);</div><div><span style="white-space:pre-wrap"> </span> </div><div><span style="white-space:pre-wrap"> </span>struct net_device *dev = skb->dev;</div><div><span style="white-space:pre-wrap"> </span>eth = (struct ethhdr *)skb_push(send_skb, ETH_HLEN);</div><div><span style="white-space:pre-wrap"> </span>printk("eth data %p\n", send_skb->data);<span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>skb_reset_mac_header(send_skb);</div><div><span style="white-space:pre-wrap"> </span>send_skb->protocol = eth->h_proto = htons(ETH_P_IP);</div><div><span style="white-space:pre-wrap"> </span>//printk("dev_addr is %p, len is %d", dev->dev_addr, ETH_ALEN);</div><div><span style="white-space:pre-wrap"> </span>printk("h_source is %p, dev_addr is %p, len is %d", eth->h_source, dev->dev_addr, ETH_ALEN);</div><div><span style="white-space:pre-wrap"> </span>memcpy(eth->h_source, dev->dev_addr, ETH_ALEN);</div><div><span style="white-space:pre-wrap"> </span>u8 dst_mac[ETH_ALEN] = DST_MAC;</div><div><span style="white-space:pre-wrap"> </span>memcpy(eth->h_dest, dst_mac, ETH_ALEN);</div><div><span style="white-space:pre-wrap"> </span>send_skb->dev = dev;</div><div><span style="white-space:pre-wrap"> </span>int result = dev_queue_xmit(send_skb);</div><div><span style="white-space:pre-wrap"> </span>printk("result is %d\n", result);</div><div><span style="white-space:pre-wrap"> </span>spin_unlock(&lock);</div></div><div>################################################################</div><div><br></div><div>But I think construct packet is wrong, and Oops is:</div><div>################################################################</div><div><div><1>BUG: unable to handle kernel paging request at ffffeae38000b520</div><div><1>IP: [<ffffffff81170bab>] kfree+0x7b/0x320</div><div><4>PGD 0 </div><div><4>Oops: 0000 [#1] SMP </div><div><4>last sysfs file: /sys/devices/system/cpu/online</div><div><4>CPU 0 </div><div><4>Modules linked in: my_mmap(U) 8021q garp stp llc fuse vmhgfs(U) vsock(U) ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 uinput microcode ppdev vmware_balloon snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore snd_page_alloc e1000 parport_pc parport sg vmci(U) i2c_piix4 i2c_core shpchp ext4 jbd2 mbcache sd_mod crc_t10dif sr_mod cdrom mptspi mptscsih mptbase scsi_transport_spi pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: my_mmap]</div><div><4></div><div><4>Pid: 0, comm: swapper Tainted: G W --------------- 2.6.32-431.el6.x86_64 #1 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform</div><div><4>RIP: 0010:[<ffffffff81170bab>] [<ffffffff81170bab>] kfree+0x7b/0x320</div><div><4>RSP: 0018:ffff88000c4037c0 EFLAGS: 00010082</div><div><4>RAX: ffffeae38000b520 RBX: ffffc9000033c400 RCX: ffffc9000033c400</div><div><4>RDX: ffffea0000000000 RSI: ffff8800378585d8 RDI: ffffc9000033c400</div><div><4>RBP: ffff88000c403820 R08: ffff880037b36020 R09: 0000000000000000</div><div><4>R10: 0000000000000000 R11: ffff88000c40384c R12: ffffffff8144fc98</div><div><4>R13: 0000000000000286 R14: 0000000000000000 R15: ffffc9000033c400</div><div><4>FS: 0000000000000000(0000) GS:ffff88000c400000(0000) knlGS:0000000000000000</div><div><4>CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b</div><div><4>CR2: ffffeae38000b520 CR3: 000000007bb47000 CR4: 00000000000007f0</div><div><4>DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000</div><div><4>DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400</div><div><4>Process swapper (pid: 0, threadinfo ffffffff81a00000, task ffffffff81a8d020)</div><div><4>Stack:</div><div><4> 3030303030303436 0000000000003082 3030303030303030 ffff88007d9e0ae0</div><div><4><d> 000000000000001d ffff88007a81d740 ffff88000c4168a8 ffff8800379b19c0</div><div><4><d> ffff8800379b19c0 ffff88007bb7b800 000000000000002a ffffc9000033c400</div><div><4>Call Trace:</div><div><4> <IRQ> </div><div><4> [<ffffffff8144fc98>] skb_release_data+0xd8/0x110</div><div><4> [<ffffffff8144f7ce>] __kfree_skb+0x1e/0xa0</div><div><4> [<ffffffff8144f88b>] consume_skb+0x3b/0x80</div><div><4> [<ffffffff814fadec>] packet_rcv+0x5c/0x440</div><div><4> [<ffffffff8145bb25>] dev_queue_xmit_nit+0x125/0x1a0</div><div><4> [<ffffffff81460346>] dev_hard_start_xmit+0x216/0x480</div><div><4> [<ffffffff8147bd0a>] sch_direct_xmit+0x15a/0x1c0</div><div><4> [<ffffffff81460858>] dev_queue_xmit+0x228/0x320</div><div><4> [<ffffffffa002a4b1>] hook_local_in+0x3e1/0x410 [my_mmap]</div><div><4> [<ffffffff8105a360>] ? check_preempt_wakeup+0x1c0/0x260</div><div><4> [<ffffffff814893a9>] nf_iterate+0x69/0xb0</div><div><4> [<ffffffff814938a0>] ? ip_rcv_finish+0x0/0x440</div><div><4> [<ffffffff81489566>] nf_hook_slow+0x76/0x120</div><div><4> [<ffffffff814938a0>] ? ip_rcv_finish+0x0/0x440</div><div><4> [<ffffffff81493f44>] ip_rcv+0x264/0x350</div><div><4> [<ffffffff8145b54b>] __netif_receive_skb+0x4ab/0x750</div><div><4> [<ffffffff8145f1b8>] netif_receive_skb+0x58/0x60</div><div><4> [<ffffffff8145f2c0>] napi_skb_finish+0x50/0x70</div><div><4> [<ffffffff81460a29>] napi_gro_receive+0x39/0x50</div><div><4> [<ffffffffa01b5a88>] e1000_receive_skb+0x58/0x90 [e1000]</div><div><4> [<ffffffffa01ba5b2>] e1000_clean_rx_irq+0x2b2/0x5e0 [e1000]</div><div><4> [<ffffffffa01b7cd1>] e1000_clean+0x1f1/0xa50 [e1000]</div><div><4> [<ffffffff81068ff5>] ? enqueue_entity+0x125/0x450</div><div><4> [<ffffffff81069384>] ? enqueue_task_fair+0x64/0x100</div><div><4> [<ffffffff81059216>] ? enqueue_task+0x66/0x80</div><div><4> [<ffffffff81460b43>] net_rx_action+0x103/0x2f0</div><div><4> [<ffffffff8107a8e1>] __do_softirq+0xc1/0x1e0</div><div><4> [<ffffffff8100c30c>] call_softirq+0x1c/0x30</div><div><4> [<ffffffff8100fa75>] do_softirq+0x65/0xa0</div><div><4> [<ffffffff8107a795>] irq_exit+0x85/0x90</div><div><4> [<ffffffff81530fe5>] do_IRQ+0x75/0xf0</div><div><4> [<ffffffff8100b9d3>] ret_from_intr+0x0/0x11</div><div><4> <EOI> </div><div><4> [<ffffffff8103eacb>] ? native_safe_halt+0xb/0x10</div><div><4> [<ffffffff810167bd>] default_idle+0x4d/0xb0</div><div><4> [<ffffffff81009fc6>] cpu_idle+0xb6/0x110</div><div><4> [<ffffffff8150cbea>] rest_init+0x7a/0x80</div><div><4> [<ffffffff81c26f8f>] start_kernel+0x424/0x430</div><div><4> [<ffffffff81c2633a>] x86_64_start_reservations+0x125/0x129</div><div><4> [<ffffffff81c26453>] x86_64_start_kernel+0x115/0x124</div><div><4>Code: 44 00 00 48 89 df e8 a5 e4 ed ff 48 c1 e8 0c 48 8d 14 c5 00 00 00 00 48 c1 e0 06 48 29 d0 48 ba 00 00 00 00 00 ea ff ff 48 01 d0 <48> 8b 10 66 85 d2 0f 88 1b 02 00 00 84 d2 0f 89 0d 02 00 00 4c </div><div><1>RIP [<ffffffff81170bab>] kfree+0x7b/0x320</div><div><4> RSP <ffff88000c4037c0></div><div><4>CR2: ffffeae38000b520</div></div><div>################################################################<br></div><div><br></div><div>My kernel version is: 2.6.32-431.el6.x86_64</div><div><br></div><div>I have confused with this question a long time.</div><div>How to fix it ? Thank you.</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div>
</blockquote></div><br></div>