I have followed the code in here and fixed the issue for printing out IP address. I perfectly worked when it reads a captured file from my machine and the results are the same with tcpdump. However, when I read another pcap file (captured from the boundary router of a big network), it gives me totally different IP addresses. I found these pcap contains VLAN in the ethernet frames. How can detect if a packet contains a vlan header?
You'd have to examine the physical layer protocol (Most likely ethernet nowadays) and determine the ethernet type (the 13th and 14th bytes of the ethernet header).You can view an example list of possible ethernet types here.
If the type is 0x0800 (IPv4) then everything should work as expected.
However, If the ethertype is 0x8100 (802.1Q) you'd have to extract the actual payload type from the VLAN header (the 17th and 18th bytes)
Here is a very crude code to bypass the upper layers starting from a base address pointing at the ethernet beginning
char *get_ip_hdr(char *base) {
// If frame is not ethernet retun NULL
uint16_t ether_type = ntohs(*(uint16_t *) (base + 12));
if (ether_type == 0x0800 ) {
return base + 14;
} else if (ether_type == 0x8100 ) {
// VLAN tag
ether_type = ntohs(*(uint16_t *) (base + 16));
if (ether_type == 0x800) {
return base + 18;
}
}
return NULL
}
Note be wary of double VLAN tagging and take the necessary similar steps to skip it as well.
Related
I am using DPDK19.11.10 on centos.
The application is working fine with HW offloading if I send only the IPV4 packet without the VLAN header.
If I add the VLAN header with IPV4, HW offloading is not working.
If capture the pcap on ubuntu gateway the IP header is corrupted with Fragmented IP packet even though we are not fragmenting IP packet.
We verified capabalities like this:
if (!(dev->tx_offload_capa & DEV_TX_OFFLOAD_VLAN_INSERT)) {
rte_panic(" VLAN offload not supported");
}
Below is my code:
.offloads = (DEV_TX_OFFLOAD_IPV4_CKSUM |
DEV_TX_OFFLOAD_UDP_CKSUM | DEV_TX_OFFLOAD_TCP_CKSUM | DEV_TX_OFFLOAD_VLAN_INSERT),
m->l2_len = L2_HDR_SIZE;
m->l3_len = L3_IPV4_HDR_SIZE;
ip_hdr->check = 0;
m->ol_flags |= PKT_TX_IPV4 | PKT_TX_IP_CKSUM;
ip_hdr = rte_pktmbuf_mtod(m, struct iphdr *);
vlan1_hdr = (struct vlan1_hdr *) rte_pktmbuf_prepend(m, sizeof(struct vlan1_hdr));
eth_hdr = (struct ethernet_hdr *) rte_pktmbuf_prepend(m, (uint16_t)sizeof(struct ethernet_hdr));
Once I received the packet in the ubuntu gateway the IP packet is corrupted as a fragmented IP packet.
The same code works fine if I removed the VLAN header.
Does anything else need to add here?
By the sound of it,
You might misunderstand the way how HW Tx VLAN offload is supposed to work;
Your code does not update m->l2_len when it inserts a VLAN header.
First of all, your code enables support for HW Tx VLAN offload, but, oddly enough, it does not actually attempt to use it. If one wants to use hardware Tx VLAN offload, they should set PKT_TX_VLAN in m->ol_flags and fill out m->vlan_tci. The VLAN header will be added by the hardware.
However, your code prepends the header itself, like if there was no intent to use a hardware offload in the first place. Your code does m->l2_len = L2_HDR_SIZE;, which, as I presume, only counts for Ethernet header. When your code prepends a VLAN header, this variable has to be updated accordingly:
m->l2_len += sizeof(struct rte_vlan_hdr);
Most DPDK NIC PMD supports HW VLAN offload (RX direction). But a limited number of PMD support the DEV_TX_OFFLOAD_VLAN_INSERT feature namely
Aquantia Atlantic
Marvell OCTEON CN9K/CN10K SoC
Cisco VIC adapter
Pensando NIC
OCTEON TX2
Wangxun 10 Gigabit Ethernet NIC and
Intel NIC - i40e, ice, iavf, ixgbe, igb
To enable HW VLAN INSERT one needs to check
if DEV_TX_OFFLOAD_VLAN_INSERT by checking get_dev_info
configure tx offload for the port with DEV_TX_OFFLOAD_VLAN_INSERT
enable MBUF descriptor with ol_flags = PKT_TX_VLAN and vlan_tci = [desired TCI in big-endian format]
This will allow the driver code in xmit function to check mbuf descriptors ol_flags for PKT_TX_VLAN and enables VLAN Insert offload to Hardware by registering the appropriate command with the Packet Descriptor before DMA.
From DPDK conditions are to be satisfied
at a given instance there should only be 1 thread access and updating the mbuf.
no modification for the mbuf is to be done on the original mbuf (with payload).
If the intention is to perform VLAN insert via SW (especially if HW or virtual NIC PMD does not support), in dpdk one has to do the following
Ensure the refcnt is 1 to prevent multiple thread access and modification on the intended buffer.
There is enough headroom to shift the packet 4 bytes to accommodate the Ether type and VLAN values.
ensure pkt_len and data_len are in bound (greater than 60 bytes and less than 4 bytes of MTU)
MBUF offload descriptors is not enabled for PKT_TX_VLAN
update data_len on the modified MBUF by 4 Bytes.
Update total pkt_len by 4.
(optional for performance consideration) prefetch the 4 bytes prior to mtod of mbuf memory address
Note: All the above things are easily achieved by using the DPDK function rte_vlan_insert. TO use the same follow the steps as
Do not configure the port with DEV_TX_OFFLOAD_VLAN_INSERT.
Update ol_flags with PKT_TX_VLAN and vlan_tci desired value.
Invoke rte_vlan_insert with the mbuf
Sample code:
/* Get burst of RX packets, from first port of pair. */
struct rte_mbuf *bufs[BURST_SIZE];
const uint16_t nb_rx = rte_eth_rx_burst(port, 0, bufs, BURST_SIZE);
if (unlikely(nb_rx == 0))
continue;
for (int i = 0; i < nb_rx; i++) {
bufs[i]->ol_flags = PKT_TX_VLAN;
bufs[i]->vlan_tci = 0x10;
rte_vlan_insert(&bufs[i]);
}
/* Send burst of TX packets, to second port of pair. */
const uint16_t nb_tx = rte_eth_tx_burst(port, 0,
bufs, nb_rx);
/* Free any unsent packets. */
if (unlikely(nb_tx < nb_rx)) {
uint16_t buf;
for (buf = nb_tx; buf < nb_rx; buf++)
rte_pktmbuf_free(bufs[buf]);
}
I’m writing a C program which builds an Ethernet/IPv4/TCP network packet, then writes the packet into a PCAP file for inspection. I build my code off the SO post here. The first version of my code worked perfectly, but it was one big main() function, and that is not portable into larger programs.
So I reorganized the code so I could port it into another program. I don’t want to get into the differences between Version 1 and Version 2 in this post. But needless to say, Version 2 works great, except for one annoying quirk. When Wireshark opened a Version 1 PCAP file, it saw that my Layer 2 was Ethernet II:
Frame 1: 154 bytes on wire (1232 bits), 154 bytes captured (1232 bits)
Ethernet II, Src: 64:96:c8:fa:fc:ff (64:96:c8:fa:fc:ff), Dst: Woonsang_04:05:06 (01:02:03:04:05:06)
Destination: Woonsang_04:05:06 (01:02:03:04:05:06)
Source: 64:96:c8:fa:fc:ff (64:96:c8:fa:fc:ff)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.10.10.10, Dst: 20.20.20.20
Transmission Control Protocol, Src Port: 22, Dst Port: 55206, Seq: 1, Ack: 1, Len: 100
SSH Protocol
But in Version 2, the Layer 2 header became 802.3 Ethernet:
Frame 1: 154 bytes on wire (1232 bits), 134 bytes captured (1072 bits)
IEEE 802.3 Ethernet
Destination: Vibratio_1c:08:00 (00:09:70:1c:08:00)
Source: 45:00:23:28:06:cf (45:00:23:28:06:cf)
Length: 64
Trailer: 050401040204000001020506040400070602040704060202…
Logical-Link Control
Data (61 bytes)
[Packet size limited during capture: Ethernet truncated]
I’m no expert in networking, but I’m guessing my Version 2 PCAP file is malformed somewhere. I should not have a Logical-Link Control header in there; my code thinks it is writing Ethernet II / IPv4 / TCP headers. At this point, my instinct is that either the PCAP Packet header (necessary to proceed every packet in a PCAP file) or my Ethernet header is incorrect, somehow. Which would tell Wireshark “the next X bytes are an Ethernet II header?"
Here’s my code, in excerpts:
The structs for the PCAP header and Ethernet frames were cribbed directly from the before-mentioned SO post. The solution in that post was to use the pcap_sf_pkthdr struct for the PCAP Packet header:
// struct for PCAP Packet Header - Timestamp
struct pcap_timeval {
bpf_int32 tv_sec; // seconds
bpf_int32 tv_usec; // microseconds
};
// struct for PCAP Packet Header
struct pcap_sf_pkthdr {
struct pcap_timeval ts; // time stamp
bpf_u_int32 caplen; // length of portion present
bpf_u_int32 len; // length this packet (off wire)
};
And the Ethernet header is from the original post:
// struct for the Ethernet header
struct ethernet {
u_char mac1[6];
u_char mac2[6];
u_short protocol; // will be ETHERTYPE_IP, for IPv4
};
There’s not much to either struct, right? I don’t really understand how Wireshark looks at this and knows the first 20 bytes of the packet are Ethernet.
Here’s the actual code, slightly abridged:
#include <netinet/in.h> // for ETHERTYPE_IP
struct pcap_sf_pkthdr* allocatePCAPPacketHdr(struct pcap_sf_pkthdr* pcapPacketHdr ){
pcapPacketHdr = malloc( sizeof(struct pcap_sf_pkthdr) );
if( pcapPacketHdr == NULL ){
return NULL;
}
uint32_t frameSize = sizeof( struct ethernet) + …correctly computed here
bzero( pcapPacketHdr, sizeof( struct pcap_sf_pkthdr ) );
pcapPacketHdr->ts.tv_sec = 0; // for now
pcapPacketHdr->ts.tv_usec = 0; // for now
pcapPacketHdr->caplen = frameSize;
pcapPacketHdr->len = frameSize;
return pcapPacketHdr;
}
void* allocateL2Hdr( packetChecklist* pc, void* l2header ){
l2header = malloc( sizeof( struct ethernet ) );
if( l2header == NULL ){
return NULL;
}
bzero( ((struct ethernet*)l2header)->mac1, 6 );
bzero( ((struct ethernet*)l2header)->mac2, 6 );
// …MAC addresses filled in later…
((struct ethernet*)l2header)->protocol = ETHERTYPE_IP; // This is correctly set
return l2header;
}
...and the code which uses the above functions...
struct pcap_sf_pkthdr* pcapPacketHdr;
pcapPacketHdr = allocatePCAPPacketHdr( pcapPacketHdr );
struct ethernet* l2header;
l2header = allocateL2Hdr( l2header );
Later, the code populates these structs and writes them into a file, along with an IPv4 header, a TCP header, and so on.
But I think my problem is that I don’t really understand how Wireshark is supposed to know that my Ethernet header is Ethernet II and not 802.3 Ethernet with an Logical-Link Header. Is that communicated in the PCAP Packet Header? Or in the ethernet frame somewhere? I’m hoping for advice. Thank you
Wireshark is supposed to know that my Ethernet header is Ethernet II and not 802.3 Ethernet with an Logical-Link Header. Is that communicated in the PCAP Packet Header?
No.
Or in the ethernet frame somewhere?
Yes.
If you want the details, see, for example, the "Types" section of the Wikipedia "Ethernet frame" page.
However, the problem appears to be that the packet you're writing to the file doesn't have the full 6-byte destination and source addresses in it - the last two bytes of the destination address are 0x08 0x00, which are the first two bytes of a big-endian value of ETHERTYPE_IP (0x0800), and the first byte of the source address is 0x45, which is the first byte of an IPv4 header for an IPv4 packet with no IP options.
Somehow, Version 1 of your program put the destination and source addresses into the data part of the pcap record, but Version 2 didn't.
Today I was investing a little more time to learn about ARP packets. To understand it's structure I tried to build one on myself in C using libpcap. I structured a simple ARP request packet and used pcap_inject function to send the packet. This function returns the number of bytes that are sent.
When I debug my code I saw that my packet was 42 bytes long. I search the Internet a bit and couldn't find a answer that tells me if this is the appropriate size for an ARP request or not. Even the wikipedia entry confused me a little. And the I discovered this post. From the answer provided by the user:
If the ARP message is to be sent in an untagged frame then the frame overhead itself is 18 bytes. That would result in a frame of
28+18=46 bytes without padding. Additional 18 bytes of padding are
necessary in this case to bloat the frame to the 64 byte length.
If the ARP message is to be sent in an 802.1Q-tagged frame then the frame overhead is 22 bytes, resulting in the total frame size of
28+22=50 bytes. In this case, the padding needs to be 14 bytes long.
If the ARP message is to be sent in a double-tagged frame then the frame overhead is 26 bytes, resulting in the total frame size of 54
bytes. In this case, the padding needs to be 10 bytes long.
My question is what do I have to do in this situation. Do I have to use padding or not?
Bellow I post the structure of my packet.
#define ETH_P_ARP 0x0806 /* Address Resolution packet */
#define ARP_HTYPE_ETHER 1 /* Ethernet ARP type */
#define ARP_PTYPE_IPv4 0x0800 /* Internet Protocol packet */
/* Ethernet frame header */
typedef struct {
uint8_t dest_addr[ETH_ALEN]; /* Destination hardware address */
uint8_t src_addr[ETH_ALEN]; /* Source hardware address */
uint16_t frame_type; /* Ethernet frame type */
} ether_hdr;
/* Ethernet ARP packet from RFC 826 */
typedef struct {
uint16_t htype; /* Format of hardware address */
uint16_t ptype; /* Format of protocol address */
uint8_t hlen; /* Length of hardware address */
uint8_t plen; /* Length of protocol address */
uint16_t op; /* ARP opcode (command) */
uint8_t sha[ETH_ALEN]; /* Sender hardware address */
uint32_t spa; /* Sender IP address */
uint8_t tha[ETH_ALEN]; /* Target hardware address */
uint32_t tpa; /* Target IP address */
} arp_ether_ipv4;
In the end I just copy each structure member in the bellow order and send the packet:
void packageARP(unsigned char *buffer, ether_hdr *frameHeader, arp_ether_ipv4 *arp_packet, size_t *bufferSize) {
unsigned char *cp;
size_t packet_size;
cp = buffer;
packet_size = sizeof(frameHeader->dest_addr)
+ sizeof(frameHeader->src_addr)
+ sizeof(frameHeader->frame_type)
+ sizeof(arp_packet->htype)
+ sizeof(arp_packet->ptype)
+ sizeof(arp_packet->hlen)
+ sizeof(arp_packet->plen)
+ sizeof(arp_packet->op)
+ sizeof(arp_packet->sha)
+ sizeof(arp_packet->spa)
+ sizeof(arp_packet->tha)
+ sizeof(arp_packet->tpa);
/*
* Copy the Ethernet frame header to the buffer.
*/
memcpy(cp, &(frameHeader->dest_addr), sizeof(frameHeader->dest_addr));
cp += sizeof(frameHeader->dest_addr);
memcpy(cp, &(frameHeader->src_addr), sizeof(frameHeader->src_addr));
cp += sizeof(frameHeader->src_addr);
/* Normal Ethernet-II framing */
memcpy(cp, &(frameHeader->frame_type), sizeof(frameHeader->frame_type));
cp += sizeof(frameHeader->frame_type);
/*
* Add the ARP data.
*/
memcpy(cp, &(arp_packet->htype), sizeof(arp_packet->htype));
cp += sizeof(arp_packet->htype);
memcpy(cp, &(arp_packet->ptype), sizeof(arp_packet->ptype));
cp += sizeof(arp_packet->ptype);
memcpy(cp, &(arp_packet->hlen), sizeof(arp_packet->hlen));
cp += sizeof(arp_packet->hlen);
memcpy(cp, &(arp_packet->plen), sizeof(arp_packet->plen));
cp += sizeof(arp_packet->plen);
memcpy(cp, &(arp_packet->op), sizeof(arp_packet->op));
cp += sizeof(arp_packet->op);
memcpy(cp, &(arp_packet->sha), sizeof(arp_packet->sha));
cp += sizeof(arp_packet->sha);
memcpy(cp, &(arp_packet->spa), sizeof(arp_packet->spa));
cp += sizeof(arp_packet->spa);
memcpy(cp, &(arp_packet->tha), sizeof(arp_packet->tha));
cp += sizeof(arp_packet->tha);
memcpy(cp, &(arp_packet->tpa), sizeof(arp_packet->tpa));
cp += sizeof(arp_packet->tpa);
*bufferSize = packet_size;
}
Is this the correct way of structuring an ARP request packet?
That's the correct structure -- except that the C compiler is free to insert padding in order to ensure structure members are placed at the most efficient boundaries. In particular, spa and tpa are not at natural 32-bit boundaries (due to the preceding 6-byte MAC address fields) and so the compiler might want to insert two bytes of padding before each.
If you are using gcc, you can ensure that doesn't happen with __attribute__((packed)):
struct {
[fields]
} __attribute__((packed)) arp_ether_ipv4;
Other compilers might have a different but equivalent mechanism (a #pragma directive for example).
The ARP payload should be 28 bytes. Adding the 14-byte ethernet header, that gives 42 total bytes. As your cite said, an 802.1Q (VLAN) header inserts an additional 4 bytes and a "double-tagged" frame (not common outside of Internet service providers) will add 2 X 4 = 8 bytes. If you're on an ordinary endpoint machine, you wouldn't typically add these headers anyway. The IT department will have configured your switches to automatically insert/remove these headers as needed.
The 42 bytes will get padded automatically to 64 bytes (Ethernet minimum packet size) by your network driver. 64 is actually 60 + the 4-byte Ethernet FCS [frame checksum]. (The post you cited is apparently including the 4-byte FCS in their calculations, which is why their numbers seem whack.)
Also, don't forget to use network byte order for all uint16_t and uint32_t fields: (ntohs and ntohl)
I would like to understand how it's possible to iterate over a packet collected with pcap.
#include <pcap.h>
#include <netinet/if_ether.h>
#include <netinet/ip.h>
#include <netinet/tcp.h>
void analyse(struct pcap_pkthdr *header, const unsigned char *packet, int verbose) {
/** Ethernet header has a fixed value, IP header and TCP header don't **/
ip_size = sizeof( struct ip );
tcp_size = sizeof( struct tcphdr );
/* Assign each pointer its correct value **/
const struct ether_header *ethernet = ( struct ether_header* ) packet;
const struct ip *ip = (struct ip*) ( packet + ETH_HLEN );
const struct tcp *tcp = (struct tcphdr*) (packet + ETH_HLEN + ip_size );
const char *payload = ( packet + ETH_HLEN + ip_size + tcp_size );
}
Can I be sure that the ethernet, ip, tcp,payload`respectively point to:
First bit of the Data link Layer (Ethernet header)
First bit of the Network Layer (IP header)
First bit of the Transport Layer (TCP header)
First bit of the payload
Thanks,
No, you cannot. I assume you are talking about the first version of the PCAP standard. There is another one called pcap-ng (next generation).
https://www.ietf.org/staging/draft-tuexen-opsawg-pcapng-02.html
File header
On the start of PCAP file there is a fixed size header. The last field in that header is the data link type.
https://tools.ietf.org/id/draft-gharris-opsawg-pcap-00.html
Hopefully the link type LINKTYPE_ETHERNET value is 0x1. If it is not, you can basically throw away the entire file.
https://www.tcpdump.org/linktypes.html
Ethernet header
So now we know that every packet in the file will the of the type ethernet. You skip the ethernet header just adding sizeof(ethhdr).
After the ethernet header the IP header might not be the next layer. You have to process the protocols as they show up in the chain.
For example, VLAN headers are placed right after the ethernet header, sometimes multiple VLAN headers in chain so you have to skip all them. This would be indicated by ethernet->h_proto==ETH_P_8021Q.
VLAN is one of them. There are other protocols and that's where it gets really complex to write something very generic to parse all them.
In 99.9% of the cases you can assume the ethernet header plus the potentially multiple VLAN headers but you have to skip each of them in sequence until ethernet->h_proto==ETH_P_IP or an unknown protocol is found, in which case you bail out.
IP Headers
If it is IPv4, skip the standard IP header and then the IP options. The IP header has variable size, although most of the time it is fixed. This is due to the options part of the IP header. What you see in the struct iphdr is only the fixed part.
You have to account for skipping the options part too so you have to add (ip->ihl & 0xF) * 4 and which will be typically equal to sizeof(iphdr) if ip->ihl&0xF is 5, which is almost always the case.
TCP header
The TCP header also contains options but in the fixed part of the header you have a count of how many 32-bit blocks the entire TCP header has. Just skip the entire header by adding tcp->th_off*4.
5 years later this is responded.
I am trying to view http traffic going to and from my loopback network adapter using libpcap. I just beginning with network programming and completely new to this library. Thanks to an answer I received previously I have been successful at detecting the link-layer type on my machine's "lo0" adapter (Mac OSx).
//lookup link-layer header type
link_layer_type = pcap_datalink(handle);
if(link_layer_type == DLT_NULL){
printf("DLT_NULL"); // this true in the case of "lo0"
}
The Programming with Pcap guide makes the assumption that each packet will contain an ethernet header. So the logic used to find a packet's payload is as follows:
ethernet = (struct sniff_ethernet*)(packet);
ip = (struct sniff_ip*)(packet + SIZE_ETHERNET);
size_ip = IP_HL(ip)*4;
if (size_ip < 20) {
printf(" * Invalid IP header length: %u bytes\n", size_ip);
return;
}
tcp = (struct sniff_tcp*)(packet + SIZE_ETHERNET + size_ip);
size_tcp = TH_OFF(tcp)*4;
if (size_tcp < 20) {
printf(" * Invalid TCP header length: %u bytes\n", size_tcp);
return;
}
}
payload = (u_char *)(packet + SIZE_ETHERNET + size_ip + size_tcp);
This logic is clearing not going to work when inspecting the contents of packet originating from the loopback interface where an ethernet header does not exists. The Link-Layer Header Types documentation states that a Link-Layer type of "DTL_NULL" contains a 4 byte header which consist of a PF_ value containing the network-layer protocol (I'm guess IPv4 in my case).
Given the above information.. how can I properly locate the packet's payload location?
Any guidance or information would be very appreciated. Thanks!
Given the above information.. how can I properly locate the packet's payload location?
For DLT_NULL, your program should extract the first 4 bytes of the packet data as a 32-bit number. If you're doing a live capture, you can extract it in the host's byte order and compare it against your OS's values of AF_INET and AF_INET6 (if it has an AF_INET6 definition; these days, most current OS versions should, as they should support IPv6); if you're reading a capture file, you'd need to byte-swap the value if pcap_is_swapped() returns a non-zero value (you can also use it for live captures; it always returns zero for live captures), and you'll need to compare against several different "IPv6" values (24, 28, and 30), each of which mean "IPv6" on some particular OS (fortunately, AF_INET is 2 on all OSes that support DLT_NULL, as they all took that value from 4.2BSD).
If the value is the IPv4 value (2, as per the above), then after those 4 bytes you have the IPv4 header for the packet. If it's one of the IPv6 values, then after those 4 bytes you have the IPv6 header for the packet. If it's not any of those values, it's some other protocol.