I am trying to view http traffic going to and from my loopback network adapter using libpcap. I just beginning with network programming and completely new to this library. Thanks to an answer I received previously I have been successful at detecting the link-layer type on my machine's "lo0" adapter (Mac OSx).
//lookup link-layer header type
link_layer_type = pcap_datalink(handle);
if(link_layer_type == DLT_NULL){
printf("DLT_NULL"); // this true in the case of "lo0"
}
The Programming with Pcap guide makes the assumption that each packet will contain an ethernet header. So the logic used to find a packet's payload is as follows:
ethernet = (struct sniff_ethernet*)(packet);
ip = (struct sniff_ip*)(packet + SIZE_ETHERNET);
size_ip = IP_HL(ip)*4;
if (size_ip < 20) {
printf(" * Invalid IP header length: %u bytes\n", size_ip);
return;
}
tcp = (struct sniff_tcp*)(packet + SIZE_ETHERNET + size_ip);
size_tcp = TH_OFF(tcp)*4;
if (size_tcp < 20) {
printf(" * Invalid TCP header length: %u bytes\n", size_tcp);
return;
}
}
payload = (u_char *)(packet + SIZE_ETHERNET + size_ip + size_tcp);
This logic is clearing not going to work when inspecting the contents of packet originating from the loopback interface where an ethernet header does not exists. The Link-Layer Header Types documentation states that a Link-Layer type of "DTL_NULL" contains a 4 byte header which consist of a PF_ value containing the network-layer protocol (I'm guess IPv4 in my case).
Given the above information.. how can I properly locate the packet's payload location?
Any guidance or information would be very appreciated. Thanks!
Given the above information.. how can I properly locate the packet's payload location?
For DLT_NULL, your program should extract the first 4 bytes of the packet data as a 32-bit number. If you're doing a live capture, you can extract it in the host's byte order and compare it against your OS's values of AF_INET and AF_INET6 (if it has an AF_INET6 definition; these days, most current OS versions should, as they should support IPv6); if you're reading a capture file, you'd need to byte-swap the value if pcap_is_swapped() returns a non-zero value (you can also use it for live captures; it always returns zero for live captures), and you'll need to compare against several different "IPv6" values (24, 28, and 30), each of which mean "IPv6" on some particular OS (fortunately, AF_INET is 2 on all OSes that support DLT_NULL, as they all took that value from 4.2BSD).
If the value is the IPv4 value (2, as per the above), then after those 4 bytes you have the IPv4 header for the packet. If it's one of the IPv6 values, then after those 4 bytes you have the IPv6 header for the packet. If it's not any of those values, it's some other protocol.
Related
I am using DPDK19.11.10 on centos.
The application is working fine with HW offloading if I send only the IPV4 packet without the VLAN header.
If I add the VLAN header with IPV4, HW offloading is not working.
If capture the pcap on ubuntu gateway the IP header is corrupted with Fragmented IP packet even though we are not fragmenting IP packet.
We verified capabalities like this:
if (!(dev->tx_offload_capa & DEV_TX_OFFLOAD_VLAN_INSERT)) {
rte_panic(" VLAN offload not supported");
}
Below is my code:
.offloads = (DEV_TX_OFFLOAD_IPV4_CKSUM |
DEV_TX_OFFLOAD_UDP_CKSUM | DEV_TX_OFFLOAD_TCP_CKSUM | DEV_TX_OFFLOAD_VLAN_INSERT),
m->l2_len = L2_HDR_SIZE;
m->l3_len = L3_IPV4_HDR_SIZE;
ip_hdr->check = 0;
m->ol_flags |= PKT_TX_IPV4 | PKT_TX_IP_CKSUM;
ip_hdr = rte_pktmbuf_mtod(m, struct iphdr *);
vlan1_hdr = (struct vlan1_hdr *) rte_pktmbuf_prepend(m, sizeof(struct vlan1_hdr));
eth_hdr = (struct ethernet_hdr *) rte_pktmbuf_prepend(m, (uint16_t)sizeof(struct ethernet_hdr));
Once I received the packet in the ubuntu gateway the IP packet is corrupted as a fragmented IP packet.
The same code works fine if I removed the VLAN header.
Does anything else need to add here?
By the sound of it,
You might misunderstand the way how HW Tx VLAN offload is supposed to work;
Your code does not update m->l2_len when it inserts a VLAN header.
First of all, your code enables support for HW Tx VLAN offload, but, oddly enough, it does not actually attempt to use it. If one wants to use hardware Tx VLAN offload, they should set PKT_TX_VLAN in m->ol_flags and fill out m->vlan_tci. The VLAN header will be added by the hardware.
However, your code prepends the header itself, like if there was no intent to use a hardware offload in the first place. Your code does m->l2_len = L2_HDR_SIZE;, which, as I presume, only counts for Ethernet header. When your code prepends a VLAN header, this variable has to be updated accordingly:
m->l2_len += sizeof(struct rte_vlan_hdr);
Most DPDK NIC PMD supports HW VLAN offload (RX direction). But a limited number of PMD support the DEV_TX_OFFLOAD_VLAN_INSERT feature namely
Aquantia Atlantic
Marvell OCTEON CN9K/CN10K SoC
Cisco VIC adapter
Pensando NIC
OCTEON TX2
Wangxun 10 Gigabit Ethernet NIC and
Intel NIC - i40e, ice, iavf, ixgbe, igb
To enable HW VLAN INSERT one needs to check
if DEV_TX_OFFLOAD_VLAN_INSERT by checking get_dev_info
configure tx offload for the port with DEV_TX_OFFLOAD_VLAN_INSERT
enable MBUF descriptor with ol_flags = PKT_TX_VLAN and vlan_tci = [desired TCI in big-endian format]
This will allow the driver code in xmit function to check mbuf descriptors ol_flags for PKT_TX_VLAN and enables VLAN Insert offload to Hardware by registering the appropriate command with the Packet Descriptor before DMA.
From DPDK conditions are to be satisfied
at a given instance there should only be 1 thread access and updating the mbuf.
no modification for the mbuf is to be done on the original mbuf (with payload).
If the intention is to perform VLAN insert via SW (especially if HW or virtual NIC PMD does not support), in dpdk one has to do the following
Ensure the refcnt is 1 to prevent multiple thread access and modification on the intended buffer.
There is enough headroom to shift the packet 4 bytes to accommodate the Ether type and VLAN values.
ensure pkt_len and data_len are in bound (greater than 60 bytes and less than 4 bytes of MTU)
MBUF offload descriptors is not enabled for PKT_TX_VLAN
update data_len on the modified MBUF by 4 Bytes.
Update total pkt_len by 4.
(optional for performance consideration) prefetch the 4 bytes prior to mtod of mbuf memory address
Note: All the above things are easily achieved by using the DPDK function rte_vlan_insert. TO use the same follow the steps as
Do not configure the port with DEV_TX_OFFLOAD_VLAN_INSERT.
Update ol_flags with PKT_TX_VLAN and vlan_tci desired value.
Invoke rte_vlan_insert with the mbuf
Sample code:
/* Get burst of RX packets, from first port of pair. */
struct rte_mbuf *bufs[BURST_SIZE];
const uint16_t nb_rx = rte_eth_rx_burst(port, 0, bufs, BURST_SIZE);
if (unlikely(nb_rx == 0))
continue;
for (int i = 0; i < nb_rx; i++) {
bufs[i]->ol_flags = PKT_TX_VLAN;
bufs[i]->vlan_tci = 0x10;
rte_vlan_insert(&bufs[i]);
}
/* Send burst of TX packets, to second port of pair. */
const uint16_t nb_tx = rte_eth_tx_burst(port, 0,
bufs, nb_rx);
/* Free any unsent packets. */
if (unlikely(nb_tx < nb_rx)) {
uint16_t buf;
for (buf = nb_tx; buf < nb_rx; buf++)
rte_pktmbuf_free(bufs[buf]);
}
I’m writing a C program which builds an Ethernet/IPv4/TCP network packet, then writes the packet into a PCAP file for inspection. I build my code off the SO post here. The first version of my code worked perfectly, but it was one big main() function, and that is not portable into larger programs.
So I reorganized the code so I could port it into another program. I don’t want to get into the differences between Version 1 and Version 2 in this post. But needless to say, Version 2 works great, except for one annoying quirk. When Wireshark opened a Version 1 PCAP file, it saw that my Layer 2 was Ethernet II:
Frame 1: 154 bytes on wire (1232 bits), 154 bytes captured (1232 bits)
Ethernet II, Src: 64:96:c8:fa:fc:ff (64:96:c8:fa:fc:ff), Dst: Woonsang_04:05:06 (01:02:03:04:05:06)
Destination: Woonsang_04:05:06 (01:02:03:04:05:06)
Source: 64:96:c8:fa:fc:ff (64:96:c8:fa:fc:ff)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.10.10.10, Dst: 20.20.20.20
Transmission Control Protocol, Src Port: 22, Dst Port: 55206, Seq: 1, Ack: 1, Len: 100
SSH Protocol
But in Version 2, the Layer 2 header became 802.3 Ethernet:
Frame 1: 154 bytes on wire (1232 bits), 134 bytes captured (1072 bits)
IEEE 802.3 Ethernet
Destination: Vibratio_1c:08:00 (00:09:70:1c:08:00)
Source: 45:00:23:28:06:cf (45:00:23:28:06:cf)
Length: 64
Trailer: 050401040204000001020506040400070602040704060202…
Logical-Link Control
Data (61 bytes)
[Packet size limited during capture: Ethernet truncated]
I’m no expert in networking, but I’m guessing my Version 2 PCAP file is malformed somewhere. I should not have a Logical-Link Control header in there; my code thinks it is writing Ethernet II / IPv4 / TCP headers. At this point, my instinct is that either the PCAP Packet header (necessary to proceed every packet in a PCAP file) or my Ethernet header is incorrect, somehow. Which would tell Wireshark “the next X bytes are an Ethernet II header?"
Here’s my code, in excerpts:
The structs for the PCAP header and Ethernet frames were cribbed directly from the before-mentioned SO post. The solution in that post was to use the pcap_sf_pkthdr struct for the PCAP Packet header:
// struct for PCAP Packet Header - Timestamp
struct pcap_timeval {
bpf_int32 tv_sec; // seconds
bpf_int32 tv_usec; // microseconds
};
// struct for PCAP Packet Header
struct pcap_sf_pkthdr {
struct pcap_timeval ts; // time stamp
bpf_u_int32 caplen; // length of portion present
bpf_u_int32 len; // length this packet (off wire)
};
And the Ethernet header is from the original post:
// struct for the Ethernet header
struct ethernet {
u_char mac1[6];
u_char mac2[6];
u_short protocol; // will be ETHERTYPE_IP, for IPv4
};
There’s not much to either struct, right? I don’t really understand how Wireshark looks at this and knows the first 20 bytes of the packet are Ethernet.
Here’s the actual code, slightly abridged:
#include <netinet/in.h> // for ETHERTYPE_IP
struct pcap_sf_pkthdr* allocatePCAPPacketHdr(struct pcap_sf_pkthdr* pcapPacketHdr ){
pcapPacketHdr = malloc( sizeof(struct pcap_sf_pkthdr) );
if( pcapPacketHdr == NULL ){
return NULL;
}
uint32_t frameSize = sizeof( struct ethernet) + …correctly computed here
bzero( pcapPacketHdr, sizeof( struct pcap_sf_pkthdr ) );
pcapPacketHdr->ts.tv_sec = 0; // for now
pcapPacketHdr->ts.tv_usec = 0; // for now
pcapPacketHdr->caplen = frameSize;
pcapPacketHdr->len = frameSize;
return pcapPacketHdr;
}
void* allocateL2Hdr( packetChecklist* pc, void* l2header ){
l2header = malloc( sizeof( struct ethernet ) );
if( l2header == NULL ){
return NULL;
}
bzero( ((struct ethernet*)l2header)->mac1, 6 );
bzero( ((struct ethernet*)l2header)->mac2, 6 );
// …MAC addresses filled in later…
((struct ethernet*)l2header)->protocol = ETHERTYPE_IP; // This is correctly set
return l2header;
}
...and the code which uses the above functions...
struct pcap_sf_pkthdr* pcapPacketHdr;
pcapPacketHdr = allocatePCAPPacketHdr( pcapPacketHdr );
struct ethernet* l2header;
l2header = allocateL2Hdr( l2header );
Later, the code populates these structs and writes them into a file, along with an IPv4 header, a TCP header, and so on.
But I think my problem is that I don’t really understand how Wireshark is supposed to know that my Ethernet header is Ethernet II and not 802.3 Ethernet with an Logical-Link Header. Is that communicated in the PCAP Packet Header? Or in the ethernet frame somewhere? I’m hoping for advice. Thank you
Wireshark is supposed to know that my Ethernet header is Ethernet II and not 802.3 Ethernet with an Logical-Link Header. Is that communicated in the PCAP Packet Header?
No.
Or in the ethernet frame somewhere?
Yes.
If you want the details, see, for example, the "Types" section of the Wikipedia "Ethernet frame" page.
However, the problem appears to be that the packet you're writing to the file doesn't have the full 6-byte destination and source addresses in it - the last two bytes of the destination address are 0x08 0x00, which are the first two bytes of a big-endian value of ETHERTYPE_IP (0x0800), and the first byte of the source address is 0x45, which is the first byte of an IPv4 header for an IPv4 packet with no IP options.
Somehow, Version 1 of your program put the destination and source addresses into the data part of the pcap record, but Version 2 didn't.
I would like to understand how it's possible to iterate over a packet collected with pcap.
#include <pcap.h>
#include <netinet/if_ether.h>
#include <netinet/ip.h>
#include <netinet/tcp.h>
void analyse(struct pcap_pkthdr *header, const unsigned char *packet, int verbose) {
/** Ethernet header has a fixed value, IP header and TCP header don't **/
ip_size = sizeof( struct ip );
tcp_size = sizeof( struct tcphdr );
/* Assign each pointer its correct value **/
const struct ether_header *ethernet = ( struct ether_header* ) packet;
const struct ip *ip = (struct ip*) ( packet + ETH_HLEN );
const struct tcp *tcp = (struct tcphdr*) (packet + ETH_HLEN + ip_size );
const char *payload = ( packet + ETH_HLEN + ip_size + tcp_size );
}
Can I be sure that the ethernet, ip, tcp,payload`respectively point to:
First bit of the Data link Layer (Ethernet header)
First bit of the Network Layer (IP header)
First bit of the Transport Layer (TCP header)
First bit of the payload
Thanks,
No, you cannot. I assume you are talking about the first version of the PCAP standard. There is another one called pcap-ng (next generation).
https://www.ietf.org/staging/draft-tuexen-opsawg-pcapng-02.html
File header
On the start of PCAP file there is a fixed size header. The last field in that header is the data link type.
https://tools.ietf.org/id/draft-gharris-opsawg-pcap-00.html
Hopefully the link type LINKTYPE_ETHERNET value is 0x1. If it is not, you can basically throw away the entire file.
https://www.tcpdump.org/linktypes.html
Ethernet header
So now we know that every packet in the file will the of the type ethernet. You skip the ethernet header just adding sizeof(ethhdr).
After the ethernet header the IP header might not be the next layer. You have to process the protocols as they show up in the chain.
For example, VLAN headers are placed right after the ethernet header, sometimes multiple VLAN headers in chain so you have to skip all them. This would be indicated by ethernet->h_proto==ETH_P_8021Q.
VLAN is one of them. There are other protocols and that's where it gets really complex to write something very generic to parse all them.
In 99.9% of the cases you can assume the ethernet header plus the potentially multiple VLAN headers but you have to skip each of them in sequence until ethernet->h_proto==ETH_P_IP or an unknown protocol is found, in which case you bail out.
IP Headers
If it is IPv4, skip the standard IP header and then the IP options. The IP header has variable size, although most of the time it is fixed. This is due to the options part of the IP header. What you see in the struct iphdr is only the fixed part.
You have to account for skipping the options part too so you have to add (ip->ihl & 0xF) * 4 and which will be typically equal to sizeof(iphdr) if ip->ihl&0xF is 5, which is almost always the case.
TCP header
The TCP header also contains options but in the fixed part of the header you have a count of how many 32-bit blocks the entire TCP header has. Just skip the entire header by adding tcp->th_off*4.
5 years later this is responded.
I have followed the code in here and fixed the issue for printing out IP address. I perfectly worked when it reads a captured file from my machine and the results are the same with tcpdump. However, when I read another pcap file (captured from the boundary router of a big network), it gives me totally different IP addresses. I found these pcap contains VLAN in the ethernet frames. How can detect if a packet contains a vlan header?
You'd have to examine the physical layer protocol (Most likely ethernet nowadays) and determine the ethernet type (the 13th and 14th bytes of the ethernet header).You can view an example list of possible ethernet types here.
If the type is 0x0800 (IPv4) then everything should work as expected.
However, If the ethertype is 0x8100 (802.1Q) you'd have to extract the actual payload type from the VLAN header (the 17th and 18th bytes)
Here is a very crude code to bypass the upper layers starting from a base address pointing at the ethernet beginning
char *get_ip_hdr(char *base) {
// If frame is not ethernet retun NULL
uint16_t ether_type = ntohs(*(uint16_t *) (base + 12));
if (ether_type == 0x0800 ) {
return base + 14;
} else if (ether_type == 0x8100 ) {
// VLAN tag
ether_type = ntohs(*(uint16_t *) (base + 16));
if (ether_type == 0x800) {
return base + 18;
}
}
return NULL
}
Note be wary of double VLAN tagging and take the necessary similar steps to skip it as well.
I'm writing network analyzer and I need to filter packets saved in file, I have written some code to filter http packets but I'm not sure if it work as it should because when I use my code on a pcap dump the result is 5 packets but in wireshark writing http in filter gives me 2 packets and if I use:
tcpdump port http -r trace-1.pcap
it gives me 11 packets.
Well, 3 different results, that's a little confusing.
The filter and the packet processing in me code is:
...
if (pcap_compile(handle, &fcode, "tcp port 80", 1, netmask) < 0)
...
while ((packet = pcap_next(handle,&header))) {
u_char *pkt_ptr = (u_char *)packet;
//parse the first (ethernet) header, grabbing the type field
int ether_type = ((int)(pkt_ptr[12]) << 8) | (int)pkt_ptr[13];
int ether_offset = 0;
if (ether_type == ETHER_TYPE_IP) // ethernet II
ether_offset = 14;
else if (ether_type == ETHER_TYPE_8021Q) // 802
ether_offset = 18;
else
fprintf(stderr, "Unknown ethernet type, %04X, skipping...\n", ether_type);
//parse the IP header
pkt_ptr += ether_offset; //skip past the Ethernet II header
struct ip_header *ip_hdr = (struct ip_header *)pkt_ptr;
int packet_length = ntohs(ip_hdr->tlen);
printf("\n%d - packet length: %d, and the capture lenght: %d\n", cnt++,packet_length, header.caplen);
}
My question is why there are 3 different result when filtering the http? And/Or if I'm filtering it wrong then how can I do it right, also is there a way to filter http(or ssh, ftp, telnet ...) packets using something else than the port numbers?
Thanks
So I have figured it out. It took a little search and understanding but I did it.
Wireshark filter set to http filter packets that have set in tcp port 80 and also flags set to PSH, ACK. After realizing this, the tcpdump command parameters which result in the same numbers of packets was easy to write.
So now the wireshark and tcpdump gives the same results
What about my code? well I figured that I actually had an error in my question, the filter
if (pcap_compile(handle, &fcode, "tcp port 80", 1, netmask) < 0)
indeed gives 11 packets (src and dst port set to 80 no matter what tcp flags are)
Now to filter the desired packets is a question of good understanding the filter syntax
or setting to filter only port 80 (21,22, ...) and then in callback function or in while loop get the tcp header and from there get the flags and use mask to see if it is the correct packet (PSH, ACK, SYN ...) the flags number are for example here