I'm making a program using pcap to parse .pcap files.
I'm actually working on the DNS protocol, i'm able to get the header and display its information. Now I'd like to display its Resource Records (Question, Answer, Authority, Additional).
I found this interesting doc: http://www.zytrax.com/books/dns/ch15/
And, as I did before for parsing the different headers, I wanted to create a structure and cast my packet in it.
Following this doc I created my structure as follow:
struct question_s {
u_short *qname;
u_short qtype;
u_short qclass;
}
and I'm casting :
struct question_s *record = (struct question_s*)(data + offset);
Where data is the packet representation, and offset is the total size of previous protocols.
Now I'm having trouble understanding some points, and as my English is not perfect, it's possible that I missed something in the documentation. Here are my questions:
As qname is of variable size, am I doing it right by making it a pointer on u_short?
All pointer are 8 bytes long, so my structure should be 12 bytes long, but where is the name in memory? Should I add 12 to my offset without taking care of the name length?
I tried to display qname, working on it as if it was a char*, but it doesn't seem to work (seg. fault), here is what I did:
void test(u_short *qname) {
for (int c = 0; qname[c] != 0; ++c)
write(1, &qname[c], 1);
}
But maybe there isn't a '\0' in the string?
May be that's an endianess issue? I use htons and htonl on all my u_short and u_int values because the network byte order isn't the same as mine, but I'm not sure it applies to pointers.
If you want to see how to dissect DNS records, first read and understand RFC 1035, and then take a look at the tcpdump code to dissect DNS records. It's harder than you think; you can't just overlay a structure on top of the raw packet data.
And you can't ever overlay a structure with a pointer in it on top of raw packet data. The pointer will almost certainly point to some bogus location in your address space; protocols don't send raw pointers over the network, as a pointer is a pointer in a particular address space, and two processes on the network will have different address spaces.
(In fact, just about everything in packet dissection is harder than people think when they first try to write code to dissect packets.)
Related
I've got a struct with 3 16-bit values in an embedded system:
struct {
uint16_t x;
uint16_t y;
uint16_t z;
};
The struct will be transmitted to another system running the same software on the same hardware. Transmission is done by sending a series of 8 8-bit values (send_data(uint8_t *data)).
My idea is just give send_data the address of the struct. The two final bytes would be considered garbage. Would this work? Is it safe?
This should be safe as long as you handle it correctly on the other end (don't write the two bytes to a random location for example).
I would also suggest you comment the issue clearly in the code so future maintainers don't run into trouble.
OR
You could change the structure for sending and receiving to include an name (eg padding) for the data at the end. This will allow you to null out those bytes before sending. Then the code should be quite clear to anyone working on it and if implemented right should have no effect on performance.
Would this work? - Yes (send_data the address of the struct.)
Is it safe? Operationally yes, but testing no.
Recommend to not send garbage. Pad the data given to send_data() as needed (2 bytes) and initialize the extra to something, say 0. This will ease debugging and make for consistent regression testing.
(Excuse me if I am not able to put the question correctly. English in not my primary language.)
I am trying to parse SyncE ESMC packet. It is Ethernet slow protocol packet.
Approach 1:
For parsing this packet, I used byte by byte approach similar to what has been done here.
Approach 2:
Other way to parse the packet would be to define a 'structure' to represent the whole packet and access individual fields to retrieve the value at particular offset.
However in this approach structure padding and alignment may come into picture(which I am not sure) but on Linux various packet headers are defined in form of structure, e.g. iphdr in ip.h. IP packets (buffer) can be type casted to 'iphdr' to retrieve ip header fields, so it must be working.
Which approach is better to parse a network packet in C?
Does structure padding and aligment make any difference while parsing a packet via approach 2? If yes, how did Linux headers overcome this problem?
Approach 1 is the best for portability. It allows you to safely avoid misaligned accesses, for example. In particular, if your machine is little-endian, this approach lets you take care of the byte-swapping very easily.
Approach 2 is sometimes convenient and often is faster code to write. If structure padding gets in your way, your compiler probably provides a flag or attribute (like __attribute__((__packed__)) or #pragma pack) to work around it. If you have a little-endian machine, however, you're still going to have to byteswap fields all over the place.
I am working on a driver for the Linux kernel. For the success of my project, I need to determine the amount of padding added to Ethernet frames smaller than the minimum size of 60 bytes (not counting the FCS). I am not generating these frames; I am receiving them on a NIC for processing.
Having a struct sk_buff, is it possible to determine the amount of trailing zeros added to the packet directly?
I can of course determine that value by going through the entire packet, figuring out where the content of the highest layer ends and then simply subtracting that position from the frame size (in this case, 60 bytes). But is there a more efficient way to do it directly from the information stored on a struct sk_buff?
EDIT: As far as I know, there is no way to check for zero padding directly using the sk_buff structure without actually looking at the ethernet header, which is simple enough.
That said, with some simple pointer arithmetic and byte subtraction, you can use the length field in IP data to figure out the padding.
This is a good reference for sk_buff:
http://vger.kernel.org/~davem/skb_data.html
And here is a good reference for the packet structure, showing the 'length' field in the bottom picture within 'data'.
http://nerdcrunch.com/wp-content/uploads/2011/05/Ethernet-Frame-Explained.png
I think this is the way it must be done, but it doesn't require parsing as you had previously maintained. The header/data structure fields are set up such that they can be referenced/stripped directly via pointer/array without parsing, and then by subtracting header+data length from raw packet length you can get the padding, all without inspecting the data.
Hope that helps.
Also, for best practice, you should probably have your driver account for both versions of 802.3 in use. You can do so by inspecting the Ethertype/length field. If the value is greater than 1536 (0x0600) than you know it's an Ethernet II type packet and the field contains an ethertype, which tells you what the ethernet packet encapsulates. There are some popular ones if you Wikipedia for "Ethertype."
For example, IP = 0x0800. If the field designates an Ethertype, you must resort to finding the data length field inside in order to find the padding. If it does not, which alot of Ethernet based LAN's still don't, then you can directly use the field specified as length to do your job.
IPv4 does almost the same, there is probably no better ways. Check ip_rcv():
len = ntohs(iph->tot_len);
if (skb->len < len) {
IP_INC_STATS_BH(dev_net(dev), IPSTATS_MIB_INTRUNCATEDPKTS);
goto drop;
} else if (len < (iph->ihl*4))
goto inhdr_error;
/* Our transport medium may have padded the buffer out. Now we know it
* is IP we can trim to the true length of the frame.
* Note this now means skb->len holds ntohs(iph->tot_len).
*/
if (pskb_trim_rcsum(skb, len)) {
IP_INC_STATS_BH(dev_net(dev), IPSTATS_MIB_INDISCARDS);
goto drop;
}
I have a small client server application in which i wish to send an entire structure over a TCP socket in C not C++. Assume the struct to be the following:
struct something{
int a;
char b[64];
float c;
}
I have found many posts saying that i need to use pragma pack or to serialize the data before sending and recieveing.
My question is, is it enough to use JUST pragma pack or just serialzation ? Or do i need to use both?
Also since serialzation is processor intensive process this makes your performance fall drastically, so what is the best way to serialize a struct WITHOUT using an external library(i would love a sample code/algo)?
You need the following to portably send struct's over the network:
Pack the structure. For gcc and compatible compilers, do this with __attribute__((packed)).
Do not use any members other than unsigned integers of fixed size, other packed structures satisfying these requirements, or arrays of any of the former. Signed integers are OK too, unless your machine doesn't use a two's complement representation.
Decide whether your protocol will use little- or big-endian encoding of integers. Make conversions when reading and writing those integers.
Also, do not take pointers of members of a packed structure, except to those with size 1 or other nested packed structures. See this answer.
A simple example of encoding and decoding follows. It assumes that the byte order conversion functions hton8(), ntoh8(), hton32(), and ntoh32() are available (the former two are a no-op, but there for consistency).
#include <stdint.h>
#include <inttypes.h>
#include <stdlib.h>
#include <stdio.h>
// get byte order conversion functions
#include "byteorder.h"
struct packet {
uint8_t x;
uint32_t y;
} __attribute__((packed));
static void decode_packet (uint8_t *recv_data, size_t recv_len)
{
// check size
if (recv_len < sizeof(struct packet)) {
fprintf(stderr, "received too little!");
return;
}
// make pointer
struct packet *recv_packet = (struct packet *)recv_data;
// fix byte order
uint8_t x = ntoh8(recv_packet->x);
uint32_t y = ntoh32(recv_packet->y);
printf("Decoded: x=%"PRIu8" y=%"PRIu32"\n", x, y);
}
int main (int argc, char *argv[])
{
// build packet
struct packet p;
p.x = hton8(17);
p.y = hton32(2924);
// send packet over link....
// on the other end, get some data (recv_data, recv_len) to decode:
uint8_t *recv_data = (uint8_t *)&p;
size_t recv_len = sizeof(p);
// now decode
decode_packet(recv_data, recv_len);
return 0;
}
As far as byte order conversion functions are concerned, your system's htons()/ntohs() and htonl()/ntohl() can be used, for 16- and 32-bit integers, respectively, to convert to/from big-endian. However, I'm not aware of any standard function for 64-bit integers, or to convert to/from little endian. You can use my byte order conversion functions; if you do so, you have to tell it your machine's byte order by defining BADVPN_LITTLE_ENDIAN or BADVPN_BIG_ENDIAN.
As far as signed integers are concerned, the conversion functions can be implemented safely in the same way as the ones I wrote and linked (swapping bytes directly); just change unsigned to signed.
UPDATE: if you want an efficient binary protocol, but don't like fiddling with the bytes, you can try something like Protocol Buffers (C implementation). This allows you to describe the format of your messages in separate files, and generates source code that you use to encode and decode messages of the format you specify. I also implemented something similar myself, but greatly simplified; see my BProto generator and some examples (look in .bproto files, and addr.h for usage example).
Before you send any data over a TCP connection, work out a protocol specification. It doesn't have to be a multiple-page document filled with technical jargon. But it does have to specify who transmits what when and it must specify all messages at the byte level. It should specify how the ends of messages are established, whether there are any timeouts and who imposes them, and so on.
Without a specification, it's easy to ask questions that are simply impossible to answer. If something goes wrong, which end is at fault? With a specification, the end that didn't follow the specification is at fault. (And if both ends follow the specification and it still doesn't work, the specification is at fault.)
Once you have a specification, it's much easier to answer questions about how one end or the other should be designed.
I also strongly recommend not designing a network protocol around the specifics of your hardware. At least, not without a proven performance issue.
It depends on whether you can be sure that your systems on either end of the connection are homogeneous or not. If you are sure, for all time (which most of us cannot be), then you can take some shortcuts - but you must be aware that they are shortcuts.
struct something some;
...
if ((nbytes = write(sockfd, &some, sizeof(some)) != sizeof(some))
...short write or erroneous write...
and the analogous read().
However, if there's any chance that the systems might be different, then you need to establish how the data will be transferred formally. You might well linearize (serialize) the data - possibly fancily with something like ASN.1 or probably more simply with a format that can be reread easily. For that, text is often beneficial - it is easier to debug when you can see what's going wrong. Failing that, you need to define the byte order in which an int is transferred and make sure that the transfer follows that order, and the string probably gets a byte count followed by the appropriate amount of data (consider whether to transfer a terminal null or not), and then some representation of the float. This is more fiddly. It is not all that hard to write serialization and deserialization functions to handle the formatting. The tricky part is designing (deciding on) the protocol.
You could use an union with the structure you want to send and an array:
union SendSomething {
char arr[sizeof(struct something)];
struct something smth;
};
This way you can send and receive just arr. Of course, you have to take care about endianess issues and sizeof(struct something) might vary across machines (but you can easily overcome this with a #pragma pack).
Why would you do this when there are good and fast serialization libraries out there like Message Pack which do all the hard work for you, and as a bonus they provide you with cross-language compatibility of your socket protocol?
Use Message Pack or some other serialization library to do this.
Usually, serialization brings several benefits over e.g. sending the bits of the structure over the wire (with e.g. fwrite).
It happens individually for each non-aggregate atomic data (e.g. int).
It defines precisely the serial data format sent over the wire
So it deals with heterogenous architecture: sending and recieving machines could have different word length and endianness.
It may be less brittle when the type change a little bit. So if one machine has an old version of your code running, it might be able to talk with a machine with a more recent version, e.g. one having a char b[80]; instead of char b[64];
It may deal with more complex data structures -variable-sized vectors, or even hash-tables- with a logical way (for the hash-table, transmit the association, ..)
Very often, the serialization routines are generated. Even 20 years ago, RPCXDR already existed for that purpose, and XDR serialization primitives are still in many libc.
Pragma pack is used for the binary compatibility of you struct on another end.
Because the server or the client to which you send the struct may be written on another language or builded with other c compiler or with other c compiler options.
Serialization, as I understand, is making stream of bytes from you struct. When you write you struct in the socket you make serialiazation.
Google Protocol Buffer offers a nifty solution to this problem. Refer here Google Protobol Buffer - C Implementaion
Create a .proto file based on the structure of your payload and save it as payload.proto
syntax="proto3"
message Payload {
int32 age = 1;
string name = 2;
} .
Compile the .proto file using
protoc --c_out=. payload.proto
This will create the header file payload.pb-c.h and its corresponding payload.pb-c.c in your directory.
Create your server.c file and include the protobuf-c header files
#include<stdio.h>
#include"payload.pb.c.h"
int main()
{
Payload pload = PLOAD__INIT;
pload.name = "Adam";
pload.age = 1300000;
int len = payload__get_packed_size(&pload);
uint8_t buffer[len];
payload__pack(&pload, buffer);
// Now send this buffer to the client via socket.
}
On your receiving side client.c
....
int main()
{
uint8_t buffer[MAX_SIZE]; // load this buffer with the socket data.
size_t buffer_len; // Length of the buffer obtain via read()
Payload *pload = payload_unpack(NULL, buffer_len, buffer);
printf("Age : %d Name : %s", pload->age, pload->name);
}
Make sure you compile your programs with -lprotobuf-c flag
gcc server.c payload.pb-c.c -lprotobuf-c -o server.out
gcc client.c payload.pb-c.c -lprotobuf-c -o client.out
I am currently working on a school project which asks me to implement a DNS client, without using any library functions.
I have got to the point where i send a DNS request and Receive the Reply. I'm getting stuck at the parsing of the reply. I receive the reply in a char* array and i want to convert it into some meaningful structure, from which i can parse the answer.
I went through the RFC and i read about the packet structure, but implementing it in C is giving me problems.
Can anyone give me any examples, in C, or maybe in any other language that explains how this is done. Or any reference to a book is also fine.
Additional Details:
So, the following are the structures that i'm using.
struct result{
int type;
struct res_ip_cname ip_cname;
struct res_error error;
struct res_mx_ns mx_ns;
};
struct res_ip_cname{
char* lst;
int sec;
char* auth_flag;
};
struct res_error{
char * info;
};
struct res_mx_ns{
char * name;
unsigned short pref;
int sec;
char* auth_flag;
};
I have a char* buffer[], where im storing the response the i receive from the server. And, i need to extract information from this buffer and populate the structure result.
Thanks,
Chander
Your structures don't look like anything I recognise from the RFCs (yes, I've written lots of DNS packet decoding software).
Look at RFC 1035 in particular - most of the structures you need can be mapped directly from the field layouts show therein.
For example, you need a header (see s4.1.1):
struct dns_header {
uint16_t query_id;
uint16_t flags;
uint16_t qdcount;
uint16_t ancount;
uint16_t nscount;
uint16_t arcount;
};
Don't forget to use ntohs() to convert the wire format of these fields into your machine's native byte order. The network order is big-endian, and most machines these days are little-endian.
You'll need a "question" structure (see s4.1.2), and a generic "resource record" structure too (see s4.1.3).
Note however that the wire format of both of these starts with a variable length "label", which can also include compression pointers (see s4.1.4). This means that you can't in these cases trivially map the whole wire block onto a C structure.
Hope this helps...
If I were you I'd be using wireshark (in combination with the RFC) to inspect the packet structure. Wireshark captures and displays the network packets flowing through your computer. It lets you see both the raw data you will be receiving, and the decoded packet structure.
For example, in the screenshot below you can see the IP address of chat.meta.stackoverflow.com returned in a DNS Response packet, rendered in three different ways. Firstly, you can see a human readable version, in the middle pane of the screen. Secondly, the highlighted text in the lower left pane shows the raw DNS packet as a series of hexadecimal bytes. Thirdly, in the highlighted text in the lower left pane, you can see the packet rendered as ASCII text (in this case, mostly but not entirely, gobbledigook).
The request format and response format are quite similar - both contain variable length fields, which I guess is what you're stuck on - but if you've managed to form a request properly, you shouldn't have too much trouble parsing the response. If you can post some more details, like where exactly you're stuck, we could help better.
My advice is don't make a meal of it. Extract QDCOUNT and ANCOUNT from the header, then skip over the header, skip QDCOUNT questions, and start parsing answers. Skipping a label is easy (just look for the first byte that's 0 or has the high bit set), but decoding one is a little bit more work (you need to follow and validate "pointers" and make sure you don't get stuck in a loop). If you're only looking up addresses (and not PTR records) then you really never need to decode labels at all.