I am writing an application that needs to get access to TCP header fields, for example, a sequence number or a TCP timestamp field.
Is it possible to get sequence numbers (or other header fields) by operating at the socket API without listening on a raw socket? (I want to avoid filtering out all the packets).
I am looking at the TCP_INFO but it has a limited information.
For example, after calling a recvmsg() and getting a data buffer, is it possible to know the sequence number of the segment that delivered the last byte in that received data buffer?
Thanks
You can try to use libpcap to capture packets. This lib allows to specify packet filter using the same syntax as in Wireshark, so you could limit captured packets to one connection only. One downside is that you would have to receive packets in normal way too, what complicated things a bit and is an additional performance overhead.
Update: you can also open raw socket and set Berkeley Packet Filter on it using socket option SO_ATTACH_FILTER. More details are here: https://www.kernel.org/doc/Documentation/networking/filter.txt . However you would have to implement TCP part of IP stack in your code too.
Related
Correct me if I'm wrong, but my understanding of sending a raw packet inevitably is defined as buffering an array of bytes in an array, and writing it to a socket. However, most example code I've seen so far tend towards sendto, rarely is send used, and I've never seen code other than my own use write. Am I missing something? What is with this apparent preoccupation with complicating code like this?
Why use send and sendto when write seems to me to be the obvious choice when dealing with raw sockets?
sendto is typically used with unconnected UDP sockets or raw sockets. It takes a parameter specifying the destination address/port of the packet. send and write don't have this parameter, so there's no way to tell the data where to go.
send is used with TCP sockets and connected UDP sockets. Since a connection has been established, a destination does not need to be specified, and in fact this function doesn't have a parameter for one.
While the write function can be used in places where send can be used, it lacks the flags parameter which can enable certain behaviors on TCP sockets. It also doesn't return the same set of error codes as send, so if things go wrong you might not get a meaningful error code. In theory you could also use write on a raw socket if the IP_HDRINCL socket option is set, but again it's not preferable since it doesn't support the same error codes as send.
Good day.
Intro.
Recently I've started to study some 'low-level' network programming as well as networking protocols in Linux. For this purpose I decided to create a small library for networking.
And now I wonder on some questions. I will ask one of them now.
As you know there are at least two protocols built on top of IP. I talk about TCP and UDP. Their implementation may differ in OS due to connection-orientation property of those.
According to man 7 udp all receive operations on UDP socket return only one packet. It is rational as different datagrams may come from different sources.
On the other hand TCP connection packets sequence may be considered as continuous byte flow.
Now, about the problem itself.
Say, I have an API for TCP connection socket and for UDP socket like:
void tcp_connection_recv(endpoint_t *ep, buffer_t *b);
void udp_recv(endpoint_t *ep, buffer_t *b);
endpoint_t type will describe the endpoint (remote for TCP connection and local for UDP). buffer_t type will describe some kind of vector-based or array-based buffer.
It is quite possible that buffer is already allocated by user and I'm not sure that this will be right for UDP to not change size of the buffer. And thus, to abstract code for TCP and UDP operations I think it will need to allocate as much buffer as needed to contain whole received data.
Also, to prevent from resizeing user buffer each socket may be maped to its own buffer (although it will be userspace buffer, but it will be hidden from user). And then on user's request data will be copied from that 'inner' buffer to user's one or read from socket if there is not enough amount.
Any suggestions or opinions?
If you want to create such API, it will depend on the service you want to provide. In TCP it will be different than UDP as TCP is stream oriented.
For TCP, tcp_connection_recv instead of reallocating a buffer, if the buffer passed by the user is not big enough, you can fill the whole buffer and then return, maybe with an output parameter, and indication that there is more data waiting to be read. Basically you can use the receive buffer that TCP connection already provides in the kernel, no need to create other buffer.
For, udp, you can request the user a number indicating the maximum datagram size it is waiting for. When you read from a UDP socket with recvfrom, if you read less data than what came in the arrived datagram, the rest of the datagram data is lost. You can read first with MSG_PEEK flag in order to find out how much data is available.
In general I wouldn't handle the buffer for the application as the application, actually the application layer protocol, is the one that knows how it expects to receive the data.
We want to receive packets from udp sockets, the udp packets have variable length and we don't know how long they really are until we receive them (parts of them exactly, length were written in the sixth byte).
We tried the function lrs_set_receive_option with MarkerEnd only to find it has no help on this issue. The reason why we want to receive by packets is that we need to respond some packet by sending back user-defined udp packets.
Is there anybody knows how achieve that?
UPDATE
The LR version seems to be v10 or v11.
We need respond an incoming udp packet by sending back a udp packet immediately.
The udp packet may be like this
| orc code | packet length | Real DATA |
Issue is we can't let loadrunner return data for each packets, sometimes it returns many packets in a buffer, sometimes it waits until timeout though when there has been an incoming packet in the socket buffer. While in the c programming language world, when calling recvfrom(udp socket) we are returned only one udp packet per time (per call) which is want we really want.
If you need raw socket support to intercept at the packet level then you are likely going to have to jump to a DLL virtual user in Visual Studio with the raw socket support.
As to your question on UDP support: Yes, a Winsock user supports both core transport types, UDP and TCP. TCP being the more common variant as connection oriented. However, packet examination is at layer 3 of the OSI model for the carrier protocol IP. The ACK should come before you receive the dataflow for your use in the script. You are looking at assembled data flows in the data.ws when you jump to the TCP and UDP level.
Now, you are likely receiving a warning on receive buffer size mismatch which is taking you down this path with a mismatch to the recording size. There is an easy way to address this. If you take your send buffer and construct it using the lrs_set_send_buffer() function, then anything that returns will be taken as correct, ignoring the previously recorded buffer size and not having to wait for a match or timeout before continuing.
I want to make a chat room using raw socket in C. I have following problems:
Q 1 : Can I use select function to handle multiple connections in case of raw sockets ?
Q 2 : Port nos in sockets are real ports or logically implemented for various applications on transport layer??
Q 3 : I am having one computer only so using lo ( local loop) as my interface. So the process which is initiating the chat has send first and then receive call, so it's receiving it's own data. How to restrict it?
Any help would be appreciated since that would help me in increasing my confidence on raw sockets.
Thanks :)
If you want this to be a real, usable chat system, stop. Don't use raw sockets. Huge mistake.
If you are just playing around because you want to put “raw sockets” under the “Experience” section of your résumé, you may read on.
You can use the select function to detect when a raw socket has a packet available to receive, and when it can accept a packet to transmit. You can pass multiple file descriptors to a single call to select if you want to check multiple raw sockets (or whatever) simultaneously.
Port numbers are part of the TCP and UDP protocols (and some other transport layer protocols). The kernel doesn't look for port numbers when receiving packets for raw sockets.
The raw(7) man page‚ states:
All packets or errors matching the protocol number specified for the raw socket are passed to this socket.
And it also states:
A raw socket can be bound to a specific local address using the bind(2) call. If it isn't bound, all packets with the specified IP protocol are received.
Therefore you probably want to at least use different IP addresses for each end of the “connection”, and bind each end to its address.
“But!” you say, “I'm using loopback! I can only use the 127.0.0.1 address!” Not so, my friend. The entire 127.0.0.0/8 address block is reserved for loopback addresses; 127.0.0.1 is merely the most commonly-used loopback address. Linux (but perhaps not other systems) responds to every address in the loopback block. Try this in one window:
nc -v -l 10150
And then in another window:
nc -s 127.0.0.1 127.0.0.2 10150
You will see that you have created a TCP connection from 127.0.0.1 to 127.0.0.2. I think you can also bind your raw sockets to separate addresses. Then, when you receive a packet, you can check whether it's from the other end's IP address to decide whether to process or discard it.
Just curious, why do you want to use raw sockets? Raw sockets (AF_INET, SOCK_RAW) allow you to send out "raw" packets, where you are responsible for crafting everything but the MAC and IP layers.
A1: There are no "connections" with raw sockets. Just packets.
A2: There are no "ports" with raw sockets. Just packets. "Port numbers" as we know them are part of the TCP or UDP protocols, both of which are above the level at which we work with raw sockets.
A3: This is not specific to raw sockets - you would have this issue regardless of your protocol selection. To really answer this, we would need to know much more about your proposed protocol, since right now, you're simply blasting out raw IP packets.
I'm sending many TCP packets each of size 50 bytes over the network. Later I found it out TCP aggregates a few 50 byte packets into a single TCP packet. My question is, is there a way to avoid TCP aggregation in 'C' program?
Packing multiple sent packets into a single TCP packet is handled using an algorithm known as Nagle's algorithm. To disable it, set the TCP_NODELAY option on your socket:
int flag = 1;
setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &flag, sizeof(flag));
Note that this decreases the efficiency of your network, and should be avoided unless you really do need each packet to be sent immediately.
TCP fundamentally does not provide a packet-based service at the application layer. Applications using TCP that need more than a pure stream must provide their own framing mechanism.
Whilst the TCP_NODELAY option will often cause each application-level write to be carried in a separate frame (unless it's larger than the MTU), there's no guarantee. In particular, if the recieve window has been reduced to zero (common on a connection where you are sending bulk data), then multiple writes can still be coalesced when the receive window opens up again.
Furthermore, even if the sending TCP does not coalesce your writes, there is no way to stop the receiving side from presenting the data from multiple TCP frames in one application-level read from the socket - this will particularly occur when your receiving application is not scheduled for some time, or when packet loss occurs on the connection.
If your application assumes that each application-level write on the sending end will be paired exactly with an application-level read on the receiving end, you are setting yourself up for rare and hard-to-reproduce bugs.
TCP does not, by definition, offer any packetization services. TCP is a byte-stream oriented protocol. The best you can do is to insert your own framing marks into your data. Then if your receiver can at least detect bad frames (aggregated data perhaps) and drop such data, that sort of gets you where you want to be.
The send-as-soon-as-possible approach (ala NODELAY) is not guaranteed to work because your receiver can still aggregate the data. This is more probable if any tcp data gets delayed for whatever reason
One way is to use the TCP_NODELAY flags in the send call. However it depends on the TCP stack if this is supported or not.