I have a very large network trace file which contains both tcp and udp packets.I want to find out the flows in the trace file.For that I have a hash function which takes in source ip address,destination ip address,source port,destination port and protocol.In case of TCP I can understand that the flow means all the packets which have the same 5 parameters same.But what does it mean in case of UDP.how does the concept of flow apply in case of UDP.? I am a novice in packet processing.And once I have idendified a flow (tcp and udp) how do I work out the direction of the flow.?Do I have to look at SYN flag ? If yes what do I do for UDP?
Netflow applies to any protocol including TCP and UDP. So to answer your question, yes, UDP packets should be treated the same as TCP.
If you do Netflow processing you might find this spec useful - very detailed and for many versions. I can confirm that it is accurate and works fine with Cisco and Juniper devices (at least version 7)
Related
Is it possible that one application uses the same Ip # and connects to three different applications using different protocols TCP and UDP?
Best regards,
Yes, one application can have several network connections using different ports and protocols all being on the same IP address.
Just open several sockets with their specific options.
The 'tricky' thing is going to be making sure that all sockets are read from and written to. You may need threads.
Port spaces are different for TCP and UDP, as they are different protocols. Nothing (but the range of values, that goes in both from 1 to 65535) makes them interchangeable. But many services that interoperate in both TCP & UDP select the same port number (if available, look in IANA for assigned numbers) to the same service. Anyway, being TCP a connection oriented, reliable protocol, without frame delimitation and UDP unreliable, packet delimited protocol, it is no much sense to use udp for some protocol if it is only specified for TCP media, or the reverse.
Both TCP & UDP can listen for packets in any (or all) of the interfaces of the host, so in order to allow several different UDP/TCP services to run in each interface, the protocols were specified with a port multiplexing/demultiplexing feature (the port numbers) This is what makes the kernel to decide to which socket to deliver the data comming from the network (demultiplexing) and what allows all packets (multiplexing) to share the same medium (the network wire)
I have a linux computer with a code in C that must communicate in UDP with 4 differents equipments. The computer sends differents commands to each equipment and receives responses, sometimes in parallel ...
I am a perfect beginner, and managed to communicate with one equipment using UDP socket. But now, i'm looking for a way to communicate with all these equipments, what i would like to call "multiple socket", but i don't know where to look/ what word to search to find a way ...
My linux computer is the client and all the equipment servers. I only have one eth port on the computer and will have to use a switch to have access to all the equipment. I would like to create functions like :
sendcmd(IPnumber, PORTnumber, cmd , ...)
readbuff(IPnumber, PORTnumber, buff, ...)
so i can choose which IP will received cmd ... i don't know if it's possible or if i need to open the socket, then close and redo the operation with another IP ...
So, if I ever managed to make myself understood, where should I look for a solution to my problem?
Thank you !
You can use a single UDP socket for your scenario. You can keep the socket open for the lifetime of your application.
UDP is not connection oriented. UDP sockets are also not classified into client sockets and server sockets. UDP sockets are always bound to a local port, either implicitly (typically for pure clients) or explicitly (which is usually the case for servers). In your case you do not care about the port for your UDP client.
To send to your four UDP server you can use sendto(). This lets you specify the destination IP address and port the UDP packet gets sent to.
To receive from your four UDP servers you can use recvfrom(). This will tell the IP address and port where the UDP packet came from.
You most likely want to have a receive loop of some kind. If you want to do anything else in your application you most likely want to either make recvfrom() non-blocking or you want to have the receive loop in its own thread. But this goes beyond your question.
The most important aspect of UDP is that it is not a protocol (despite its name which is misleading). It is one puzzle piece for a protocol. It is a tool to develop your own protocol. But I assume you already have a specific protocol at hand defined by your peripherals.
When using a PF_PACKET type of socket with protocol type ETH_P_IP, the man packet documentation talks about a socket option for multicast. The socket option is PACKET_ADD_MEMBERSHIP.
Assuming you use PACKET_ADD_MEMBERSHIP socket option on a PF_PACKET socket correctly, what features and benefits and use cases is this socket option for?
Right now I receive all incoming IP packets so I look at each packet to see if it has the correct IP dst-address and UDP dst-port and I skip over all the other packets. Would using PACKET_ADD_MEMBERSHIP socket option mean I don't need to do my own filter because the kernel or driver would filter for me?
I dug into the linux-kernel source and traced down the code a little bit. I found that the ethernet-mac-address you pass in via setsockopt() is added to a list of ethernet-mac-addresses. And then the list is sent to the network-device hardware to do something... but I can't find any authoritative documentation telling me what happens next.
My educated guess is that the ethernet-mac-address list is used by the hardware to filter at the layer-2 ethernet protocol (i.e. the hardware only accepts packets that have a destination ethernet-address that matches one on the list). If there is some good documentation I would welcome that.
(I'm more familiar with TCP/UDP sockets and so this looks very similar to AF_INET type of socket's IP_ADD_MEMBERSHIP socket option... so I was expecting IGMP reports to be generated which would start multicast traffic from the router... but I found out experimentally that no IGMP reports are generated when you use this socket option.)
Your guess is correct. PACKET_ADD_MEMBERSHIP should add addresses to the NIC's hardware filter. As you've surmised, it's intended to allow you to receive multicasts for a number of different addresses without incurring the load(*) of full promiscuous mode.
(* With modern full duplex ethernet, there's generally not a lot of traffic coming to the NIC that it wouldn't want to receive anyway, unless it's in a virtualized environment.)
Note that there is also a separate PACKET_MR_UNICAST which does not appear in the packet(7) man page but works analogously. I would use the appropriate one (unicast vs multicast) for the type of address you're filtering on, as it's conceivable (though unlikely) that a driver would refuse to put a unicast address into the multicast filtering table.
All that being said, you'll still need to keep your software filtering as backup. There are some older drivers that don't implement MAC filtering at all (particularly for multiple unicast addresses). The core kernel or the driver handles this by simply turning on promiscuous mode if the feature isn't available.
As for the relationship with IP_ADD_MEMBERSHIP, the IP_ADD_MEMBERSHIP code will automatically construct the appropriate multicast MAC address and add it to the interface. See ip_mc_filter_add.
I was looking the most straightforward tutorial on making a tiny network sniffer and found this one. I followed it, but the method advised to sniff packets is:
sock_raw = socket( AF_PACKET , SOCK_RAW , htons(ETH_P_ALL)) ;;
while(1)
{
data_size = recvfrom(sock_raw , buffer , 65536 , 0 , &saddr , &saddr_size);
}
With the rational:
A raw socket when put in recvfrom loop receives all incoming packets.
This is because it is not bound to a particular address or port.
It seemed to me that this would only monitor network traffic coming in and out of my own computer, and not the whole LAN. A test-run confirmed my intuition.
Is it correct? Can this method only sniff packets coming in and out of my laptop?
Which approach should I take to sniff all network traffic (ie: netsniff-ng, Wireshark)?
I want to avoid using libpcap in this case.
That depends on your network, especially the types and configuration of the routers and switches that connect everything. In most installations, computers are not connected directly to each other. Instead, they talk to a switch. The switch will only send you the packets which are meant for your computer - the rationale is that it doesn't make sense for the switch to elaborately push electrons towards you which you'll only throw away.
But a computer can tell a switch to go into "promiscuous mode". That will configure your port of the switch to send you a copy of every packet that the switch sees. There are two catches:
The switch still can't see packets which other switches/routers don't send to it. Promiscuous mode doesn't propagate - otherwise, you'd quickly get a copy of every packet sent over the Internet anywhere. Not even Google or the NSA could handle this kind of traffic.
If your switch isn't the plug&play kind, your system admin will have disabled this feature.
[EDIT] Wireshark uses libpcap which is a low-level library which does everything from configuring your network device to fetching and filtering the packets that it can see. So you should try to find the source code of this library to understand better how it is done.
Related:
FedEx Bandwidth: When - if ever - will the bandwidth of the Internet surpass that of FedEx?
I am currently working on a programming assignment. The assignment is to implement a client,network emulator, and server. The client passes packets to a network emulator, and the network emulator passes to the server. Vice-versa applies as well. The prerequisite for the assignment is that I may only use raw sockets. So I will create my own IP and UDP headers. I have tested my packets with wireshark. They are all correct and in the proper format(it reads them properly).
Another requirement is that the emulator, client and server all have specific ports they must be bound to. Now, I do not understand how to bind a raw socket to a specific port. All my raw sockets receive all traffic on the host address they are bound to. According to man pages, and everywhere else on the internet, including "Unix Network Programming" by Richard Stevens, this is how they are supposed to work. My teacher has not responded to any of my emails and I probably will not be able to ask him until Tuesday.I see two options in front of me. First I can use libpcap to filter from a specific device and then output to my raw socket. I feel this is way out of scope for our assignment though. Or I can filter them after I receive them from the socket. This apparently has a lot of overhead because all the packets are being copied/moved through the kernel. At least, that is my understanding(please feel free to correct me if i'm wrong).
So my question is:
Is their an option or something I can set for this? Where the raw socket will bind to a port? Have I missed something obvious?
Thank you for your time.
--
The man page for raw(7) says:
A raw socket can be bound to a specific local address using the bind(2) call. If it isn't bound all packets with the specified IP protocol are received. In addition a RAW socket can be bound to a specific network device using SO_BINDTODEVICE; see socket(7).
Edit: You cannot bind a raw socket to a specific port because "port" is a concept in TCP and UDP, not IP. Look at the header diagrams for those three protocols and it should become obvious: you are working at a lower level, where the concept of port is not known.
I would think you're expected to filter the packets in your software. It sounds like the exercise is to learn what the different components of the IP stack do by recreating a simplified piece of it in user space. Normally in the kernel, the IP code would process all packets, verify the IP headers, reassemble fragments, and check the protocol field. If the protocol field is 17 (udp), then it passes it to the UDP code (every UDP packet). It's up to the UDP code to then validate the UDP header and determine if any applications are interested in them based on the destination port.
I imagine your project is expected to more or less mimic this process. Obviously none of it will be as efficient as doing it in the kernel, but since the assignment is to write (part of) an IP stack in user-space, I'd guess efficiency isn't the point of the exercise.