First a little background: I have an idea for a chat application. I don't want the user to need to put in connection details, so I was thinking I can send a custom ethernet packet type. That packet is received by peers; generating a database of every peer within the user's application. I don't want to overcomplicate generating my raw packets. What is the minimum amount of data required in order to send a successfully received packet? Do any special circumstances exist where a minimal packet will transmit properly with some hardware, and not other hardware?
Let me clarify, and be explicit: I am asking what is the minimum amount of information required in the packet for that packet to be transmitted, and received. An example of information that can be found in an ethernet packet is an IP address, a hardware address, time-to-live, et cetera.
What is the mimimum amount of data required in order to send a
successfully received packet?
You don't need to use IP at all if you aren't going to send this over the internet.
On your LAN you could get by with a custom network layer if you like.
There are also existing nonip protocols (IPX) which may be suiting your needs. IPX is relatively simple to set up but people will be mad at you if they ever catch you using it.
(I also have no idea if windows supports IPX since winxp).
That would be the hard way.
The easier way would be to use UDP/IP. UDP is designed to be minimal.
The wiki page for UDP will tell you the header size of a UDP datagram.
Don't forget to add the IP header (wiki again) and ethernet frame (ibid.) to this.
Finally you need a payload.
There are many many options. It depends largely on the network infrastructure you intend to use. If your infrastructure is IP over Ethernet then you'll be stuck with that!
Do any special circumstances exist where a minimal packet will
transmit properly with some hardware, and not other hardware?
If you adhere to the standards: No. If you flaunt the standards: Very probably.
In your latter question you seem to be hinting you would like to employ 'clever' tricks to minimize packet size (ie, missing out non-critical parts of the header(s)), but I repeat: DO NOT GO AGAINST THE STANDARDS.
People will be very very dissapointed with you if the catch you doing this and you will lose all your friends.
Related
I'm a beginner to C, I've recently decided to make migrate this project to C from using Scapy/Python, solely because I want better performance. I wish to send layer 2 data, specifically beacon frames to advertise an access point.
So far I have found that I need to (or rather could) use libpcap and a Linux header called ieee80211.h that pre-defines packets, that's all I could gather from the other questions. I've found other information which says I should use raw sockets instead of libpcap? I'm not sure if this is all I need. Most of the information and tutorials I have found on Google refer to packet sniffing, not sending.
How do I define a custom frame and/or packet (e.g. a beacon frame or association request) and then simply send it to wlan0 etc.?
Thought I might update this. I used libpcap.
You just need to create a handle with your device, set it to monitor mode successfully (important), check the data link type (e.g. 802.11 with Radiotap for layer 2) then use pcap_sendpacket(handle, packetArrayContainingHex, size);. Hard part is forming legal packets that aren't rejected or dropped, taking a look in wireshark helps.
This link might help. It basically opens the driver at raw packet level and creates the whole packet to send over the wire, as your question suggests.
Is there a way to view all the IPv4 packets sent to a Linux computer?
I know I can capture the packets at the ethernet level using libpcap. This can work, but I don't really want to defragment the IPv4 packets. Does libpcap provide this functionality and I'm just missing it?
One thing that kinda works is using a tun device. I can capture all the IPv4 traffic by routing all traffic to the tun device via something like ip route add default via $TUN_IP dev $TUNID. This also stops outbound traffic though, which is not what I want.
I just want to see the IPv4 packets, not intercept them. (Or, even better, optionally intercept them.)
Edit: I'm specifically looking for a programmatic interface to do this. E.g. something I can use from within a C program.
Yes, you can see all the packets that arrive at your network interface. There are several options to access or view them. Here a small list of possible solutions, where the first one is the easiest and the last one the hardest to utilize:
Wireshark
I'd say this is pretty much the standard when it comes to protocol analyzers with a GUI (uses libpcap). It has tons of options, a nice GUI, great filtering capabilities and reassembles IP datagrams. It uses libpcap and can also show the raw ethernet frame data. For example it allows you to see layer 2 packets like ARP. Furthermore you can capture the complete data arriving at your network interface in a file that can later be analyzed (also in Wireshark).
tcpdump
Very powerful, similar features like Wireshark but a command line utility, which also uses libpcap. Can also capture/dump the complete interface traffic to a file. You can view the dumped data in Wireshark since the format is compatible.
ngrep
This is known as the "network grep" and is similar to tcpdump but supports regular expressions (regex) to filter the payload data. It allows to save captured data in the file format supported by Wireshark and tcpdump (also uses libpcap).
libnids
Quotation from the official git repository:
"Libnids is a library that provides a functionality of one of NIDS
(Network Intrusion Detection System) components, namely E-component. It means
that libnids code watches all local network traffic [...] and provides convenient information on them to
analyzing modules of NIDS. Libnids performs:
assembly of TCP segments into TCP streams
IP defragmentation
TCP port scan detection"
libpcap
Of course you can also write your own programs by using the library directly. Needless to say, this requires more efforts.
Raw or Packet Sockets
In case you want to do all the dirty work yourself, this is the low level option, which of course also allows you to do everything you want. The tools listed above use them as a common basis. Raw sockets operate on OSI layer 3 and packet sockets on layer 2.
Note: This is not meant to be a complete list of available tools or options. I'm sure there are much more but these are the most common ones I can think of.
Technically you have to make a copy of the received packet via libpcap. To be more specific, what you can do is to get packets with libpcap, that way the packets will be kind of blocked, so you need to re send them to the destination. Lets say that you want to make a Fire-Wall or something, what you should do is to have a layer that can work like getting the package and then send it to the destination, in between you can make a copy of what you got for further processes. In order to make the intercept option, you need to create some predefined rules, i.e. the ones that violates the rules will not be send again to their destination.
But that needs a lot of efforts and I don't think you want to waist your life on it.
Wire-shark as mentioned by #Barmar can do the job already.
If you need some kind of command line interface option I would say that "tcpdump" is one of the best monitoring tools. for example for capturing all ipv4 HTTP packets to and from port 80 the command will be:
tcpdump 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
for more information and options see tcpdump
Please be specific if you need to write a program for it, then we can help about how to do it.
I had two send()s in my C program and looking at wireshark, I realized they were sent out as one TCP/IP message. I am assuming this is some sort of TCP/IP optimization that determined they were small enough that they could be sent out together. However, I am rebuilding an old program from scratch and I am building my tool based on it's TCP/IP traffic: MTU limitations, internal protocol design, etc. So if the old tool sends out two separate messages, I need to send out two separate messages.
So does anyone know what specifically it is doing in the background(besides simple optimization) and if there is a flag or something that needs to be enabled/disabled so that I get a 1 to 1 ratio of C send()s and TCP/IP transmission? For now all I can do to keep them separated is to put a sleep(1) after every send().
Thanks.
You can set TCP_NODELAY in setsockopt to disable Nagle's algorithm, to prevent your OS from combining small packets. However, it's important for you to realize that TCP is a stream-oriented protocol, and individual "packets" are not intended to be meaningfully separated. Routers along the way are free to combine or split TCP packets (though this is uncommon, due to the extra processing required), and the receiving OS will not necessarily read exactly one sent packet per recv(). If you want to delineate packets of information in TCP, you'll need to use a header structure to report how many of the following bytes belong to that packet.
I'm trying to make a custom packet using C using the TCP/IP protocol. When I say custom, I mean being able to change any value from the packet; ex: MAC, IP address and so on.
I tried searching around but I can't find anything that is actually guiding me or giving me example source codes.
How can I create a custom packet or where should I look for guidance?
A relatively easy tool to do this that is portable is libpcap. It's better known for receiving raw packets (and indeed it's better you play with that first as you can compare received packets with your hand crafted ones) but the little known pcap_sendpacket will actually send a raw packet.
If you want to do it from scratch yourself, open a socket with AF_PACKET and SOCK_RAW (that's for Linux, other OS's may vary) - for example see http://austinmarton.wordpress.com/2011/09/14/sending-raw-ethernet-packets-from-a-specific-interface-in-c-on-linux/ and the full code at https://gist.github.com/austinmarton/1922600 . Note you need to be root (or more accurately have the appropriate capability) to do this.
Also note that if you are trying to send raw tcp/udp packets, one problem you will have is disabling the network stack automatically processing the reply (either by treating it as addressed to an existing IP address or attempting to forward it).
Doing this sort of this is not as simple as you think. Controlling the data above the IP layer is relatively easy using normal socket APIs, but controlling data below is a bit more involved. Most operating systems make changing lower-level protocol information difficult since the kernel itself manages network connections and doesn't want you messing things up. Beyond that, there are other platform differences, network controls, etc that can play havoc on you.
You should look into some of the libraries that are out there to do this. Some examples:
libnet - http://libnet.sourceforge.net/
libdnet - http://libdnet.sourceforge.net/
If your goal is to spoof packets, you should read up on network-based spoofing mitigation techniques too (for example egress filtering to prevent spoofed packets from exiting a network).
im working on a project with two clients ,one for sending, and the other one for receiving udp datagrams, between 2 machines wired directly to each other.
each datagram is 1024byte in size, and it is sent using winsock(blocking).
they are both running on a very fast machines(separate). with 16gb ram and 8 cpu's, with raid 0 drives.
im looking for tips to maximize my throughput , tips should be at winsock level, but if u have some other tips, it would be great also.
currently im getting 250-400mbit transfer speed. im looking for more.
thanks.
Since I don't know what else besides sending and receiving that your applications do it's difficult to know what else might be limiting it, but here's a few things to try. I'm assuming that you're using IPv4, and I'm not a Windows programmer.
Maximize the packet size that you are sending when you are using a reliable connection. For 100 mbs Ethernet the maximum packet is 1518, Ethernet uses 18 of that, IPv4 uses 20-64 (usually 20, thought), and UDP uses 8 bytes. That means that typically you should be able to send 1472 bytes of UDP payload per packet.
If you are using gigabit Ethernet equiptment that supports it your packet size increases to 9000 bytes (jumbo frames), so sending something closer to that size should speed things up.
If you are sending any acknowledgments from your listener to your sender then try to make sure that they are sent rarely and can acknowledge more than just one packet at a time. Try to keep the listener from having to say much, and try to keep the sender from having to wait on the listener for permission to keep sending.
On the computer that the sender application lives on consider setting up a static ARP entry for the computer that the receiver lives on. Without this every few seconds there may be a pause while a new ARP request is made to make sure that the ARP cache is up to date. Some ARP implementations may do this request well before the ARP entry expires, which would decrease the impact, but some do not.
Turn off as many users of the network as possible. If you are using an Ethernet switch then you should concentrate on the things that will introduce traffic to/from the computers/network devices on which your applications are running reside/use (this includes broadcast messages, like many ARP requests). If it's a hub then you may want to quiet down the entire network. Windows tends to send out a constant stream of junk to networks which in many cases isn't useful.
There may be limits set on how much of the network bandwidth that one application or user can have. Or there may be limits on how much network bandwidth the OS will let it self use. These can probably be changed in the registry if they exist.
It is not uncommon for network interface chips to not actually support the maximum bandwidth of the network all the time. There are chips which may miss packets because they are busy handling a previous packet as well as some which just can't send packets as close together as Ethernet specifications would allow. Additionally the rest of the system might not be able to keep up even if it is.
Some things to look at:
Connected UDP sockets (some info) shortcut several operations in the kernel, so are faster (see Stevens UnP book for details).
Socket send and receive buffers - play with SO_SNDBUF and SO_RCVBUF socket options to balance out spikes and packet drop
See if you can bump up link MTU and use jumbo frames.
use 1Gbps network and upgrade your network hardware...
Test the packet limit of your hardware with an already proven piece of code such as iperf:
http://www.noc.ucf.edu/Tools/Iperf/
I'm linking a Windows build, it might be a good idea to boot off a Linux LiveCD and try a Linux build for comparison of IP stacks.
More likely your NIC isn't performing well, try an Intel Gigabit Server Adapter:
http://www.intel.com/network/connectivity/products/server_adapters.htm
For TCP connections it has been shown that using multiple parallel connections will better utilize the data connection. I'm not sure if that applies to UDP, but it might help with some of the latency issues of packet processing.
So you might want to try multiple threads of blocking calls.
As well as Nikolai's suggestion of send and recv buffers, if you can, switch to overlapped I/O and have many recvs pending, this also helps to minimise the number of datagrams that are dropped by the stack due to lack of buffer space.
If you're looking for reliable data transfer, consider UDT.