So, for a CS project I'm supposed to sniff a network stream and build files from that stream. For example, if the program is pointed to ~/dumps/tmp/ then the directory structure would be this:
~/dumps/tmp
/192.168.0.1/
page1.html
page2.html
[various resources for pages1 & 2]
downloaded file1
/192.168.0.2/
so on and so forth.
I'm doing this in C & pcap on linux (since I already know C++, and figure the learning experience would be good).
Thus far, I've been looking at various header formats for TCP/IP
TCP header
As I figure, I can sort the packets by their dst/src and then order them correctly by sequence and acknowledgement windows.
But that leaves me with a big ? as to how do I figure out how packets a-z are part of an html file and A-Z part of some random file being downloaded etc?
Also, what other kind of header formats should I be looking up? Currently, I have:
I'd post more hyperlink pictures, but I apparently need reputation to do that, sorry
TCP, Ethernet, UDP, and I'll get around to things like FTP (but I'm pretty sure FTP is built on top of TCP, as is HTTP)
So, in short, how do I find files in a network stream, and am I missing any major protocols that I'll need to be able to read?
REPLY
I can't figure out how to reply, so this will have to do.
I have used pcap on several occasions, and will do so again for this project, but I won't use any of Wiresharks stuff (although it is a great program) because I want to no kidding learn this kind of stuff.
Yeah, I'll look into the OSI layer, any suggestions on a good site that covers common protocols?
And I guess I should stop, before this 'question' becomes a discussion.
Where a file begins and ends is not in TCP. You have to deal with the protocol carried over TCP. For example, for HTTP, you have to read the Content-Length header in the HTTP header, which should be equal to the length of the HTTP body (the full html page). Then you accumulate the body over 1 or more TCP packets until you have the total content, as indicated by the Content-Length header.
Since this is a school assignment, you may be limited as to what tools you can use, but you might want to look into Wireshark. If I were given this task as a real-world project, I'd take Wireshark and look into how to use its stream extraction and protocol parsing capabilities and just wrap something around them to automate them and get the desired result.
You need to open a raw socket over a promiscuous Ethernet device. Then use libpcap to store and analyze the packets.
As this is for CS school, I would start with the OSI Model which gives you a good overview and logical structure of network protocols.
Files are on level 6 (MIME) and 7 (various).
Then you need to go through each protocol and check how to determine which contain files and how you can capture them.
Related
I'm a beginner to C, I've recently decided to make migrate this project to C from using Scapy/Python, solely because I want better performance. I wish to send layer 2 data, specifically beacon frames to advertise an access point.
So far I have found that I need to (or rather could) use libpcap and a Linux header called ieee80211.h that pre-defines packets, that's all I could gather from the other questions. I've found other information which says I should use raw sockets instead of libpcap? I'm not sure if this is all I need. Most of the information and tutorials I have found on Google refer to packet sniffing, not sending.
How do I define a custom frame and/or packet (e.g. a beacon frame or association request) and then simply send it to wlan0 etc.?
Thought I might update this. I used libpcap.
You just need to create a handle with your device, set it to monitor mode successfully (important), check the data link type (e.g. 802.11 with Radiotap for layer 2) then use pcap_sendpacket(handle, packetArrayContainingHex, size);. Hard part is forming legal packets that aren't rejected or dropped, taking a look in wireshark helps.
This link might help. It basically opens the driver at raw packet level and creates the whole packet to send over the wire, as your question suggests.
Is there a way to view all the IPv4 packets sent to a Linux computer?
I know I can capture the packets at the ethernet level using libpcap. This can work, but I don't really want to defragment the IPv4 packets. Does libpcap provide this functionality and I'm just missing it?
One thing that kinda works is using a tun device. I can capture all the IPv4 traffic by routing all traffic to the tun device via something like ip route add default via $TUN_IP dev $TUNID. This also stops outbound traffic though, which is not what I want.
I just want to see the IPv4 packets, not intercept them. (Or, even better, optionally intercept them.)
Edit: I'm specifically looking for a programmatic interface to do this. E.g. something I can use from within a C program.
Yes, you can see all the packets that arrive at your network interface. There are several options to access or view them. Here a small list of possible solutions, where the first one is the easiest and the last one the hardest to utilize:
Wireshark
I'd say this is pretty much the standard when it comes to protocol analyzers with a GUI (uses libpcap). It has tons of options, a nice GUI, great filtering capabilities and reassembles IP datagrams. It uses libpcap and can also show the raw ethernet frame data. For example it allows you to see layer 2 packets like ARP. Furthermore you can capture the complete data arriving at your network interface in a file that can later be analyzed (also in Wireshark).
tcpdump
Very powerful, similar features like Wireshark but a command line utility, which also uses libpcap. Can also capture/dump the complete interface traffic to a file. You can view the dumped data in Wireshark since the format is compatible.
ngrep
This is known as the "network grep" and is similar to tcpdump but supports regular expressions (regex) to filter the payload data. It allows to save captured data in the file format supported by Wireshark and tcpdump (also uses libpcap).
libnids
Quotation from the official git repository:
"Libnids is a library that provides a functionality of one of NIDS
(Network Intrusion Detection System) components, namely E-component. It means
that libnids code watches all local network traffic [...] and provides convenient information on them to
analyzing modules of NIDS. Libnids performs:
assembly of TCP segments into TCP streams
IP defragmentation
TCP port scan detection"
libpcap
Of course you can also write your own programs by using the library directly. Needless to say, this requires more efforts.
Raw or Packet Sockets
In case you want to do all the dirty work yourself, this is the low level option, which of course also allows you to do everything you want. The tools listed above use them as a common basis. Raw sockets operate on OSI layer 3 and packet sockets on layer 2.
Note: This is not meant to be a complete list of available tools or options. I'm sure there are much more but these are the most common ones I can think of.
Technically you have to make a copy of the received packet via libpcap. To be more specific, what you can do is to get packets with libpcap, that way the packets will be kind of blocked, so you need to re send them to the destination. Lets say that you want to make a Fire-Wall or something, what you should do is to have a layer that can work like getting the package and then send it to the destination, in between you can make a copy of what you got for further processes. In order to make the intercept option, you need to create some predefined rules, i.e. the ones that violates the rules will not be send again to their destination.
But that needs a lot of efforts and I don't think you want to waist your life on it.
Wire-shark as mentioned by #Barmar can do the job already.
If you need some kind of command line interface option I would say that "tcpdump" is one of the best monitoring tools. for example for capturing all ipv4 HTTP packets to and from port 80 the command will be:
tcpdump 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
for more information and options see tcpdump
Please be specific if you need to write a program for it, then we can help about how to do it.
I am asked for an assignment that requires implementation of FTP protocol. I have gone through the documentation given at RFC959.
I am confused with a couple of implementation details
1)If a file needs to be transferred, what function can be used. can a simple send() function be used for a non text file.
2) Is it possible to get a good tutorial that speaks about implementing Modes and file structures, and to specify, which are essential.
hope to get a reply soon.
FTP transfers file through a plain TCP connection, and you can transfer any kind of file with it. There is no difference between text files and binary files, they are all just sequence of bytes.
For the file transmission is sufficient to open a connection and call the write function many times until the entire file is transmitted (check the return value of write to know how many bytes it sent).
The rest of the FTP protocol is text based and is sent to a different port.
There is a good tutorial on using FTP directly through netcat, that can be useful to understand how things work. Understanding active and passive mode can also be useful, since you are going to implement at least one of them.
Also, use wireshark to follow a TCP stream and see the data you are sending/receiving, it can be very useful in debugging.
The protocol implementation won't give you a file structure. The protocol is here to define some rules and states.
The dev/prog part is up to you. You just need to respect the FTP protocol in order to gain the normalization and the compatibility with other client/server.
Best regards
I'm trying to make a custom packet using C using the TCP/IP protocol. When I say custom, I mean being able to change any value from the packet; ex: MAC, IP address and so on.
I tried searching around but I can't find anything that is actually guiding me or giving me example source codes.
How can I create a custom packet or where should I look for guidance?
A relatively easy tool to do this that is portable is libpcap. It's better known for receiving raw packets (and indeed it's better you play with that first as you can compare received packets with your hand crafted ones) but the little known pcap_sendpacket will actually send a raw packet.
If you want to do it from scratch yourself, open a socket with AF_PACKET and SOCK_RAW (that's for Linux, other OS's may vary) - for example see http://austinmarton.wordpress.com/2011/09/14/sending-raw-ethernet-packets-from-a-specific-interface-in-c-on-linux/ and the full code at https://gist.github.com/austinmarton/1922600 . Note you need to be root (or more accurately have the appropriate capability) to do this.
Also note that if you are trying to send raw tcp/udp packets, one problem you will have is disabling the network stack automatically processing the reply (either by treating it as addressed to an existing IP address or attempting to forward it).
Doing this sort of this is not as simple as you think. Controlling the data above the IP layer is relatively easy using normal socket APIs, but controlling data below is a bit more involved. Most operating systems make changing lower-level protocol information difficult since the kernel itself manages network connections and doesn't want you messing things up. Beyond that, there are other platform differences, network controls, etc that can play havoc on you.
You should look into some of the libraries that are out there to do this. Some examples:
libnet - http://libnet.sourceforge.net/
libdnet - http://libdnet.sourceforge.net/
If your goal is to spoof packets, you should read up on network-based spoofing mitigation techniques too (for example egress filtering to prevent spoofed packets from exiting a network).
TL;DR available at the bottom
I've been trying to figure out a way to get two laptops (both running Ubuntu) to be able to pass basic messages back and forth without the need for them to be connected via a wireless network,either by an AP or ad-hoc. I want to reiterate here that ad-hoc networking is not what I'm looking for, I've seen many similar questions here with that as the answer.
I guess what I'm asking is: how do I achieve this? All I really need is for one computer to be able to send a packet, and then for another to pick it up via a packet sniffer of some kind.
Currently: I have both laptops in monitor mode (via a mon0 interface created from aircrack-ng's airmon-ng)so that they can sniff nearby traffic (with Wireshark, tcpdump,tcpcump.org's sample libpcap code, and opening a raw socket and just printing out all the packets. I tried each just because I thought one could be doing something differently/leaving something out). I also have a very basic program that consists of opening a raw socket to send crafted ethernet frames out to the air, but I can't get my two machines to see the other's packets. The sniffer running on each machine can only see the packets going out of that machine (in addition to nearby beacons/control traffic from wifi in the area).
Some things to note that might be important are:
-the packets I'm sending out appear in Wireshark (only on the sending machine) as malformed 802.11 packets (probably because I'm just filling them with junk data for now). I was under the impression that my other laptop would also see them as malformed packets, but it gets nothing
-the sockets I'm using are from a call to socket(PF_PACKET,SOCK_RAW,ETH_P_ALL). Raw sockets are something I just recently was aware of, so I could be misunderstanding how they work, but my impression is that I can craft a layer 2 packet by hand and ship out straight out to the wire/air.
If you're curious as to why I want to do something like this, it's part curiosity, part research for a project I'm working on. I want to streamline / automate the process of setting up an ad-hoc network, and what I'm trying to do here is for the laptops to do a small exchange to figure out the specifics of the adhoc network they are about to create and then make/join that network automatically, instead of either one person explicitly setting up the network OR having both people pre-decide the name, etc of the network and have both computers constantly trying to connect to that specific one.
I'm more interested if I'm going about this process in the right way rather than if my code works or not, if someone thinks me posting my (very basic, taken from another post on Stack Overflow) raw socket code will help, I can.
Edit: I am more than happy to post a complete set of code with instructions if I can get this working. I couldn't find much helpful info on this topic on the internet, and I'd love to put it up for future people trying to do the same thing.
TL;DR I want to send out a packet from one laptop and pick it up on another via a packent sniffer of some sort. No wifi network or ad-hoc network involved. Something akin to spoofing an AP's beacon frame (or similar) for the purpose of sending small amounts of data.
Edit 2:After some thought, perhaps what I'm looking for is some kind of raw 802.11 use? Having direct control of the wifi radio? Is such a thing possible?
I found out I was able to send packets out through my monitor mode interface as long as I had correct 802.11 with radiotap headers. I think the problem I was originally experiencing (not being able to sniff the packets) was because they were malformed and thus not actually getting sent out.
I was able to accomplish this by adapting the example code found here, courtesy of someone named Evan Jones, except I did not need to use an Atheros based card or Madwifi drivers, everything worked fine with the mon0 interface created with aircrack-ng.
I am certain that Apple Mac do this. Apple call it 'bonjour'. There may well be a proper IETF spec for it. This is an Article on Bonjour this is Wikipedia on an open component of bonjour which might help get you moving.