can we in a network gateway write a program with libpcap to extract destination web address and other info like squid? in c language.
is any better way? if yo can give me some example and guide.
libcap could not be used for realtime monitoring as it has big impact on network performance. I think it can perform at a maximum of 25% of the effective bandwidth. You can observe that by using wireshark (based on libpcap).
You can configure squid to be a transparent proxy. This can be done on Linux by editing the IPTable rules. This way all traffic goes via squid, which can then log or do whatever you want. Additionally no traffic is missed, but performance will be effected.
Yes you can quite easily. Just capture packets on port 80, and look in the first one or two for the HTTP request. The HTTP request is easily parsed.
Please read the libpcap documentation and look at some of the examples for getting started.
Related
I have a zigbee2mqtt / home assistant setup working fine, and I'd like to try to make my own simple devices to connect to that network.
I got an xbee 3 board, and using micropython to start with I was able to connect to my network.
However the "interview" fails. The xbee receives a message with a cluster 0, profile 260 (home automation) and endpoint 230 (command). Not sure what the payload contains, that's not a string :
{'profile': 260, 'dest_ep': 230, 'broadcast': False, 'sender_nwk': 0, 'source_ep': 1, 'payload': b'\x10\x02\x00\x05\x00\x04\x00\x07\x00', 'sender_eui64': b"\x00\x12K\x00\x18\xe2I'", 'cluster': 0}
My question is what should I answer for the interview to succeed ?
I'm making only a basic sensor, I'd like to just report 1 weight reading periodically. I'm assuming I need to send back something saying I have one endpoint, on some cluster (not sure which, I guess something in the 400s) but I don't know what the format should be.
I couldn't find much info on this (baring how to use things like the Zigbee Cluster Library, which aren't python), any pointers or examples of end devices I could take a look at to understand how this interview process works ?
Unfortunately digi's examples all seem to involve xbee devices talking to each other, I couldn't find any examples of how to make a regular end device.
Thanks !
EDIT: Just found this great page which explains how this all works. Still need to figure out the exact bits I'll need and try it out, but now I know where to start !
This sounds a lot like the ZCL, and I'm not aware of an Open Source Python implementation of that protocol. Digi has an Open Source ANSI C Library that includes a ZCL implementation. If you can read C code, you might be able to decode that payload to see what it's asking. You might also need to handle some of the ZDO/ZDP (Zigbee Data Object/Device Profile) protocol on endpoint 0, by setting ATAO=3 (IIRC). There's also ZDO/ZDP code in that C library. (Full disclosure: I wrote most of the code in that library, including the Zigbee layer. But I haven't worked with Zigbee in a long time, so I'm rusty on protocol details.)
My recommendation would be to just hardcode hand-generated responses as much as possible. Figure out the expected format for requests, and determine what works as a response. If you can sniff the 802.15.4 traffic, or have your zigbee2mqtt gateway log activity with an existing device, you might be able to use its responses as a starting point for your implementation.
Is there a way to view all the IPv4 packets sent to a Linux computer?
I know I can capture the packets at the ethernet level using libpcap. This can work, but I don't really want to defragment the IPv4 packets. Does libpcap provide this functionality and I'm just missing it?
One thing that kinda works is using a tun device. I can capture all the IPv4 traffic by routing all traffic to the tun device via something like ip route add default via $TUN_IP dev $TUNID. This also stops outbound traffic though, which is not what I want.
I just want to see the IPv4 packets, not intercept them. (Or, even better, optionally intercept them.)
Edit: I'm specifically looking for a programmatic interface to do this. E.g. something I can use from within a C program.
Yes, you can see all the packets that arrive at your network interface. There are several options to access or view them. Here a small list of possible solutions, where the first one is the easiest and the last one the hardest to utilize:
Wireshark
I'd say this is pretty much the standard when it comes to protocol analyzers with a GUI (uses libpcap). It has tons of options, a nice GUI, great filtering capabilities and reassembles IP datagrams. It uses libpcap and can also show the raw ethernet frame data. For example it allows you to see layer 2 packets like ARP. Furthermore you can capture the complete data arriving at your network interface in a file that can later be analyzed (also in Wireshark).
tcpdump
Very powerful, similar features like Wireshark but a command line utility, which also uses libpcap. Can also capture/dump the complete interface traffic to a file. You can view the dumped data in Wireshark since the format is compatible.
ngrep
This is known as the "network grep" and is similar to tcpdump but supports regular expressions (regex) to filter the payload data. It allows to save captured data in the file format supported by Wireshark and tcpdump (also uses libpcap).
libnids
Quotation from the official git repository:
"Libnids is a library that provides a functionality of one of NIDS
(Network Intrusion Detection System) components, namely E-component. It means
that libnids code watches all local network traffic [...] and provides convenient information on them to
analyzing modules of NIDS. Libnids performs:
assembly of TCP segments into TCP streams
IP defragmentation
TCP port scan detection"
libpcap
Of course you can also write your own programs by using the library directly. Needless to say, this requires more efforts.
Raw or Packet Sockets
In case you want to do all the dirty work yourself, this is the low level option, which of course also allows you to do everything you want. The tools listed above use them as a common basis. Raw sockets operate on OSI layer 3 and packet sockets on layer 2.
Note: This is not meant to be a complete list of available tools or options. I'm sure there are much more but these are the most common ones I can think of.
Technically you have to make a copy of the received packet via libpcap. To be more specific, what you can do is to get packets with libpcap, that way the packets will be kind of blocked, so you need to re send them to the destination. Lets say that you want to make a Fire-Wall or something, what you should do is to have a layer that can work like getting the package and then send it to the destination, in between you can make a copy of what you got for further processes. In order to make the intercept option, you need to create some predefined rules, i.e. the ones that violates the rules will not be send again to their destination.
But that needs a lot of efforts and I don't think you want to waist your life on it.
Wire-shark as mentioned by #Barmar can do the job already.
If you need some kind of command line interface option I would say that "tcpdump" is one of the best monitoring tools. for example for capturing all ipv4 HTTP packets to and from port 80 the command will be:
tcpdump 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
for more information and options see tcpdump
Please be specific if you need to write a program for it, then we can help about how to do it.
This may not be in the right location, so tell me and I'll move it.
I am a recent EE grad and I was hired to build a system that exists on a SoC with a simple 32-bit processor. The system basically monitors several external devices and performs some DSP on it, and then is supposed to send the results using a WiFi device (in my case I have the ESP8266 using UDP) to an email server for logging/notification.
I have been trying to find a library that I can use, but my uC can only program in C and I have it set up for UDP, and everything is in C++ using some other protocol, or something else completely.
I am great at DSP, decent at SoC's and uC's, but when it come to this email server communication thing I am at a loss.
I have successfully configured everything for the sensors, the datapath, the DSP, and connected the system to my WiFi via UDP, but I have yet to figure out how to send data to any servers.
Could someone help me understand how I should go about this?
I have looked into some simple SMTP commands such as HELO, MAIL, RCPT, DATA, etc. but I cannot understand how I actually should implement them in my code.
When I send out the WiFi data via UDP what type of data do I send and how do I format it? Do I need to send any other kind of flags? How should I expect the response? I also know the data has to be transformed into base 64 which is confusing me further.
I am also not super familiar with UDP to begin with, I have been using libraries that are part of the SoC's default library to connect to my WiFi.
I know these may either seem like obvious or stupid questions but it is were I no longer have any knowledge, and everything I find online doesn't make sense, or doesn't attempt to explain it, just gives a pre-made solution
I have found the RFC2821 but it doesn't get any clearer.
I know that's a lot but any help at all would be a lifesaver!
Since you are asking this question, I'm assuming that you are not booting and running an OS suitable for micro-controllers such as an embedded variant of Linux or such. If you were, you would simply be able to take advantage of possibly built in applications or other existing code.
But you don't mention having written an Ethernet stack, so are you using some other library or operating environment which might have some of the functionality needed for an implementation of SMTP?
If you don't and really do need to write your own SMTP client to run directly on the processor you are using, then you should be able to find plenty of examples of source code for this. A quick google search of How To Write an SMTP client showed a few articles with some example code. One article seems to be an exact hit, but you need to look at it further.
However, I would highly suggest just sitting down with a telnet client and connect to an SMTP server you are allowed to use and try the commands you need to just send a message. If you only need to send text, you don't need to get involved in MIME encoding or anything like that.
The question is, does pCap library allow capturing packets that are generated by the local system? something like the netfitler hook NF_IP_LOCAL_OUT but in user-space? If pCap cannot support this, is there any well-supported library that can?
Looking on the web, some people mentioned that pCap has a function called setDirection which sets which traffic we're capturing according to traffic direction but many people said this function is only on Windows; and I am kinda limited in time to learn about pCap just to test if it can do what I need to do.
The question is, does pCap library allow capturing packets that are generated by the local system?
Yes. In fact, it captures them by default. You can, in newer versions of libpcap, disable that by calling pcap_setdirection(), but, by default, it captures both incoming and outgoing packets on the interface on which you're capturing. pcap_setdirection() exists in newer versions of WinPcap, but it just returns an error; there's a flag for the WinPcap-only pcap_open() that lets you disable capturing outgoing packets.
I've certainly use tcpdump (on linux) to capture traffic on local interface and originating from the own machine. Since tcpdump uses libpcap, this must be possible.
I'm afraid this is a rather rubbish answer, because I can't tell you exactly HOW to configure libpcap to capture your local packets. However, I would suggest that tcpdump is a good starting point - either by simply using tcpdump itself, or look at the code [which is probably quite large and complex, of course, but if you can figure out what settings you need to make tcpdump do what you want, then you can perhaps add some code to tcpdump to print the settings it uses for the same thing].
Again, slight apology for not "giving you the code".
Also dumpcap -i < capture interface> captures the live traffic,
Dumpcap is a network traffic dump tool. It lets you capture packet data from a live network and write the packets to a file. Dumpcap's native capture file format is libpcap format, which is also the format used by Wireshark, tcpdump and various other tools.
TL;DR available at the bottom
I've been trying to figure out a way to get two laptops (both running Ubuntu) to be able to pass basic messages back and forth without the need for them to be connected via a wireless network,either by an AP or ad-hoc. I want to reiterate here that ad-hoc networking is not what I'm looking for, I've seen many similar questions here with that as the answer.
I guess what I'm asking is: how do I achieve this? All I really need is for one computer to be able to send a packet, and then for another to pick it up via a packet sniffer of some kind.
Currently: I have both laptops in monitor mode (via a mon0 interface created from aircrack-ng's airmon-ng)so that they can sniff nearby traffic (with Wireshark, tcpdump,tcpcump.org's sample libpcap code, and opening a raw socket and just printing out all the packets. I tried each just because I thought one could be doing something differently/leaving something out). I also have a very basic program that consists of opening a raw socket to send crafted ethernet frames out to the air, but I can't get my two machines to see the other's packets. The sniffer running on each machine can only see the packets going out of that machine (in addition to nearby beacons/control traffic from wifi in the area).
Some things to note that might be important are:
-the packets I'm sending out appear in Wireshark (only on the sending machine) as malformed 802.11 packets (probably because I'm just filling them with junk data for now). I was under the impression that my other laptop would also see them as malformed packets, but it gets nothing
-the sockets I'm using are from a call to socket(PF_PACKET,SOCK_RAW,ETH_P_ALL). Raw sockets are something I just recently was aware of, so I could be misunderstanding how they work, but my impression is that I can craft a layer 2 packet by hand and ship out straight out to the wire/air.
If you're curious as to why I want to do something like this, it's part curiosity, part research for a project I'm working on. I want to streamline / automate the process of setting up an ad-hoc network, and what I'm trying to do here is for the laptops to do a small exchange to figure out the specifics of the adhoc network they are about to create and then make/join that network automatically, instead of either one person explicitly setting up the network OR having both people pre-decide the name, etc of the network and have both computers constantly trying to connect to that specific one.
I'm more interested if I'm going about this process in the right way rather than if my code works or not, if someone thinks me posting my (very basic, taken from another post on Stack Overflow) raw socket code will help, I can.
Edit: I am more than happy to post a complete set of code with instructions if I can get this working. I couldn't find much helpful info on this topic on the internet, and I'd love to put it up for future people trying to do the same thing.
TL;DR I want to send out a packet from one laptop and pick it up on another via a packent sniffer of some sort. No wifi network or ad-hoc network involved. Something akin to spoofing an AP's beacon frame (or similar) for the purpose of sending small amounts of data.
Edit 2:After some thought, perhaps what I'm looking for is some kind of raw 802.11 use? Having direct control of the wifi radio? Is such a thing possible?
I found out I was able to send packets out through my monitor mode interface as long as I had correct 802.11 with radiotap headers. I think the problem I was originally experiencing (not being able to sniff the packets) was because they were malformed and thus not actually getting sent out.
I was able to accomplish this by adapting the example code found here, courtesy of someone named Evan Jones, except I did not need to use an Atheros based card or Madwifi drivers, everything worked fine with the mon0 interface created with aircrack-ng.
I am certain that Apple Mac do this. Apple call it 'bonjour'. There may well be a proper IETF spec for it. This is an Article on Bonjour this is Wikipedia on an open component of bonjour which might help get you moving.