I'm building a LoRa network where the server and the end-device need to communicate using a protocol which normally transmits data via UDP. Due to the fact that these two protocols act totally different I need to find a way to combine those two.
One solution I found is to create my own socket API which provides send, receive, bind, ... functions. But here I'm actually struggling.
In which scope do I need to write my socket? Is it enough to just edit the functions and rely on the other given parameters such as the address families? Or must I define my own AF and if so where/how is this achieved.
I'm looking forward to your answers / ideas.
According to the LoRaWAN specification and my limited experience, LoRaWAN is not suitable for such situation. If you still wanna use UDP packets over LoRaWAN, here are some tips for your question.
In which scope do I need to write my socket?
You may use sendUnconfirm function since this function does not need ACK from gateway. And port in LoRaWAN could play the role of bind in UDP socket.
Is it enough to just edit the functions and rely on the other given
parameters such as the address families?
LoRaWAN server has its own features and structure. Usually, a LoRaWAN server is consist of packet_forwarder, LoRaWAN server and LoRaWAN application server. You may use these features to build you own application on LoRaWAN applicaiton server. It could save your a lot of time.
It is highly recommended to read LoRaWAN specification (Get it here) and TTN LoRaWAN wiki to help you get a better understanding in LoRaWAN.
Related
So I have a PC connected to a micro-controller via a serial cable and an Ethernet cable. Initially the PC sends a byte across the serial cable to the micro-controller. This results in the micro-controller sending back a UDP datagram via the Ethernet cable.
I want to know whether the code running on my PC should be a server or a client?
Per Wikiepdia Client/Server:
The server component provides a function or service to one or many
clients, which initiate requests for such services
And Master/Slave:
Master/slave is a model of asymmetric communication or control where
one device or process controls one or more other devices or processes
and serves as their communication hub
The above scenario looks like the Master/Slave. In the initial, 'idle' case, there is no "SERVER" that is waiting ("listening") for requests. Only when the PC activate the micro-controller they will start communication (via UDP).
You could use either term depending on what you were talking about. As other people have noted, client and server are terms used to describe how distinct parties are involved in a service. The terms can be useful in some situations (e.g. a web server and the browser as a client) but in other situations it's a less useful term (e.g. peer-to-peer protocols).
Presumably you're on stackoverflow because you're dealing with code.
In this case it's useful to be more precise and I'd suggest using terms to match whatever primitives are exposed by your language. Most will use/expose Posix sockets as their standard API, and hence you'd want to talk about/use connect or accept (potentially after binding first). Note that these calls work across TCP and UDP (except accept), but the semantics of sending and recving on the resulting connected sockets will obviously be different.
Is there a way to view all the IPv4 packets sent to a Linux computer?
I know I can capture the packets at the ethernet level using libpcap. This can work, but I don't really want to defragment the IPv4 packets. Does libpcap provide this functionality and I'm just missing it?
One thing that kinda works is using a tun device. I can capture all the IPv4 traffic by routing all traffic to the tun device via something like ip route add default via $TUN_IP dev $TUNID. This also stops outbound traffic though, which is not what I want.
I just want to see the IPv4 packets, not intercept them. (Or, even better, optionally intercept them.)
Edit: I'm specifically looking for a programmatic interface to do this. E.g. something I can use from within a C program.
Yes, you can see all the packets that arrive at your network interface. There are several options to access or view them. Here a small list of possible solutions, where the first one is the easiest and the last one the hardest to utilize:
Wireshark
I'd say this is pretty much the standard when it comes to protocol analyzers with a GUI (uses libpcap). It has tons of options, a nice GUI, great filtering capabilities and reassembles IP datagrams. It uses libpcap and can also show the raw ethernet frame data. For example it allows you to see layer 2 packets like ARP. Furthermore you can capture the complete data arriving at your network interface in a file that can later be analyzed (also in Wireshark).
tcpdump
Very powerful, similar features like Wireshark but a command line utility, which also uses libpcap. Can also capture/dump the complete interface traffic to a file. You can view the dumped data in Wireshark since the format is compatible.
ngrep
This is known as the "network grep" and is similar to tcpdump but supports regular expressions (regex) to filter the payload data. It allows to save captured data in the file format supported by Wireshark and tcpdump (also uses libpcap).
libnids
Quotation from the official git repository:
"Libnids is a library that provides a functionality of one of NIDS
(Network Intrusion Detection System) components, namely E-component. It means
that libnids code watches all local network traffic [...] and provides convenient information on them to
analyzing modules of NIDS. Libnids performs:
assembly of TCP segments into TCP streams
IP defragmentation
TCP port scan detection"
libpcap
Of course you can also write your own programs by using the library directly. Needless to say, this requires more efforts.
Raw or Packet Sockets
In case you want to do all the dirty work yourself, this is the low level option, which of course also allows you to do everything you want. The tools listed above use them as a common basis. Raw sockets operate on OSI layer 3 and packet sockets on layer 2.
Note: This is not meant to be a complete list of available tools or options. I'm sure there are much more but these are the most common ones I can think of.
Technically you have to make a copy of the received packet via libpcap. To be more specific, what you can do is to get packets with libpcap, that way the packets will be kind of blocked, so you need to re send them to the destination. Lets say that you want to make a Fire-Wall or something, what you should do is to have a layer that can work like getting the package and then send it to the destination, in between you can make a copy of what you got for further processes. In order to make the intercept option, you need to create some predefined rules, i.e. the ones that violates the rules will not be send again to their destination.
But that needs a lot of efforts and I don't think you want to waist your life on it.
Wire-shark as mentioned by #Barmar can do the job already.
If you need some kind of command line interface option I would say that "tcpdump" is one of the best monitoring tools. for example for capturing all ipv4 HTTP packets to and from port 80 the command will be:
tcpdump 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
for more information and options see tcpdump
Please be specific if you need to write a program for it, then we can help about how to do it.
I want to write an Application layer protocol that uses TCP to return certain ASCII Text when a GET request is sent. I read the first HTTP specification and the SMTP specification but am still unclear on how to write the protocol connecting the two computers itself. How could I write this in C? Are there any tutorials or examples I could look at?
The heart of any communication protocol is the interface control document(ICD), which will describe the message structures that are allowed, like what is the size of your header, data, crc field etc. It is from this document you create the C structures. Usually people use bit fields to encapsulate the message fields appropriately. When you use existing communication methods, for example Ethernet you have the TCP or UDP sockets to send and receive the data. You can encode your messages in them. If you want to develop a new communication protocol then you have to make a logic of your own and embed it over existing media and proceed.
You're asking two questions. Your first question is "How can I create a new communications protocol", and your second question is "How can I implement this in C".
These are both far too generic to be good questions per the charter of this forum.
The answer to "How can I create a new communications protocol" is, as millimoose already pointed out, simple: A protocol is a document specifying the set of rules for how entities can communicate. Decide what a conversation should look like, starting from the "hello" or equivalent, specifying every possible request and every possible response, and every possible error response, through to how to say goodbye (and how to deal with a connection that gets dropped with saying goodbye), and write that all down. The SMTP protocol is actually a fairly good example of exactly that, in fact. (A TCP-based internet protocol will also typically specify a default TCP port to operate the protocol over.)
The answer to "How can I implement this in C", now that you have a fully specified protocol, is the same as the answer to "How can I implement this in Java", "How can I implement this in REBOL", or "How can I implement this in TCL": Write a basic server app that speaks the server half of the protocol and a basic client app that speaks the client half of the protocol.
(Of course, you might actually have been intending to ask "Regardless of the specific protocol, how can I write in C a server and client that communicate with each other?". This is also an excessively generic question, which can be answered through judicious searching on google.)
In practice, it is much better to use an existing TCP stack (e.g. tcp(7) socket implementation, at least on Linux), then use some HTTP server library above it, like e.g. libonion. Reinventing your TCP stack and your HTTP server layer would take you more than a year of work.
I am unable to understand or grasp rather; what it means to program at a lower layer in socket programming. I am used to working with tcp/udp/file system sockets. These are all wrapped around their own protocol specifications ... which as i understand would make it working at the application layer in the stack.
In the project i am on , i have seen some files which are "named" LinkLayer, TransportLayer... but i don't see any more calls other than standard socket calls....send /recv/ seletct...
Does the fact you are setting a socket options mean you are programming at a lower level ? Is it just restricted to it? Or are there other API's which grant you access at the representation in kernel ?
Typically this refers to using SOCK_RAW sockets, which requires you to assemble your own packet headers, calculate checksums, etc. You still use send/recv/etc. but now you are responsible for making sure every bit is in the right place.
You can use SOCK_RAW sockets to implement protocols other than TCP or UDP, or to do things with the Internet protocols that higher-level interfaces don't accommodate (like tweaking the TTL of your packets to implement something like traceroute).
This usually means working on a lower OSI-Layer, for example, not directly sending TCP-streams or UDP-packets, but crafting own IP or even Ethernet packets or other low-layer protocols which would - in normal case - be handled by the operating system.
This can be done done via specific socket options which enable you to receive or send data on any layer, even layer 2 (Data Link).
I have a Linux application that talks TCP, and to help with analysis and statistics, I'd like to modify the data in some of the TCP packets that it sends out. I'd prefer to do this without hacking the Linux TCP stack.
The idea I have so far is to make a bridge which acts as a "TCP packet modifier". My idea is to connect to the application via a tun/tap device on one side of the bridge, and to the network card via raw sockets on the other side of the bridge.
My concern is that when you open a raw socket it still sends packets up to Linux's TCP stack, and so I couldn't modify them and send them on even if I wanted to. Is this correct?
A pseudo-C-code sketch of the bridge looks like:
tap_fd = open_tap_device("/dev/net/tun");
raw_fd = open_raw_socket();
for (;;) {
select(fds = [tap_fd, raw_fd]);
if (FD_ISSET(tap_fd, &fds)) {
read_packet(tap_fd);
modify_packet_if_needed();
write_packet(raw_fd);
}
if (FD_ISSET(raw_fd, &fds)) {
read_packet(raw_fd);
modify_packet_if_needed();
write_packet(tap_fd);
}
}
Does this look possible, or are there other better ways of achieving the same thing? (TCP packet bridging and modification.)
There were some apps I used years ago to do some TCP/IP packet manipulation for testing a firewall: fragoute and fragtest. Looks like they haven't been touched in years, but they might give you some ideas of what to do in your code.
You might want to consider using a LD_PRELOAD library to hook the functions that it uses to send the data out (send(), write() etc).
That wouldn't involve any kernel messing-around at all.
Another option is to NAT the outbound connections to a local proxy which can read the data, make whatever modifications, and send it all out to the real destination (with some options to prevent it being NAT'd again and going round in circles)
You can use the click modular router. It is a software router implemented entirely in C++. Click allows you to capture packets as they pass through elements in the router where you can modify or collect statistics as needed. As a kernel module, you completely override the linux routing mechanism and as a userland binary you simply get a duplicate (as you mention in your post) of each packet from the interface. Packets can be directed through the Click graph by way of pcap filters and a variety of other mechanisms.
If you are headed down the bridge route, I think this provides the most direct support for what you are looking to do as you can use tun/tap, to/from host or to/from device capture methods as you require.