I want to write an Application layer protocol that uses TCP to return certain ASCII Text when a GET request is sent. I read the first HTTP specification and the SMTP specification but am still unclear on how to write the protocol connecting the two computers itself. How could I write this in C? Are there any tutorials or examples I could look at?
The heart of any communication protocol is the interface control document(ICD), which will describe the message structures that are allowed, like what is the size of your header, data, crc field etc. It is from this document you create the C structures. Usually people use bit fields to encapsulate the message fields appropriately. When you use existing communication methods, for example Ethernet you have the TCP or UDP sockets to send and receive the data. You can encode your messages in them. If you want to develop a new communication protocol then you have to make a logic of your own and embed it over existing media and proceed.
You're asking two questions. Your first question is "How can I create a new communications protocol", and your second question is "How can I implement this in C".
These are both far too generic to be good questions per the charter of this forum.
The answer to "How can I create a new communications protocol" is, as millimoose already pointed out, simple: A protocol is a document specifying the set of rules for how entities can communicate. Decide what a conversation should look like, starting from the "hello" or equivalent, specifying every possible request and every possible response, and every possible error response, through to how to say goodbye (and how to deal with a connection that gets dropped with saying goodbye), and write that all down. The SMTP protocol is actually a fairly good example of exactly that, in fact. (A TCP-based internet protocol will also typically specify a default TCP port to operate the protocol over.)
The answer to "How can I implement this in C", now that you have a fully specified protocol, is the same as the answer to "How can I implement this in Java", "How can I implement this in REBOL", or "How can I implement this in TCL": Write a basic server app that speaks the server half of the protocol and a basic client app that speaks the client half of the protocol.
(Of course, you might actually have been intending to ask "Regardless of the specific protocol, how can I write in C a server and client that communicate with each other?". This is also an excessively generic question, which can be answered through judicious searching on google.)
In practice, it is much better to use an existing TCP stack (e.g. tcp(7) socket implementation, at least on Linux), then use some HTTP server library above it, like e.g. libonion. Reinventing your TCP stack and your HTTP server layer would take you more than a year of work.
Related
I'm a beginner to C, I've recently decided to make migrate this project to C from using Scapy/Python, solely because I want better performance. I wish to send layer 2 data, specifically beacon frames to advertise an access point.
So far I have found that I need to (or rather could) use libpcap and a Linux header called ieee80211.h that pre-defines packets, that's all I could gather from the other questions. I've found other information which says I should use raw sockets instead of libpcap? I'm not sure if this is all I need. Most of the information and tutorials I have found on Google refer to packet sniffing, not sending.
How do I define a custom frame and/or packet (e.g. a beacon frame or association request) and then simply send it to wlan0 etc.?
Thought I might update this. I used libpcap.
You just need to create a handle with your device, set it to monitor mode successfully (important), check the data link type (e.g. 802.11 with Radiotap for layer 2) then use pcap_sendpacket(handle, packetArrayContainingHex, size);. Hard part is forming legal packets that aren't rejected or dropped, taking a look in wireshark helps.
This link might help. It basically opens the driver at raw packet level and creates the whole packet to send over the wire, as your question suggests.
I'm building a LoRa network where the server and the end-device need to communicate using a protocol which normally transmits data via UDP. Due to the fact that these two protocols act totally different I need to find a way to combine those two.
One solution I found is to create my own socket API which provides send, receive, bind, ... functions. But here I'm actually struggling.
In which scope do I need to write my socket? Is it enough to just edit the functions and rely on the other given parameters such as the address families? Or must I define my own AF and if so where/how is this achieved.
I'm looking forward to your answers / ideas.
According to the LoRaWAN specification and my limited experience, LoRaWAN is not suitable for such situation. If you still wanna use UDP packets over LoRaWAN, here are some tips for your question.
In which scope do I need to write my socket?
You may use sendUnconfirm function since this function does not need ACK from gateway. And port in LoRaWAN could play the role of bind in UDP socket.
Is it enough to just edit the functions and rely on the other given
parameters such as the address families?
LoRaWAN server has its own features and structure. Usually, a LoRaWAN server is consist of packet_forwarder, LoRaWAN server and LoRaWAN application server. You may use these features to build you own application on LoRaWAN applicaiton server. It could save your a lot of time.
It is highly recommended to read LoRaWAN specification (Get it here) and TTN LoRaWAN wiki to help you get a better understanding in LoRaWAN.
I'm trying to make a custom packet using C using the TCP/IP protocol. When I say custom, I mean being able to change any value from the packet; ex: MAC, IP address and so on.
I tried searching around but I can't find anything that is actually guiding me or giving me example source codes.
How can I create a custom packet or where should I look for guidance?
A relatively easy tool to do this that is portable is libpcap. It's better known for receiving raw packets (and indeed it's better you play with that first as you can compare received packets with your hand crafted ones) but the little known pcap_sendpacket will actually send a raw packet.
If you want to do it from scratch yourself, open a socket with AF_PACKET and SOCK_RAW (that's for Linux, other OS's may vary) - for example see http://austinmarton.wordpress.com/2011/09/14/sending-raw-ethernet-packets-from-a-specific-interface-in-c-on-linux/ and the full code at https://gist.github.com/austinmarton/1922600 . Note you need to be root (or more accurately have the appropriate capability) to do this.
Also note that if you are trying to send raw tcp/udp packets, one problem you will have is disabling the network stack automatically processing the reply (either by treating it as addressed to an existing IP address or attempting to forward it).
Doing this sort of this is not as simple as you think. Controlling the data above the IP layer is relatively easy using normal socket APIs, but controlling data below is a bit more involved. Most operating systems make changing lower-level protocol information difficult since the kernel itself manages network connections and doesn't want you messing things up. Beyond that, there are other platform differences, network controls, etc that can play havoc on you.
You should look into some of the libraries that are out there to do this. Some examples:
libnet - http://libnet.sourceforge.net/
libdnet - http://libdnet.sourceforge.net/
If your goal is to spoof packets, you should read up on network-based spoofing mitigation techniques too (for example egress filtering to prevent spoofed packets from exiting a network).
Most of the applications I've seen that use TCP, do roughly the following to connect to remote host:
get the hostname (or address) from the configuration/user input (textual)
either resolve the hostname into address and add the port, or use getaddrinfo()
from the above fill in the sockaddr_* structure with one of the remote addresses
use the connect() to get the socket connected to the remote host.
if fails, possibly go to (3) and retry - or just complain about the error
(2) is blocking in the stock library implementation, and the (4) seems to be most frequently non-blocking, which seems to give a room for a lot of somewhat similar yet different code that serves the purpose to asynchronously connect to a remote host by its hostname.
So the question: what are the good reasons not to have the additional single call like following:
int sockfd = connect_by_name(const char *hostname, const char *servicename)
?
I can come up with three:
historic: because that's what the API is
provide for custom per-application policy mechanism for address selection/connection retry: this seems a bit superficial, since for the common case ("get me a tube to talk to remote host") the underlying OS should know better
provide the visual feedback to the user about the exact step involved ("name resolution" vs "connection attempt"): this seems rather important, lookup+connection attempt may take time
Only the last of them seems to be compelling enough to rewrite the resolve/connect code for every client app (as opposed to at least having and using a widely used library that would implement the connect_by_name() semantics in addition to the existing sockets API), so surely there should be some more reasons that I am missing ?
(one of the reasons behind the question is that this kind of API would appear to help the portability to IPv6, as well as possibly to other stream transport protocols significantly)
Or, maybe such a library exists and my google-fu failed me ?
(edited: corrected the definition to look like it was meant to look, thanks LnxPrgr3)
Implementing such an API with non-blocking characteristics within the constraints of the standard library (which, crucially, isn't supposed to start its own threads or processes to work asynchronously) would be problematic.
Both the name lookup and connecting part of the process require waiting for a remote response. If either of these are not to block, then that requires a way of doing asychronous work and signalling the change in state of the socket to the calling application. connect is able to do this, because the work of the connect call is done in the kernel, and the kernel can mark the socket as readable when the connect is done. However, name lookup is not able to do this, because the work of a name lookup is done in userspace - and without starting a new thread (which is verboten in the standard library), giving that name lookup code a way to be woken up to continue work is a difficult problem.
You could do it by having your proposed call return two file descriptors - one for the socket itself, and another that you are told "Do nothing with this file descriptor except to check regularly if it is readable. If this file descriptor becomes readable, call cbn_do_some_more_work(fd)". That is clearly a fairly uninspiring API!
The usual UNIX approach is to provide a set of simple, flexible tools, working on a small set of object types, that can be combined in order to produce complex effects. That applies to the programming API as much as it does to the standard shell tools.
Because you can build higher level APIs such as the one you propose on top of the native low level APIs.
The socket API is not just for TCP, but can also be used for other protocols that may have different end point conventions (i.e. the Unix-local protocol where you have a name only and no service). Or consider DNS which uses sockets to implement itself. How does the DNS code connect to the server if the connection code relies on DNS?
If you would like a higher level abstraction, one library to check out is ACE.
There are several questions in your question. For instance, why not
standardizing an API with such connect_by_name? That would certainly
be a good idea. It would not fit every purpose (see the DNS example
from R Samuel Klatchko) but for the typical network program, it would
be OK. A paper exploring such APIs is "Simplifying Internet Applications Development
With A Name-Oriented Sockets Interface" by Christian Vogt. Note
that another difficulty for such an API would be "callback"
applications, for instance a SIP client asking to be called back: the
application has no easy way to know its own name and therefore often
prefer to be called back by address, despite the problems it make, for
instance with NAT.
Now, another question is "Is it possible to build such
connect_by_name subroutine today?" Partly yes (with the caveats
mentioned by caf) but, if written in userspace, in an ordinary
library, it would not be completely "name-oriented" since the Unix
kernel still manages the connections using IP addresses. For instance,
I would expect a "real" connect_by_name routine to be able to
survive renumbering (for instance because a mobile host renumbered),
which is quite difficult to do in userspace.
Finally, yes, it already exists a lot of libraries with similar
semantics. For a HTTP client (the most common case for a program whose
network abilities are not the main feature, for instance a XML
processor), you have Neon and libcURL. With libcURL, you can
simply write things like:
#define URL "http://www.velib.paris.fr/service/stationdetails/42"
...
curl_easy_setopt(curl, CURLOPT_URL, URL);
result = curl_easy_perform(curl);
which is even higher-layer than connect_by_name since it uses an
URL, not a domain name.
I need to decode a packet sent using TIBCO-RV and pull fields out of the header and skip over the message body. I have not been able to any examples or documentation. Does anybody know of any open source applications that might do this or if there is a Wireshark dissector out there somewhere?
Maybe you should try applying for a license and getting the official documentation. According to Wikipedia:
TIBCO provides messaging APIs in C,
C++, Java, Visual BASIC , Perl and
.NET to receive data feeds on MS Excel
spreadsheets and other applications of
choice.
Failing that, you could perhaps dive into the TIBCO:RV Perl module.
The methods which TibcoRV implements reliable mutli-cast are propriety, but one would assume easy to reverse engineer. I don't believe any of the official documentation goes into detail on the packet level detail. It's quite easy to get the data out if you have the API.
Several things come to mind:
Is the client on your machine running? This is required in order to create the multicast subscription (unless you are using broadcast mode). Otherwise, you need to have some client subscribe to the multicast channel, or your switch shouldn't forward the traffic.
Generally, you will have a single rrd running locally. You have TCP traffic between the RRD and your app. You can use an app like socketsniff to view the traffic between the two.