I believe that my question is:
Is there a simple user-mode TCP stack on PC operating systems that could be used to exchange data over a lossy serial link with a Linux-based device?
Here is more context:
I have a Linux-based device connected via a serial link to a PC. The serial link is lossy so data being sent between the two devices sometimes needs to be retransmitted. Currently the system uses a custom protocol that includes framing, addressing (for routing to different processes within the Linux device), and a not-so-robust retransmission algorithm.
On the Linux device side, it would be convenient to replace the custom protocol, implement SLIP over the serial link and use TCP for all communications. The problem is that on the PC-side, we're not sure how to use the host's TCP stack without pulling in general IP routing that we don't need. If there were a user-mode TCP stack available, it seems like I could integrate that in the PC app. The only TCP stacks that I've found so far are for microcontrollers. They could be ported, but it would be nice if there were something more ready-to-go. Or is there some special way to use the OS's built in TCP stack without needing administrative privileges or risking IP address conflicts with the real Ethernet interfaces.
Lastly, just to keep the solution focused on TCP, yes, there are other solutions to this problem such as using HDLC or just fixing our custom protocol. However, we wanted to explore the TCP route further in case it was an option.
It appears that the comments have already answered your question, but perhaps to clarify; No you can not use TCP without using IP. TCP is built on top of IP, and it isn't going to work any other way.
PPP is a good way of establishing an IP connection over a serial link, but if you do not have administrative access on both sides of the computer this could be difficult. 172.16.x, 10.x, and 192.168.x are defined as being open for local networks, so you should be able to find a set of IP addresses that does not interfere with the network operation of the local computer.
From the point of view of no configuration, no dependencies, comping up with your own framing / re-transmit protocol should not be too hard, and is probably your best choice if you don't need inter-operability. That being said kermit, {z,y,z}modem would provide both better performance and a standard to code against.
Lastly, you may be able to use something like socat to do protocol translation. I.e. connect a serial stream to a TCP port. That wouldn't address data reliability / re-transmission, but it may be the interface you are looking to program against.
Related
if I malfunctioned my computer TCP stack in kernel just commenting out single important line of code or compiling with excluding TCP stack and install. Then What are the steps in I need to take in coding. Do I need to Implement Ethernet (if creates layer 2 Socket in my stack implementation) then do I need to implement IP because that comes with if I am using layer 2 socket then implement TCP (this is what final code will be)
If that's so
Then what are the supported protocols I need to implement to cover ethernet layer, Ip layer and (TCP layer -- that I can know I may have used it in code)
Can anyone please tell the machining protocols with each layer
whats before?-->ethernet --> ip -- tcp --> http <--protocol names please that may be needed
the whole thing is just for as a research thing or learning for my self or just may be programming
I'll assume you want to be able to do something like request https://example.com from a web server.
For this the following procotols are diretly necessary:
HTTPS (i.e. HTTP and TLS), but that's usually already implemented in user space, no change here
DNS to resolve example.com to an IP address. Usually implemented in user space as well, based on UDP
UDP to run DNS on top of. Usually implemented in the kernel on top of IP.
TCP to run HTTPS on top of. Usually implemented in the kernel on top of IP.
IP to run both TCP and UDP on top of. Usually implemented in the kernel on top of Ethernet.
Ethernet to send/receive IP packages. Usually implemented in the kernel, with the help of hardware-specific NIC drivers.
But even if you had implemented those, you wouldn't be done. For example, you wouldn't know what your local IP address is.
To do this, you'd implement DHCP. You could get away without implementing this, if you just configured your IP address to a fixed value (as long as it's within the allowed range of your network and doesn't conflict with other devices, this is not a problem).
But when you try to send your first IP packet to the DNS server, you'd realize that you don't know what Ethernet address you should send that to. So you'd need to implement ARP as well. Again, you could theoretically have a static routing table and "fake" the ARP responses by your router this way, but this could be way trickier than just hard-coding the IP adress.
I may have missed some protocol (and I'm sure comments will pop up, if I did), but that should be roughly everything that you'd need to get going.
I am working a C program that is uses sockets to implement tcp networking in a server application that I am working on. I was wondering is it possible to disable the tcp/ip stack of the kernel so my system do not interfere with incoming connection sync requests and IP packets.
Or I must compile kernel to disable it please tell if this is the case.
On this question How to create a custom packet in c?
it says
Also note that if you are trying to send raw tcp/udp packets, one problem you will have is disabling the network stack automatically processing the reply (either by treating it as addressed to an existing IP address or attempting to forward it).
If thats the case then how can it be possible.
Or is there any tool or program in Linux that can be used to achieve this like this comment Disable TCP/IP Stack from user-space
There is of course the counterintuitive approach of using additional networking functionality to disable normal networking functionality: netfilter. There are a few iptables matches/targets which might prove beneficial to you (e.g., the “owner” match that may deny or accept based on PID or UID). This still means the functionality is in the kernel, it just limits it.
if someone knows from right above then how can this be done are there any commands?
Well, you could compile yourself a kernel without networking :)
A couple of options
Check out the DPDK project (https://www.linuxjournal.com/content/userspace-networking-dpdk). DPDK passes the Physical NIC to User space via UIO driver to igb_uio|uio_pci_generic|vfio-pci. Thus eliminates Kernel Stack.
Use XDP supported NIC with either Zero-Copy or Driver-mode. with eBPF running one can push the received packets directly to User space bypassing the kernel stack.
Unless this is a homework project, remember: don't invent, reuse.
[EDIT-based on comment] Userspace TCP-IP stack have custom sock-API to read/write into the socket. So with either LD_PRELOAD or source file change, one can use the same application.
I want to be able to simulate an incoming packet on a certain physical network interface.
Specifically, given an array of bytes and an interface name, I want to be able to make that interface think a packet containing those bytes arrived from another interface (most likely on another machine).
I've implemented the code that prepares the packet, but I'm unsure what the next step is.
I should point out that I actually need to feed the interface with my bytes, and not use a workaround that might produce a similar results in other machines (I've seen answers to other questions mentioning the loopback interface and external tools). This code is supposed to simulate traffic on a machine that's expecting to receive traffic from certain sources via specific interfaces. Anything else will be ignored by the machine.
I'm going to stick my neck out and say this is not possible without kernel modifications, and possibly driver modifications. Note that:
There are plenty of ways of generating egress packets through a particular interface, including libpcap. But you want to generate ingress packets.
There are plenty of ways of generating ingress packets that are not through a physical interface - this is what tap/tun devices are for.
If you modify the kernel to allow direct injection of packets into a device's receive queue, that may have unexpected effects, and is still not going to be an accurate simulation of the packets arriving in hardware (e.g. they will not be constrained to the same MTU etc). Perhaps you can build an iptables extension that fools the kernel into thinking the packet came from a different interface; I'm not sure that will do what you need though.
If all you need is simulation (and you are happy with a complete simulation), build a tap/tun driver, and rename the tap interface to eth0 or similar.
Depending on which network layer you're trying to simulate, there may be a work-around.
I have had success getting ip packets into the ingress queue with an ethernet 'hairpin'. That is, by setting the source and destination MAC address to the local interface, sending the packet results in it first appearing as an egress packet, then being 'hairpinned' and also appearing as an ingress packet.
This at least works under linux using pcapplusplus (libpcap under the hood), with my wireless interface. Your millage may vary.
This will obviously only suit your needs if you're OK with modifying the ethernet header, ie only simulating a higher layer.
Here is a snippet of c++ where I spoof a rst tcp packet for a local socket:
//always use the actual device source MAC, even if we're spoofing the remote rst
// this produces a 'hairpin' from the egress to the ingress on the interface so the tcp stack actually processes the packet
// required because the tcp stack doesn't process egress packets (at least on a linux wireless interface)
pcpp::EthLayer eth(localMAC,localMAC);
pcpp::IPv4Layer ip(remoteIP, localIP);
pcpp::TcpLayer tcp(remotePort, localPort);
pcpp::Packet pac(60);
ip.getIPv4Header()->timeToLive = 255;
tcp.getTcpHeader()->rstFlag = 1;
tcp.getTcpHeader()->ackFlag = 1;
tcp.getTcpHeader()->ackNumber = pcpp::hostToNet32(src.Ack);
tcp.getTcpHeader()->sequenceNumber = pcpp::hostToNet32(src.Seq);
pac.addLayer(ð);
pac.addLayer(&ip);
pac.addLayer(&tcp);
pac.computeCalculateFields();
dev->sendPacket(&pac);
EDIT: the same code works on windows on an ethernet interface. It doesn't seem to do the same 'hairpin' judging from wireshark, but the tcp stack does process the packets.
Another solution is to create a new dummy network device driver, which will have the same functionality as the loopback interface (i.e. it will be dummy). After that you can wrap up a creation of simple tcp packet and specify in the source and destination addresses the addresses of the two network devices.
It sounds a little hard but it's worth trying - you'll learn a lot for the networking and tcp/ip stack in linux.
Per http://www.solacesystems.com/blog/kernel-bypass-revving-up-linux-networking:
[...]a network driver called OpenOnload that use “kernel bypass” techniques to run the application and network driver together in user space and, well, bypass the kernel. This allows the application side of the connection to process many more messages per second with lower and more consistent latency.
[...]
If you’re a developer or architect who has fought with context switching for years kernel bypass may feel like cheating, but fortunately it’s completely within the rules.
What are the functions needed to do such kernel bypassing?
A TCP offload engine will "just work", no special application programming needed. It doesn't bypass the whole kernel, it just moves some of the TCP/IP stack from the kernel to the network card, so the driver is slightly higher level. The kernel API is the same.
TCP offload engine is supported by most modern gigabit interfaces.
Alternatively, if you mean "running code on a SolarFlare network adapter's embedded processor/FPGA 'Application Onload Engine'", then... that's card-specific. You're basically writing code for an embedded system, so you need to say which kind of card you're using.
Okay, so the question is not straight forward to answer without knowing how the kernel handles the network stack.
In generel the network stack is made up of a lot of layers, with the lowest one being the actual hardware, typically this hardware is supported by means of drivers (one for each network interface), the nic's typically provide very simple interfaces, think recieve and send raw data.
On top of this physical connection, with the ability to recieve and send data is a lot of protocols, which are layered as well, near the bottem is the ip protocol, which basically allows you to specify the reciever of your information, while at the top you'll find TCP which supports stable connections.
So in order to answer your question, you most first figure out which part of the network stack you'll need to replace, and what you'll need to do. From my understanding of your question it seems like you'll want to keep the original network stack, and then just sometimes use your own, and in that case you should really just implement the strategy pattern, and make it possible to state which packets should be handled by which toplevel of the network stack.
Depending on how the network stack is implemented in linux, you may or may not be able to achieve this, without kernel changes. In a microkernel architecture, where each part of the network stack is implemented in its own service, this would be trivial, as you would simply pipe your lower parts of the network stack to your strategy pattern, and have this pipe the input to the required network toplevel layers.
Do you perhaps want to send and recieve raw IP packets?
Basically you will need to fill in headers and data in a ip-packet.
There are some examples here on how to send raw ethernet packets:
:http://austinmarton.wordpress.com/2011/09/14/sending-raw-ethernet-packets-from-a-specific-interface-in-c-on-linux/
To handle TCP/IP on your own, i think that you might need to disable the TCP driver in a custom kernel, and then write your own user space server that reads raw ip.
It's probably not that efficient though...
Can someone possibly explain (within the size of a stackoverflow answer) the code required in order to emulate a network interface? I just know that there is virtualization software out there like Qemu that does this specific type of hardware emulation, but have no idea how this would work. Lots of books will show you how to create a program that listens on a TCP socket, but not create a host that gets its own IP address.
VirtualBox is open source. As a VM, with networking support, it should be sufficient to demonstrate to you what to do, along with a working implementation. https://www.virtualbox.org/wiki/Downloads
It's really depends what do you mean and what do you want to achieve. If you want emulate some real hardware you need via hypervisor's primitive emulate the most aspects mentioned in datasheet of corresponding adapter, if you want introduce some service, e.g. DNS or HTTP service visible in internal network: you need port teach some user land stack (e.g. LWIP or Slirp, or part if you need UDP only or lower) to communicate with hypervisor's internal network.