I'm trying code TCP server in C language. I just noticed accept() function returns when connection is already established.
Some clients are flooding with random data some clients are just sending random data for one time, after that I want to close their's current connection and future connections for few minutes (or more, depends about how much load program have).
I can save bad client IP addresses in a array, can save timings too but I cant find any function for abort current connection or deny future connections from bad clients.
I found a function for windows OS called WSAAccept that allows you deny connections by user choice, but I don't use windows OS.
I tried code raw TCP server which allows you access TCP packet from begin including all TCP header and it doesn't accept connections automatically. I tried handle connections by program side including SYN ACK and other TCP signals. It worked but then I noticed raw TCP server receiving all packets in my network interface, when other programs using high traffic it makes my program laggy too.
I tried use libnetfilter which allows you filter whole traffic in your network interface. It works too but like raw TCP server it also receiving whole network interface's packets which is making it slow when there is lot of traffic. Also I tried compare libnetfilter with iptables. libnetfilter is slower than iptables.
So in summary how I can abort client's current and future connection without hurt other client connections?
I have linux with debian 10.
Once you do blacklisting on packet level you could get very fast vulnerable to very trivial attacks based on IP spoofing. For a very basic implementation an attacker could use your packet level blacklisting to blacklist anyone he wants by just sending you many packets with a fake source IP address. Usually you don't want to touch these filtering (except you really know what you are doing) and you just trust your firewall etc. .
So I recommend really just to close the file descriptor immediately after getting it from accept.
What I'm doing
I'm implementing a websocket server on a stellaris board as the title says. At the moment I'm able to establish connection to the client and send a few frames.
The way I'm implementing the websocket
The way I'm developing it is something like a master slave communication. Whenever the client sends a string, the server decodes it and then answers. At the moment I'm simply responding to a character 'e', which is designed to be just a counter. The thing is that I implemented the websocket on the client side to send 'e' whenever it receives a message and then displays the message on the page.
The problem
The problem is that it does about 15 transactions and then I can see the communication being re-transmitted from and to the stellaris board and then the communication closes. After the connection closes I noticed that that I can't access any other page on the board. It simply doesn't respond anymore.
My assumptions of what may be causing it
This lead me to believe that the transactions are being too fast and there may be an implementation bug, lwIP bug or hardware bug (I'm using the enet_io example as base).
My assumptions on how to fix it
After seeing this I can imagine that what I need is to control the string being sent to the microcontroller so that it sends once a second, or maybe even less, because at the moment it was doing something like 1000 transactions per second and sometimes more.
The question
So ... after my trials I still have a few questions that need to be answered. Do websockets need this kind of relationship? Where client asks and server serves? Or can I simply stream data from the server to the client as long as the connection is open? Is my supposition that slowing down my rates will work?
Do websockets need this kind of relationship [request-response]? Where client asks and server serves? Or can I simply stream data from the server to the client as long as the connection is open?
The Websocket protocol doesn't require a request-response model (except for the connection establishing handshake).
The server can stream data to the client without worrying about any response or request from the client.
However, it's common practice to get a response or a ping from a client once in a while, just to know they're alive.
This allows the client to renew a connection if a message or ping fails to reach the server - otherwise the client might not notice an abnormally dropped connection (it will just assume no updates are being sent because there's no new data).
It also allows the server to know a connection is still alive even when no information is being exchanged.
Is my supposition that slowing down my rates will work?
I guess this question becomes less relevant due to the first question's answer... however, I should probably note that the web socket client (often a browser) will have limited resources and a different memory management scheme.
Browsers are easy to overwhelm with too much data because they often keep references to all the exchanges since the page was loaded (or refreshed).
This is especially true when logging events to a browser's console.
C Language TCP server/client.. I want to assign a new socket for a particular client which requested my server from 8080 lets say the new socket is 8081 to get further request, and want to free the previous socket(8080) so that the other clients will request my server from 8080. is there any way of doing it in C language. (OS Ubuntu) Thanks
Your problem statement is incorrect. You can't do this even if you wanted to. The way that TCP sockets work is that accept() gives you a new socket for the incoming client connection, on the same port you are listening to. That's all you need and it's all you can get. You can't 'allocate a new socket' to the client on a new port without engaging in another TCP handshake with him, which would be nothing but a complete waste of time when you already have a connection to him. This does not preclude another connection bring accepted while this one is open. You need to read a TCP Sockets networking tutorial.
Mat and EJP have said the pertinent things above, but I thought it might help others to describe the situation more verbosely.
A TCP/IP connection is identified by a four-tuple: target IP address, target TCP port number, source IP address, and source TCP port number. The kernel will keep track of established connections based on these four things. A single server port (and IP address) can be connected to thousands of clients at the same time, limited in practice only by the resources available.
When you have a listening TCP socket, it is bound to some IP address (or wildcard address) and TCP port. Such a socket does not receive data, only new connections. When accept() is called, the server notes the new four-tuple of the connection, and hands off the file descriptor that represents that connection (as the accept() return value). The original socket is free to accept new connections. Heck, you can even have more than one thread accepting new connections if you want to, although establishing new connections in Linux is so fast you shouldn't bother; it's just too insignificant to worry about.
If establishing the connection at application level is resource-intensive -- this is true for for example encrypted connections, where agreeing to an encryption scheme and preparing the data structures needed takes typically several orders of magnitude more CPU resources than a simple TCP connection --, then it is natural to wish to avoid that overhead. Let's assume this is the point in OP's question: to avoid unnecessary application-level connection establishment when a recent client needs another connection.
The preferred solution is connection multiplexing. Simply put, the application-level protocol is designed to allow multiple data streams via a single TCP connection.
The OP noted that it would be necessary/preferable to keep the existing application protocol intact, i.e. that the optimization should be completely on the server side, transparent to the clients.
This turns the recommended solution to a completely new direction. We should not talk about application protocols, but how to efficiently implement the existing one.
Before we get to that, let's take a small detour.
Technically, it is possible to use the kernel packet filtering facilities to modify incoming packets to use a different port based on the source IP address, redirecting requests from specific IP addresses to separate ports, and making those separate ports otherwise inaccessible. Technically possible, but quite complex to implement, and with very questionable benefits.
So, let's ignore the direction OP assumed would bring the desired benefits, and look at the alternatives. Or, actually, the common approach used.
Structurally, your application has
- A piece of code accepting new connections
- A piece of code establishing the application-level resources needed for that connection
- A piece of code doing the communication with the client (serving the response to the client, per the client's request)
There is no reason for these three pieces to be consecutive, or even part of the same code flow. Use data structures to your advantage.
Instead of treating new incoming connections (accept()ed) as equal, they can be simply thrown into separate pools based on their source IP addresses. (Or, if you are up to it, have a data structure which clusters source IP addresses together, but otherwise keeps them in the order they were received.)
Whenever a worker completes a request by a client, it checks if that same client has new incoming connections. If yes, it can avoid most if not all of the application-level connection establishment by checking that the new connection matches the application-level parameters of the old one. (You see, it is possible that even if the source IP address is the same, it could be a completely different client, for example if the clients are under the same VPN or NATted subnet.)
There are quite a few warts to take care of, for example how to keep the priorities, and avoid starving new IP addresses if known clients try to hog the service.
For protocols like HTTP, where the client sends the request information as soon as the server accepts the connection, there is an even better pattern to apply: instead of connection pools, have request pools. A single thread or a thread pool can receive the requests (they may span multiple packets in most protocols), without acting on them; only detecting when the request itself is complete. (A careful server will limit the number of pending requests, and the number of incomplete request, to avoid vulnerability to DOS.)
When the requests are complete, they are grouped, so that the same "worker" who serves one request, can serve another similar request with minimal overhead. Again, some careful thought is needed to avoid the situation where a prolific client hogs the server resources by sending a lot of requests, but it's nothing some careful thought and testing won't resolve.
One question remains:
Do you need to do this?
I'd wager you do not. Apache, which is one of the best HTTP servers, does not do any of the above. The performance benefits are not considered worth the extra code complexity. Could you write a new HTTP server (or a server for whatever protocol you're working with), and use a scheme similar to above, to make sure you can use your hardware as efficiently as possible? Sure. You don't even need to be a wizard, just do some research and careful planning, and avoid getting caught in minute details, keeping the big picture in mind at all times.
I firmly believe that code maintainability and security is more important than efficiency, especially when writing an initial implementation. The information gained from the first implementation has thus far always changed how I perceive the actual "problem"; similar to opening new eyes. It has always been worth it to create a robust, easy to develop and maintain, but not necessarily terribly efficient implementation, for the first generation. If there is someone willing to support the development of the next generation, you not only have the first generation implementation to compare (and verify and debug) against, but also all the practical knowledge gained.
That is also the reason old hands warn so often against premature optimization. In short, you end up optimizing resource waste and personal pain, not the implementation you're developing.
If I may, I'd recommend the OP back up a few steps, and actually describe what they intend to implement, what the observed problem with the implementation is, and suggestions on how to fix and avoid the problem. The current question is like asking how to better freeze a banana, as it keeps shattering when you hammer nails with it.
I am developing a server application that needs to send a lot of data to the client. However, client can get disconnected at any time and send()/write() on socket will return an error in this case. I would like to check how much data has been actually delivered before a client get disconnected to be able to continue sending data from the place where it left off when the client reconnect.
Is it possible to check it using sockets API?
No, the sockets API does not give you this information. In fact, it is not possible in general to know this. Depending on the particular way in which the connection failed, the TCP stack on one side generally can't know how much data successfully made it to the other side. The only thing it can know is how much data was acknowledged, which is not the same thing. And considering that other things than TCP/IP might have failed (the local OS, the remote OS, the remote process, the remote application logic), the amount of data that has been acknowledged at the TCP level probably doesn't mean much anyway.
You need to use an end-to-end application protocol to have the remote end acknowledge the data it has received and successfully processed (and committed, if applicable).
Recently I've been doing research on using C language to do network socket programming. I'm wondering if I write a program to listen for incoming data, is this "before" or "after" the firewall?
What I understand is a web server like nginx, lighttpd or cherokee uses socket programming to listen for data, and yet I can set up a firewall like (OpenBSD's) "pf" to control the incoming data, so it seems like socket probgramming in C is "after" the firewall.
BUT, if that is true, then how are firewalls written? How do they listen to incoming data from specific ports?
Firewalls are implemented by the kernel, in a different part of the networking code. Essentially it amounts to a selection of "hooks" (which can be accessed either via kernel space or user space or both) that notify upon activity.
That activity can either be incoming (ingress) or outgoing (egress) depending upon who originated the packet. For each packet and usually each connection for stateful, connection oriented protocols the firewall is given an opportunity to re-write, veto (e.g. return an error) or simply silently drop a given packet or connection. (Implementations vary and the actions available can be more complex).
The key thing is that the interfaces is quite different to the normal sockets interface - you're told that things are happening and asked what you'd like to do in relation to that, but you're not given the same accept/listen/connect style interface that's normally used for sockets programming.
On Linux for example the firewall is implemented as the input/output "filter" boxes in this packet flow diagram, whereas your sockets code happens in the red layer at the top labelled "protocol/application layer"