I'm currently working on a distributed networking project for some networking practice and the idea is to send a file from my server to a few different clients (after breaking up the file) and the clients will find the frequency of a string and return it back.
The problem I'm running into is how to identify each client and send data to each one.
The solution I've been working on to identify each client by their port. The problem arises as to how I handle multiple connections and ports. I know I have to use send() to send the data to a port once I open a connection and etc. but I have no idea how to do this across multiple connections ( I can do this with a single client and server but not with multiple clients)
Does anyone have any suggestions from a high level standpoint? I got one suggestion from a friend who said:
Open a socket
Listen for connections
When a connection request is received, spawn a new thread to handle the connection.
The main process will go back to step 2 to listen for new connections, while the new thread
will handle all data flow with the associated client.
But I'm not really sure I understand this... I've also been referencing http://shoe.bocks.com/net/#socket
Thanks
Your friend is correct. Follow first three steps (mentioned by him) and then you need to:
After spawning thread, send data (read from file) to new socket.
Once entire file is finished, you should disconnect and exit thread. On client side, you should handle disconnect and probably exit.
NOTES:
Also, you can use sendfile() instead of send() if you wish. You can use select() if you wish to handle all connections without spawning threads.
Refer http://beej.us/guide/bgnet/ for details.
EDIT:
how to identify each client? Ans: This is classical port discovery problem but in your case its simple. Server should be listening on well known port (say 12345) and all the clients will connect to it. Once they are connected, server has all sockfds. You need to use these sockfds to send data and identify them.
If you check out networkComms.net, an open source network communication library, once you have created a connection with a client you can keep track of that specific client by looking at it's NetworkIdentifier tag, a guid unique to each client.
If you will be sending large files to all of your clients also check out the included DistributedFileSystem which is specifically designed for that purpose.
Related
Context: this is a web/sqlite application. One process receives new data over TCP, and feed them to a SQLite database. Other processes (number is variable) are launched as required as clients connect and request updates over HTML5's server-side events interface (this might change to websocket in the future).
The idea is to force the client apps to block, and to find a way for the server to create a notification that will wakeup all awaiting clients.
Note that the clients aren't fork'ed from the server.
I'm hoping for a solution that:
doesn't require clients to register themselves to the server
allows the server to broadcast even if no client is listening - and doesn't create a huge pile of unprocessed notifications
allows clients to detect that server isn't present
allows clients to define a custom timeout (maximum wait time for an event)
Solutions checked:
sqlite3_update_hook() - only works within a single process (damned, that would have been sleek)
signals: I still have nightmares about the last time I used signals. Maybe signalfd would be better (server creates a folder, client create unique files, and server notifies all files in that folder)
iNotify - didn't read enough on this one
semaphores / locks / shared memory - can't think of a non-hacked way to use these. The server could update a shared memory area with the row ID of the line just inserted in the DB, but then what?
I'm sure I'm missing something obvious - but what? At this time, polling seems to be the best option!
Thanks.
Just as a suggestion can you try message queues? multiple clients can connect to the same queue and receive one broadcast message, each client can have its own message queue if it requires communication with the server.
Message queues are implemented by Linux OS and they are very reliable. I personally use message queues to pass messages from several clients to a central routing daemon, clients being responsible of processing and returning the altered data.
Firstly, please excuse my probable butchering the technical terminology. I've been thrown into socket IO with little formal education and I know that I am bungling words left and right.
I'm trying to build a client and server in C that enables multiple clients to connect to one another. The general procedure goes something like this:
1) Server has one port that is constantly listening and accepting connections
2) A client connects on that port
3) Server creates a new socket (same address, different port number), tells the client to connect to that socket, and closes the connection with the client.
4) The client connects to the designated socket and provides the server with a channel it would like to be on
5) Server places that socket on the designated channel
6) Repeat steps 2 through 5 for each client that connects to the server
/* all of the above has been coded already */
7) Once a channel has 2 or more members, I'd like to have each member port be able to broadcast to all other ports in the same channel (and thus the clients communicate with each other)
In this situation, all involved sockets on the same channel have the same address and DIFFERENT port numbers. Everything I've read and researched about broadcasting and multicasting revolves around each communicator having the same port number and different addresses.
Is there a way to do the communication that I'm hoping to do, in C?
I would think you want to use the listen() and accept() functions for TCP. You can do what you describe and have clients talk to each other, but all traffic will run through the server as a hub.
If you want all clients to be able to talk to every other client, you have a few options:
Server is the hub for all data and passes it between clients for you
Clients maintain direct connections to the other clients and pass data to each other in order to facilitate the hub. This means lots of data copying.
Broadcast or multicast (UDP). This is only possible over a local network, as internet routers will block multicast and broadcast traffic.
I would probably go with #1.
Remember that each client has it's own IP address, so for a client to communicate with another client, and not involving the server, it would need to open a new connection with the other client, send data and then close the connection. While doable, I do not think this idea would scale very well.
I do agree with Syplex that having the server act as a relay hub is probably the best, and certainly has the potential to scale well. So the data-flow would be something like this:
a client receives a message that is to be retransmitted to all the other clients.
this message is passed to all other instances of your server process
each of these instances of your server process sends out the message.
The issue becomes how you are implementing you server, and you do have two models that fit what you describe:
(1) you are using a multi-treaded server, in which each new connection causes a thread to be spawned to handle the communication between the client and the server.
(2) you are using a forking server, in which the server forks a new process to communicate with the client.
In case (1) you would be interested in intra-process communication (message queue for example) while in case (2) you would be interested in inter-process communication (named pipes or shared memory for example).
At this point there are two many variables to give a concise answer. I hope this helps gets you started and at least gives you somewhere to start looking.
I want two functionalities to be implemented on my udp server application.
Creating thread that continuously receives data coming from any client.
Creating a thread that continuously sends data on server socket after specific time period and waits for reply from client. (I implemented this to make aure that whenever any client goes down, the data is not received back from client and server comes to know that client is down.)
Now, the problem I am facing is that since two threads are sharing same connected socket, whenever both threads try to access this socket simultaneously, a deadlock is established.
One of the solution I found was to create two sockets. One that continuously receives data, and the other socket that is meant for sending data from server time to time to clients and wait for their response, but since Server can must be bind()ed and I have bind()ed my socket to INADDR_ANY once, how would I create a separate socket for sending data from server and waiting for replies from client.
Please help me with this complication.
Also do let me know if there is some other better way of its implementation.
Thanks in advance :)
You will have to use non-blocking net functions and use a mutex to ensure no two threads access the socket at once.
A single thread may, however, be enough, if you use non-blocking functions. Using many threads will probably not improve performance, but may make the code more readable.
I am writing a server application that simply connects a local serial port to multiple network connected clients. I am using linux and C for the server application because the equipment for the program is a router with limited memory.
I have everything setup for multiple clients to connect and send data to the serial port using a fork() process for each connection.
My problem lies in getting data incoming on the serial port out to the multiple (varing number) client connections. my problem lies in designing a way for each active socket to get all of the incoming data, and to only get it once. Any help?
Sounds like you need a data queue (buffer) for each connected client. Each time data comes in on the port, you post it to the back of each client's queue. The clients then read the data from the front of their respective queues. Since all the clients will probably read at different rates/times, this will ensure all of them get a copy of the data only once, and you won't get hung up waiting for any one client while more data comes in. Of course, you'll need to allocate a certain amount of memory for each connected client's queue (I'm not sure how many clients you're expecting, and you did say your available memory is limited), and you need to consider what to do if a queue gets full before the client reads all of it.
Presumably you keep a list or some other reference of/to connected clients, why not just loop over that for each bit of information and send it to all of them?
A thread per socket design might not be the best way to solve this. An event driven asynchronous approach should be a much better fit. However, if you must do it with threads, and given that serial ports are slow anyway, building a pipe between the thread listening to the serial port and all the threads talking to the network clients is the most practical. You could do fancy things with rwlocks to move the data, but you'll still need a way for the network threads to wait on both the socket and the data from the serial port, so you need to use file descriptors for both and something like poll.
But seriously, this would likely be much easier and would perform better without the threads. Think of it as a main loop which waits on poll which is watching the network and the serial port, determines which event occurred, and distributes data accordingly. It should be easier all around once you get the idea.
I've been working on a complex server-client system in C and I'm not sure how to implement the socket communication.
In a nutshell, the system is a server application which communicates with a database and uses a UNIX socket to communicate with one or more child processes created with fork(). The purpose of the children is to run game servers. The process of launching a game server is like this:
The server/"manager" identifies a game server in the database that is to be made. (Assume database communication is already sorted.)
The manager forks a child (the "game controller").
The game controller sets up two pipe pairs, then forks, replacing its child's stdin with a pipe, and it's stdout and stderr with another pipe.
The game controller's child then runs execlp() to begin running the actual game server executable.
My experience with sockets is fairly minimal. I have used select() on a server application before to 'multiplex' numerous clients, as demonstrated by the simple example in the GNU C documentation here.
I now have a new challenge, as the system must be able to do more: the manager needs to be able to arbitrarily send commands to the game controller children (that it will find by periodically checking the database) and get replies, but also expect incoming arbitrary commands/errors from them and send replies back.
So, I need a sort-of "context" system, where sockets are meaningful only between themselves. In other words, when a command is sent from the manager to the game controller, each party needs to be aware of who is asking and know what the reply is (and, therefore, which command it is a reply to).
Because select() is only useful for knowing when we have incoming data, and a thread should block on it, would I need another thread that sends data and gets the replies? Will this require each game controller, although technically a 'client', to use a listening socket and use select() as well?
I hope I've explained the system and the problem concisely; I will add more detail if required. Thanks!
Ok, I am still not really sure I understand exactly where your trouble is, so I will just spout off some things about writing a client/server app. If I am off track, just let me know.
The way that the server will know which clients corresponds to which socket is that the clients will tell the server. Essentially, you need to have a log-in protocol. When the game controller connects to the server, it will send a message that says "Hi, i am registering as controller foo1 on host xyz, port abc..." and whatever else the server needs to know about its clients. The server will keep a data structure that maps sockets to client metadata, state, etc. Whenever it gets a new message, it can easily map from the incoming host/port to its metadata. Or your protocol can require that on each incoming message, the will client send the name it registered with as a field.
Handling the request/response can be done several ways. First lets deal with the networking part of it on the server side. One way to manage this, as you mentioned, is by using select (or poll, or epoll) to multiplex the sockets. This is actually usually considered the more complicated way to do things. Another way is to spawn off a thread (or fork a process, which is less common these days) for each incoming client. Each spawned thread can read its own assigned socket, responding to messages one at a time without worrying about the fact that there are other clients besides the own it is dealing with. This simple one to one thread to socket model breaks down if there are many clients, but if that is not the case, then it is worth consideration.
Part 2 really covers only the client sending the server a message, and the server replying. What happens when the server wants to initiate communication? How does it do it and how does the client handle it? Also, how do you model the model the communication at the application level, meaning assuming we have the read/write part down, how do we know what to send? You will probably want to model things in terms of state machines. There is also a lot more to deal with like what happens when a client crashes? What about when the server crashes? Also, what if you really have your heart set of using select, perhaps because you expect many client? I will try to add more to this answer tomorrow.