I have few questions as below regarding Hiredis for C:
I read somewhere that hiredis does not do connection pooling. Is there any library available for hiredis which does connection pooling or may be an alternative to hiredis which has connection pooling.
For other question I'll give the background below:
In the code I plan to use the same context through out until the application stops, don't know is this the correct way and what are the complications of doing this. Please correct me if I am doing anything wrong. The way I want to do is, the app forks few child once started. I will create the context in the child initialization and use it for all the requests the process serves. Also, I can check the reply of redisCommand and try to reconnect as:
reply = redisCommand(c,"set %s %s","key","value");
if(reply == NULL)
{
printf("Connection error.\n");
c = redisConnect(hostname,port);
// checks
}
**Q:**How can I check if context is still valid, means the connection is not lost? I see REDIS_CONNECTED in hiredis.h but not sure this is what can tell this. I printed this variable after creating context which printed 2 and then I stopped the DB but still it printed 2 and program segfaulted. Does hiredis handles reconnection.
EDIT:
I am no expert in C, I would appreciate if I can get some useful links from where I can learn how to create and use connection pools in C.
Thanks
Related
Hello and good evening,
I am writing a client & server pair of programs in C. Right now I got both of them running and by providing access to ports on my server through router settings, I can establish local and global connections for a barebones chat back and forth. Here is what both programs do until now.
Client:
create a socket
create an sockaddr_in (port and IP address)
connect the socket with the sockaddr_in to the server
go into a loop for read / write to the server, one at a time
Server:
create a socket
create a sockaddr_in (port and IP address)
bind the socket to sockaddr_in
listen to request on the socket
accept a request
go into a loop for read / write to the server, one at a time
My question:
What do I have to consider for handling and error-handling these individual steps correctly?
What I got until now is code like this:
printf("trying to connect to: IPv4 <%s> port <%s> ...", argv[1], argv[2]);
if(connect(net_socket, (struct sockaddr *)&server_address, sizeof server_address) == -1)
{
fprintf(stderr, "\ntcp connection failed.\n");
exit(4);
}
printf(" success.\nconnection to server: IPv4 <%s> port <%s>\n", argv[1], argv[2]);
Code like this is wrapped around all the steps these programs do from socket creation to connect and accept. And they work given the right input.
I want to scale my programs:
simultaneous read and write for both sides
multiple clients to server instead of client and server
correct data encapsulation
etc.
For that I need to know how to do the backbone correct and I want to get the basics straight before I start multithreading, forking, selecting or building whatnot around the core.
I don't want to overscale this post, hence I will only post my code if it is requested at all. Thank you very much for reading and replying in advance.
First off, it's always good practice to wrap all error/loging functions so they can be modified for later scaling. I do this so, later I can add file/line to my errors.
void perr(char *message)
{
fprintf(stderr, message);
}
On to your question. It's a very practice-rich answer. (Better asked on this board. But I'll help you anyways.
You're right to have the backbone as solid as possible before expanding. The best practice for a well made application is to have it very 'flat' in scope. (not to many if's in if's). Doing this will also provide you with a better understanding of the flow of code and more atomic error reporting:
main()
{
if(!(soc = createsocket(...)) return 1;
if(!bindsocket(soc))
{
deletesocket(soc);
return 1;
}
// Main server loop
while(req = accept_request(soc))
{
proc_request(req);
}
plog("No longer accepting request");
return 0;
}
Notice how I have no error reporting in the driver function (main in this example). This is because all error reporting is handled INSIDE of the called functions (createsocket(), bindsocket(), ect).
The driver function is only concerned if the called functions succeeded or not. The details of each failure are best described in the function who's arguments lead to the failure.
Here are my tips:
Avoid exit(), your code needs to be ready to use return all the way up to main() if something goes wrong. There are execptions (threads and child procs, but you get the idea)
Good programs are good at failing
For a server/client relationship, the server is typically single threaded and only loops through all client requests (computers are faster than you think, unless you're expecting around 5k requests).
State Machines are extremely useful in your situation. Think of the states your program is going to be in between start and stop, lay out what methods need to be in each state and have an overall plan what happens with errors in each state. (ie, opened, establishing, running, stopping)
Context
Debian64bits.
Thought I understood socket implications but not.
Worried about the management of slow clients.
Read and fiddled with that code epoll edge triggered
Imagine two clients:
A: very slow network
B: very fast network.
Question
In edge triggered mode what would happen when we start reading from A and then B also sends some datas to read while we are in the read loop for A?
Will B have to wait for us to read all the datas coming from A?
Or should we create a buffer (not in the example given) to store A datas and concatenate those until the message is complete and return immediately?
Or is that buffer already managed by epoll?
The underlying is not clear to me..
It doesn't matter if the network is slow, the reading part will stop whenever it has read all data that has reached the server so far.
There is a bug (or at least a portability issue) in it though. The code only checks for EAGAIN when reading data. It should also check for EWOULDBLOCK, since some platforms return that error code when all data has been read.
Edit:
Ok, so if i want to put the incoming datas into a database, how could i manage it? Will i be able to read the whole datas?
If you get an incomplete message you would need to store it until the entire message has been received. I would malloc a char* for each incoming connection with a message fragment.
When a message is complete, you can process it, store it in the database or whatever you desire.
Note that the beginning of the next message may already be available, so you can't stop reading until you get EWOULDBLOCK or EAGAIN.
Another way to treat the problem is to create a separate thread for each connection.
Which approcah is best? Well, that would depend on the application (number of simultaneous users etc).
I am totally new to socket programming and I want to program a combined TCP/UDP-Server socket in C but I don't know how to combine those two.
So at the moment, I do know how TCP- and UDP-Server/-Clients work and I have already coded the Clients for TCP and UDP. I also know that I have to use the select()-function somehow, but I don't know how to do it.
I have to read two numbers, which are sent to the TCP-/UDP-Server with either TCP- or UDP-Clients and then do some calculations with these numbers and then print the result on the server.
Does anyone know a tutorial for that or an example code or can help me with that?
Or at least a good explanation of the select() function.
Basically, use an event loop. It works like this:
Is there anything I need to do now? If so, do it.
Compute how long until I next need to do something.
Call select specifying all sockets I'm willing to read from in the read set and all sockets I'm trying to write to in the write set.
If we discovered any sockets that are ready for reading, read from them.
If we discovered any sockets that are ready from writing, try to write to them. If we wrote everything we need to write, remove them from the write set.
Go to step 1.
Generally, to write to a socket, you follow this logic:
Am I already trying to write to this socket? If so, just add this to the queue and we're done.
Try to write the data to the socket. If we sent it all, we're done.
Save the leftover in the queue and add this socket to our write set.
Three things to keep in mind:
You must set all sockets non-blocking.
Make sure to copy your file descriptor sets before you pass them to select because select modifies them.
For TCP connections, you will probably need your own write queue.
The idea is to mix inside your server a TCP part and a UDP part.
Then you multiplex the inputs. You could use the old select(2) multiplexing call, but it has limitations (google for C10K problem). Using the poll(2)
multiplexing call is preferable.
You may want to use some event loop libraries, like libev (which uses select or poll or some fancier mechanisms like epoll). BTW, graphical toolkits (e.g. GTK or Qt) also provide their own even loop machinery.
Read some good Linux programming book like the Advanced Linux Programming
book (available online) which has good chapters about multiplexing syscalls and event loops. These are too complex to be explained well in a few minutes in such an answer. Books explain them better.
1) Simple write a tcp/udp server code, and when receive the message, just print it out.
2) substitute print code to process_message() function.
Then you have successfully combine TCP and UDP server to the same procedure.
Be careful with your handling procedure, it's should be cope with parellel execution.
You may try this stream_route_handler, it is c/c++ application, you can add tcp/udp handler in your single c/c++ application. This has been using by transportation heavy traffic route, and logging service purpose.
Example of using
void read_data(srh_request_t *req);
void read_data(srh_request_t *req) {
char *a = "CAUSE ERROR FREE INVALID";
if (strncmp( (char*)req->in_buff->start, "ERROR", 5) == 0) {
free(a);
}
// printf("%d, %.*s\n", i++, (int) (req->in_buff->end - req->in_buff->start), req->in_buff->start);
srh_write_output_buffer_l(req, req->in_buff->start, (req->in_buff->end - req->in_buff->start));
// printf("%d, %.*s\n", i++, (int) (req->out_buff->end - req->out_buff->start), req->out_buff->start);
}
int main(void) {
srh_instance_t * instance = srh_create_routing_instance(24, NULL, NULL);
srh_add_udp_fd(instance, 12345, read_data, 1024);
srh_add_tcp_fd(instance, 3232, read_data, 64);
srh_start(instance);
return 0;
}
If you are using C++ program, you may like this sample code.
stream route with spdlog
i am learning to use SO_SNDTIMEO and SO_RCVTIMEO to check the timeout.
It is easy to use with read socket. But when i want to check write timeout, it always return successful. Here is what i did:(all in blocking mode)
close the client read socket and exit before server start write
terminate the client before server start write
unplug the cable of server after accept but before write
well, it seems all these case write just return sucessfully.
I think the reason should be that port is resource managed by os, and at the client side, after program gone, the tcp connection still shows FIN_WAIT2 state.
so, is there any convenient way to simulate some cases that write can receive errors such as EPIPE, EAGAIN?
How to get the error EAGAIN?
To get the error EAGAIN, you need to be using Non-Blocking Sockets. With Non-Blocking sockets, you need to write huge amounts of data (and stop receiving data on the peer side), so that your internal TCP buffer gets filled and returns this error.
How to get the error EPIPE?
To get the error EPIPE, you need to send large amount of data after closing the socket on the peer side. You can get more info about EPIPE error from this SO Link. I had asked a question about Broken Pipe Error in the link provided and the accepted answer gives a detailed explanation. It is important to note that to get EPIPE error you should have set the flags parameter of send to MSG_NOSIGNAL. Without that, an abnormal send can generate SIGPIPE signal.
Additional Note
Please note that it is difficult to simulate a write failure, as TCP generally stores the data that you are trying to write into it's internal buffer. So, if the internal buffer has sufficient space, then you won't get an error immediately. The best way is to try to write huge amounts of data. You can also try setting a smaller buffer size for send by using setsockopt function with SO_SNDBUF option
You can simulate errors using fault injection. For example, libfiu is a fault injection library that comes with an example project that allows you to simulate errors from POSIX functions. Basically it uses LD_PRELOAD to inject a wrapper around the regular system calls (including write), and then the wrapper can be configured to either pass through to the real system call, or return whatever error you like.
You could set the receive buffer size to be really small on one side, and send a large buffer on the other. Or on the one side set the send buffer small and try to send a large message.
Otherwise the most common test (I think) is to let the server and client talk for a while, and then remove a network cable.
I'm learning my way about socket programming in C (referring to Beej).
Here is a simple multi-user chat server i'm trying to implement:
http://pastebin.com/gDzd0WqP
On runtime, it gives Bus Error. It's coming from the lines 68-78.
Help me trace the source of the problem?
in fact, WHY is my code even REACHING that particular region? I've just run the server. no clients have connected.. :#
ps - i know my code is highly unreliable (no error checks anywhere), but i WILL do that at a later stage, i just want to TEST the functionality of the code before implementing it in all it's glory ;)
line 81
msg[MSG_SIZE] = '\0';`
overruns your buffer. Make it
msg[MSG_SIZE - 1] = '\0';`
You also need to check the return value of all the calls that can fail, that's line 39,42,45,68 and 80
Edit: And if you'd checked for errors, likely you'd seen the accept() call fail, likely due to the socket not being in listen mode - that is, you're missing a call to listen()
Another thing to consider is that you can't necessarily copy fd_set variables by simple assignment. The only portable way to handle them is to regenerate the fd_set from scratch by looping over a list of active file descriptors each time.