I m developing a C application running on linux system (kernel 3.4.11)
In my application, I m opening a server socket on a thread. And I m opening a fork() in which I execute a shell command with execvp() in the main thread.
Opening a fork() will inherit the socket file descriptor in the child. And this could cause problems according to many topics on the net. In my case If I close my application I can see with netstat that the socket is assigned to another daemon (another random daemon).
In fact there is many solutions for a such issue:
1) close the socket on the beginning of the fork() child:
if ((pid = fork()) == -1)
return -1;
if (pid == 0) {
/* child */
close(socket_desc);
2) Use fcntl() and FD_CLOEXEC when opening the socket in the parent
socket_desc = socket(AF_INET , SOCK_STREAM , 0);
fcntl(socket_desc, F_SETFD, fcntl(fd, F_GETFD) | FD_CLOEXEC);
3) Use O_CLOEXEC in the socket() function:
socket_desc = socket(AF_INET , SOCK_STREAM|O_CLOEXEC , 0);
What's the best solution? and why?
Any other better solution is welcome.
If you control all the code, these make no difference.
Use the first solution if someone else gives you a socket you don't know how it was created.
Use the second, if you write a module to which someone else transmit you a socket, and you want to pass it to another module you don't control but you suspect to fork()/exec().
Use the third if you are the creator of the socket and need to pass it to another module that you suspect to fork()/exec().
Related
for (int t = 0; t < physicalCoreCount; t++) {
int pid = fork();
if (pid==0) {
// setting up epoll
epoll_fd = epoll_create1(0);
event.data.fd = listenSocketfd;
event.events = EPOLLIN | EPOLLET;
ret = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, listenSocketfd, &event);
events = (epoll_event*)calloc(LISTENQ, sizeof(event));
//*****/
while (1) {
int ndfs = epoll_wait(epoll_fd, events, curfdCount, -1);
if (ndfs==-1) {
continue;
}
for (int i=0; i < ndfs; i++) {
if (events[i].data.fd == listenSocketfd) { // original listener
int new_connfd = accept(events[i].data.fd, (sockaddr*)&clientaddr, &clientlen);
if (new_connfd==-1) {
if (errno==EAGAIN || errno==EWOULDBLOCK) {
continue;
}
else exitPerrorLog("accept()");
}
set_non_block(new_connfd);
event.events = EPOLLIN | EPOLLET;
event.data.fd = new_connfd;
if (epoll_ctl(epoll_fd, EPOLL_CTL_ADD, new_connfd, &event) < 0)
exitPerrorLog("epoll_ctl");
clientaddrOfFd[new_connfd] = new sockaddr_in();
memcpy(clientaddrOfFd[new_connfd], &clientaddr, sizeof(clientaddr));
curfdCount++;
}
else {
process(events[i].data.fd, clientaddrOfFd[events[i].data.fd]);
//epoll_ctl(epoll_fd, EPOLL_CTL_DEL, events[i].data.fd, &event);
if (curfdCount > 10000) curfdCount = 10000; //curfdCount--;
}
}
}
}
}
The problem appears because I'm trying to implement persistent connection (not closing after response). However, child 0 can accept() a socket file descriptor X, process it, but later child 1 can accept() the same file descriptor. Since I didn't close the connection on child 0 (to implement HTTP1.1 keep-alive), now 2 children are reading/writing to the same file descriptor.
What's a way to prevent this problem? Thank you.
Edit: Important update: so the main problem is that I thought 2 same FD means it's the same connection, I want to avoid that since it would lead to 2 children reading/writing the same thing. If that situation doesn't happen (read/write overlap) then I think the question is solved. Can someone confirm?
You are creating multiple epoll instances and registering for edge-triggered events on the listening socket in each one. Naturally, you will get an event from each one when a new connection becomes available to accept. However, only one process can successfully accept each connection. As observed in comments, two different children might accept connections that are assigned the same file descriptor number in the respective processes, but that doesn't mean they refer to the same socket.
You have several options, but prominent among them are:
use a single epoll instance, shared by all the processes. You can get this automatically by having the parent create it before forking off any of the children. In this case, only one child will receive each edge-triggered event. Of course, if the children intend to register for events that should be received only by them, then this isn't going to work very well.
just accept (no pun intended) that multiple processes will receive events when a connection becomes available, and deal with it. That appears to be what you're doing now (by ignoring EAGAIN and EWOULDBLOCK errors from accept()), and I see no particular reason why you shouldn't keep doing it.
I have a multi-threaded application and have got a way to do a telnet, ssh on to this application. In my application, I do one of the init script restart using the custom system() call below. It seems like, the child process is still active. I am saying this because If I logout from telnet session still the process hangs i.e. it cannot logout. This happens only when I restart the script using this system call. Is there something wrong with my system() function?
int system(const char *command)
{
int wait_val, pid;
struct sigaction sa, save_quit, save_int;
sigset_t save_mask;
syslog(LOG_ERR,"SJ.. calling this system function\r\n");
if (command == 0)
return 1;
memset(&sa, 0, sizeof(sa));
sa.sa_handler = SIG_IGN;
/* __sigemptyset(&sa.sa_mask); - done by memset() */
/* sa.sa_flags = 0; - done by memset() */
sigaction(SIGQUIT, &sa, &save_quit);
sigaction(SIGINT, &sa, &save_int);
__sigaddset(&sa.sa_mask, SIGCHLD);
sigprocmask(SIG_BLOCK, &sa.sa_mask, &save_mask);
if ((pid = vfork()) < 0) {
perror("vfork fails: ");
wait_val = -1;
goto out;
}
if (pid == 0) {
sigaction(SIGQUIT, &save_quit, NULL);
sigaction(SIGINT, &save_int, NULL);
sigprocmask(SIG_SETMASK, &save_mask, NULL);
struct sched_param param;
param.sched_priority = 0;
sched_setscheduler(0, SCHED_OTHER, ¶m);
setpriority(PRIO_PROCESS, 0, 5);
execl("/bin/sh", "sh", "-c", command, (char *) 0);
_exit(127);
}
#if 0
__printf("Waiting for child %d\n", pid);
#endif
if (wait4(pid, &wait_val, 0, 0) == -1)
wait_val = -1;
out:
sigaction(SIGQUIT, &save_quit, NULL);
sigaction(SIGINT, &save_int, NULL);
sigprocmask(SIG_SETMASK, &save_mask, NULL);
return wait_val;
}
Any ideas on how to debug whether this system call is getting hanged or not?
I realized this happens because file descriptors are inherited upon fork .
Since my custom system() is nothing but fork() and exec(). There are plenty of sockets in my application. These socket file descriptors gets inherited by the child process.
My assumption here is that "Child process can't exit because it is waiting for parent process to close the file descriptors or those file descriptors are in a state where it can be closed". Not sure what those states are though.
So, here is the interesting link I found -
Call system() inside forked (child) process, when parent process has many threads, sockets and IPC
Solution -
linux fork: prevent file descriptors inheritance
Not sure, I can do this in a big application where sockets are opened at thousand of places. So, here is what I did.
My Solution -
I created a separate process/daemon that listens for the command from the parent application. This communication is based on socket. Since, it is a separate application/daemon it doesn't affect the main application which is running multiple threads and has a lot of opened sockets. This worked for me.
I believe that this problem will be fixed once I do -
fcntl(fd, F_SETFD, FD_CLOEXEC);
Any comments are welcome here.
Is this a fundamental problem in Linux, C i.e.
all file descriptors are inherited by default?
Why linux/kernel allow this? What advantage do we get out of it?
Is it possible to bind and listen to multiple ports in Linux in one application?
For each port that you want to listen to, you:
Create a separate socket with socket.
Bind it to the appropriate port with bind.
Call listen on the socket so that it's set up with a listen queue.
At that point, your program is listening on multiple sockets. In order to accept connections on those sockets, you need to know which socket a client is connecting to. That's where select comes in. As it happens, I have code that does exactly this sitting around, so here's a complete tested example of waiting for connections on multiple sockets and returning the file descriptor of a connection. The remote address is returned in additional parameters (the buffer must be provided by the caller, just like accept).
(socket_type here is a typedef for int on Linux systems, and INVALID_SOCKET is -1. Those are there because this code has been ported to Windows as well.)
socket_type
network_accept_any(socket_type fds[], unsigned int count,
struct sockaddr *addr, socklen_t *addrlen)
{
fd_set readfds;
socket_type maxfd, fd;
unsigned int i;
int status;
FD_ZERO(&readfds);
maxfd = -1;
for (i = 0; i < count; i++) {
FD_SET(fds[i], &readfds);
if (fds[i] > maxfd)
maxfd = fds[i];
}
status = select(maxfd + 1, &readfds, NULL, NULL, NULL);
if (status < 0)
return INVALID_SOCKET;
fd = INVALID_SOCKET;
for (i = 0; i < count; i++)
if (FD_ISSET(fds[i], &readfds)) {
fd = fds[i];
break;
}
if (fd == INVALID_SOCKET)
return INVALID_SOCKET;
else
return accept(fd, addr, addrlen);
}
This code doesn't tell the caller which port the client connected to, but you could easily add an int * parameter that would get the file descriptor that saw the incoming connection.
You only bind() to a single socket, then listen() and accept() -- the socket for the bind is for the server, the fd from the accept() is for the client. You do your select on the latter looking for any client socket that has data pending on the input.
In such a situation, you may be interested by libevent. It will do the work of the select() for you, probably using a much better interface such as epoll().
The huge drawback with select() is the use of the FD_... macros that limit the socket number to the maximum number of bits in the fd_set variable (from about 100 to 256). If you have a small server with 2 or 3 connections, you'll be fine. If you intend to work on a much larger server, then the fd_set could easily get overflown.
Also, the use of the select() or poll() allows you to avoid threads in the server (i.e. you can poll() your socket and know whether you can accept(), read(), or write() to them.)
But if you really want to do it Unix like, then you want to consider fork()-ing before you call accept(). In this case you do not absolutely need the select() or poll() (unless you are listening on many IPs/ports and want all children to be capable of answering any incoming connections, but you have drawbacks with those... the kernel may send you another request while you are already handling a request, whereas, with just an accept(), the kernel knows that you are busy if not in the accept() call itself—well, it does not work exactly like that, but as a user, that's the way it works for you.)
With the fork() you prepare the socket in the main process and then call handle_request() in a child process to call the accept() function. That way you may have any number of ports and one or more children to listen on each. That's the best way to really very quickly respond to any incoming connection under Linux (i.e. as a user and as long as you have child processes wait for a client, this is instantaneous.)
void init_server(int port)
{
int server_socket = socket();
bind(server_socket, ...port...);
listen(server_socket);
for(int c = 0; c < 10; ++c)
{
pid_t child_pid = fork();
if(child_pid == 0)
{
// here we are in a child
handle_request(server_socket);
}
}
// WARNING: this loop cannot be here, since it is blocking...
// you will want to wait and see which child died and
// create a new child for the same `server_socket`...
// but this loop should get you started
for(;;)
{
// wait on children death (you'll need to do things with SIGCHLD too)
// and create a new children as they die...
wait(...);
pid_t child_pid = fork();
if(child_pid == 0)
{
handle_request(server_socket);
}
}
}
void handle_request(int server_socket)
{
// here child blocks until a connection arrives on 'server_socket'
int client_socket = accept(server_socket, ...);
...handle the request...
exit(0);
}
int create_servers()
{
init_server(80); // create a connection on port 80
init_server(443); // create a connection on port 443
}
Note that the handle_request() function is shown here as handling one request. The advantage of handling a single request is that you can do it the Unix way: allocate resources as required and once the request is answered, exit(0). The exit(0) will call the necessary close(), free(), etc. for you.
In contrast, if you want to handle multiple requests in a row, you want to make sure that resources get deallocated before you loop back to the accept() call. Also, the sbrk() function is pretty much never going to be called to reduce the memory footprint of your child. This means it will tend to grow a little bit every now and then. This is why a server such as Apache2 is setup to answer a certain number of requests per child before starting a new child (by default it is between 100 and 1,000 these days.)
I am trying to make a simple client-server chat program. On the client side I spin off another thread to read any incomming data from the server. The problem is, I want to gracefully terminate that second thread when a person logs out from the main thread. I was trying to use a shared variable 'running' to terminate, problem is, the socket read() command is a blocking command, so if I do while(running == 1), the server has to send something before the read returns and the while condition can be checked again. I am looking for a method (with common unix sockets only) to do a non-blocking read, basically some form of peek() would work, for I can continually check the loop to see if I'm done.
The reading thread loop is below, right now it does not have any mutex's for the shared variables, but I plan to add that later don't worry! ;)
void *serverlisten(void *vargp)
{
while(running == 1)
{
read(socket, readbuffer, sizeof(readbuffer));
printf("CLIENT RECIEVED: %s\n", readbuffer);
}
pthread_exit(NULL);
}
You can make socket not blockable, as suggested in another post plus use select to wait input with timeout, like this:
fd_set input;
FD_ZERO(&input);
FD_SET(sd, &input);
struct timeval timeout;
timeout.tv_sec = sec;
timeout.tv_usec = msec * 1000;
int n = select(sd + 1, &input, NULL, NULL, &timeout);
if (n == -1) {
//something wrong
} else if (n == 0)
continue;//timeout
if (!FD_ISSET(sd, &input))
;//again something wrong
//here we can call not blockable read
fcntl(socket, F_SETFL, O_NONBLOCK);
or, if you have other flags:
int x;
x=fcntl(socket ,F_GETFL, 0);
fcntl(socket, F_SETFL, x | O_NONBLOCK);
then check the return value of read to see whether there was data available.
note: a bit of googling will yield you lots of full examples.
You can also use blocking sockets, and "peek" with select with a timeout. It seems more appropriate here so you don't do busy wait.
The best thing is likely to get rid of the extra thread and use select() or poll() to handle everything in one thread.
If you want to keep the thread, one thing you can do is call shutdown() on the socket with SHUT_RDWR, which will shut down the connection, wake up all threads blocked on it but keep the file descriptor valid. After you have joined the reader thread, you can then close the socket. Note that this only works on sockets, not on other types of file descriptor.
Look for function setsockopt with option SO_RCVTIMEO.
I'm trying to develop a simple "telnet/server" daemon which have to run a program on a new socket connection.
This part working fine.
But I have to associate my new process to a pty, because this process have some terminal capabilities (like a readline).
The code I've developped is (where socketfd is the new socket file descriptor for the new input connection) :
int masterfd, pid;
const char *prgName = "...";
char *arguments[10] = ....;
if ((pid = forkpty(&masterfd, NULL, NULL, NULL)) < 0)
perror("FORK");
else if (pid)
return pid;
else
{
close(STDOUT_FILENO);
dup2(socketfd, STDOUT_FILENO);
close(STDIN_FILENO);
dup2(socketfd, STDIN_FILENO);
close(STDERR_FILENO);
dup2(socketfd, STDERR_FILENO);
if (execvp(prgName, arguments) < 0)
{
perror("execvp");
exit(2);
}
}
With that code, the stdin / stdout / stderr file descriptor of my "prgName" are associated to the socket (when looking with ls -la /proc/PID/fd), and so, the terminal capabilities of this process doesn't work.
A test with a connection via ssh/sshd on the remote device, and executing "localy" (under the ssh connection) prgName, show that the stdin/stdout/stderr fd of this process "prgName" are associated to a pty (and so the terminal capabilities of this process are working fine).
What I am doing wrong?
How to associate my socketfd with the pty (created by forkpty) ?
Thank
Alex
You must write some code to transfer data from the socket to the master pty and vice versa. It's usually a parent process' job. Note that the data transfer must be bidirectional. There are many options: a select()-driven cycle to track both the masterfd and the socketfd
(just as hint, very bad code, not for production!!! Missing error and eof checks!!!)
for (;;) {
FD_ZERO(&set);
FD_SET(masterfd,&set);
FD_SET(socketfd,&set);
select(...,&set,...);
if (FD_ISSET(masterfd,&set)) {
read(masterfd,&c,1);
write(socketfd,&c,1);
}
if (FD_ISSET(sockerfd,&set)) {
read(sochetfd,&c,1);
write(masterfd,&c,1);
}
or a pair of threads, one for socketfd->masterfd and one for masterfd->sockefd transfers.
(just as hint, very bad code, not for production!!!)
/*thread 1 */
while (read(masterfd,&c,1) > 0)
write(socketfd,&c,1);
/*thread 2 */
while (read(socketfd,&c,1) > 0)
write(masterfdfd,&c,1);
Anyway you must add some code in the parent side of the branch.
Regards
---EDIT---
Of course, you must not redirect fd 0,1 and 2 to socketfd in the child process.