Multi-Process/Multi-Threaded C Server reads null-string from socket - c

I'm working on a Server in C using a Multi-Process / Multi-Threaded architecture. At the start I've a main process that forks 10 different processes. Each process creates a pool of threads. Each process is, also, the controller of its pool (creates new thread / extends the pool when needed).
The processes also listen on the socket (whose access is controlled by the main process with an unnamed semaphore in a shared memory area) and pass the socket file descriptor to one of the thread in their pool when a connection is acquired. The chosen thread (which is awakened by a pthread_cond_signal) reads from the socket with a recv using the option MSG_DONTWAIT.
ssize_t readn, writen;
size_t nleft;
char *buff;
buff = malloc(sizeof(char) * BUFF_SIZE);
if (buff == NULL) {
perror("malloc");
return EXIT_FAILURE;
}
char *ptr = buff;
/*if (fcntl(connsd, F_SETFL, O_NONBLOCK) == -1) { // set to non-blocking
perror("fcntl");
exit(EXIT_FAILURE);
}*/
errno = 0;
nleft = BUFF_SIZE;
while(nleft > 0) {
if ((readn = recv(connsd, ptr, nleft, MSG_DONTWAIT)) < 0) {
if (errno == EAGAIN || errno == EWOULDBLOCK) {
*ptr = '\0';
break;
}
else if (errno == EINTR)
readn = 0;
else {
perror("read");
return EXIT_FAILURE;
}
}
else if (readn == 0)
break;
if (buff[strlen(buff)-1] == '\0') {
break;
}
nleft -= readn;
ptr += readn;
}
The problem is that sometimes, when I try to connect to the server using Chrome (or Firefox) 3 threads seems to receive the HTTP Request but each one of them simply closes the connection because of this portion of code.
if (buff[strlen(buff)-1] != '\0') {
printf("%s\n",buff);
fflush(stdout);
buff[strlen(buff)-1] = '\0';
}
errno = 0;
if (strlen(buff) < 1) {
perror("No string");
if (shutdown(connsd,SHUT_RDWR) < 0) {
perror("shutdown");
return EXIT_FAILURE;
}
if (close(connsd) < 0) {
perror("close");
return EXIT_FAILURE;
}
return EXIT_FAILURE;
}
Other times we have 3 threads with different behaviours: one of them receives and reads from the socket the first HTTP Request (GET / HTTP/1.1). The second one is empty (request received (because it is awakened (?)) no string read). The third one receives and reads another HTTP Request (GET /favicon.ico HTTP/1.1).
Where is the problem that cause these behaviours? I can add other portions of code if needed.
Thank you very much for your time.

Related

FTP socket for command is blocked after uploading a file to server via data socket

My FTP server is IIS. I closed the data socket after STOR a file, however, I found the cmd socket is blocked. It's strange. It seems the server data socket is stil waiting for data.
this is my code
int client_data_socket = enter_passvie_mode(client_cmd_socket, client_cmd_port + 1, send_buffer, recv_buffer);
FILE *fp;
if ((fp = fopen(filename, "rb")) == NULL)
{
close(client_data_socket);
printf("open file failed\n");
exit(1);
}
size_t char_size = sizeof(char);
char data_buffer[FILE_READ_BUFFER_SIZE];
int numread;
for (;;)
{
bzero(data_buffer, FILE_READ_BUFFER_SIZE);
numread = fread(data_buffer, char_size, FILE_READ_BUFFER_SIZE, fp);
if (numread < 0)
{
printf("read file failed\n");
break;
}
else if (numread > 0)
{
int length = send(client_data_socket, data_buffer, numread, 0);
if (length == 0)
{
break;
}
else if (length < 0)
{
if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
{
continue;
}
printf("[PUT] command send data failed\n");
exit(1);
}
}
if (numread == FILE_READ_BUFFER_SIZE) continue;
else {
break;
}
}
close(client_data_socket);
printf("close data socket\n");
fclose(fp);
exit(0);
command
after put a file, other command is blocked, it shows socket for command is blocked. Should I do anything other to notify the server that data transmission ends?
source code
I have found the answer:
close will send FIN packet and destroy fd immediately when socket is not shared with other processes.
In my case, parent process shares client_data_socket with the child process. So I need to use shutdown function to send FIN.

C socket: FD_ISSET return true with no data waiting

As a school project, I'm implementing a IRC server and I've been stuck on this one problem for the day.
My server use select to choose which client is sending data, it then read one command from this user (commands are \r\n separated), parse it and execute it before passing to the next user.
A user can send multiple command at once like so :
"command1\r\ncommand2\r\n"
If this happen, i want the first command to be executed and the second to stay in the stream so that it can be read at the next select() call. (this way, each user only execute one comamnd per "turn").
To do this, I have a FILE *stream per client that stay open as long as it is connected.
This work perfectly (if I send the double comamnd specified above, the two commands are executed one after the other).
The problem is after that after that, select() continue to tell me that there is something to read in the socket (FD_ISSET of the fd return 1), so my receive function is called for that fd and the getline() in it fail (without setting errno) and it loop like this forever.
I don't understand why the select still consider that there is something to read in the socket and why the getline is failing instead of blocking.
Any idea ?
edit: I'm not allowed to use non blocking socket or fork() for this project
The "main" loop:
while (g_run_server)
{
fd_max = g_socket_fd;
FD_ZERO(fds);
FD_SET(g_socket_fd, fds);
tmp = users;
while (tmp)
{
FD_SET(tmp->fd, fds);
fd_max = (tmp->fd > fd_max ? tmp->fd : fd_max);
tmp = tmp->next;
}
if (select(fd_max + 1, &fds, NULL, NULL, NULL) < 0)
break;
if (FD_ISSET(g_socket_fd, &fds))
accept_new_user(&hdl, &users);
handle_clients(&hdl, &fds);
}
The functions to handle and read client input :
static bool recv_and_execute(t_handle *hdl)
{
char *raw;
size_t len;
len = 0;
raw = NULL;
if (!hdl->sender->stream &&
(hdl->sender->stream = fdopen(dup(hdl->sender->fd), "r")) == NULL)
return (false);
if (getline(&raw, &len, hdl->sender->stream) != -1)
{
printf("Received \"%s\"\n", raw);
parse_cmd(hdl, raw);
exec_cmd(hdl);
}
else
printf("Getline failed %s\n", strerror(errno));
free(raw);
return (true);
}
int handle_clients(t_handle *hdl, fd_set *fds)
{
t_user *tmp;
tmp = *hdl->users;
while (tmp)
{
if (FD_ISSET(tmp->fd, fds))
{
printf("fd %d is ready to be read\n", tmp->fd);
hdl->sender = tmp;
recv_and_execute(hdl);
FD_CLR(tmp->fd, fds);
tmp = tmp->next;
if (hdl->sender->status == DEAD)
del_user(hdl->users, hdl->sender);
}
else
tmp = tmp->next;
}
return (0);
}
And this is the output when I connect one client and send "USER foo\r\nUSER no bo dy :wa\r\n" :
fd 4 is ready to be read
Received "NICK foo
"
[DEBUG] Executing "NICK" with params "foo" "(null)" "(null)" "(null)"
[INFO] Nickname successfully changed.
fd 4 is ready to be read
Received "USER no bo dy :wa
"
[DEBUG] Executing "USER" with params "no" "bo" "dy" ":wa"
[INFO] User registered.
fd 4 is ready to be read
Getline failed Success
fd 4 is ready to be read
Getline failed Success
fd 4 is ready to be read
Getline failed Success
fd 4 is ready to be read
Getline failed Success
fd 4 is ready to be read
Getline failed Success
continue like this....
Edit : I edited my receive function based on the comment of alk:
static bool recv_and_execute(t_handle *hdl)
{
char *raw;
size_t len;
ssize_t nread;
len = 0;
raw = NULL;
if (!hdl->sender->stream &&
(hdl->sender->stream = fdopen(dup(hdl->sender->fd), "r")) == NULL)
return (false);
errno = 0;
if ((nread = getline(&raw, &len, hdl->sender->stream)) > 0)
{
printf("Received \"%s\"\n", raw);
parse_cmd(hdl, raw);
exec_cmd(hdl);
}
else if (errno != 0)
printf("getline failed %s\n", strerror(errno));
else {
printf("EOF reached\n");
fclose(hdl->sender->stream);
hdl->sender->stream = NULL;
}
printf("nread = %zd\n", nread);
free(raw);
return (true);
}
So this time, when EOF is reach (getline return -1), I close the stream and set it to NULL to be reopened the next time select find data on the socket fd. But even when I close the stream, select still detect that there is data available and the inifinite loop continue :/
fd 4 is ready to be read
Received "NICK foo^M
"
nread = 10
fd 4 is ready to be read
Received "USER no bo dy :wa^M
"
nread = 19
fd 4 is ready to be read
EOF reached
nread = -1
fd 4 is ready to be read
EOF reached
nread = -1
fd 4 is ready to be read
EOF reached
nread = -1
and so on...
I'm pretty sure you're using the select wrong. I show you a simple code example on how to use it (I don't handle many errors) and you can edit it as you need.
/*
* If you read at man select bugs you can see
* that select could return that someone is
* ready but it isn't true
*/
int fd_c;
fd_set rdset;
fd_set set; /*That's the mask you'll use when new requests arrive*/
FD_ZERO(&set); /*Clears the mask*/
FD_SET(g_socket_fd,&set); /*Set the listening socket as ready*/
while(g_run_server){
/*
* YOU MUST INITIALIZATE IT EVERY LOOP
* read # man select
*/
rdset = set;
if(select((fd_num_max+1),(&rdset),NULL,NULL,NULL) < 0){
perror("Failed on select\n");
exit(EXIT_FAILURE);
}
/*You go through the ready clients in the rdset*/
for(fd=0;fd<=fd_num_max;fd++) {
if(FD_ISSET(fd,&rdset)) { /*If the bit is set*/
if(fd == fd_skt) { /*If it's a new client*/
/*You can handle the new client here or delegete it to someone*/
fd_c=accept(fd_skt,NULL,0); /*File descriptor of new client*/
FD_SET(fd_c,&set);
if(fd_c > fd_num_max) fd_num_max = fd_c;
}else { /*If it's a request from an existing client*/
/*You can handle the new request here or delegete it to someone*/
FD_SET(fd,&set);
}
}
}
}
You should also modify static bool recv_and_execute(t_handle *hdl) that way:
errno = 0;
if ((nread = getline(&raw, &len, hdl->sender->stream)) != -1){
printf("Received \"%s\"\n", raw);
parse_cmd(hdl, raw);
exec_cmd(hdl);
}else{
if( errno == 0){
printf("EOF reached\n");
fclose(hdl->sender->stream);
hdl->sender->stream = NULL;
}else{
printf("getline failed %s\n", strerror(errno));
exit(EXIT_FAILURE); /*You must handle it in some way, exiting or doing something*/
}
}

How to make non-blocking OpenSSL connection?

I want make a non-blocking OpenSSL connection
On this connection - if no data available for read, then entire program execution flow make stop on SSL_read(). I want so that if no data available for read it give me the returns values like WANT_READ and i know no more data available.
char *sslRead (connection *c)
{
const int readSize = 1024;
char *rc = NULL;
int r;
int received, count = 0;
int ReallocSize = 0;
char buffer[1024];
if (c)
{
while (1)
{
if (!rc)
{
rc = malloc (readSize + 1);
if (rc == NULL)
printf("the major error have happen. leave program\n");
}
else
{
ReallocSize = (count + 1) * (readSize + 1);
rc = realloc (rc, ReallocSize);
}
// if i have no data available for read after reading data,
// this call will not return anything and wait for more data
// i want change this non blocking connections
received = SSL_read (c->sslHandle, buffer, readSize);
buffer[received] = '\0';
if (received <= 0)
{
printf(" received equal to or less than 0\n");
switch (SSL_get_error(c->sslHandle, r))
{
case SSL_ERROR_NONE:
printf("SSL_ERROR_NONE\n");
break;
case SSL_ERROR_ZERO_RETURN:
printf("SSL_ERROR_ZERO_RETURN\n");
break;
case SSL_ERROR_WANT_READ:
printf("SSL_ERROR_WANT_READ\n");
break;
default:
printf("error happens %i\n", r);
}
break;
}
count++;
}
}
return rc;
}
here is how i make connection
connection *sslConnect (void)
{
connection *c;
c = malloc (sizeof (connection));
c->sslHandle = NULL;
c->sslContext = NULL;
c->socket = tcpConnect ();
if (c->socket)
{
// Register the error strings for libcrypto & libssl
SSL_load_error_strings ();
// Register the available ciphers and digests
SSL_library_init ();
// New context saying we are a client, and using SSL 2 or 3
c->sslContext = SSL_CTX_new (SSLv23_client_method ());
if (c->sslContext == NULL)
ERR_print_errors_fp (stderr);
// Create an SSL struct for the connection
c->sslHandle = SSL_new (c->sslContext);
if (c->sslHandle == NULL)
ERR_print_errors_fp (stderr);
// Connect the SSL struct to our connection
if (!SSL_set_fd (c->sslHandle, c->socket))
ERR_print_errors_fp (stderr);
// Initiate SSL handshake
if (SSL_connect (c->sslHandle) != 1)
ERR_print_errors_fp (stderr);
}
else
{
perror ("Connect failed");
}
return c;
}
thanks you very much.
Creating a non-blocking socket is a pre-requisite to a non-blocking connect...
The following steps summarize: (see complete description in site linked below)
1) Call the fcntl() API to retrieve the socket descriptor's current flag settings into a local variable.
2) In that local variable, set the O_NONBLOCK (non-blocking) flag on. (being careful not to tamper with the other flags)
3) Call the fcntl() API to set the flags for the descriptor to the value in our local variable.
( read more on non-blocking sockets techniques here )
Assuming an existing socket, the following implements the steps outlined above:
BOOL SetSocketBlockingEnabled(SOCKET fd, BOOL blocking)
{
if (fd < 0) return FALSE;
#ifdef WIN32
unsigned long mode = blocking ? 0 : 1;
return (ioctlsocket(fd, FIONBIO, &mode) == 0) ? TRUE : FALSE;
#else
int flags = fcntl(fd, F_GETFL, 0);
if (flags < 0) return false;
flags = blocking ? (flags&~O_NONBLOCK) : (flags|O_NONBLOCK);
return (fcntl(fd, F_SETFL, flags) == 0) ? TRUE : FALSE;
#endif
}
Once you have a non-blocking socket, then see this post explaining how to do a non-blocking connect

Close socket after select()

I'm coding an IRC client and I would like implement a "/server" command to switch the connection of my client to an other server.
Before initialize the new connection I want to close the sockect's fd but the close() call fail. Anybody could say me why ?
Here is my code :
/* Main execution loop */
FD_ZERO(&irc->rdfs);
FD_SET(STDIN_FILENO, &irc->rdfs);
FD_SET(irc->socket_fd, &irc->rdfs);
if ((select(irc->socket_fd + 1, &irc->rdfs, NULL, NULL, NULL)) == -1)
{
if ((close(irc->socket_fd)) == -1)
exit(usage(CLOSE_ERROR));
exit(usage(SELECT_ERROR));
}
if (FD_ISSET(STDIN_FILENO, &irc->rdfs))
{
fgets(irc->buffer, SIZE - 1, stdin);
{
p = strstr(irc->buffer, RET);
if (p != NULL)
*p = 0;
else
irc->buffer[SIZE - 1] = 0;
}
write_on_server(irc, irc->buffer); /* The function where I call switch_server() in */
}
else if (FD_ISSET(irc->socket_fd, &irc->rdfs))
{
if ((read_on_server(irc)) == 0)
exit(usage(SERVER_DISCONNECT));
puts(irc->buffer);
}
And here is where I'm trying to close my socket's fd :
void switch_server(t_irc *irc)
{
if ((close(irc->socket_fd)) == -1) /* This is the close which fail */
exit(EXIT_FAILURE);
}
void write_on_server(t_irc *irc, const char * buffer)
{
if (!(strncmp("/server", buffer, strlen("/server"))))
switch_server(irc);
else
if ((send(irc->socket_fd, buffer, strlen(buffer), 0)) < 0)
{
if ((close(irc->socket_fd)) == -1)
exit(usage(CLOSE_ERROR));
exit(usage(CLIENT_SEND_ERROR));
}
}
Thanks a lot.
If you want to know why a syscall like close() failed, use perror() to print an error message to stderr, or strerror(errno) to convert the error code to a string and output it some other way.
Almost certainly the socket FD is invalid. You need to call perror() on that, and on the select() failure.

C Socket Server, memory increases with file writes

I have a simple server program written in C, and the program is running on an Ubuntu Linux distribution. The program is intended to listen for messages sent from the client, write those messages to a file (each message goes into a separate file), and send an acknowledgement back to the client once the message has been received and stored.
I have noticed that, as the server continues to receive and store messages, the amount of available system memory quickly decreases and continues to decrease until messages have stopped. Memory remains constant when no messages are being sent. However, I have also noticed that I can free up the memory again by deleting the written files from disk (I can do this even while the server is still running). I am therefore led to believe that the memory issue has something to do with my file operations, though I can't see any issue with the code that writes the files.
Can someone help?
NOTE: I am observing the memory usage with "top".
I have included an excerpt from the program. The below function handles input from the client and writes that information to file. This is where I currently believe the problem to be:
void handleinput (int sock)
{
char filename[strlen(tempfolder) + 27];
generatefilename(filename);
int rv;
int n = 1;
int received = 0;
char buffer[BUFFER_SIZE];
FILE *p = NULL;
fd_set set;
char response[768];
struct timeval timeout;
timeout.tv_sec = 360;
timeout.tv_usec = 0;
FD_ZERO(&set);
FD_SET(sock, &set);
bzero(buffer, BUFFER_SIZE);
bzero(response, sizeof response);
rv = select(sock + 1, &set, NULL, NULL, &timeout);
if (rv == -1)
{
error("error on select in handleinput");
close(sock);
exit(1);
}
else if (rv == 0)
{
close(sock);
exit(0);
}
else
{
n = read(sock, buffer, BUFFER_SIZE-1);
if (n <= 0)
{
close(sock);
exit(0);
}
}
// open file
if (n != 0)
{
p = fopen(filename, "a");
if (p == NULL)
{
error("ERROR writing message to file");
close(sock);
exit(1);
}
}
// loop until full message is received
while (n != 0)
{
if (n < 0)
{
error("ERROR reading from socket");
close(sock);
exit(1);
}
received = 1;
// write content to file
fwrite(buffer, strlen(buffer), 1, p);
if (buffer[strlen(buffer)-1] == 0x1c)
{
break;
}
bzero(buffer, BUFFER_SIZE);
rv = select(sock + 1, &set, NULL, NULL, &timeout);
if (rv == -1)
{
error("ERROR select in loop in handleinput");
close(sock);
exit(1);
}
else if (rv == 0)
{
close(sock);
exit(0);
}
else
{
n = read(sock, buffer, BUFFER_SIZE-1);
}
}
// close file if we opened it earlier
if (p != NULL)
{
fclose(p);
}
// send acknowledgement back to client
if (received == 1)
{
generateResponse(response, filename);
n = write(sock, response, strlen(response));
if (n < 0)
{
error("ERROR writing to socket");
close(sock);
exit(1);
}
}
}
Its because of the caching mechanism of writing. IF you have a lot of clients trying to write files, the IO buffer fills in kernel memory, but doesn't actually write to that file until you close the socket or the buffer fills. You can fix this just by flushing the buffer. What some other people have suggested is to get rid of the write and read wrappers in stdio, and just use the kernel call write, since this will help performance and will probably flush the buffer automatically.
You can flush with fsync() btw.

Resources