I have developed a tcp server in my one embedded device using lwip+freeRTOS.
Now this is the flow how I communicate with other device ( Other device has Linux system and let's name it LinuxDevice) using my device ( let's name it lwipDevice)
Send UDP packet from lwipDevice to LinuxDevice to initiate that I am ready to share my information.
Now LinuxDevice recognises this message successfully and sends one TCP packet (which contain command) to lwipDevice for telling to send its information.
But at lwipDevice side this message is not received. So it will not send any response to LinuxDevice. And steps 1-3 repeat again and again.
Now this is code of lwipDevice for TCP server:
long server_sock=-1;
#define FAIL 1
#define PASS 0
int CreateTcpServerSocket(long *pSock, int port)
{
struct sockaddr_in sin;
int addrlen = sizeof(sin);
int e;
struct linger linger;
linger.l_linger=0;
int i = 1;
*pSock = socket(AF_INET, SOCK_STREAM, 0);
if (*pSock == -1)
{
printf("*** createTcpSercerSocket:open sock error,port %d\n",port);
return FAIL;
}
memset((char *)&sin, 0, sizeof(sin));
sin.sin_family = AF_INET;
sin.sin_len = sizeof(sin);
sin.sin_addr.s_addr = htonl(INADDR_ANY); /* wildcard IP address */
sin.sin_port = htons(port);
e = bind(*pSock, (struct sockaddr*)&sin, addrlen);
if (e != 0)
{
printf("error %d binding tcp listen on port\n");
closesocket(*pSock);
*pSock = -1;
return FAIL;
}
lwip_ioctl(*pSock, FIONBIO, &i); //Set Non blocking mode
e = listen(*pSock, 2);
if (e != 0)
{
pprintf("error :listen on TCP server\n");
closesocket(*pSock);
*pSock = -1;
return FAIL;
}
return PASS;
}
void vTCPTask(void *parm)
{
struct sockaddr client; /* for BSDish accept() call */
int clientsize;
long sock;
if(CreateTcpServerSocket(&server_sock, 8000) == FAIL) //Here server created successfully
{
printf("Fail to create server!!!!!\n");
server_sock=-1;
}
while(1)
{
// some code for other stuff
sock= accept(server_sock, &client, &clientsize); //This line always fails and reurn -1
if(sock != -1)
{
printf("accepted socket:\n\n");
//...now receive data from client....
// send some data to client
}
// some code for other stuff
//sleep for 15 seconds
}
}
int main()
{
//...initilization of lwip stack
//....some more code
//...................
xTaskCreate(vTCPTask, (signed char const *) "tcptask",
356, NULL, 3, (xTaskHandle *)&xNotifierServiceTaskHandle);
/* Start the scheduler */
vTaskStartScheduler();
return 1
}
I have checked lwip_accept function and it will return from this condition:
if (netconn_is_nonblocking(sock->conn) && (sock->rcvevent <= 0))
{
LWIP_DEBUGF(SOCKETS_DEBUG, ("lwip_accept(%d): returning EWOULDBLOCK\n", s));
sock_set_errno(sock, EWOULDBLOCK);
return -1;
}
EDIT:
I know that netconn_is_nonblocking(sock->conn) condition will always true because have set socket in non blocking mode. But why sock->rcvevent always zero even LinuxDevice already send packet to it?
EDIT:
For testing purpose have commented all other stuff code in task ( see //some code for other stuff ) then socket is successfully accepted and i try to receive the packet but now problem is it's now stuck in lwip_recvfrom function (Note: LinuxDevice continue send packets). So have further debug more and found that it stuck in function sys_arch_mbox_fetch ( function call flow:: lwip_recvfrom\netconn_recv\netconn_recv_data\sys_arch_mbox_fetch).
Does anyone have an idea what is wrong with it?
You have configured the socket as non-blocking, so the accept() call will never block. If there is no incoming connection pending it will return the EWOULDBLOCK error code, which is what you see.
Finally I figured out what is cause of issue.
In lwipopt.h file there is macro like
/* Non-static memory, used with DMA pool */
#ifdef __CODE_RED
#define MEM_SIZE (6 * 1024)
#else
#define MEM_SIZE (24 * 1024)
#endif
I have defined _CODE_RED. So MEM_SIZE will (6 * 1024). Now when i change that memory size to (16 * 1024) then everything working fine.
Now all the time connection accepted and after that i am able to send/recv tcp packets successfully.
Where do you set rcvevent? Your code doesn't reveal it. I suppose it's the result of recv (or read). Reading from a non-blocking that has no available data (haven't yet received data) returns EAGAIN, which evaluates true in your rcvevent <= 0 condition. You have to manually check these specific error codes.
But why sock->rcvevent always zero even LinuxDevice already send packet to it?
Have you tried sending data with telnet or netcat to be sure the error is in your server and not in your client? Maybe your client is not sending to the correct destination, or something else.
Related
I am trying to write a basic TCP server that streams serial data to a client. The server would connect to a serial device, read data from said device, and then transmit it as a byte stream to the client. Writing the TCP server is no problem. The issue is that the server will crash when a client disconnects. In other languages, like Python, I can simply wrap the write() statement in a try-catch block. The program will try to write to the socket, but if the client has disconnected then an exception will be thrown. In another project, this code snippet worked for me:
try:
client_socket.send(bytes(buf, encoding='utf8'))
except Exception as e:
logger.info("Client disconnected: %s", client_id)
I can handle client disconnects in my C code, but only by first reading from the socket and checking if the read is equal to 0. If it is, then my client has disconnected and I can carry on as usual. The problem with this solution is that my client has to ping back to the server after every write, which is less than ideal.
Does anyone know how to gracefully handle TCP client disconnects in C? My example code is shown below. Thank you!
// Define a TCP socket
int sockfd = socket(AF_INET, SOCK_STREAM, 0);
// Allow for the backlog of 100 connections to the socket
int backlog = 100;
// Supply a port to bind the TCP server to
short port = 9527;
// Set up server attributes
struct sockaddr_in servaddr;
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
servaddr.sin_port = htons(port);
// Set the socket so that we can bind to the same port when we exit the program
int flag = 1;
if (setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, &flag, sizeof(flag)) == -1) {
perror("setsockopt fail");
}
// Bind the socket to the specified port
int res = bind(sockfd, (struct sockaddr *)&servaddr, sizeof(servaddr));
if (res < 0) {
perror("bind fail");
exit(1);
}
// Listen for incoming connections
if (listen(sockfd, backlog) == -1) {
perror("listen fail");
exit(1);
} else {
printf("Server listening on port\n", port);
}
for(;;) {
// Wait for incoming connection
struct sockaddr_in cliaddr;
socklen_t len = sizeof(cliaddr);
int connfd = accept(sockfd, (struct sockaddr *)&cliaddr, &len);
if (-1 == connfd) {
perror("Could not accept incoming client");
continue;
}
//Resolving Client Address
char buff[INET_ADDRSTRLEN + 1] = {0};
inet_ntop(AF_INET, &cliaddr.sin_addr, buff, INET_ADDRSTRLEN);
uint16_t cli_port = ntohs(cliaddr.sin_port);
printf("connection from %s, port %d\n", buff, cli_port);
for(;;) {
// Read from serial device into variable here, then send
if(send(connfd, "Data...Data...Data\n", 19, 0) < 0) {
printf("Client disconnected...\n");
break;
}
}
}
Looks like a duplicate of this, this and this.
Long story short you can't detect the disconnection until you perform some write (or read) on that connection. More exactly, even if it seems there is no error returned by send, this is not a guarantee that this operation was really sent and received by the client. The reason is that the socket operations are buffered and the payload of send is just queued so that the kernel will dispatch it later on.
Depending on the context, the requirements and the assumptions you can do something more.
For example, if you are under the hypothesys that you will send periodic message at constant frequency, you can use select and a timeout approach to detect an anomaly.
In other words if you have not received anything in the last 3 minutes you assume that there is an issue.
As you can easily found, this and this are a good read on the topic.
Look at that for a far more detailed explanation and other ideas.
What you call the ping (intended as a message that is sent for every received packet) is more similar to what is usually known as an ACK.
You only need something like that (ACK/NACK) if you also want to be sure that the client received and processed that message.
Thanks to #emmanuaf, this is the solution that fits my project criteria. The thing that I was missing was the MSG_NOSIGNAL flag, referenced here.
I use Mashpoe's C Vector Library to create a new vector, which will hold all of my incoming client connections.
int* client_array = vector_create();
I then spawn a pthread that continually reads from a serial device, stores that data in a variable, and then sends it to each client in the client list
void* serve_clients(int *vargp) {
for(;;) {
// Perform a microsleep
sleep(0.1);
// Read from the Serial device
// Get the size of the client array vector
int client_vector_size = vector_size(vargp);
for(int i = 0 ; i < client_vector_size ; i++) {
// Make a reference to the socket
int* conn_fd = &vargp[i];
/*
In order to properly handle client disconnects, we supply a MSG_NOSIGNAL
flag to the send() call. That way, if the client disconnects, we will
be able to detect this, and properly remove them from the client list.
Referenced from: https://beej.us/guide/bgnet/html//index.html#sendman
*/
if (send(*conn_fd, "Reply from server\n", 18, MSG_NOSIGNAL) < 0) {
printf("Client disconnected...\n");
// Close the client connection
close(*conn_fd);
// Remove client socket from the vector
vector_remove(vargp, i);
// Decrement index and client_server_size by 1
i--;
client_vector_size--;
}
}
}
}
To spawn the pthread:
// Spawn the thread that serves clients
pthread_t serving_thread;
pthread_create(&serving_thread, NULL, serve_clients, client_array);
When a new connection comes in, I simply add the new connection to the client vector
while(1) {
// Wait for incoming connection
struct sockaddr_in cliaddr;
socklen_t len = sizeof(cliaddr);
int connfd = accept(sockfd, (struct sockaddr *)&cliaddr, &len);
if (-1 == connfd) {
perror("Could not accept incoming client");
continue;
}
//Resolving Client Address
char buff[INET_ADDRSTRLEN + 1] = {0};
inet_ntop(AF_INET, &cliaddr.sin_addr, buff, INET_ADDRSTRLEN);
uint16_t cli_port = ntohs(cliaddr.sin_port);
printf("connection from %s:%d -- Connfd: %d\n", buff, cli_port, connfd);
// Add client to vector list
vector_add(&client_array, connfd);
}
In the end, we have a TCP server that can multiplex data to many clients, and handle when those clients disconnect.
I've been attempting TCP hole punching for a while now and forums don't seem be helping much when it comes to TCP based approach and C programming language. Following were the main references from internet,
a. http://www.brynosaurus.com/pub/net/p2pnat/
b. https://wuyongzheng.wordpress.com/2013/01/31/experiment-on-tcp-hole-punching/
My setup is
Client A -- NAT-A -- Internet -- NAT-B -- Client B.
Assuming that client A knows B's public and private endpoint, and B knows A's endpoints ( I have written a server 'S' that exchanges endpoint information among peers), and given that both the NATs are NOT symmetric, will it suffice (to achieve TCP hole punching), if both the clients attempt to connect() to each other's public endpoint ( for the above setup) repeatedly?
If not, what exactly has to be done to achieve tcp hole punching?
I have two threads on each clients , one that makes a connect call repeatedly to other client, and the other that listens to incoming connection from other client. I have made sure that sockets in both the threads are bound to the local port that was given to the peer. Also, I see that both the NATs preserve port mapping i.e., local and public ports are same. Yet, my program isn't working.
Is it so that the rendezvous server 'S' that I mentioned above has a role to play in punching a hole or creating a NAT mapping that will allow SYN requests to pass through, to the peers. If yes, what has to be done?
Relevant sections of the code are attached.
connect_with_peer() is the entry point, after server 'S' provides the peer's public ip:port tuple, which is given to this function along with local port to which binding is done. This function spawns a thread ( accept_handler() ) which also binds to the local port and listens for incoming connection from the peer.
connect_with_peer() returns a socket , if connect() [ main thread ] or accept() [ child thread ], is successful.
Thanks,
Dinkar
volatile int quit_connecting=0;
void *accept_handler(void *arg)
{
int i,psock,cnt=0;
int port = *((int *)arg);
ssize_t len;
int asock,opt,fdmax;
char str[BUF_SIZE];
struct sockaddr_in peer,local;
socklen_t peer_len = sizeof(peer);
fd_set master,read_fds; // master file descriptor list
struct timeval tv = {10, 0}; // 10 sec timeout
int *ret_sock = NULL;
struct linger lin;
lin.l_onoff=1;
lin.l_linger=0;
opt=1;
//Create socket
asock = socket(AF_INET , SOCK_STREAM, IPPROTO_TCP);
if (asock == -1)
{
fprintf(stderr,"Could not create socket");
goto quit_ah;
}
else if (setsockopt(asock, SOL_SOCKET, SO_LINGER, &lin,
(socklen_t) sizeof lin) < 0)
{
fprintf(stderr,"\nTCP set linger socket options failure");
goto quit_ah;
}
else if (setsockopt(asock, SOL_SOCKET, SO_REUSEADDR | SO_REUSEPORT, &opt,
(socklen_t) sizeof opt) < 0)
{
fprintf(stderr,"\nTCP set csock options failure");
goto quit_ah;
}
local.sin_family = AF_INET; /* host byte order */
local.sin_port = htons(port); /* short, network byte order */
local.sin_addr.s_addr = INADDR_ANY; /* auto-fill with my IP */
bzero(&(local.sin_zero), 8); /* zero the rest of the struct */
fprintf(stderr,"\naccept_handler: binding to port %d",port);
if (bind(asock, (struct sockaddr *)&local, sizeof(struct sockaddr)) == -1) {
perror("accept_handler bind error :");
goto quit_ah;
}
if (listen(asock, 1) == -1) {
perror(" accept_handler listen");
goto quit_ah;
}
memset(&peer, 0, sizeof(peer));
peer.sin_addr.s_addr = inet_addr(peer_global_address);
peer.sin_family = AF_INET;
peer.sin_port = htons( peer_global_port );
FD_ZERO(&master); // clear the master and temp sets
FD_SET(asock, &master);
fdmax = asock; // so far, it's this one
// Try accept
fprintf(stderr,"\n listen done; accepting next ... ");
while(quit_connecting == 0){
read_fds = master; // copy it
if (select(fdmax+1, &read_fds, NULL, NULL, &tv) == -1) {
perror("accept_handler select");
break;
}
// run through the existing connections looking for data to read
for(i = 0; i <= fdmax; i++) {
if (FD_ISSET(i, &read_fds)) { // we got one!!
if (i == asock) {
// handle new connections
psock = accept(asock, (struct sockaddr *)&peer, (socklen_t*)&peer_len);
if (psock == -1) {
perror("accept_handler accept");
} else {
fprintf(stderr,"\n Punch accept in thread succeeded soc=%d....",psock);
quit_connecting = 1;
ret_sock = malloc(sizeof(int));
if(ret_sock){
*ret_sock = psock;
}
}
}
}
} // end for
}
quit_ah:
if(asock>=0) {
shutdown(asock,2);
close(asock);
}
pthread_exit((void *)ret_sock);
return (NULL);
}
int connect_with_peer(char *ip, int port, int lport)
{
int retval=-1, csock=-1;
int *psock=NULL;
int attempts=0, cnt=0;
int rc=0, opt;
ssize_t len=0;
struct sockaddr_in peer, apeer;
struct sockaddr_storage from;
socklen_t peer_len = sizeof(peer);
socklen_t fromLen = sizeof(from);
char str[64];
int connected = 0;
pthread_t accept_thread;
long arg;
struct timeval tv;
fd_set myset;
int so_error;
struct linger lin;
lin.l_onoff=1;
lin.l_linger=0;
opt=1;
//Create socket
csock = socket(AF_INET , SOCK_STREAM, IPPROTO_TCP);
if (csock == -1)
{
fprintf(stderr,"Could not create socket");
return -1;
}
else if (setsockopt(csock, SOL_SOCKET, SO_LINGER, &lin,
(socklen_t) sizeof lin) < 0)
{
fprintf(stderr,"\nTCP set linger socket options failure");
}
#if 1
else if (setsockopt(csock, SOL_SOCKET, SO_REUSEADDR | SO_REUSEPORT, &opt,
(socklen_t) sizeof opt) < 0)
{
fprintf(stderr,"\nTCP set csock options failure");
}
#endif
quit_connecting = 0;
///////////
if( pthread_create( &accept_thread , NULL , accept_handler , &lport) < 0)
{
perror("could not create thread");
return 1;
}
sleep(2); // wait for listen/accept to begin in accept_thread.
///////////
peer.sin_family = AF_INET; /* host byte order */
peer.sin_port = htons(lport); /* short, network byte order */
peer.sin_addr.s_addr = INADDR_ANY; /* auto-fill with my IP */
bzero(&(peer.sin_zero), 8); /* zero the rest of the struct */
fprintf(stderr,"\n connect_with_peer: binding to port %d",lport);
if (bind(csock, (struct sockaddr *)&peer, sizeof(struct sockaddr)) == -1) {
perror("connect_with_peer bind error :");
goto quit_connect_with_peer;
}
// Set non-blocking
arg = fcntl(csock, F_GETFL, NULL);
arg |= O_NONBLOCK;
fcntl(csock, F_SETFL, arg);
memset(&peer, 0, sizeof(peer));
peer.sin_addr.s_addr = inet_addr(ip);
peer.sin_family = AF_INET;
peer.sin_port = htons( port );
//Connect to remote server
fprintf(stderr,"\n Attempting to connect/punch to %s; attempt=%d",ip,attempts);
rc = connect(csock , (struct sockaddr *)&peer , peer_len);
if(rc == 0){ //succeeded
fprintf(stderr,"\n Punch Connect succeeded first time....");
} else {
if (errno == EINPROGRESS) {
while((attempts<5) && (quit_connecting==0)){
tv.tv_sec = 10;
tv.tv_usec = 0;
FD_ZERO(&myset);
FD_SET(csock, &myset);
if (select(csock+1, NULL, &myset, NULL, &tv) > 0) {
len = sizeof(so_error);
getsockopt(csock, SOL_SOCKET, SO_ERROR, &so_error, (socklen_t *)&len);
if (so_error == 0) {
fprintf(stderr,"\n Punch Connect succeeded ....");
// Set it back to blocking mode
arg = fcntl(csock, F_GETFL, NULL);
arg &= ~(O_NONBLOCK);
fcntl(csock, F_SETFL, arg);
quit_connecting=1;
retval = csock;
} else { // error
fprintf(stderr,"\n Punch select error: %s\n", strerror(so_error));
goto quit_connect_with_peer;
}
} else {
fprintf(stderr,"\n Punch select timeout: %s\n", strerror(so_error));
}
attempts++;
}// end while
} else { //errorno is not EINPROGRESS
fprintf(stderr, "\n Punch connect error: %s\n", strerror(errno));
}
}
quit_connect_with_peer:
quit_connecting=1;
fprintf(stderr,"\n Waiting for accept_thread to close..");
pthread_join(accept_thread,(void **)&psock);
if(retval == -1 ) {
if(psock && ((*psock) != -1)){
retval = (*psock); // Success from accept socket
}
}
fprintf(stderr,"\n After accept_thread psock = %d csock=%d, retval=%d",psock?(*psock):-1,csock,retval);
if(psock) free(psock); // Free the socket pointer , not the socket.
if((retval != csock) && (csock>=0)){ // close connect socket if accept succeeded
shutdown(csock,2);
close(csock);
}
return retval;
}
First, read this very similar question:
TCP Hole Punching
And read the part after EDIT2 (excerpt here). That's possibly the cause of failure.
Once the second socket has successfully bound, the behavior for all
sockets bound to that port is indeterminate.
Don't worry linux has similar limitations in socket(7) with SO_REUSEADDR:
For AF_INET sockets this means that a socket may bind, except when
there is an active listening socket bound to the address. When the
listening socket is bound to INADDR_ANY with a specific port then it is
not possible to bind to this port for any local address
I don't think that listening after instead of before will make a difference.
You don't have to try and open twice your connection.
Summary of steps to establish a TCP connection:
Left side: (a client C connecting to a server S) is the usual case, right side is the simultaneous connection of two peers A and B (what you're trying to do):
C A B
\ (SYN) \ /
\ (SYN)\ /(SYN)
> S X
/ / \
/(SYN+ACK) / \
/ A < > B
C< \ /
\ (SYN+ACK)\ / (SYN+ACK)
\(ACK) X
\ / \
\ / \
> S A < > B
ESTABLISHED ESTABLISHED
references:
https://www.rfc-editor.org/rfc/rfc793#section-3.4 figure 8.
correction for fig 8 line 7:
https://www.rfc-editor.org/rfc/rfc1122#page-87 (section 4.2.2.10)
The difference is the simultaneous SYN2/SYN+ACK2 instead of SYN/SYN+ACK/ACK (in my tests with two linux peers, usually only the "first" answers with SYN+ACK because it's never that simultaneous. It doesn't really matter).
Both peers actively initiate a connection. They're not initially waiting for a connection and you don't have to call listen()/accept() at all. You don't have to use any threads at all.
Each peer should exchange (through S) their intended local port for the other to use (and with the help of S they'll exchange their public IP), with the assumption the port won't be translated.
Now you just try and connect with your 4-uple of informations. each will binds with (INADDR_ANY,lport) and connect to (peer_global_address,peer_global_port) while simultanously B does the same. At the end there is an UNIQUE connection established between both sides.
both NAT boxes will see outgoing packets and prepare a reverse path.
Now what can go wrong?
A NAT box can't cope with the expected packet having a SYN instead of the more common SYN+ACK. Sorry, if that happens you might be out of luck. TCP protocol allows for this case and it's mandatory (rfc 1122 section 4.2.2.10 above). If the other NAT box is fine it should still work (once a SYN+ACK is sent back).
A NAT device (from a peer doing the request too late, say NAT-B in front of B) answers with a RST packet instead of silently dropping the still unknown packet like most NAT devices are doing. A receives RST and aborts the connection. Then B sends it and a similar fate happens. The faster the ping round-trip, the easier you'd get this. To avoid this, either:
if you can control one of the NAT devices, have it drop the packet instead of sending a RST.
be really synchronized (use NTP, exchange a precise date in sub-milliseconds of intended action between the peers through S, or wait the next multiple of 5 seconds to start)
drop the outgoing RST packet with a custom (and temporary) firewall rule on A and/or B (better than dropping the incoming RST, because the NAT devices can decide to close the expectation when they see it)
I can just tell I could have TCP hole punching working reliably "by hand" simply using netcat between two peers set like in your case.
Eg on Linux with netcat: type simultaneously those on the two peers A and B each in a private LAN behind their NAT device. With usual NAT devices (which drop unknown packets), no need for any perfect synchronization, even 5s between those two commands is fine (of course the first will be waiting):
host-a$ nc -p 7777 public-ip-host-b 8888
host-b$ nc -p 8888 public-ip-host-a 7777
When it's done, both netcat have established the SAME UNIQUE connection together, there aren't two connections established. No retry (no loop) was needed. Of course the programs will have used a connect(), and the OS may have sent multiple SYN packets as an automatic retry mechanism during the connect() if the second command (and thus connect() ) is delayed. This is at system/kernel level, not at your level.
I hope this helped so you can simplify your program and have it work. Remember, no need to listen(), accept(), having to fork, use threads. You don't even need a select(), just have connect() block normally without O_NONBLOCK.
I am trying to understand why my function dosnt sending the all string (Its send only 53576 elements from 365568:
This is the function I am using in the client side:
#define DATASIZEBUFFER 4000// 365568
void DieWithError(char *errorMessage);/* Error handling function */
void TcpClient ( char *servIP , unsigned short echoServPort , Hash_t *HashData)//(int argc, char *argv[])
{
int sock; //Socket descriptor
struct sockaddr_in ServAddr; //Echo server address
int bytesRcvd, totalBytesRcvd; //Bytes read in single recv()
//and total bytes read
// Create a reliable, stream socket using TCP
if ((sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0)
DieWithError(" socket () failed") ;
// Construct the server address structure
memset(&ServAddr, 0, sizeof(ServAddr)); /* Zero out structure */
ServAddr.sin_family = AF_INET; /* Internet address family */
ServAddr.sin_addr.s_addr = inet_addr(servIP);/* Server IP address */
ServAddr.sin_port = htons(echoServPort); /* Server port */
// Establish the connection to the server
if (connect(sock, (struct sockaddr *) &ServAddr, sizeof(ServAddr)) < 0)
DieWithError(" connect () failed") ;
for (;;)
{
// Send the string to the server //
if (send(sock, HashData->array , HashData->elementNumber, 0) != HashData->elementNumber)
{
printf ("Bytes Nedded to recived: %ld\nAnd (DATASIZEBUFFER) is %d\n", HashData->elementNumber , DATASIZEBUFFER);
DieWithError("send() sent a different number of bytes than expected");
}
}
send() does not guarantee that it would send all the data.
From send man page:
On success, these calls return the number of bytes sent. On error,
-1 is returned, and errno is set appropriately.
You can write a loop around send() and invoke it multiple times until all data is sent (or, error is returned). It could be something like the following (please modify it based on your needs):
size_t
Send(int sockfd, const void *buf, size_t len, int flag) {
size_t sent_total = 0;
for (int sent_now = 0; sent_total != len; sent_total += sent_now) {
sent_now = send(sockfd, buf + sent_total, len - sent_total, flag);
if (sent_now == -1) break;
}
if (sent_total != len) {
LOG("send requested = %zu, sent = %zu", len, sent_total);
}
return sent_total;
}
Update to address #Myst's comments:
Although the question did not mention it explicitly, I assumed that the sockets used are blocking, since there are no fcntl call. With that in mind, the following from send() man page explains the situation:
When the message does not fit into the send buffer of the socket,
send() normally blocks, unless the socket has been placed in
nonblocking I/O mode.
In nonblocking mode it would fail with the
error EAGAIN or EWOULDBLOCK in this case. The select(2) call may be
used to determine when it is possible to send more data.
For non-blocking socket, the design need to be different and is outside the scope of this discussion.
I'm writing a peer 2 peer chat application using TCP. This application includes client and server part in one file. I'm using select() without using fork(), pthread to handle connections. This is my mechanism of application, I run the application on a same host by running application on different terminals:
Initialize the first node, let say P1, as ./p2p portToListen.
This node will open a socket on portToListen, let say ServerSock1, to listen connections from other peers.
And then it goes to a while loop with a select() function to wait for events.
Initialize the second node, let say P2, as ./p2p portToListen portToConnectTo (I don't specify IP address here since I'm running on a local machine).
This also opens a new socket, let say ServerSock2, on "portToListen" as the first one, opens a new socket, let say ClientSock2 to connect to the first one. And then goes into the while loop.
When P2 connects to P1, at P1, it also accepts a new connection with a socket, ClientSock1.
After some phases for setting network information and exchanging configuration of the network(P1 sends information of current group to P2 and waits for ACK from P2), they are able to send chatting messages.
My question is, in this case, do I need to close the socket every time a peer sends/receives configuration information(not chatting message). For example, after P1 sends information of current group, do I need to close both ServerSock1 and ClientSock1, and after P2 sends ACK, do I need to close both ServerSock2 and ClientSock2? I think, ServerSock1,2 should always be opened? And only be closed outside of while loop?
If I do that, select() will not work since there is no socket to monitor events. When two peers would like to send chatting message, they need to open sockets again for Server side and Client side, send information to set up a new connection, by calling (socket(), bind(), listener(), accept(), connect()), send some chatting messages and close() sockets again.
Furthermore, if I want to send a broadcasting messages to other peers in a same group, I need to open sockets of peers again, send a chatting message and close() every socket?
In general, since I'm using only select(), what is a correct way to close() and open sockets? If it's possible, can you give me a general scenario for this? I really appreciate for your any comments. Thanks very much. Below is my general code:
int main(void)
{
int sock;
fd_set socks;
fd_set readsocks;
int maxsock;
int reuseaddr = 1; /* True */
struct addrinfo hints, *res;
/* Get the address info */
memset(&hints, 0, sizeof hints);
hints.ai_family = AF_INET;
hints.ai_socktype = SOCK_STREAM;
if (getaddrinfo(NULL, PORT, &hints, &res) != 0) {
perror("getaddrinfo");
return 1;
}
/* Create the socket */
sock = socket(res->ai_family, res->ai_socktype, res->ai_protocol);
if (sock == -1) {
perror("socket");
return 1;
}
/* Enable the socket to reuse the address */
if (setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &reuseaddr, sizeof(int)) == -1) {
perror("setsockopt");
return 1;
}
/* Bind to the address */
if (bind(sock, res->ai_addr, res->ai_addrlen) == -1) {
perror("bind");
return 1;
}
freeaddrinfo(res);
/* Listen */
if (listen(sock, BACKLOG) == -1) {
perror("listen");
return 1;
}
/* Set up the fd_set */
FD_ZERO(&socks);
FD_SET(sock, &socks);
FD_SET(0, &socks);
maxsock = sock;
if (argc > 2)
{
clientSock2 = ConnectToServer(IPaddres, portToConnect);
FD_SET(clientSock2, &socks);
}
/* Main loop */
while (1) {
unsigned int s;
readsocks = socks;
if (select(maxsock + 1, &readsocks, NULL, NULL, NULL) == -1) {
perror("select");
return 1;
}
for (s = 0; s <= maxsock; s++) {
if (FD_ISSET(s, &readsocks)) {
printf("socket %d was ready\n", s);
if (s == sock) {
/* New connection */
int clientSock1;
struct sockaddr_in their_addr;
size_t size = sizeof(struct sockaddr_in);
clientSock1 = accept(sock, (struct sockaddr*)&their_addr, &size);
if (newsock == -1) {
perror("accept");
}
else {
printf("Got a connection from %s on port %d\n",
inet_ntoa(their_addr.sin_addr), htons(their_addr.sin_port));
FD_SET(clientSock1, &socks);
if (clientSock1 > maxsock) {
maxsock = clientSock1;
}
}
}
else {
/* Handle send, recv() information of network */
handle(s, &socks);
}
}
}
if (FD_ISSET(0, &readset) { // Handle input
// Sending chatting message
}
}
close(sock);
return 0;
}
You should reuse the same connection. You may need to adjust your protocol so you know exactly where a request and a response finish and a new one starts.
No, you don't have to close() anything unless you want to terminate a connection or stop listening for new ones.
The listen() socket will continue to listen for new connection attempts even after you accept() one, just keep checking for new connections with select() and accept() them aswell, you don't have to recreate this socket.
An estabilished TCP session is a two-way stream of data where you can safely send() and recv() multiple times as long as you need to communicate with that peer. Reconnecting after each block of information is pointless.
Use select() in the accepted connections too to check if there is information ready to recv().
Note you have to rebuild the fd_set every time you call select() on it. The way your code it doing it is wrong.
I have a problem with a server socket under Linux. For some reason unknown to me the server socket vanishes and I get a Bad file descriptor error in the select call that waits for an incomming connection. This problem always occurs when I close an unrelated socket connection in a different thread. This happens on an embedded Linux with 2.6.36 Kernel.
Does anyone know why this would happen? Is it normal that a server socket can simply vanish resulting in Bad file descriptor?
edit:
The other socket code implements a VNC Server and runs in a completely different thread. The only thing special in that other code is the use of setjmp/longjmp but that should not be a problem.
The code that create the server socket is the following:
int server_socket = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);
struct sockaddr_in saddr;
memset(&saddr, 0, sizeof(saddr));
saddr.sin_family = AF_INET;
saddr.sin_addr.s_addr = htonl(INADDR_ANY);
saddr.sin_port = htons(1234);
const int optionval = 1;
setsockopt(server_socket, SOL_SOCKET, SO_REUSEADDR, &optionval, sizeof(optionval));
if (bind(server_socket, (struct sockaddr *) &saddr, sizeof(saddr)) < 0) {
perror("bind");
return 0;
}
if (listen(server_socket, 1) < 0) {
perror("listen");
return 0;
}
I wait for an incomming connection using the code below:
static int WaitForConnection(int server_socket, struct timeval *timeout)
{
fd_set read_fds;
FD_ZERO(&read_fds);
int max_sd = server_socket;
FD_SET(server_socket, &read_fds);
// This select will result in 'EBADFD' in the error case.
// Even though the server socket was not closed with 'close'.
int res = select(max_sd + 1, &read_fds, NULL, NULL, timeout);
if (res > 0) {
struct sockaddr_in caddr;
socklen_t clen = sizeof(caddr);
return accept(server_socket, (struct sockaddr *) &caddr, &clen);
}
return -1;
}
edit:
When the problem case happens i currently simply restart the server but I don't understand why the server socket id should suddenly become an invalid file descriptor:
int error = 0;
socklen_t len = sizeof (error);
int retval = getsockopt (server_socket, SOL_SOCKET, SO_ERROR, &error, &len );
if (retval < 0) {
close(server_socket);
goto server_start;
}
Sockets (file descriptors) usually suffer from the same management issues as raw pointers in C. Whenever you close a socket, do not forget to assign -1 to the variable that keeps the descriptor value:
close(socket);
socket = -1;
As you would do to C pointer
free(buffer);
buffer = NULL;
If you forget to do this yo can later close socket twice, as you would free() memory twice if it was a pointer.
The other issue might be related to the fact that people usually forget: file descriptors in UNIX environment start from 0. If somewhere in the code you have
struct FooData {
int foo;
int socket;
...
}
// Either
FooData my_data_1 = {0};
// Or
FooData my_data_2;
memset(&my_data_2, 0, sizeof(my_data_2));
In both cases my_data_1 and my_data_2 have a valid descriptor (socket) value. And later, some piece of code, responsible for freeing FooData structure may blindly close() this descriptor, that happens to be you server's listening socket (0).
1- close your socket:
close(sockfd);
2- clear your socket file descriptor from select set:
FD_CLR(sockfd,&master); //opposite of FD_SET
You don't distinguish the two error cases in your code, both can fail select or accept. My guess is that you just have a time out and that select returns 0.
print retval and errno in an else branch
investigate the return value of accept seperately
ensure that errno is reset to 0 before each of the system calls
In Linux once you create a connection and it get closed then you have to wait for some time before making new connection.
As in Linux, socket doesn't release the port no. as soon as you close the socket.
OR
You reuse the socket, then bad file descriptor want come.