Understanding USBIP bind function - c

I am trying to understand the USBIP tool code from Linux (https://github.com/torvalds/linux/tree/master/tools/usb/usbip).
USBIP has a command that attaches a remote USB from an IP to a client system. If you look at the code (https://github.com/torvalds/linux/blob/master/tools/usb/usbip/src/usbip_bind.c#L130) below, which binds the USB, if the function calls close(sockfd); and close the socket, then how the communication is done between client and server for USB data.
From Docs (https://docs.kernel.org/usb/usbip_protocol.html):
Once the client knows the list of exported USB devices it may decide
to use one of them. First the client opens a TCP/IP connection to the
server and sends an OP_REQ_IMPORT packet. The server replies with
OP_REP_IMPORT. If the import was successful the TCP/IP connection
remains open and will be used to transfer the URB traffic between the
client and the server.
It says connection remains open if import is successful, then what is the purpose of close(sockfd);. I can also see sockfd is sent inside query_import_device(sockfd, busid); but if it is closed how this is used?
static int attach_device(char *host, char *busid)
{
int sockfd;
int rc;
int rhport;
// creates a TCP connection to the specified host on the specified port.
sockfd = usbip_net_tcp_connect(host, usbip_port_string);
if (sockfd < 0) {
err("tcp connect");
return -1;
}
// sends a query to the connected host to import the device specified by "busid".
rhport = query_import_device(sockfd, busid);
if (rhport < 0)
return -1;
// closes the previously established TCP connection.
close(sockfd);
// records details of the connection, such as the host and port, the busid of the device, and the assigned rhport.
rc = record_connection(host, usbip_port_string, busid, rhport);
if (rc < 0) {
err("record connection");
return -1;
}
return 0;
}

If you look at the source code of usbip helper program, query_import_device(), it establishes the connection and then calls import_device() to finish it.
In turn, import_device() calls usbip_vhci_attach_device(), that calls usbip_vhci_attach_device2()... that eventually writes the sockfd and some extra data to the /sys/*/attach pseudo file for the given device.
To see what happens next we need to go to kernel mode. So, looking at the source code of the usbip, at function attach_store(). This function parses the sockfd back into an integer file descriptor and, most notably, does this:
struct socket *socket;
socket = sockfd_lookup(sockfd, &err);
This converts a file descriptor into a real kernel socket object. And it increases the reference count. Note how if any of the further checks fail, it calls sockfd_put(socket); to decrease that reference count.
But if the function succeeds, the socket is stored into the device. It is just as if an userland program had done a call to dup(): it doesn't matter if the original sockfd is closed, the kernel keeps the actual socket opened as long as it is needed.
You can see that the value of sockfd is also stored in the device, but it isn't actually used for anything other than reading back from /sys, so it doesn't matter if it is no longer a valid file descriptor.
So the close is actually necessary because the kernel dups the socket internally, that keeps the connection opened. And you do not want to keep an extra FD referencing the same socket, if you try to use that socket for anything that could mess up the device connection. The responsible thing to do is just to close it.

Related

systemd socket activation, sd_listen_fds return 0 fd

I am writing a sample systemd service to test few things.
unfortunately, the test code is not working.
(i expected it to be simple ..)
$>cat test.socket
[Unit]
Description=TEST-SERVICE
[Socket]
ListenStream=5575
Accept=yes
[Install]
WantedBy=sockets.target
$>cat test#.service
[Unit]
Description=TEST-SERVICE
Requires=test.socket
[Service]
StandardInput=socket
PIDFile=/var/run/test.pid
ExecStart=/usr/bin/server_d
Type=simple
[Install]
WantedBy=default.target
The server code looks like below
int main(int argc, char *argv[]){
char buffer[255]; //Temporary buffer to read and write data
int sockfd, newsockfd ; //File descriptor for Sockets
struct sockaddr_in cli_addr; //Structure to hold Client details
socklen_t cli_len; //Client length
int nfd = sd_listen_fds(0);
if (nfd != 1) {
fprintf(stderr, "No or too many file descriptors received.\n");
fprintf(stderr, "FD Count: %d.\n",nfd);
exit(1);
}
sockfd = SD_LISTEN_FDS_START + 0;
The output comes when I connect to the service as below.
>telnet 15.218.114.55 5575
Trying 15.218.114.55...
Connected to 15.218.114.55.
Escape character is '^]'.
No or too many file descriptors received.
FD Count: 0.
Connection closed by foreign host.
It looks to me like sd_listen_fds(0) is returning 0.
what could be the reason for it here?
Thanks for the help in advance.
There are 2 different ways a socket unit works. With Accept=yes, systemd will do the bind() and listen() calls for each LISTEN...=... socket, then wait with select() on them all. Whenever there is a connection, a socket becomes ready, and systemd will do an accept() call on the socket connection, then pass the new socket fd to the service unit ready to use with read/write etc. Systemd then carries on waiting for new connections, and will start a new service with each of them similarly.
In this case the (template) service does not use sd_listen_fds(). If you have StandardInput=socket, then you can simply read/write fd 0.
The second type of socket unit has Accept=no (the default if not specified). Systemd will do the bind() and listen() calls for each LISTEN...=... socket, then wait with select() on them all. As soon as the first socket is ready, it will start the service once only and pass all the open socket file descriptors to it. This is where the sd_listen_fds() api comes in. It says how many socket file descriptors got passed. They start at 3 (SD_LISTEN_FDS_START).
The service application now has to find out which of the passed sockets was ready. Typically, there is only one LISTEN line in the socket unit, so only one fd is passed, and so it is easy (otherwise simply do a select()).
The service application now has to do an accept() on the ready socket, and can then read/write to the new fd. Normally, it would also continue to wait for more connections on the listened-to sockets it was given, do more accepts, and so on.

C sockets: using shutdown() to stop server running in background

I have implemented end_server() method using some volatile flag should_server_end = true/false. I have used non-blocking connection sockets to enable checking this flag between consecutive recv() calls. It works fine. But I have read about using shutdown(sock, SHUT_RDWR) called from the main thread that can stop the server (and its connections) running in the background. I would like to try this approach in my app and implement some alternative methods instead of end_server() like shutdown_server().
I have tried something like this:
int pasv_sock = server_info_sock(server_info);
if(shutdown(pasv_sock, SHUT_RDWR) < 0) {
fprintf(stderr, "shutdown: failed! %s\n", strerror(errno));
return FAILURE;
}
But now I am getting error message:
shutdown: failed! Socket is not connected
which means shutdown() return this error code:
ENOTCONN
The specified socket is not connected.
1. Can I only use shutdown on active (connection) sockets and not on passive (server) socket. Should I just use close()?
Next I change shutdown() to close() on passive socket, and then nothing happens. No errors but as in the previous method with shutdown connection still works correctly and I can send() and recv() packets of data.
2. Does it mean that close()-ing passive socket only stops possibility of making new connections with the server (server will no longer accept connections?)
So I have changed the code to something like this:
static void shutdown_conn_sock_data_handler(void *data, size_t data_size) {
sock_fd_t *conn_sock = (sock_fd_t *) data;
printf("Connection sock: %d closing...!\n", *conn_sock);
if(shutdown(*conn_sock, SHUT_RDWR) < 0) {
fprintf(stderr, "shutdown: failed! %s\n", strerror(errno));
return;
}
}
server_info_set_force_shut_down(server_info, 1);
const linked_list_t *conn_socks = server_info_conn_socks(server_info);
linked_list_travers(conn_socks, shutdown_conn_sock_data_handler);
int pasv_sock = server_info_sock(server_info);
if(close(pasv_sock) < 0) {
fprintf(stderr, "close: failed! %s\n", strerror(errno));
return FAILURE;
}
return SUCCESS;
}
It works now but this need also some flag to give the hint information about the closed server, otherwise, it will be closed with some error message as trying to accept new connections on the already closed passive socket.
So before trying to accept a new connection I need to check like this:
while(1) {
if(server_info_should_shut_down(server_info)) {
return CLOSED;
}
if(server_info_force_shut_down(server_info)) {
return FORCE_CLOSED;
}
As you can see such a force close approach doesn't differ much from lazy shutdown when I just set volatile should_shut_down flag and wait for the server to detect this and close in a regular way. The only benefit is that I possibly no longer have to have:
non-blocking connection sockets in connection_handlers (this functions are supplied by client code using server api)
before each client code need to set:
fcntl(sock_fd, F_SETFL, O_NONBLOCK);
to enable server self-closing.
*client - means programmer using server API, not client side of TCP communication.
moreover there was need to place after each recv failing without new request data
if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) {
// call to recv() on non-blocking socket result with nothing to receive
continue;
}
and client-code needs to add in connection_handler in between each client-side request:
if(server_info_should_shut_down(server_info))
return CLOSED;
So implementing this shutdown_server() method instead of end_server()
I can hide implementation details inside server API and allow user of this API to provide simpler and cleaner connection handler. Just recv/send logic without need to inject some special code that enables the server to be closable!
3. Is it this new approach with shutdown() correct? Didn't I missed anything?
Can I only use shutdown on active (connection) sockets and not on passive (server) socket.
Yes.
Should I just use close()?
Yes.
Next I change shutdown() to close() on passive socket, and then nothing happens. No errors but as in the previous method with shutdown connection still works correctly and I can send() and recv() packets of data.
Correct. That's how it works.
Does it mean that close()-ing passive socket only stops possibility of making new connections with the server (server will no longer accept connections?)
Yes. It doesn't affect existing accepted sockets.
Is it this new approach with shutdown() correct? Didn't I missed anything?
You should not shutdown the sockets for output. That will cause errors at both ends: this end, because it may write to a shutdown socket, and the other end because it will receive a truncation.
All you need to to is shutdown each accepted socket for input (i.e. SHUT_RD). That will cause the next recv() on that socket to return zero,meaning the peer disconneceted, whereupon the existing code should already close the socket and exit the thread.

"tcp session control protocol" in linux kernel?

I'm programming in Linux environment.
I want to establish a TCP socket between two PCs, and multiplexing it.
To be specific, here is the pseudo-code:
void func_in_process0()
{
socket s = socket(AF_INET, SOCK_STREAM, 0);
connect(s, "1.1.1.1", 8080);
socket s1 = ADD_CHANNEL(s, 1); // key part
SEND_FILE_DESCRIPTOR_TO_PROCESS(s1, process1);
socket s2 = ADD_CHANNEL(s, 2); // key part
SEND_FILE_DESCRIPTOR_TO_PROCESS(s2, process2);
WAIT_FOR_PROCESS_TO_FINISH(process1);
WAIT_FOR_PROCESS_TO_FINISH(process2);
close(s);
}
void func_in_process1()
{
socket s1 = GET_SOCKET_FROM_PROCESS0();
send(s1, "abcdefg");
close(s1);
}
void func_in_process2()
{
socket s2 = GET_SOCKET_FROM_PROCESS0();
send(s2, "abcdefg");
close(s2);
}
The key part is how to implement function ADD_CHANNEL.
I found a document TCP Session Control Protocol which provides exactly the function I want.
But I don't think it is implemented in Linux kernel.
I can implement this protocol in userspace, but variable "s1" and "s2" will not be file descriptor and can't be sent to other processses.
Edit:
process0, process1 and process2 are all on localhost.
socket "s" is a client socket.
1.1.1.1 is a remote PC, who is the server. so proc1 and proc2 both sends packet to 1.1.1.1 through the CHANNEL.
the server is CHANNEL aware too.
Edit2:
I read the SCP Protocol, but it still has no in-kernel implementation.
I learned the SCTP multi-streaming, which is implemented in kernel, but there's no API that creates sub-socket so the problem remains.
Sure, they can be file descriptors. Create two pipes and give one end to each of the other two processes. Your 'multiplexor' will read from, and write two, the other ends of the two bidirectional pipes.

how to kill a tcp connection in a tcp server program if no FIN/ACK or RST received

I wrote a tcp server program(linux c) and run it on host B
if host A establishes a TCP connection with host B
then A shutdown without sending FIN/ACK
how do I write source codes inside tcp server to kill this tcp connection?
use raw socket to craft s fake RST?
or other ways?
thanks!
Just close() the server's end of the socket once it has determined that the connection is no longer available. Eventually, the socket will time out internally and start reporting errors to read/write operations, at which time you can then close it. If you do not want to wait that long then implement a timeout in your own code, either as a keepalive/ping command in your protocol, or just as a simple timer that keeps track of the last time the client exchanged any data with the server. If the timeout period expires, close the socket regardless of its actual state.
I'm not sure in this case, but if you create a binary file with all the open connections:
FILE *ptr_myfile;
ptr_myfile=fopen("test.bin","wb"); //opened sockets
//.. your code ..
sockfd[k] = socket(AF_INET, SOCK_STREAM, 0);
fwrite(&sockfd[k], sizeof(sockfd[k]), 1, ptr_myfile_soc);
//.. your code ..
close(ptr_myfile);
and if host A shuts down without sending FIN/ACK, in another executable you will read from that file and close the related sockets:
fread(&sockfd[k],sizeof(sockfd[k]),1,ptr_myfile_soc);
printf("closing %d \n",k );
close(sockfd[k]);

Server program is stuck at send

I am building a server client model in C. The clients connects to the server and they start exchanging data. However, the user can end the client at any time in the program, but the server is not notified about it. The server keeps sending that data even after the client is closed.
I was in the impression that send function will return -1 if the server is unable to send the data, but my server program just stuck at send
if((byteSent = send(new_fd, fileContents, strlen(fileContents), 0)) == -1){ //
the program just halts at the above line.
How do I overcome this problem?
//Code
exitT = 0;
//execution_count = 1;
for(i=0;i<execution_count;i++)
{
sleep(time_delay);
//getting the current time on the server machine
time_t t;
time(&t);
char *time=ctime(&t);
printf("The Execution time at server = %s\n",time);
system(exec_command);
/*Open the file, get file size, read the contents and close the file*/
// Open the file
fp = fopen(fileName,"r");
// Get File Size
fseek(fp,0,SEEK_END);
dataLength = ftell(fp);
rewind(fp);
fileContents = (char*)malloc(dataLength+1);
// Read File
fread(fileContents,1,dataLength,fp);
fileContents[dataLength] = '\0';
// Close file
fclose(fp);
printf("sockfd = %d \n",new_fd);
// send file length to client
rc=send(new_fd, &dataLength, sizeof(dataLength), 0) ;
printf("length of client data = %d \n",rc);
printf("sockfd = %d \n",new_fd);
// send time to client
rc=send(new_fd, time, strlen(time), 0) ;
printf("length of client time = %d \n",rc);
usleep(20000);
// Send file contents to Client
while(dataLength>0){
printf("sockfd = %d \n",new_fd);
if((byteSent = send(new_fd, fileContents, strlen(fileContents), 0)) == -1){
printf("bytes sent = %d \n",byteSent);
exitT = 1;
break;
}
dataLength-=byteSent;
}
//Delete the log file
sprintf(deleteCommand,"rm %s",fileName);
system(deleteCommand);
if(exitT == 1)
break;
}
bzero(fileName,sizeof(fileName));
bzero(exec_command,sizeof(exec_command));
bzero(deleteCommand,sizeof(deleteCommand));
//decClientNum();
kill(parent_id,SIGALRM);
close(new_fd); // parent doesn't need this
printf("STATUS = CLOSED\n");
exit(0);
}
Thanks
I assume you are coding for a Linux or Posix system.
When a syscall like send fails it returns -1 and sets the errno; you very probably should use errno to find out why it failed.
You could use strace to find out which syscalls are done by your sever, or some other one. Of course, use also the gdb debugger.
You very probably need to multiplex inputs or outputs. The system calls doing that are poll, select (and related ppoll and pselect). Read e.g. the select_tut(2) man page.
You may want to use (or at least to study the source code of) existing event oriented libraries like libevent, libev etc.. (Both Gtk and Qt frameworks provide also their own, which might be used even outside of GUI applications).
I strongly suggest reading about advanced unix programming and unix network programing (and perhaps also about advanced linux programming).
maybe you're using a tcp protocol and the server is waiting for an ACK. Try using udp if you want your connection to be asynchronous.
From the man page: No indication of failure to deliver is implicit in a send(). Locally detected errors are indicated by a return value of -1.
Proably something like this might help: http://stefan.buettcher.org/cs/conn_closed.html
I think I am pretty late in the party, but I think this answer might help someone.
If space is not available at the sending socket to hold the message to be transmitted, and the socket file descriptor does not have O_NONBLOCK set, send() shall block until space is available.
When send() function gets stuck, there might be a situation like, TCP window size has become 0. It happens when the other end of the connection is not consuming received data.
There might be a scenario like this, the receiving end process is running by GDB and segfault occurred.
The TCP connection remains established.
Data is being send continuously.
The receiver end is not consuming it.
Consequently the receiver TCP window size will keep decreasing and you can send data till it is greater than zero. Once it becomes 0, send() function will get stuck forever.
As the situation mentioned in the question is not a scenario of closed connection. When a process writes something on a closed TCP connection, it receives a signal SIGPIPE. Default handler of SIGPIPE terminates the process. So, in a closed connection scenario if you are not using your own SIGPIPE handler then process should be terminated by default handler whenever something is written on the socket.

Resources