I followed a tutorial on how to make two processes on Linux communicate using the Linux Sockets API, and that's the code it showed to make it happen:
Connecting code:
char* socket_path = "\0hidden";
int fd = socket(AF_UNIX, SOCK_STREAM, 0);
struct sockaddr_un addr;
memset(&addr, 0x0, sizeof(addr));
addr.sun_family = AF_UNIX;
*addr.sun_path = '\0';
strncpy(addr.sun_path+1, socket_path+1, sizeof(addr.sun_path)-2);
connect(fd, (struct sockaddr*)&addr, sizeof(addr));
Listening code:
char* socket_path = "\0hidden";
struct sockaddr_un addr;
int fd = socket(AF_UNIX, SOCK_STREAM, 0);
memset(&addr, 0x0, sizeof(addr));
addr.sun_family = AF_UNIX;
*addr.sun_path = '\0';
strncpy(addr.sun_path+1, socket_path+1, sizeof(addr.sun_path)-2);
bind(fd, (struct sockaddr*)&addr, sizeof(addr));
listen(fd, 5);
Basically, I have written a web server for a website in C, and a database management system in C++, and making them communicate (after a user's browser sends an HTTP request to my web server, which it's listening for using an AF_INET family socket, but that's not important here, just some context) using this mechanism.The database system is listening with its socket, and the web server connects to it using its own socket. It's been working perfectly fine.
However, I never understood what the purpose of a null byte at the beginning of the socket path is. Like, what the heck does "\0hidden" mean, or what does it do? I read the manpage on sockets, it says something about virtual sockets, but it's too technical for me to get what's going on. I also don't have a clear understanding of the concept of representing sockets as files with file descriptors. I don't understand the role of the strncpy() either. I don't even understand how the web server finds the database system with this code block, is it because their processes were both started from executables in the same directory, or is it because the database system is the only process on the entire system listening on an AF_UNIX socket, or what?
If someone could explain this piece of the Linux Sockets API that has been mystifying me for so long, I'd be really grateful. I've googled and looked at multiple places, and everyone simply seems to be using "\0hidden" without ever explaining it, as if it's some basic thing that everyone should know. Like, am I missing some piece of theory here or what? Massive thanks to anybody explaining in advance!
This is specific to the Linux kernel implementation of the AF_UNIX local sockets. If the character array which gives a socket name is an empty string, then the name doesn't refer to anything in the filesystem namespace; the remaining bytes of the character array are treated as an internal name sitting in the kernel's memory. Note that this name is not null-terminated; all bytes in the character array are significant, regardless of their value. (Therefore it is a good thing that your example program is doing a memset of the structure to zero bytes before copying in the name.)
This allows applications to have named socket rendezvous points that are not occupying nodes in the filesystem, and are therefore are more similar to TCP or UDP port numbers (which also don't sit in the file system). These rendezvous points disappear automatically when all sockets referencing them are closed.
Nodes in the file system have some disadvantages. Creating and accessing them requires a storage device. To prevent that, they can be created in a temporary filesystem that exists in RAM like tmpfs in Linux; but tmpfs entries are almost certainly slower to access and take more RAM than a specialized entry in the AF_UNIX implementation. Sockets that are needed temporarily (e.g. while an application is running) may stay around if the application crashes, needing external intervention to clean them up.
hidden is probably not a good name for a socket; programs should take advantage of the space and use something quasi-guaranteed not to clash with anyone else. The name allows over 100 characters, so it's probably a good idea to use some sort of UUID string.
The Linux Programmer's Manual man page calls this kind of address "abstract". It is distinct and different from "unnamed".
Any standard AF_UNIX implementation provides "unnamed" sockets which can be created in two ways: any AF_UNIX socket that has been created with socket but not given an address with bind is unamed; and the pair of sockets created by socketpair are unnamed.
For more information, see
man 7 unix
in some GNU/Linux distro that has the Linux Man Pages installed.
\0 just puts a NUL character into the string. As a NUL characters is used to terminate a string, to all C string functions socket_path looks like an empty string, while in fact it is not but they would stop processing it after the first character.
So im memory socket_path actually looks like this:
char socket_path[] = { `\0`, `h`, `i`, `d`, `d`, `e`, `n`, `\0` };
As all strings automatically get a terminating NUL attached.
The line
strncpy(addr.sun_path+1, socket_path+1, sizeof(addr.sun_path)-2);
copies the bytes of socket_path to the socket address structure addr, yet skipping the first (NUL) byte as well as the last one (also NUL). Thus the address of the socket effectively is just the word "hidden".
But as the first byte is left out from the addr.sun_path as well and this byte has been initialized to NUL by memset before, the actual path is still \0hidden.
So why would anyone do that? Probably to hide the socket, as normally systems show UNIX sockets in the file system as actual path entries but no file system I'm aware of can handle the \0 character. So if the name has a \0 character, it won't appear in the file system, yet such a characters is only allowed as the very first characters, otherwise the system would still try to create that path entry and fail and thus the socket creating would fail. Only as the first characters, the system will not even try to create it, which means you cannot see that socket by just calling ls in terminal and whoever wants to connect to it needs to know the name.
Note that this is not POSIX conform, as POSIX expects UNIX sockets to always appear in the file system and thus only characters that are legal for the file system in use are allowed as socket name. This will only work on Linux.
Related
I want to write basic chat program. I don't release that, I just want to learn socket programming. The chat program will be between client and server.
server code:
//bla bla code
new_socket = accept(server_fd, (struct sockaddr*)&address,(socklen_t*)&addrlen);
char server_msg[128];
char client_msg[128];
int running = 1;
while(running){
fgets(server_msg, 64, stdin);
send(new_socket, server_msg, strlen(server_msg), 0);
recv(new_socket, client_msg, 128, 0);
}
client code:
char server_msg[128];
char client_msg[128];
int running = 1;
while(running){
fgets(client_msg, 64, stdin);
send(server_socket, client_msg, strlen(client_msg), 0);
recv(server_socket, server_msg, 128, 0);
}
Questions:
Is the new socket fd used only once? That means; Will I do create a new socket for each sending and receiving. Or can I use this forever?
If first question answer is "FOREVER", Will I do something to new socket fd? I don't know maybe clear.
The above code is not working as expected. As I expected. Actually, The code is working very well :D. How do I perform interrupt operations such as getting input, sending messages, receiving messages?
My English is not good, I'm sorry.
Is the new socket fd used only once? That means; Will I do create a new socket for each sending and receiving. Or can I use this forever?
you open a TCP socket connection, so, once in the connected state (the ESTABLISHED state) it remains there until one side closes it (and the other side will notice it by reading 0 bytes as result, this explanation is oversimplified to make it simpler to your case, normally you should detect the end of file in the socket, so you can close it, and accept(2) another connection)
If first question answer is "FOREVER", Will I do something to new socket fd? I don't know maybe clear.
Your program is not a chat program, as the server uses the SAME socket to write what it has read, so it will return the message sent to the same client that issued it (acting more as an echo server, than a chat)
The above code is not working as expected. As I expected. Actually, The code is working very well :D. How do I perform interrupt operations such as getting input, sending messages, receiving messages?
Normally, a chat server should wait for a message to arrive at several sockets (by means of the select(2) system call) and will determine which socket needs attention. As a result, the server will read(2) from that socket (probably in a non blocking way, to avoid being blocked on a read to a socket, that will not be interrupted if data enters on another socket) The chat server should write what it has read over all the other sockets it has connections from, so every other user of your chat system receives a copy of the message (sending back the message to the sender is optional, but not very elegant, the issuer of the message should do local echo of it, although, to ensure the user sees his messages interspersed between the ones from other users)
I recommend you to read the book "Unix Network Programming" from Richard Stevens. It describes perfectly how to use the sockets API and IMHO it is the best reference you can get to learn how to use the socket interface.
One final note:
Don't use plain numbers hardwired in your code, as it is error prone (you declare a buffer string to be 128 bytes, but then read only 64, making 64 unused bytes in the array, you can use sizeof buffer as to make the compiler to use the amount declared in the variable, or you can #define BUFFSIZE 128 and then declare char buffer[BUFFSIZE]; and later do a fgets(buffer, BUFFSIZE, socket_FILE_descriptor); if you use a constant to size a buffer, use a constant name to name it :) and use it everywhere, in case you decide to give it a different value, all the occurences of it will change, instead of having to reviste all places in your code where you have used the number 64.
I'm sending some ping packets via a raw socket in C, on my linux machine.
int sock_fd = socket(AF_INET, SOCK_RAW, IPPROTO_RAW);
This means that I specify the IP packet header when I write to the socket (IP_HDRINCL is implied).
Writing to the socket with send fails, telling me I need to specify an address.
If I use sendto then it works. For sendto I must specify a sockaddr_in struct to use, which includes the fields sin_family, sin_port and sin_addr.
However, I have noticed a few things:
The sin_family is AF_INET - which was already specified when the socket was created.
The sin_port is naturally unused (ports are not a concept for IP).
It doesn't matter what address I use, so long as it is an external address (the IP packet specifies 8.8.8.8 and the sin_addr specifies 1.1.1.1).
It seems none of the extra fields in sendto are actually used to great extent. So, is there a technical reason why I have to use sendto instead of send or is it just an oversight in the API?
Writing to the socket with send fails, telling me I need to specify an address.
It fails, because the send() function can only be used on connected sockets (as stated here). Usually you would use send() for TCP communication (connection-oriented) and sendto() can be used to send UDP datagrams (connectionless).
Since you want to send "ping" packets, or more correctly ICMP datagrams, which are clearly connectionless, you have to use the sendto() function.
It seems none of the extra fields in sendto are actually used to great
extent. So, is there a technical reason why I have to use sendto
instead of send or is it just an oversight in the API?
Short answer:
When you are not allowed to use send(), then there is only one option left, called sendto().
Long answer:
It is not just an oversight in the API. If you want to send a UDP datagram by using an ordinary socket (e.g. SOCK_DGRAM), sendto() needs the information about the destination address and port, which you provided in the struct sockaddr_in, right? The kernel will insert that information into the resulting IP header, since the struct sockaddr_in is the only place where you specified who the receiver will be. Or in other words: in this case the kernel has to take the destination info from your struct as you don't provide an additional IP header.
Because sendto() is not only used for UDP but also raw sockets, it has to be a more or less "generic" function which can cover all the different use cases, even when some parameters like the port number are not relevant/used in the end.
For instance, by using IPPROTO_RAW (which automatically implies IP_HDRINCL), you show your intention that you want to create the IP header on your own. Thus the last two arguments of sendto() are actually redundant information, because they're already included in the data buffer you pass to sendto() as the second argument. Note that, even when you use IP_HDRINCL with your raw socket, the kernel will fill in the source address and checksum of your IP datagram if you set the corresponding fields to 0.
If you want to write your own ping program, you could also change the last argument in your socket() function from IPPROTO_RAW to IPPROTO_ICMP and let the kernel create the IP header for you, so you have one thing less to worry about. Now you can easily see how the two sendto()-parameters *dest_addr and addrlen become significant again because it's the only place where you provide a destination address.
The language and APIs are very old and have grown over time. Some APIs can look weird from todays perspective but you can't change the old interfaces without breaking a huge amount of existing code. Sometimes you just have to get used to things that were defined/designed many years or decades ago.
Hope that answers your question.
The send() call is used when the sockets are in a TCP SOCK_STREAM connected state.
From the man page:
the send() call may be used only when the socket is in a connected
state (so that the intended recipient is known).
Since your application obviously does not connect with any other socket, we cannot expect send() to work.
In addition to InvertedHeli's answer, the dest_addr passed in sendto() will be used by kernel to determine which network interface to used.
For example, if dest_addr has ip 127.0.0.1 and the raw packet has dest address 8.8.8.8, your packet will still be routed to the lo interface.
I'm not an expert in C programming, but I'm trying to write a fairly simple program using sendmsg() and recvmsg() to send a message between a client and a server (both are on the same machine, so basically I'm sending a message to localhost).
After initialising the required structures (as in the iovec and the msghdr) and succesfully connecting the client to the server, my sendmsg() call fails with "no buffer space avaliable" errno.
This is what linux man reports about this type of error:
The output queue for a network interface was full. This generally indicates that the interface has stopped sending, but maybe caused by transient congestion. (Normally, this does not occur in Linux. Packets are just silently dropped when a device queue overflows.)
I looked around on the Internet and as a result I found out that sendmsg() is not widely used, and nobody could relate with this type of error. The only useful advice I found was to check a possible excess of open sockets, but again I always close EVERY socket I create.
So I'm stuck, basically because being quite the noob I don't know exactly where to look to fix this kind of problem.
If anybody knows how to proceed, it would be great.
(And please don't tell me not to use sendmsg(), because the whole purpose of my work is to understand this syscall, and not send a message to myself)
Here's the code I've written so far on pastebin: client and server
--SOLVED--
Thank you very much. I've been able to solve the problem and I fixed other mistakes I made, so here's the functioning code for sendmsg() and recvmsg() working message-passing: Client and Server
As others have pointed out, iovlen should be 1. But also, you want to zero out mh before initializing some of its fields since you're probably sending in garbage in the uninitialized fields and the syscall gets confused. Also, it doesn't really make sense to set msg_name and msg_namelen since you're connected and can't change your mind about where to send the data anyway.
This is what works for me in your client code:
/* The message header contains parameters for sendmsg. */
memset(&mh, 0, sizeof(mh));
mh.msg_iov = iov;
mh.msg_iovlen = 1;
printf("mh structure initialized \n");
The msg_iovlen field contains the number of elements in the iov array, not its size in bytes.
The system interpreted the following uninitialized memory as iov elements, ended up with a packet that is larger than the socket buffer space available, and thus refused to send the data.
Okay, so in your code I found this:
mh.msg_iovlen = sizeof(iov);
Which sets the msg_iovlen member to the size of struct iovec. But the documentation says this about this field:
size_t msg_iovlen; /* # elements in msg_iov */
So your code is wrong, it tells sendmsg() that it's going to send way more elements than you actually initialize.
This may be a bit difficult to enumerate succinctly but I will give it my best on my novice understanding of the domain and problem.
I have 2 processes, one stream server who first unlinks, creates a socket descriptor, binds, listens, and accepts on a local unix socket. The job of the server is to accept a connection, send arbitrary data, and also receive arbitrary data. The client process' job is to do the same as the server with the exception of the initial setup; create a socket descriptor, and connect to the unix socket.
Upon launching the server, I can verify the unix socket is being created. Upon launching the client, I receive a connect() error stating the file or directory doesn't exist or invalid. And yes, attempting to locate the unix socket as before, the file no longer exists...
Does anyone know why or where in the bug may lie that is causing this behavior?
If code snippets would be helpful to clarify, I can certainly post those as well.
struct addrinfo * server;
int sockfd;
sockfd = socket( server->ai_family, server->ai_socktype, server->ai_protocol );
if( connect(sockfd, server->ai_addr, server->ai_addrlen) == 0 )
return sockfd;
else
perror("connect()");
It's probably also worth noting that I'm using a modified version of getaddrinfo to populate the addrinfo struct for the unix domain specifically.
Following the server startup, check that the socket file exists on the client system i.e. make sure that the file you're going to use in the sun_path field of the struct sockaddr_un passed into the connect on the client exists. This entry must match the one that was created in the server and passed into the bind. Also make sure that you are populating the sun_family field in both the client and the server with AF_UNIX.
In the client do not perform any creation/deletion of the socket file - i.e there should not be an unlink anywhere in the client code related to the location of the server socket.
These are the general processes I would follow to ensure that the code is doing the right thing. There is a sample server/client in the old, but still reliable Beej's guide to UNIX IPC which is probably the simplest example you should be comparing to.
Edit Based on the discussion in the comments, it turns out that the custom getaddrinfo call is the culprit in the deletion of the unix socket file. This is because there is server-side logic in the code which checks if hints->ai_flags & AI_PASSIVE is set. If this is the case, then it unlinks the socket file, as it expects the software to be performing a bind (as in be a server). The logic about the AI_PASSIVE flag is codified in the RFC, and in that case, the bind would fail if the file does not exist.
If the AI_PASSIVE flag is specified, the returned address information
shall be suitable for use in binding a socket for accepting incoming
connections for the specified service (i.e., a call to bind()).
However, the end sentence of that paragraph states:
This flag is ignored if the nodename argument is not null
So it seems like the logic is slightly incorrect in this case of the call getaddrinfo( "/local", "/tmp/socket", hints, &server), as the nodename parameter is not null.
I am writing an API which includes IPC functions which send data to another process which may be local or on another host. I'd really like the send function to be as simple as:
int mySendFunc(myDataThing_t* thing, int sd);
without the caller having to know -- in the immediate context of the mySendFunc() call -- whether sd leads to a local or remote process. It seems to me that if I could so something like:
switch (socketFamily(sd)) {
case AF_UNIX:
case AF_LOCAL:
// Send without byteswapping
break;
default:
// Use htons() and htonl() on multi-byte values
break;
}
It has been suggested that I might implement socketFamily() as:
unsigned short socketFamily(int sd)
{
struct sockaddr sa;
size_t len;
getsockname(sd, &sa, &len);
return sa.sa_family;
}
But I'm a little concerned about the efficiency of getsockname() and wonder if I can afford to do it every time I send.
See getsockname(2). You then inspect the struct sockaddr for the family.
EDIT: As a side note, its sometimes useful to query info as well, in this case info libc sockets
EDIT:
You really can't know without looking it up every time. It can't be simply cached, as the socket number can be reused by closing and reopening it. I just looked into the glibc code and it seems getsockname is simply a syscall, which could be nasty performance-wise.
But my suggestion is to use some sort of object-oriented concepts. Make the user pass a pointer to a struct you had previously returned to him, i.e. have him register/open sockets with your API. Then you can cache whatever you want about that socket.
Why not always send in network byte order?
If you control the client and server code I have a different suggestion, which I've used successfully in the past.
Have the first four bytes of your message be a known integer value. The receiver can then inspect the first four bytes to see if it matches the known value. If it matches, then no byte swapping is needed.
This saves you from having to do byte swapping when both machines have the same endianness.