Currently I have a daemon listening for TCP/UDP packets on port X using the recvfrom system call:
ssize_t recvfrom(int sockfd, void *buf, size_t len, int flags,
struct sockaddr *src_addr, socklen_t *addrlen);
and then the data is send over a message queue with the msgsnd system call:
int msgsnd(int msqid, struct msgbuf *msgp, int msgsz, int msgflg);
I would like to replace the msgsnd system call with RabbitMQ and send the messages using amqp protocol. The problem I am facing is the format of the data received and send over the RabbitMQ. When I receive data on port then I try to send it like this:
ssize_t recvfrom(
sockfd, msgbufInst.msgContent, len, flags, src_addr, addrlen
);
send_over_rabbitmq(msgbufInst.msgContent);
where send_over_rabbitmq is pretty much
send_over_rabbitmq(char *rawData){
amqp_bytes_t payload;
payload.len = sizeof(rawData);
payload.bytes = rawData;
int result = amqp_basic_publish((*channel)->conn, 1,
amqp_cstring_bytes("myExchange"),
amqp_cstring_bytes(""), 0, 0, NULL, payload);
}
The payload is send and received successfully but some data is lose. When I consume the message from the queue the payload is not the same as when I used the system call to send/consume messages. I am sure that it has to do with the data types that I am using but can't figure it out yet.
As I noted in a comment:
In the send_over_rabbitmq() function, the value of sizeof(rawData) is probably 8, and maybe only 4 — it's the size of the pointer variable, and not the length of the data that it points at. That probably isn't what you want.
You'll need to send more information to the send_over_rabbitmq() function — most noticeably, the length of the data received from recvfrom(). That means capturing the information — you should be checking the return value from recvfrom() anyway before trying to relay information via RabbitMQ.
Related
I'm wondering on what basis src_addr.sin_port while receiving socket data gets filled up in the last but one argument of recvfrom function, syntax of which is
#include <sys/types.h>
#include <sys/socket.h>
ssize_t recvfrom(int sockfd, void *buf, size_t len, int flags,
struct sockaddr *src_addr, socklen_t *addrlen);
And also what's the significance of it, does it help the receiver in any way?
The IP and port that is populated after a successful call to recvfrom is the source IP and port of the received datagram. This can be logged so the user knows where the datagram came from.
Also, if a datagram needs to be sent back to this endpoint, src_addr can be passed to sendto to specify the destination IP and port.
I'm writing a linux C client-server programs that communicates with each other with unix domain sockets and passes couple of buffers each time.
I'm using ioverctors but for some reasons the server program only receives the first io vector.
Any idea ?
I attached the relevant code snippets.
Client code:
struct iovec iov[2];
struct msghdr mh;
int rc;
char str1[] = "abc";
char str2[] = "1234";
iov[0].iov_base = (caddr_t)str1;
iov[0].iov_len = sizeof(str1);
iov[1].iov_base = (caddr_t)str2;
iov[1].iov_len = sizeof(str2);
memset(&mh, 0, sizeof(mh));
mh.msg_iov = iov;
mh.msg_iovlen = 2;
n = sendmsg(sockfd, &mh, 0); /* no flags used*/
if (n > 0) {
printf("Sendmsg successfully executed\n");
}
}
Server code:
{
struct sockaddr_un *client_sockaddr = (sockaddr_un *)opq;
struct msghdr msg;
struct iovec io[2];
char buf[16];
char buf2[16];
io[0].iov_base = buf;
io[0].iov_len = sizeof(buf);
io[1].iov_base = buf2;
io[1].iov_len = sizeof(buf2);
msg.msg_iov = io;
msg.msg_iovlen = 2;
int len = recvmsg(sock, &msg, 0);
if (len > 0) {
printf("recv: %s %d %s %d\n", msg.msg_iov[0].iov_base, msg.msg_iov[0].iov_len, msg.msg_iov[1].iov_base, msg.msg_iov[1].iov_len);
}
return 0;
}
The output i'm getting from the server:
recv: abc 16 16
sendmsg(), writev(), pwritev(), and pwritev2() do not operate on multiple buffers, but one discontiguous buffer. They operate exactly as if you'd allocate a large enough temporary buffer, gather the data there, and then do the corresponding syscall on the single temporary buffer.
Their counterparts recvmsg(), readv(), preadv(), and preadv2() similarly do not operate on multiple buffers, only on one discontiguous buffer. They operate exactly as if you'd allocate a large enough temporary buffer, receive data into that buffer, then scatter the data from that buffer to the discontiguous buffer parts.
Unix domain datagram (SOCK_DGRAM) and seqpacket (SOCK_SEQPACKET) sockets preserve message boundaries, but stream sockets (SOCK_STREAM) do not. That is, using a datagram or seqpacket socket you receive each message as it was sent. With a stream socket, message boundaries are lost: two consecutively sent messages can be received as a single message, and you can (at least in theory) receive a partial message now and the rest later.
You can use the Linux-specific sendmmsg() function to send several messages in one call (using the same socket). If you use an Unix domain datagram or seqpacket socket, these will then retain their message boundaries.
Each message is described using a struct mmsghdr. It contains struct msghdr msg_hdr; and unsigned int msg_len;. msg_hdr is the same as you use when sending a single message using e.g. sendmsg(); you can use more than one iovec for each message, but the recipient will receive them concatenated into a single buffer (but can scatter that buffer using e.g. recvmsg()). msg_len will be filled in by the sendmmsg() call: the number of bytes sent for that particular message, similar to the return value of e.g. sendmsg() call when no errors occur.
The return value from the sendmmsg() call is the number of messages sent successfully (which may be fewer than requested!), or -1 if an error occurs (with errno indicating the error as usual). Thus, you'll want to write a helper function or a loop around sendmmsg() to make sure you send all the messages. For portability, I recommend a helper function, because you can then provide another based on a loop around sendmsg() for use when sendmmsg() is not available.
The only real benefit of sendmmsg() is that you need fewer syscalls to send a large number of messages: it boosts efficiency in certain situations, that's all.
I want to send data in varying sizes over UDP. The size of data to be sent is not fixed. I have the following scenario:
unsigned char buffer[BUFFERSIZE];
int bytes = fill_buffer(buffer, sizeof(buffer)): // Returns number of filled bytes.
sendto(socket, buffer, bytes, 0, (struct sockaddr *)&server, sizeof(server))
In the example above, receiving side does not know how many bytes to receive. I also thought of first sending the number of bytes to receive and then sending the data. But in that case, I don't know what would happen if the packets arrive out-of-order.
Sender side would be
sendto(socket, &bytes, sizeof(bytes), 0, (struct sockaddr *)&server, sizeof(server))
sendto(socket, buffer, bytes, 0, (struct sockaddr *)&server, sizeof(server))
Receving side would be
recvfrom(socket, &bytes, sizeof(bytes), 0, NULL, NULL)
recvfrom(socket, buffer, bytes, 0, NULL, NULL)
But could it be that sent data comes out-of-order?
I think you can send both in a single datagram if you add a message header.
The sender only sends the amount of payload data it has.
The receiver always requests the maximum payload size but examines the header and the return from recvfrom to determine the actual length.
Here's some rough code that illustrates what I'm thinking of:
struct header {
u32 magic_number;
u32 seq_no;
u32 msg_type;
u32 payload_length;
} __attribute__((__packed__));
#define MAXPAYLOAD 1024
struct message {
struct header info;
unsigned char payload[MAXPAYLOAD];
} __attribute__((__packed__));
void
sendone(int sockfd,const void *buf,size_t buflen)
{
struct message msg;
static u32 seqno = 0;
memcpy(&msg.payload[0],buf,buflen);
msg.info.magic_number = 0xDEADADDE;
msg.info.seq_no = seqno++;
msg.info.payload_length = buflen;
sendto(sockfd,&msg,sizeof(struct header) + buflen,...);
}
ssize_t
getone(int sockfd,void *buf,size_t buflen)
{
struct message msg;
ssize_t rawlen;
ssize_t paylen;
static u32 seqno = 0;
rawlen = recvfrom(sockfd,&msg,sizeof(struct header) + MAXPAYLOAD,...);
paylen = msg.info.payload_length;
if (rawlen != (sizeof(struct header) + paylen))
// error ...
memcpy(buf,&msg.payload[0],paylen);
return paylen;
}
The receiver can check the magic number and sequence number to look for corruption or missing/dropped packets, etc.
In fact, you can probably get more efficiency by using sendmsg and recvmsg since they allow you to send a single message using a scatter/gather list. (i.e.) The data would not have to be copied in/out using memcpy from the message struct [you'd only need struct header], so closer to zero copy buffering.
Another option may be to use the MSG_PEEK flag with the recvfrom/recvmsg. I've never used this myself, but it would be something like:
Do recvmsg with length of sizeof(struct header) with MSG_PEEK flag
Do second recvmsg with length of sizeof(struct header) + msg.info.payload_length
This is just a nicety of not having to always provide a maximum sized buffer. Since it involves two syscalls, it may be a bit slower. But, it might allow allow some tricks with selecting a payload buffer from a pool, based on the type of message and/or length
Unlike TCP which is a stream-based protocol, meaning that calls to recv don't exactly correspond to a call to send, UDP is packet based meaning that each recvfrom matches with exactly one sendto. This also means you need to take care of how large each message you send is.
If you send a UDP datagram that is larger that what can be contained in a IP packet, the UDP message will be fragmented across multiple UDP packets, increasing the chance of data loss. That's something you want to avoid. Also, if you're using IPv6, you'll get an error when you attempt to send because IPv6 doesn't support fragmentation.
What does this mean in relation to what you're doing? It means that, roughly speaking, your messages shouldn't be any larger than about 1450 bytes, so you can use that value as the size of your input buffer. Then you can use the return value of recvfrom to see how many bytes were actually read. If your messages are larger than that, you should break them up into multiple messages.
As with any UDP based protocol, you need to account for the case where messages get lost and they need to be retransmitted, or if messages come out of order.
Actually answer to this question was quite simple.
Given:
unsigned char buffer[BUFFERSIZE];
int bytes = fill_buffer(buffer, sizeof(buffer)): // Returns number of filled bytes.
sendto(socket, buffer, bytes, 0, (struct sockaddr *)&server, sizeof(server))
The return value of recvfrom tells us how many bytes are received, although we make a full read,
int bytesReceived = recvfrom(socket, buffer, sizeof(buffer), 0, NULL, NULL);
// Process bytesReceived number of bytes in the buffer
Below program recieves/captures the packet from the internet and writes into a file (logfile). In my other program i use read() func instead of recvfrom(),but just sending the the buffer doesn't process the packets correctly. Hence i thought of sending two more fields which are in the recvfrom function
int main()
{
int saddr_size,data_size;
struct sockaddr saddr;
unsigned char *buffer = (unsigned char *) malloc(1024);
while(1)
{
saddr_size = sizeof saddr;
//Receive a packet
data_size = recvfrom(sock_raw , buffer , 1024, 0 , &saddr ,(socklen_t*)&saddr_size);
int cont= write(logfile,buffer,data_size);
}
return 0;
}
In the above program, i need to define structure to &saddr and (socklen_t*)&saddr_size). that is instead of sending just the buffer i need to send value of &saddr and (socklen_t*)&saddr_size). How can i do it?i mean to say how to define struct to &sddr and (socklen_t*)&saddr_size)?? please somebody guide me.
No you don't send those fields, those are automatically filled in by the recvfrom function from the information in the received packet.
I am writing a simple web server and client using UDP and so far: the programs can connect to each other, the client can send a request, the server can read the request, the server can recognize the client's IP address and client's port, and the server can send a message back to the client
My problem is that my client code gets stuck waiting in the rcvfrom function, even after the server has sent a response.
Here is the function that is supposed to pick up the server message and return the number of bytes read by the socket:
ssize_t receive_from_server(rdp_socket *rsocket, char *buffer, size_t buf_len){
socklen_t sendsize = sizeof(rsocket->server_addr);
bzero(&(rsocket->server_addr), sendsize);
//STUCK HERE:
return recvfrom(rsocket->sockfd, buffer, buf_len, 0,
(struct sockaddr*)&(rsocket->server_addr), &sendsize);
}
I set the sockopts for both SO_SNDTIMEO and SO_RCVTIMEO to timeout after a few seconds.
Question:
In the short term future I will be adding acknowledgements (ACKs) for reliable data transfer. I imagine that missing ACKs could be the issue but I'm just wondering if, to the trained eye, it looks like a different problem.
Are ACKs necessary for a timeout to work?
How can I synchronize my client and server so that they can actually communicate with each other?
Since UDP does not provide reliability, you will need to implement retransmission of missing data. Since it looks like this is a client request server response model, the easiest retransmission implementation for you may be to resend the request when you time out waiting for the response, and wait for the response again. You may want to implement a retry counter and give up after a certain number of retries.
If the SO_RCVTIMEO and SO_SNDTIMEO socket options do not seem to be taking effect, it may be those options are not implemented for that type of socket. Check the return value of the setsockopt()call to make sure they succeeded.
As a workaround, you can change your receive_from_server() function to use poll() or select() to wait for a readable event for some amount of time, instead of blocking in recvfrom().
ssize_t receive_from_server(rdp_socket *rsocket, char *buffer, size_t buf_len){
struct pollfd pfd = { rsocket->sockfd, POLLIN };
int pollresult = poll(&pfd, 1, RECV_TIMEOUT_SECONDS * 1000);
if (pollresult > 0) {
socklen_t sendsize = sizeof(rsocket->server_addr);
bzero(&(rsocket->server_addr), sendsize);
return recvfrom(rsocket->sockfd, buffer, buf_len, MSG_DONTWAIT,
(struct sockaddr*)&(rsocket->server_addr), &sendsize);
}
if (pollresult == 0) {
errno = ETIME;
}
return -1;
}