We use RPC (udp socket) in our application and we noticed that RPC retransmit it messages when it was not received (or confirmed) by the target application.
Does RPC present the order of messages? let say we have message 1 and message 2, does it wait for message 1 to be confirmed by the receiver and then sends message 2?
also I wasn't been able to find how many retries it does by default and if the sending failed after x retries does it log it somewhere so we can inspect it?
Thanks
RPC library has a call to control how long to wait before retry the request:
struct timeval tv;
clnt_control(cl, CLSET_TIMEOUT, (char *) &tv);
when you invoke an rpc call, you provide total timeout:
enum clnt_stat clnt_call(CLIENT *clnt, unsigned long procnum,
xdrproc_t inproc, char *in,
xdrproc_t outproc, char *out,
struct timeval tout);
if you divide tout by value you set with clnt_control, you will get number of retries.
The sync/async behavior depends on your application only.
Related
I try to send a few small (8 byte) messages through inproc pair socket. However after a few tries zmq_send() blocks. When I stop debugger I see following stack trace:
libc.so.6!__GI___poll(struct pollfd * fds, nfds_t nfds, int timeout)
libzmq.so.5!poll(int __timeout, nfds_t __nfds, pollfd * __fds)
libzmq.so.5!zmq::signaler_t::wait(zmq::signaler_t * const this, int timeout_)
libzmq.so.5!zmq::mailbox_t::recv(zmq::mailbox_t * const this, zmq::command_t * cmd_, int timeout_)
libzmq.so.5!zmq::socket_base_t::process_commands(zmq::socket_base_t * const this, int timeout_, bool throttle_)
libzmq.so.5!zmq::socket_base_t::send(zmq::socket_base_t * const this, zmq::msg_t * msg_, int flags_)
libzmq.so.5!s_sendmsg(int flags_, zmq_msg_t * msg_, zmq::socket_base_t * s_)
libzmq.so.5!zmq_send(void * s_, const void * buf_, size_t len_, int flags_)
Why does it happen? What commands does ZMQ try to to process? Why does it call recv()? Is it because of high water mark? I suppose it's something different because I send small amount of data and water mark shouldn't be reached yet. And if water mark is the only explanation then how can I measure it?
Generally, this will happen when the socket reaches some sort of limit on outstanding sent data; it will then block waiting for an acknowledgement message allowing it to continue.
The precise behavior depends on the zmq_socket type as well as any other configuration you have done on it (setting the high-water mark, for example).
It turned out it was bug in logic of my code. After sending some messages, next were sent by different thread using different socket. That socket was not connected so zmq_send() was waiting for connection to be established. This is why its pipe was NULL. Thanks everybody for help.
Currently I have a daemon listening for TCP/UDP packets on port X using the recvfrom system call:
ssize_t recvfrom(int sockfd, void *buf, size_t len, int flags,
struct sockaddr *src_addr, socklen_t *addrlen);
and then the data is send over a message queue with the msgsnd system call:
int msgsnd(int msqid, struct msgbuf *msgp, int msgsz, int msgflg);
I would like to replace the msgsnd system call with RabbitMQ and send the messages using amqp protocol. The problem I am facing is the format of the data received and send over the RabbitMQ. When I receive data on port then I try to send it like this:
ssize_t recvfrom(
sockfd, msgbufInst.msgContent, len, flags, src_addr, addrlen
);
send_over_rabbitmq(msgbufInst.msgContent);
where send_over_rabbitmq is pretty much
send_over_rabbitmq(char *rawData){
amqp_bytes_t payload;
payload.len = sizeof(rawData);
payload.bytes = rawData;
int result = amqp_basic_publish((*channel)->conn, 1,
amqp_cstring_bytes("myExchange"),
amqp_cstring_bytes(""), 0, 0, NULL, payload);
}
The payload is send and received successfully but some data is lose. When I consume the message from the queue the payload is not the same as when I used the system call to send/consume messages. I am sure that it has to do with the data types that I am using but can't figure it out yet.
As I noted in a comment:
In the send_over_rabbitmq() function, the value of sizeof(rawData) is probably 8, and maybe only 4 — it's the size of the pointer variable, and not the length of the data that it points at. That probably isn't what you want.
You'll need to send more information to the send_over_rabbitmq() function — most noticeably, the length of the data received from recvfrom(). That means capturing the information — you should be checking the return value from recvfrom() anyway before trying to relay information via RabbitMQ.
I am writing a kernel module which needs to do some packet filtering work at the IP layer. What I need to do is intercept all IP packets, and on some outgoing packets, I need to withhold them for a small amount of time (like a few dozen milliseconds) for analysis before sending them on their way.
I've got the kernel module up and running, at the moment it accepts all incoming packets and returns NF_QUEUE on all outgoing packets. I can then pick up those packets using libnetfilter in userspace (like with a Python script and NetfilterQueue) but the problem is that the Python library takes a queue number, and my kernel module queues up the packets in queue #0. How do I configure that?!
This is what my kernel module's outgoing packet hook looks like (cobbled up from various sources):
static struct nf_hook_ops nfho_send;
unsigned int hook_send_func(unsigned int hooknum, struct sk_buff **skb, const struct net_device *in, const struct net_device *out, int (*okfn)(struct sk_buff *))
{
return NF_QUEUE;
}
/* in init_module() */
nfho_send.hook = hook_send_func;
nfho_send.hooknum = NF_INET_POST_ROUTING;
nfho_send.pf = NFPROTO_IPV4;
nfho_send.priority = NF_IP_PRI_LAST;
nf_register_hook(&nfho_send);
Where do I specify which queue the packets will end up on? What is the significance of queue numbers? I can't be processing random stray packets queued by another hook than mine, is that already done by default?
Thanks! Also, I am using Linux 3.x.
Use the NF_QUEUE_NR(x) macro. You pass it a single argument (the queue number), and it will build your return value for you. Specifically, it will left-shift the queue number by 16 bits and then AND it with NF_QUEUE.
If you're interested, the implementation is in include/uapi/linux/netfilter.h
I am writing a simple web server and client using UDP and so far: the programs can connect to each other, the client can send a request, the server can read the request, the server can recognize the client's IP address and client's port, and the server can send a message back to the client
My problem is that my client code gets stuck waiting in the rcvfrom function, even after the server has sent a response.
Here is the function that is supposed to pick up the server message and return the number of bytes read by the socket:
ssize_t receive_from_server(rdp_socket *rsocket, char *buffer, size_t buf_len){
socklen_t sendsize = sizeof(rsocket->server_addr);
bzero(&(rsocket->server_addr), sendsize);
//STUCK HERE:
return recvfrom(rsocket->sockfd, buffer, buf_len, 0,
(struct sockaddr*)&(rsocket->server_addr), &sendsize);
}
I set the sockopts for both SO_SNDTIMEO and SO_RCVTIMEO to timeout after a few seconds.
Question:
In the short term future I will be adding acknowledgements (ACKs) for reliable data transfer. I imagine that missing ACKs could be the issue but I'm just wondering if, to the trained eye, it looks like a different problem.
Are ACKs necessary for a timeout to work?
How can I synchronize my client and server so that they can actually communicate with each other?
Since UDP does not provide reliability, you will need to implement retransmission of missing data. Since it looks like this is a client request server response model, the easiest retransmission implementation for you may be to resend the request when you time out waiting for the response, and wait for the response again. You may want to implement a retry counter and give up after a certain number of retries.
If the SO_RCVTIMEO and SO_SNDTIMEO socket options do not seem to be taking effect, it may be those options are not implemented for that type of socket. Check the return value of the setsockopt()call to make sure they succeeded.
As a workaround, you can change your receive_from_server() function to use poll() or select() to wait for a readable event for some amount of time, instead of blocking in recvfrom().
ssize_t receive_from_server(rdp_socket *rsocket, char *buffer, size_t buf_len){
struct pollfd pfd = { rsocket->sockfd, POLLIN };
int pollresult = poll(&pfd, 1, RECV_TIMEOUT_SECONDS * 1000);
if (pollresult > 0) {
socklen_t sendsize = sizeof(rsocket->server_addr);
bzero(&(rsocket->server_addr), sendsize);
return recvfrom(rsocket->sockfd, buffer, buf_len, MSG_DONTWAIT,
(struct sockaddr*)&(rsocket->server_addr), &sendsize);
}
if (pollresult == 0) {
errno = ETIME;
}
return -1;
}
I have successfully created the message queue by using the following command:
msgIdHareTurtle = msgget(keyHareTurtle, 0644 | IPC_CREAT | O_NONBLOCK);
Now I want to send the queue to some other process I used,
msgsnd(msgIdHareTurtle, (struct msgbuf *)&bufHareTurtle, sizeof(int), IPC_NOWAIT);
and I try to receive it in different process by:
msgrcv(msgIdHareTurtle, (struct msgbuf *)&bufHareTurtle, sizeof(int), 0, IPC_NOWAIT);
my structure bufHareTurtle is of following type:
typedef struct smsgbuf{
long mtype;
unsigned int position;
} smsgbuf;
My question: The sending was successful and the program(both the processes) is running too but whenever I am sending an unsigned integer for example 2 , I AM ALWAYS GETTING THE RECEIVED VALUE (IN LATTER PROCESS) AS 0 EVRYTIME. Could somebody tell me what is the error in this code or what could be possible error elsewhere.
The problem was there in synchronization. The sending to the queue was delayed due to sleep inserted in between. I corrected it and the error was gone