I am working on a Client-Server Architecture on a single local machine with Linux 3.16 using AF_UNIX IPC Sockets. I have implemented a send_all and recv_all Functions to ensure sending/receiving all the wanted data and I use clock_gettime() with CLOCK_MONOTONIC to make time measurement just before the send-call on the client side and after the receive-call on the server side. Every packet I send is very small (11 Byte + the struct timespec). When I calc the delta of these two values after each transmission, I get fluctuating results by the factor ~15 while the shortest is around 150µs and the longest is above 2 milliseconds. While the <200µs would be fast enough for my needs, the >2ms are already very critical, so how can this varying latency be explained and is there a way to make the latency consistently as fast as possible for Sockets?
I am using SOCK_STREAM in the initialization and I thought that Nagle's Algorithm has to do something with the problem, but TCP_NODELAY is not settable via setsockopt() when using AF_UNIX (it seems to only work in AF_INET?). Also I'm thinking about using SOCK_DGRAM, but before I jump into that (for me) new area, I would like to know if I can even solve my problem with it.
My essential Functions for the Sockets are mostly inspired by Beej's Guide, but here some code anyway:
/* Packet Definition */
typedef struct Packet_ {
Byte opcode;
Byte from;
Byte to;
Byte payload[8];
timespec timestamp;
} Packet;
// Send
void send_all(SOCKET s, void* data, size_t len)
{
unsigned char* buffer_ptr = (unsigned char*)data;
unsigned int out_counter = 0;
while(out_counter < len)
{
out_counter += send(s, buffer_ptr, len - out_counter, 0);
}
}
// Receive
void recv_all(SOCKET s, void* buffer, size_t len)
{
unsigned char* buffer_ptr = (unsigned char*)buffer;
unsigned int in_counter = 0;
while(in_counter < len)
{
in_counter += recv(s, buffer_ptr + in_counter, len - in_counter, 0);
}
}
// Set Packet Header
void setPacketHeader(Packet* packet, Byte receivers, Byte opcode)
{
packet->from = this->get_ident();
packet->to = receivers;
packet->opcode = opcode;
}
// Set Packet Payload
void setPacketPayload(Packet* packet, void* payload, int pl_size)
{
clock_gettime(CLOCK_MONOTONIC, &(packet->timestamp));
memcpy(&(packet->payload), payload, pl_size);
}
Thanks for your time
Related
According to the documentation for bpf_perf_event_output found here: http://man7.org/linux/man-pages/man7/bpf-helpers.7.html
"The flags are used to indicate the index in map for which the value must be put, masked with BPF_F_INDEX_MASK."
In the following code:
SEC("xdp_sniffer")
int xdp_sniffer_prog(struct xdp_md *ctx)
{
void *data_end = (void *)(long)ctx->data_end;
void *data = (void *)(long)ctx->data;
if (data < data_end) {
/* If we have reached here, that means this
* is a useful packet for us. Pass on-the-wire
* size and our cookie via metadata.
*/
/* If we have reached here, that means this
* is a useful packet for us. Pass on-the-wire
* size and our cookie via metadata.
*/
__u64 flags = BPF_F_INDEX_MASK;
__u16 sample_size;
int ret;
struct S metadata;
metadata.cookie = 0xdead;
metadata.pkt_len = (__u16)(data_end - data);
/* To minimize writes to disk, only
* pass necessary information to userspace;
* that is just the header info.
*/
sample_size = min(metadata.pkt_len, SAMPLE_SIZE);
flags |= (__u64)sample_size << 32;
ret = bpf_perf_event_output(ctx, &my_map, flags,
&metadata, sizeof(metadata));
if (ret)
bpf_printk("perf_event_output failed: %d\n", ret);
}
return XDP_PASS;
}
It works as you would expect and stores the information for the given CPU number.
However, suppose I want all packets to be sent to index 1.
I swap
__u64 flags = BPF_F_INDEX_MASK;
for
__u64 flags = 0x1ULL;
The code compiles correctly and throws no errors, however no packets get saved at all anymore. What am I doing wrong if I want all of the packets to be sent to index 1?
Partial answer: I see no reason why the packets would not be sent to the perf buffer, but I suspect the error is on the user space code (not provided). It could be that you do not “open” the perf event for all CPUs when trying to read from the buffer. Have a look at the man page for perf_event_open(2): check that the combination of values for pid and cpu allows you to read data written for CPU 1.
As a side note, this:
__u64 flags = BPF_F_INDEX_MASK;
is misleading. The mask should be used to mask the index, not to set its value. BPF_F_CURRENT_CPU should be used instead, the former only happens to work because the two enum attributes have the same value.
I am interested in creating a DNS (using UDP protocol to send it) response packet, however I found limited information how to create your own packet.
Most tutorials are like this https://opensourceforu.com/2015/03/a-guide-to-using-raw-sockets/
They use structs to fill in the fields and connect them into 1 sequence. But I am concerned that the compiler can pad the struct, making it "corrupted" (make the packet longer then it should be)
I fully know that there are struct attributes, that don't allow the compiler to pad structs, but I don't want to use them
Can anyone point me some resources on packet creation. I can use Libpcap and raw sockets
You do it like this:
// helper function to add uint32_t to a buffer
char *append_uint32(char *buf_position, uint32_t value) {
// network protocols usually use network byte order for numbers,
// htonl is POSIX function so you may have to make your own on other platform
// http://pubs.opengroup.org/onlinepubs/9699919799/functions/htonl.html
value = htonl(value);
memcpy(buf_postion, &value, sizeof value);
return buf_position + sizeof value;
}
// example code using the function:
// generate packet with numbers 0...9 in network byte order
void func() {
char buf[sizeof(int32_t) * 10];
char *bptr = buf;
for(uint32_t i=0; i<10; ++i) {
bptr = append_uint32(bptr, i);
}
// do something with buf (use malloc instead of stack if you want return it!)
}
I am writing a small analysis tool using libpcap that sniffs traffic on an ethernet device and performs some sort of analysis on the received packets. In order to do so, I have the obvious libpcap loop:
void packet_loop(u_char *args, const struct pcap_pkthdr *header,
const u_char *packetdata) {
int size = (int)header->len;
//Before we map the buffer to the ethhdr struct,
//we check if the size fits
if (ETHER_HDR_LEN > size)
return;
const struct ethhdr *ethh = (const struct ethhdr *)(packetdata);
//If this protocol is IPv4 and the packet size is bigger than
//ETH hdr size
if (ETHERTYPE_IP == ntohs(ethh->h_proto)) {
//Before we map the buffer to the iph struct,
//we check if the size fits
if (ETHER_HDR_LEN + (int)sizeof(struct iphdr) > size)
return;
const struct iphdr *iph = (const struct iphdr*)
(packetdata + sizeof(struct ethhdr));
//If this protocol isn't UDP and the header length
//isn't 5 (20bytes)
if (IPPROTO_UDP != iph->protocol && 5 != iph->ihl)
return;
//eval_udp(packetdata, size);
const struct udphdr *udph = (const struct udphdr*)
(packetdata + sizeof(struct ethhdr) +
sizeof(struct iphdr));
if (DATA_SRCPORT == ntohs(udph->uh_sport) &&
DATA_DESTPORT == ntohs(udph->uh_dport)) {
analyse_data(packetdata);
}
}
}
that calls the follwoing code snipped on receival of a specific packet type. As you can see, I am using a static variable to keep track of the previous packet, in order to compare two.
void analyse_data(const uint8_t *packet)
{
if (!packet)
return;
static const uint8_t *basepacket;
//If there was no packet to base our analysis on, we will wait for one
if (!basepacket) {
basepacket = packet;
return;
}
const struct dataheader *basedh = (const struct dataheader *)
(__OFFSETSHERE__ + basepacket);
const struct dataheader *dh = (const struct dataheader *)
(__OFFSETSHERE__ + packet);
printf("%d -> %d\n", ntohs(basedh->sequenceid),
ntohs(dh->sequenceid));
basepacket = packet;
return;
}
struct dataheader is a regular struct, just like etthdr. I would expect a constant printout like:
0 -> 1
1 -> 2
2 -> 3
Unfortunately, I get a different printout, which is mostly right. But around every 20th-40th packet, I see the following behavior (example):
12->13
13->14
0->15
15->16
...
It is maybe interesting to note that this does NOT occcur, when I receive only packets of the specific type I look after (8-10 Mbit/s). Nevertheless, as soon as I use my tool in the "regular" network environment (around 100Mbit/s), I get this behavior. I checked my if statement, that filters the packet it works flawlessly (checking UDP source and destination ports). Wireshark also shows me that there is not a single packet on those ports that is not of that specific type.
libpcap controls the packet data it passes in to your packet_loop. Once packet_loop returns, you have no guarantee what the pointers for the packet data point to - libpcap could throw the packet away, or it could reuse the same space for a new packet.
This means if you want to compare 2 packets, you must make a copy of the 1. packet - you cannot save the pointer from one call to packet_loop and expect that pointer to be valid and point to the same packet in future calls to packet_loop. So your code could be changed to e.g.
void analyse_data(const uint8_t *packet, int size )
{
if (!packet)
return;
static const uint8_t basepacket[1024*64];
static int has_basepacket;
//If there was no packet to base our analysis on, we will wait for one
if (!has_basepacket){
if (size < sizeof basepacket) {
memcpy(basepacket, packet, size);
has_basepacket = 1;
}
return;
}
...
Also, make sure your verify the sizes everywhere. Just because the ethernet type says it is an IPv4 packet, doesn't mean you can trust it to contain a full IP packet. Just because the IP header says it is 20 bytes, doesn't mean you can trust it to contain a full IP packet, and so on for all the layers you attempt to decode.
I'm trying to access a SPI sensor using the SPIDEV driver but my code gets stuck on IOCTL.
I'm running embedded Linux on the SAM9X5EK (mounting AT91SAM9G25). The device is connected to SPI0. I enabled CONFIG_SPI_SPIDEV and CONFIG_SPI_ATMEL in menuconfig and added the proper code to the BSP file:
static struct spi_board_info spidev_board_info[] {
{
.modalias = "spidev",
.max_speed_hz = 1000000,
.bus_num = 0,
.chips_select = 0,
.mode = SPI_MODE_3,
},
...
};
spi_register_board_info(spidev_board_info, ARRAY_SIZE(spidev_board_info));
1MHz is the maximum accepted by the sensor, I tried 500kHz but I get an error during Linux boot (too slow apparently). .bus_num and .chips_select should correct (I also tried all other combinations). SPI_MODE_3 I checked the datasheet for it.
I get no error while booting and devices appear correctly as /dev/spidevX.X. I manage to open the file and obtain a valid file descriptor. I'm now trying to access the device with the following code (inspired by examples I found online).
#define MY_SPIDEV_DELAY_USECS 100
// #define MY_SPIDEV_SPEED_HZ 1000000
#define MY_SPIDEV_BITS_PER_WORD 8
int spidevReadRegister(int fd,
unsigned int num_out_bytes,
unsigned char *out_buffer,
unsigned int num_in_bytes,
unsigned char *in_buffer)
{
struct spi_ioc_transfer mesg[2] = { {0}, };
uint8_t num_tr = 0;
int ret;
// Write data
mesg[0].tx_buf = (unsigned long)out_buffer;
mesg[0].rx_buf = (unsigned long)NULL;
mesg[0].len = num_out_bytes;
// mesg[0].delay_usecs = MY_SPIDEV_DELAY_USECS,
// mesg[0].speed_hz = MY_SPIDEV_SPEED_HZ;
mesg[0].bits_per_word = MY_SPIDEV_BITS_PER_WORD;
mesg[0].cs_change = 0;
num_tr++;
// Read data
mesg[1].tx_buf = (unsigned long)NULL;
mesg[1].rx_buf = (unsigned long)in_buffer;
mesg[1].len = num_in_bytes;
// mesg[1].delay_usecs = MY_SPIDEV_DELAY_USECS,
// mesg[1].speed_hz = MY_SPIDEV_SPEED_HZ;
mesg[1].bits_per_word = MY_SPIDEV_BITS_PER_WORD;
mesg[1].cs_change = 1;
num_tr++;
// Do the actual transmission
if(num_tr > 0)
{
ret = ioctl(fd, SPI_IOC_MESSAGE(num_tr), mesg);
if(ret == -1)
{
printf("Error: %d\n", errno);
return -1;
}
}
return 0;
}
Then I'm using this function:
#define OPTICAL_SENSOR_ADDR "/dev/spidev0.0"
...
int fd;
fd = open(OPTICAL_SENSOR_ADDR, O_RDWR);
if (fd<=0) {
printf("Device not found\n");
exit(1);
}
uint8_t buffer1[1] = {0x3a};
uint8_t buffer2[1] = {0};
spidevReadRegister(fd, 1, buffer1, 1, buffer2);
When I run it, the code get stuck on IOCTL!
I did this way because, in order to read a register on the sensor, I need to send a byte with its address in it and then get the answer back without changing CS (however, when I tried using write() and read() functions, while learning, I got the same result, stuck on them).
I'm aware that specifying .speed_hz causes a ENOPROTOOPT error on Atmel (I checked spidev.c) so I commented that part.
Why does it get stuck? I though it can be as the device is created but it actually doesn't "feel" any hardware. As I wasn't sure if hardware SPI0 corresponded to bus_num 0 or 1, I tried both, but still no success (btw, which one is it?).
UPDATE: I managed to have the SPI working! Half of it.. MOSI is transmitting the right data, but CLK doesn't start... any idea?
When I'm working with SPI I always use an oscyloscope to see the output of the io's. If you have a 4 channel scope ypu can easily debug the issue, and find out if you're axcessing the right io's, using the right speed, etc. I usually compare the signal I get to the datasheet diagram.
I think there are several issues here. First of all SPI is bidirectional. So if yo want to send something over the bus you also get something. Therefor always you have to provide a valid buffer to rx_buf and tx_buf.
Second, all members of the struct spi_ioc_transfer have to be initialized with a valid value. Otherwise they just point to some memory address and the underlying process is accessing arbitrary data, thus leading to unknown behavior.
Third, why do you use a for loop with ioctl? You already tell ioctl you haven an array of spi_ioc_transfer structs. So all defined transaction will be performed with one ioctl call.
Fourth ioctl needs a pointer to your struct array. So ioctl should look like this:
ret = ioctl(fd, SPI_IOC_MESSAGE(num_tr), &mesg);
You see there is room for improvement in your code.
This is how I do it in a c++ library for the raspberry pi. The whole library will soon be on github. I'll update my answer when it is done.
void SPIBus::spiReadWrite(std::vector<std::vector<uint8_t> > &data, uint32_t speed,
uint16_t delay, uint8_t bitsPerWord, uint8_t cs_change)
{
struct spi_ioc_transfer transfer[data.size()];
int i = 0;
for (std::vector<uint8_t> &d : data)
{
//see <linux/spi/spidev.h> for details!
transfer[i].tx_buf = reinterpret_cast<__u64>(d.data());
transfer[i].rx_buf = reinterpret_cast<__u64>(d.data());
transfer[i].len = d.size(); //number of bytes in vector
transfer[i].speed_hz = speed;
transfer[i].delay_usecs = delay;
transfer[i].bits_per_word = bitsPerWord;
transfer[i].cs_change = cs_change;
i++
}
int status = ioctl(this->fileDescriptor, SPI_IOC_MESSAGE(data.size()), &transfer);
if (status < 0)
{
std::string errMessage(strerror(errno));
throw std::runtime_error("Failed to do full duplex read/write operation "
"on SPI Bus " + this->deviceNode + ". Error message: " +
errMessage);
}
}
I am trying to send data between a client/Server, the data looks like
typedef Struct Message
{ int id;
int message_length;
char* message_str;
}message;
I am trying to Write and Read this message between a client and server constantly updating the elements in this struct. I have heard Writev may do the trick. i want to send a
message to the server and then the server pulls out the elements and uses those elements as conditionals to execute the proper method?
Assuming you want to do the serialization yourself and not use Google Protocol Buffers or some library to handle it for you, I'd suggest writing a pair of functions like this:
// Serializes (msg) into a flat array of bytes, and returns the number of bytes written
// Note that (outBuf) must be big enough to hold any Message you might have, or there will
// be a buffer overrun! Modifying this function to check for that problem and
// error out instead is left as an exercise for the reader.
int SerializeMessage(const struct Message & msg, char * outBuf)
{
char * outPtr = outBuf;
int32_t sendID = htonl(msg.id); // htonl will make sure it gets sent in big-endian form
memcpy(outPtr, &sendID, sizeof(sendID));
outPtr += sizeof(sendID);
int32_t sendLen = htonl(msg.message_length);
memcpy(outPtr, &sendLen, sizeof(sendLen));
outPtr += sizeof(sendLen);
memcpy(outPtr, msg.message_str, msg.message_length); // I'm assuming message_length=strlen(message_str)+1 here
outPtr += msg.message_length;
return (outPtr-outBuf);
}
// Deserializes a flat array of bytes back into a Message object. Returns 0 on success, or -1 on failure.
int DeserializeMessage(const char * inBuf, int numBytes, struct Message & msg)
{
const char * inPtr = inBuf;
if (numBytes < sizeof(int32_t)) return -1; // buffer was too short!
int32_t recvID = ntohl(*((int32_t *)inPtr));
inPtr += sizeof(int32_t);
numBytes -= sizeof(int32_t);
msg.id = recvID;
if (numBytes < sizeof(int32_t)) return -1; // buffer was too short!
int32_t recvLen = ntohl(*((int32_t *)inPtr));
inPtr += sizeof(int32_t);
numBytes -= sizeof(int32_t);
msg.message_length = recvLen; if (msg.message_length > 1024) return -1; /* Sanity check, just in case something got munged we don't want to allocate a giant array */
msg.message_str = new char[msg.message_length];
memcpy(msg.message_str, inPtr, numBytes);
return 0;
}
With these functions, you are now able to convert a Message into a simple char-array and back at will. So now all you have to do is send the char-array over the TCP connection, receive it at the far end, and then Deserialize the array back into a Message struct there.
One wrinkle with this is that your char arrays will be variable-length (due to the presence of a string which can be different lengths), so your receiver will need some easy way to know how many bytes to receive before calling DeserializeMessage() on the array.
An easy way to handle that is to always send a 4-byte integer first, before sending the char-array. The 4-byte integer should always be the size of the upcoming array, in bytes. (Be sure to convert the integer to big-endian first, via htonl(), before sending it, and convert it back to native-endian on the receiver via htonl() before using it).
Okay, I'll take a stab at this. I'm going to assume that you have a "message" object on the sending side and what you want to do is somehow send it across to another machine and reconstruct the data there so you can do some computation on it. The part that you may not be clear on is how to encode the data for communications and then decode it on the receiving side to recover the information. The simplistic approach of just writing the bytes contained in a "message" object (i.e. write(fd, msg, sizeof(*msg), where "msg" is a pointer to an object of type "message") won't work because you will end up sending the value of a virtual address in the memory of one machine to different machine and there's not much you can do with that on the receiving end. So the problem is to design a way to pass an two integers and a character string bundled up in a way that you can fish them back out on the other end. There are, of course, many ways to do this. Does this describe what you are trying to do?
You can send structs over socket, but you have to serialize them before sending the struct using boost serialization.
Here is a sample code :
#include<iostream>
#include<unistd.h>
#include<cstring>
#include <sstream>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
using namespace std;
typedef struct {
public:
int id;
int message_length;
string message_str;
private:
friend class boost::serialization::access;
template <typename Archive>
void serialize(Archive &ar, const unsigned int vern)
{
ar & id;
ar & message_length;
ar & message_str;
}
} Message;
int main()
{
Message newMsg;
newMsg.id = 7;
newMsg.message_length = 14;
newMsg.message_str="Hi ya Whats up";
std::stringstream strData;
boost::archive::text_oarchive oa(strData);
oa << newMsg;
char *serObj = (char*) strData.str().c_str();
cout << "Serialized Data ::: " << serObj << "Len ::: " << strlen(serObj) << "\n";
/* Send serObj thru Sockets */
/* recv serObj from socket & deserialize it */
std::stringstream rcvdObj(serObj);
Message deserObj;
boost::archive::text_iarchive ia(rcvdObj);
ia >> deserObj;
cout<<"id ::: "<<deserObj.id<<"\n";
cout<<"len ::: "<<deserObj.message_length<<"\n";
cout<<"str ::: "<<deserObj.message_str<<"\n";
}
you can compile the program by
g++ -o serial boost.cpp /usr/local/lib/libboost_serialization.a
you must have libboost_serialization.a statically compiled in your machine.
Keeping the sockets 'blocking' will be good and you have to devise for reading these structs from recv buffer.