I'm pretty new to DPDK.
I have a packet that I need to duplicate several times
meaning I want to deep copy mbuf
the only mbuf api command I've found :
static struct rte_mbuf* rte_pktmbuf_clone(struct rte_mbuf *md,struct rte_mempool *mp )
does not seems to deep-copy it by just pointing to the same data payload.
my question is how can I properly deep copy mbuf
I did some digging , I didn't find any simple API.
this is what I came up with :
going over all mbuf segments and copying it to new mbuf
please note this code won't work directly , you need to implement get_data_from_mbuf and mbuf_free
rte_mbuf *myJumboMbuf = m_pPoolMgr->getJPktPool()->getMBuf();
for (rte_mbuf *m = (rte_mbuf*)mbuf; m != NULL; m = mbuf_api_next(m))
{
uint16_t u16DataLen = mbuf_api_get_data_len(m);
char* buffer = rte_pktmbuf_append(myJmboMbuf, u16DataLen);
if (!buffer)
{
mbuf_free(myJmboMbuf);
ret = ERR_NO_MEM;
}
rte_memcpy(buffer, get_data_from_mbuf(m, void*), u16DataLen);
}
Related
I am working with the library libusb which is working nicely, but now I am trying to use its structures in a structure of my own. I think the problem is just the way I copy the structures, so there is probably no need to understand how libusb works.
I have my structure containing libusb structures:
struct device {
libusb_device *device_handled;
libusb_device_handle *handle;
int port[7];
};
typedef struct device device;
and my function:
int myfunction(device *device_element)
{
libusb_device *tempdevice;
libusb_device_handle *temphandle;
device_element = malloc(sizeof(device));
//my code here where I use tempdevice and temphandle
(&device_element)->device_handled = tempdevice;
(&device_element)->handle = temphandle;
}
The error issued come from the two last lines and I don't really understand why.
‘device_element’ is a pointer; did you mean to use ‘->’?
(&device_element)->handle = temphandle;
^~
->
you don't need to add & before device_element because malloc returns a pointer to the allocated structure. So just use:
(device_element)->device_handled = tempdevice;
(device_element)->handle = temphandle;
You don't need the & operator there.
Just use
device_element->device_handled = tempdevice;
device_element->handle = temphandle;
I am trying to write a remapping target for usage with DM.
I followed instructions from several places (including this Answer) all essentially giving the same code.
This is ok, but not enough for me.
I need to modify "in transit" data of struct bio being remapped.
This means I need to make a deep-clone of the bio, including the data; apparently the provided functions (e.g.: bio_clone_bioset()) do not copy data at all, but point iovec's to the original pages/offsets.
I tried some variations of the following scheme:
void
mt_copy(struct bio *dst, struct bio *src) {
struct bvec_iter src_iter, dst_iter;
struct bio_vec src_bv, dst_bv;
void *src_p, *dst_p;
unsigned bytes;
unsigned salt;
src_iter = src->bi_iter;
dst_iter = dst->bi_iter;
salt = src_iter.bi_sector;
while (1) {
if (!src_iter.bi_size) {
break;
}
if (!dst_iter.bi_size) {
break;
}
src_bv = bio_iter_iovec(src, src_iter);
dst_bv = bio_iter_iovec(dst, dst_iter);
bytes = min(src_bv.bv_len, dst_bv.bv_len);
src_p = kmap_atomic(src_bv.bv_page);
dst_p = kmap_atomic(dst_bv.bv_page);
memcpy(dst_p + dst_bv.bv_offset, src_p + src_bv.bv_offset, bytes);
kunmap_atomic(dst_p);
kunmap_atomic(src_p);
bio_advance_iter(src, &src_iter, bytes);
bio_advance_iter(dst, &dst_iter, bytes);
}
}
static struct bio *
mt_clone(struct bio *bio) {
struct bio *clone;
clone = bio_clone_bioset(bio, GFP_KERNEL, NULL);
if (!clone) {
return NULL;
}
if (bio_alloc_pages(clone, GFP_KERNEL)) {
bio_put(clone);
return NULL;
}
clone->bi_private = bio;
if (bio_data_dir(bio) == WRITE) {
mt_copy(clone, bio);
}
return clone;
}
static int
mt_map(struct dm_target *ti, struct bio *bio) {
struct mt_private *mdt = (struct mt_private *) ti->private;
bio->bi_bdev = mdt->dev->bdev;
bio = mt_clone(bio);
submit_bio(bio->bi_rw, bio);
return DM_MAPIO_SUBMITTED;
}
This, however, does not work.
When I submit_bio() using the cloned bio I do not get the .end_io call and the calling task becomes blocked ("INFO: task mount:488 blocked for more than 120 seconds."). This with a READ request consisting of a single iovec (1024 bytes). In this case, of course the in buffers do not need copying because they should be overwritten; I need to copy back the incoming data unto the original buffers after the request has completed... but I don't get there.
I'm quite evidently missing some piece, but I'm unable to understand what.
Note: I didn't do any optimization (e.g.: use smarter allocation strategies) specifically because I need to get the basics first.
Note: I corrected a mistake (thanks #RuslanRLaishev), unfortunately ininfluent; see my own answer.
It's correct ?
if (bio_alloc_pages(**bio**, GFP_KERNEL)) {
bio_put(clone);
return NULL;
}
or
if (bio_alloc_pages(**clone**, GFP_KERNEL)) {
bio_put(bio);
return NULL;
}
It turns out bio_clone_bioset() and friends do not copy the callback address to call when request is over.
Trivial solution is to add clone->bi_end_io = bio->bi_end_io; before the end of mt_clone().
Unfortunately this is not enough to make the code functional because it turns out upper layers can spawn thousands of inflight requests (i.e.: requests queued and preprocessed before the previous ones complete) leading to memory starvation. Trying to slow upper layers by returning DM_MAPIO_REQUEUE does not seem to work (see: https://unix.stackexchange.com/q/410525/130498). This has nothing to do with current question, however.
I'm new to networking and trying to write pcap files comparison tool as a way to learn network programming
this is a snippet of my code (I writing dll to be used later in c#)
ptt_packet PcapDPI(ptt_ethernetheader puc_PacketData , size_t size)
{
uint8_t *head = (uint8_t*)puc_PacketData;
ptt_ethernetheader ether_header;
ptt_ipheader ip_header;
ptcp_header_t tcp_header;
pudp_header_t udp_header;
ether_header = (ptt_ethernetheader)head;
head += sizeof(*ether_header);
if (0x0800 == ntohs(ether_header->type))/*IP*/
{
ip_header = (ptt_ipheader)head;
head += sizeof(*ip_header);
/* now we need to check if tcp or udp*/
if (ntohs(0x06) == ipProtocolNtohsRes)/*TCP*/
{
//tcp_header = (ptcp_header_t)((uint8_t*)puc_PacketData + 20);
tcp_header = (ptcp_header_t)head;
head += sizeof(*tcp_header);
}
else if (ntohs(0x11) == ipProtocolNtohsRes)/*UDP*/
{
udp_header = (pudp_header_t)head;
head += sizeof(*udp_header);
}
}
As you can see I've started parsing the headers one after the other
(eth->ip->tcp/udp)
My question is: is there is a way to maybe use wireshark dissectors outside of wireshark
or even better: does anyone knows about an open source lib that do all the dirty job of populating packet struct
I'm working on INI-style configuration parser for some project, and I gets next trouble.
I have 3 structures:
typedef struct {
const char* name;
unsigned tract;
int channel;
const char* imitation_type;
} module_config;
typedef struct {
int channel_number;
int isWorking;
int frequency;
int moduleCount;
} channel_config;
typedef struct {
int mode;
module_config* module;
channel_config* channel;
} settings;
And I have function for handling data in my INI-file (I working under inih parser): [pasted to pastebin cause too long]. Finally, in main(), I did the next:
settings* main_settings;
main_settings = (settings*)malloc(sizeof(settings));
main_settings->module = (module_config*)malloc(sizeof(module_config));
main_settings->channel = (channel_config*)malloc(sizeof(channel_config));
if (ini_parse("test.ini", handler, &main_settings) < 0) {
printf("Can't load 'test.ini'\n");
return 1;
}
In result, binary crashes with memory fault. I think (no, I KNOW), what I'm incorrectly allocating the memory in handler(), but I does not understand, where I do it wrong. I spent all night long trying to understand memory allocating, and I'm very tired, but now me simply interestingly, what I'm doing wrong, and HOW to force this working fine.
P.S. Sorry for ugly english
The problem seems to be related to the reallocation of your structs:
pconfig = (settings *) realloc(pconfig, (module_count + channel_count) * sizeof(channel_config));
pconfig->module = (module_config *) realloc(pconfig->module, module_count * sizeof(module_config));
pconfig->channel = (channel_config *) realloc(pconfig->channel, channel_count * sizeof(channel_config));
First of all, you must not reallocate the main settings struct. Since your handler will always be called with the original pconfig value, the reallocation of the module and channel arrays has no effect, and you'll access freed memory.
Also when reallocating the module and channel arrays you should allocate count + 1 elements, since the next invocation of handler might assign to the [count] slot.
So try to replace the three lines above with:
pconfig->module = (module_config *) realloc(pconfig->module, (module_count + 1) * sizeof(module_config));
pconfig->channel = (channel_config *) realloc(pconfig->channel, (channel_count + 1) * sizeof(channel_config));
I'm need to implement a few functions that read messages from different devices that have different interface possibilities and different message structure. (but the messages have pretty much the same data)
Eg
Device_A {
message type: A
iface 1: tcp
}
Device_B {
message type: B
iface 1: serial
iface 2: tcp
}
... and so on
In my main...
struct msg_data;
while(user_wants_to_read) {
read_msg(); // reads and sets data in msg_data
do_work(msg_data);
}
In an OO Language I would use the strategy pattern. I think I could do this with a void* read_func;?
I'm inexperienced in C and I want to learn to program this like a good C programmer would do. What sort of design pattern/functions should I implement?
It sounds like you got two or more different abstractions to solve for:
Different stream sources (TCP vs. Serial). Is the the TCP protocol the same for device A and device B?
Different message types that are structurally different but semantically the same.
Different device classes (device A vs Device B)
I would focus on a strategy pattern with factories for reading from a stream. And then perhaps an adapter or strategy pattern for getting more data into message objects. But I wouldn't get held up on "which design pattern". More likely, just think in terms of interfaces.
So to start, perhaps abstracting out the serial and TCP streaming into different implementations with the same interface. One implementation that knows how connect and read bytes from a TCP socket without regard to the message contents. Another that knows how to read from a serial port. They should have the same "interface". Here's a lightweight example of a a "byte stream interface" with some hacked up socket code thrown. Forgive me if this doesn't compile. I might have a typo valid in C++ by wrong in C. In any case, it's just an example demonstrating interfaces through function table pointers.
My thinking on suggesting this is, "how would I implement this in C++?" And then I'm transposing my answer to pure "C". (Note: I'm likely making some declaration mistakes below.)
struct ByteStreamer;
typedef int (*ReadFunc)(ByteStreamer*, char* buffer, int count);
typedef int (*OpenFunc)(ByteStreamer*, char* url); // maybe 'open' isn't needed if it's handled by the factory
typedef int (*CloseFunc)(ByteStreamer*);
typedef void (*DisposeFunc)(ByteStreamer*);
typedef struct _ByteStreamer
{
ReadFunc readfunc;
OpenFunc openfunc;
CloseFunc closefunc;
DisposeFunc dispose;
// private data meant for the "class"
void* instancedata;
} ByteStreamer;
struct _tcpconnection
{
int socket;
sockaddr_in addrRemote;
} TCPConnection;
struct _serialconnection
{
int filehandle;
int baud;
} SerialConnection;
// ---------------------------------------
ByteStream* CreateStreamForTCP(const sockaddr_in *pAddr) // pass additional parameter as needed
{
ByteStreamer* pStream = (ByteStreamre*)malloc(sizeof(ByteStreamer));
TCPConnection* pTCPConnection = (TCPConnection*)malloc(sizeof(TCPConnection*));
pTCPConnection->socket = -1;
pTCPConnection->addrRemote = *pAddr;
pStream->instancedata = pTCPConnection;
pStream->ReadFunc = TCPRead;
pStream->OpenFunc = TCPOpen;
pStream->CloseFunc = TCPClose;
pStream->DisposeFunc = TCPDispose;
pStream->type = STREAM_TYPE_TCP;
return pStream;
}
int TCPRead(ByteStream* pStream, char* buffer, int count)
{
return recv(((TCPConnection*)pStream->instancedata)->socket, buffer, count, 0);
}
int TCPOpen(ByteStream* pStream, char* url)
{
// it's up to you if you want to encapsulate the socket address in url or in the instance data
TCPConnection* pConn = (TCPConnection*)(pStream->instancedata);
int sock = socket(AF_INET, SOCK_STREAM, 0);
connect(&pConn->addrRemote, sizeof(pConn->addrRemote));
return (pConn->sock >= 0); // true/false return;
}
void TCPClose(ByteStream* pStream)
{
TCPConnection* pConn = (TCPConnection*)(pStream->instancedata);
close(pConn->sock);
}
void TCPDispose(ByteStream* pStream)
{
free(pStream->instancedata);
free(pStream);
}
Now replace all the TCP code above with an equivalent serial port implementation. It would also be a good idea to implement a "file stream" (or "in memory stream") version of the ByteStream struct. Because it will be very useful in unit tests for higher level code.
So after you get all the byte stream implementations worked out, then move onto parsing device specific messages.
typedef struct _Message_A
{
// A specific data fields
} Message_A;
struct _Message_B
{
// B specific data fields
} Message_B;
struct Message
{
// commonality between Message_A and Message_B
};
typedef (*ReadMessageFromStream)(MessageReader* pReader, Message* pMsg); // pStream is an in-param, pMSg is an out-param.
typedef (*MessageReaderDispose)();
struct MessageReader
{
ReadMessageFromStream reader;
MessageReaderDispose dispose;
// -----------------------------
ByteStream* pStream;
void *instancedata;
};
// function to read a "Message_A" from a stream - and then transpose it to the generic Message type
int ReadMessage_A(ByteStream* pStream, Message* pMsg);
// function to read a "Message_B" from a stream - and then transpose it to the generic Message type
int ReadMessage_B(ByteStream* pStream, Message* pMsg);
So what's really cool about implementing ReadMessage_A and ReadMessage_B is that you can pass that "file stream" implementation of ByteStream and make some really good unit tests. So when you plug in the TCP or serial version, it has a high chance of just working (assuming your TCP and serial code are tested seperately).
And then perhaps a factory method off each class for creating the uber ReadMessageFromStream:
MessageReader* CreateTCPReaderForDeviceA(DeviceA* pA, sockaddr_in* pAddr)
{
MessageReader *pMR = (vMessageReader*)malloc(sizeof(MessageReader));
pMR->pStream = CreateStreamForTCP(pAddr);
pMR->pStream->Open();
pMR->reader = ReadMessage_A;
return pMR;
}
MessageReader* CreateSerialReaderForDeviceB(DeviceB* pB, int comport)
{
MessageReader *pMR = (vMessageReader*)malloc(sizeof(MessageReader));
pMR->pStream = CreateStreamForSerial(comport);
pMR->pStream->Open();
pMR->reader = ReadMessage_B;
return pMR;
}
And then your main loop looks something like the following:
if ((type == DEVICE_A) && (source == TCP))
pReader = CreateTCPReaderForDeviceA(pDevice, &addr)
else if ((type == DEVICE_B) && (source == SERIAL))
pReader = CreateSerialReaderForDeviceB(pDeviceB, 1);
// read the message
Message msg;
pReader->reader(pReader, &msg);
pReader->Dispose(); // free all the data allocated and close connections/files
Wooh.... I'm tired of typing this point. hope this helps.
I would agree with #rsaxvc. Function pointers are probably the best way to go about this. A google search turned up this: Strategy pattern in C
And for your message struct, you could use nested struct to emulate OO class inheritance
struct base {
// common members
}
struct child1 {
struct base;
// other data members
}
or simplely:
struct child2 {
// same data members as base
// other data members
}
use a base* parameter