I have to prepare a string which has to be sent over socket, that string is constructed using different data types int, char, unsigned char etc... As i have used it in many places my callgrind report says most of the CPU consumed by sprintf and strcat.
Can some one tell me alternate for this sprintf and strcat idea?
Below is the piece of code
pData_temp = (char *)malloc(200);
pData = (char *)malloc(500);
sprintf(pData_temp, "String to be sent over socket at time %u $%04x",seconds,id);
strncpy(pData,pData_temp,strlen(pData_temp);
for in t(i=0; i < 1000; i++)
{
sprintf(pData_temp,"%02x%04x%08x%08x%08x",var1,var2,var3,var4,var5);
strcat(pData,pData_temp);
}
sprintf(pData_temp,"\n");
strcat(pData,pData_temp)
sock_send(pData,strlen(pData);
free(pData);
free(pData_temp);
Any help appriciated.
Regards
You're using Shlemiel the painter's algorithm.
Instead, you can advance pData by the length you write, instead of strcating to it over and over again.
sprintf returns the number you need to add.
But as #alk points out in the comments, it might be even better just to write pData_temp directly to the socket - don't bother making one big string at all.
You have a buffer overflow in the code. You only allocate 500 bytes for pData and you concatenate 1000 30byte strings to it.
Does the data really need to be ASCII? You can improve the efficiency of the algorithm by using binary data rather than ASCII:-
struct Data // turn off padding!
{
char var1;
short var2;
int var3, var4, var5;
};
void somefunc ()
{
int i;
for (i = 0 ; i < 1000 ; ++i)
{
struct Data data;
// set up the members of data
socket_write (&data, sizeof data); // not a real function, pseudo-code!
}
}
Related
Let's consider following piece of code:
int len = 100;
char *buf = (char*)malloc(sizeof(char)*len);
printf("Appended: %s\n",struct_to_string(some_struct,buf,len));
Someone allocated amount of memory in order to get it filled with string data. The problem is that string data taken from some_struct could be ANY length. So what i want to achieve is to make struct_to_string function do the following:
Do not allocate any memory that goes outside (so, buf has to be allocated outside of the function, and passed)
Inside the struct_to_string I want to do something like:
char* struct_to_string(const struct type* some_struct, char* buf, int len) {
//it will be more like pseudo code to show the idea :)
char var1_name[] = "int l1";
buf += var1_name + " = " + some_struct->l1;
//when l1 is a int or some non char, I need to cast it
char var2_name[] = "bool t1";
buf += var2_name + " = " + some_struct->t1;
// buf+= (I mean appending function) should check if there is a place in a buf,
//if there is not it should fill buf with
//as many characters as possible (without writting to memory) and stop
//etc.
return buf;
}
Output should be like:
Appended: int l1 = 10 bool t1 = 20 //if there was good amount of memory allocated or
ex: Appended: int l1 = 10 bo //if there was not enough memory allocated
To sum up:
I need a function (or couple of functions) that adds given strings to the base string without overwritting base string;
do nothing when base string memory is full
I can not use C++ libraries
Another things that I could ask but are not so important right now:
Is there a way (in C) iterate through structure variable list to get their names, or at least to get their values without their names? (for example iterate through structure like through array ;d)
I do not normally use C, but for now I'm obligated to do, so I have very basic knowledge.
(sorry for my English)
Edit:
Good way to solve that problem is shown in post below: stackoverflow.com/a/2674354/2630520
I'd say all you need is the standard strncat function defined in the string.h header.
About the 'iterate through structure variable list' part, I'm not exactly sure what you mean. If your talking about iterating over the structure's members, a short answer would be : you can't introspect C structs for free.
You need to know beforehand what structure type you're using so that the compiler know at what offset in the memory it can find each member of your struct. Otherwise it's just an array of bytes like any other.
Don't mind asking if I wasn't clear enough or if you want more details.
Good luck.
So basically I did it like here: stackoverflow.com/a/2674354/2630520
int struct_to_string(const struct struct_type* struct_var, char* buf, const int len)
{
unsigned int length = 0;
unsigned int i;
length += snprintf(buf+length, len-length, "v0[%d]", struct_var->v0);
length += other_struct_to_string(struct_var->sub, buf+length, len-length);
length += snprintf(buf+length, len-length, "v2[%d]", struct_var->v2);
length += snprintf(buf+length, len-length, "v3[%d]", struct_var->v3);
....
return length;
}
snprintf writes as much as possible and discards everything left, so it was exactly what I was looking for.
I have a struct object that comprises of several primitive data types, pointers and struct pointers. I want to send it over a socket so that it can be used at the other end. As I want to pay the serialization cost upfront, how do I initialize an object of that struct so that it can be sent immediately without marshalling? For example
struct A {
int i;
struct B *p;
};
struct B {
long l;
char *s[0];
};
struct A *obj;
// can do I initialize obj?
int len = sizeof(struct A) + sizeof(struct B) + sizeof(?);
obj = (struct A *) malloc(len);
...
write(socket, obj, len);
// on the receiver end, I want to do this
char buf[len];
read(socket, buf, len);
struct A *obj = (struct A *)buf;
int i = obj->i;
char *s = obj->p->s[0];
int i obj.i=1; obj.p.
Thank you.
The simplest way to do this may be to allocate a chunk of memory to hold everything. For instance, consider a struct as follows:
typedef struct A {
int v;
char* str;
} our_struct_t;
Now, the simplest way to do this is to create a defined format and pack it into an array of bytes. I will try to show an example:
int sLen = 0;
int tLen = 0;
char* serialized = 0;
char* metadata = 0;
char* xval = 0;
char* xstr = 0;
our_struct_t x;
x.v = 10;
x.str = "Our String";
sLen = strlen(x.str); // Assuming null-terminated (which ours is)
tLen = sizeof(int) + sLen; // Our struct has an int and a string - we want the whole string not a mem addr
serialized = malloc(sizeof(char) * (tLen + sizeof(int)); // We have an additional sizeof(int) for metadata - this will hold our string length
metadata = serialized;
xval = serialized + sizeof(int);
xstr = xval + sizeof(int);
*((int*)metadata) = sLen; // Pack our metadata
*((int*)xval) = x.v; // Our "v" value (1 int)
strncpy(xstr, x.str, sLen); // A full copy of our string
So this example copies the data into an array of size 2 * sizeof(int) + sLen which allows us a single integer of metadata (i.e. string length) and the extracted values from the struct. To deserialize, you could imagine something as follows:
char* serialized = // Assume we have this
char* metadata = serialized;
char* yval = metadata + sizeof(int);
char* ystr = yval + sizeof(int);
our_struct_t y;
int sLen = *((int*)metadata);
y.v = *((int*)yval);
y.str = malloc((sLen + 1) * sizeof(char)); // +1 to null-terminate
strncpy(y.str, ystr, sLen);
y.str[sLen] = '\0';
As you can see, our array of bytes is well-defined. Below I have detailed the structure:
Bytes 0-3 : Meta-data (string length)
Bytes 4-7 : X.v (value)
Bytes 8 - sLen : X.str (value)
This kind of well-defined structure allows you to recreate the struct on any environment if you follow the defined convention. To send this structure over the socket, now, depends on how you develop your protocol. You can first send an integer packet containing the total length of the packet which you just constructed, or you can expect that the metadata is sent first/separately (logically separately, this technically can still all be sent at the same time) and then you know how much data to receive on the client-side. For instance, if I receive metadata value of 10 then I can expect sizeof(int) + 10 bytes to follow to complete the struct. In general, this is probably 14 bytes.
EDIT
I will list some clarifications as requested in the comments.
I do a full copy of the string so it is in (logically) contiguous memory. That is, all the data in my serialized packet is actually full data - there are no pointers. This way, we can send a single buffer (we call is serialized) over the socket. If simply send the pointer, the user receiving the pointer would expect that pointer to be a valid memory address. However, it is unlikely that your memory addresses will be exactly the same. Even if they are, however, he will not have the same data at that address as you do (except in very limited and specialized circumstances).
Hopefully this point is made more clear by looking at the deserialization process (this is on the receiver's side). Notice how I allocate a struct to hold the information sent by the sender. If the sender did not send me the full string but instead only the memory address, I could not actually reconstruct the data which was sent (even on the same machine we have two distinct virtual memory spaces which are not the same). So in essence, a pointer is only a good mapping for the originator.
Finally, as far as "structs within structs" go, you will need to have several functions for each struct. That said, it is possible that you can reuse the functions. For instance, if I have two structs A and B where A contains B, I can have two serialize methods:
char* serializeB()
{
// ... Do serialization
}
char* serializeA()
{
char* B = serializeB();
// ... Either add on to serialized version of B or do some other modifications to combine the structures
}
So you should be able to get away with a single serialization method for each struct.
This answer is besides the problems with your malloc.
Unfortunately, you cannot find a nice trick that would still be compatible with the standard. The only way of properly serializing a structure is to separately dissect each element into bytes, write them to an unsigned char array, send them over the network and put the pieces back together on the other end. In short, you would need a lot of shifting and bitwise operations.
In certain cases you would need to define a kind of protocol. In your case for example, you need to be sure you always put the object p is pointing to right after struct A, so once recovered, you can set the pointer properly. Did everyone say enough already that you can't send pointers through network?
Another protocolish thing you may want to do is to write the size allocated for the flexible array member s in struct B. Whatever layout for your serialized data you choose, obviously both sides should respect.
It is important to note that you cannot rely on anything machine specific such as order of bytes, structure paddings or size of basic types. This means that you should serialize each field of the element separately and assign them fixed number of bytes.
You should serialize the data in a platform independent way.
Here is an example using the Binn library (my creation):
binn *obj;
// create a new object
obj = binn_object();
// add values to it
binn_object_set_int32(obj, "id", 123);
binn_object_set_str(obj, "name", "Samsung Galaxy Charger");
binn_object_set_double(obj, "price", 12.50);
binn_object_set_blob(obj, "picture", picptr, piclen);
// send over the network
send(sock, binn_ptr(obj), binn_size(obj));
// release the buffer
binn_free(obj);
If you don't want to use strings as keys you can use a binn_map which uses integers as keys. There is also support for lists. And you can insert a structure inside another (nested structures). eg:
binn *list;
// create a new list
list = binn_list();
// add values to it
binn_list_add_int32(list, 123);
binn_list_add_double(list, 2.50);
// add the list to the object
binn_object_set_list(obj, "items", list);
// or add the object to the list
binn_list_add_object(list, obj);
Interpret your data and understand what you want to serialize. You want to serialize an integer and a structure of type B (recursivelly, you want to serialize an int, a long, and an array of strings). Then serialize them. The length you need it sizeof(int) + sizeof(long) + ∑strlen(s[i])+1.
On the other hand, serialization is a solved problem (multiple times actually). Are you sure you need to hand write a serialization routine ? Why don't you use D-Bus or a simple RPC call ? Please consider using them.
I tried the method provided by #RageD but it didn't work.
The int value I got from deserialization was not the original one.
For me, memcpy() works for non-string variables. (You can still use strcpy() for char *)
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef struct A {
int a;
char *str;
} test_struct_t;
char *serialize(test_struct_t t) {
int str_len = strlen(t.str);
int size = 2 * sizeof(int) + str_len;
char *buf = malloc(sizeof(char) * (size+1));
memcpy(buf, &t.a, sizeof(int));
memcpy(buf + sizeof(int), &str_len, sizeof(int));
memcpy(buf + sizeof(int) * 2, t.str, str_len);
buf[size] = '\0';
return buf;
}
test_struct_t deserialize(char *buf) {
test_struct_t t;
memcpy(&t.a, buf, sizeof(int));
int str_len;
memcpy(&str_len, buf+sizeof(int), sizeof(int));
t.str = malloc(sizeof(char) * (str_len+1));
memcpy(t.str, buf+2*sizeof(int), str_len);
t.str[str_len] = '\0';
return t;
}
int main() {
char str[15] = "Hello, world!";
test_struct_t t;
t.a = 123;
t.str = malloc(strlen(str) + 1);
strcpy(t.str, str);
printf("original values: %d %s\n", t.a, t.str);
char *buf = serialize(t);
test_struct_t new_t = deserialize(buf);
printf("new values: %d %s\n", new_t.a, new_t.str);
return 0;
}
And the output of the code above is:
original values: 123 Hello, world!
new values: 123 Hello, world!
#Shahbaz is right I would think you actually want this
int len = sizeof(struct A);
obj = (struct A *) malloc(len);
But also you will run into problems when sending a pointer to another machine as the address the pointer points to means nothing on the other machine.
I'm trying to split a char* to an array of char* in C.
I'm used to program in Java / PHP OO. I know several easy way to do that in these languages but in C... I'm totally lost. I often have segfault for hours x)
I'm using TinyXML and getting info from XML File.
Here's the struct where we find the array.
const int MAX_GATES = 64;
typedef struct {
char *name;
char *firstname;
char *date;
char *id;
char *gates[MAX_GATES];
} UserInfos;
And here's where I fill this struct :
UserInfos * infos = (UserInfos*)malloc(1024);
infos->firstname = (char*)malloc(256);
infos->name = (char*)malloc(128);
infos->id = (char*)malloc(128);
infos->date = (char*)malloc(128);
sprintf(infos->firstname, "%s", card->FirstChild("firstname")->FirstChild()->Value());
sprintf(infos->name, "%s", card->FirstChild("name")->FirstChild()->Value());
sprintf(infos->date, "%s", card->FirstChild("date")->FirstChild()->Value());
sprintf(infos->id, "%s", card->FirstChild("filename")->FirstChild()->Value());
////////////////////////
// Gates
char * gates = (char*) card->FirstChild("gates")->FirstChild()->Value();
//////////////////////////
The only problem is on 'gates'.
The input form XML looks like "gate1/gate2/gate3" or just blank sometimes.
I want gate1 to be in infos->gates[0] ; etc.
I want to be able to list the gates array afterwards..
I always have a segfault when I try.
Btw, I don't really now how to initialize this array of pointers. I always initialize all gates[i] to NULL but It seems that I've a segfault when I do
for(int i=0;i
Thanks for all.
It's OK when I've only pointers but when String(char*) / Arrays / Pointers are mixed.. I can't manage =P
I saw too that we can use something like
int *myArray = calloc(NbOfRows, NbOfRows*sizeof(int));
Why should we declare an array like that.. ? x)
Thanks!
The problem that people frequently have with XML is that they assume all the elements are available. That's not always safe. Thus this statement:
sprintf(infos->firstname, "%s", card->FirstChild("firstname")->FirstChild()->Value());
Isn't safe to do because you don't actually know if all of those
functions actually return valid objects. You really need something
like the following (which is not optimized for speed, as I don't
know the tinyXML structure name being returned at each point and thus
am not storing the results once and am rather calling each function
multiple times:
if (card->FirstChild("firstname") &&
card->FirstChild("firstname")->FirstChild()) {
sprintf(infos->firstname, "%s", card->FirstChild("firstname")->FirstChild()->Value());
}
And then, to protect against buffer overflows from the data you should
really be doing:
if (card->FirstChild("firstname") &&
card->FirstChild("firstname")->FirstChild()) {
infos->firstname[sizeof(infos->firstname)-1] = '\0';
snprintf(infos->firstname, sizeof(infos->firstname)-1, "%s", card->FirstChild("firstname")->FirstChild()->Value());
}
Don't you just love error handling?
As to your other question:
I saw too that we can use something like int *myArray =
calloc(NbOfRows, NbOfRows*sizeof(int)); Why should we declare an array
like that.. ? x)
calloc first initializes the resulting memory to 0, unlike malloc.
If you see above where I set the end of the buffer to '\0' (which is
actually 0), that's because malloc returns a buffer with potentially
random (non-zero) data in it. calloc will first set the entire buffer
to all 0s first, which can be generally safer.
so I have the following enum method in c:
enum enum_type GetInfo (int socket, unsigned char *data)
{
}
and at the api I can find this:
Received data is written to pointer *data....
So if I'm doing something like this:
unsigned char *data;
enum_type enum1;
enum1 = GetInfo (int socket, data);
I got an segmentation fault.
What's my problem?
Thanks,
Simon
Your problem is that you haven't allocated space for data but try to write to it. Do
unsigned char *data = malloc(sizeof(unsigned char) * MYBUFLENGTH);
and then pass data to GetInfo. At the end do not forget to
free(data);
Alternatively you could allocate space on stack (available in C99, some compilers support it as extension even with earlier versions of the C Standard)
unsigned char data[MYBUFLENGTH];
In this case you should not worry about memory management.
you need to allocate memory to store the data in.
for instance:
unsigned char data[10000]; /* allocate 10000 bytes */
enum_type enum1;
enum1 = GetInfo(socket, data);
If you don't understand what's going on, I recommend spending time to read up on pointers.
not sure how big the info is, but try the following
unsigned char data[512] = {0};
enum_type enum1;
enum1 = GetInfo (socket, data);
this makes sure, that data points to a valid memory address on the stack.
It's maybe because GetInfo wants to write to a buffer pointed by data, and you just pass the pointer without allocating any memory space. Allocate memory and point it to data like this:
// I assume you need 1000 bytes
data = (unsigned char*)malloc(1000*sizeof(unsigned char));
I have a packet from a server which is parsed in an embedded system. I need to parse it in a very efficient way, avoiding memory issues, like overlapping, corrupting my memory and others variables.
The packet has this structure "String A:String B:String C".
As example, here the packet received is compounded of three parts separated using a separator ":", all these parts must be accesibles from an structure.
Which is the most efficient and safe way to do this.
A.- Creating an structure with attributes (partA, PartB PartC) sized with a criteria based on avoid exceed this sized from the source of the packet, and attaching also an index with the length of each part in a way to avoid extracting garbage, this part length indicator could be less or equal to 300 (ie: part B).
typedef struct parsedPacket_struct {
char partA[2];int len_partA;
char partB[300];int len_partB;
char partC[2];int len_partC;
}parsedPacket;
The problem here is that I am wasting memory, because each structure should copy the packet content to each the structure, is there a way to only save the base address of each part and still using the len_partX.
How about replacing the (:) with a 0, and add a null to the end - then you have three char * to pass around. You will need to deal with 0 length strings, but that might solve it
To avoid corrupting memory and other variables, you generally declare large data buffers as statics and place them at file scope, then allocate a separate RAM segment for them. Having them sitting on the stack is a bad idea in any embedded system.
You need to consider whether there is an alignment requirement for the CPU and whether the code should be portable or not. The compiler is free to add any number of padding bytes anywhere in that struct, meaning you may not be able to do this:
parsedPacket pp;
memcpy(&pp, raw_data, sizeof(parsedPacket )) ;
For this reason, structs are generally a bad choise for storing data packages. The safest solution is this:
/* packet.h */
typedef struct parsedPacket_struct {
uint8_t* partA;
uint8_t* partB;
uint8_t* partC;
uint16_t len_partA;
uint16_t len_partB;
uint16_t len_partC;
}parsedPacket;
#define MAX_PART_A 2
#define MAX_PART_B 300
#define MAX_PART_C 2
void packet_receive (parsedPacket* packet);
/* packet.c */
static uint8 partA[MAX_PART_A];
static uint8 partB[MAX_PART_B];
static uint8 partC[MAX_PART_C];
void packet_receive (parsedPacket* packet)
{
/* receive data from server */
...
packet->len_partA = ...;
packet->len_partB = ...;
packet->len_partC = ...;
packet->partA = partA;
packet->partB = partB;
packet->partC = partC;
memcpy(partA, A_from_server, packet->len_partA);
memcpy(partB, B_from_server, packet->len_partB);
memcpy(partC, C_from_server, packet->len_partC);
}
This can be extended to contain several static buffers if needed, ie a static array of arrays for each buffer. As you are dealing with large amounts of data in an embedded system, you can never allow the program to stack the buffers at a whim. The maximum amount of copies of a received packet must be determined during program design.
I'm not sure why you think your approach is wasting memory, but here's what I would do if I were feeling especially hacky:
typedef struct {
char *a, *b, *c;
char data[1]; // or 0 if your compiler lets you, or nothing in C99
} parsedPacket;
This is called a flexible array member. Basically, when you allocate memory for your struct, you do this:
parsedPacket *p = malloc(offsetof(parsedPacket, data[N]));
N above becomes the amount of data your array needs, i.e. how long the string you read is. This allocates the struct so that the data member has enough size for your entire string of data. Then, copy the string you recieve into this member, replace ':' characters with '\0', and set a to the first string (i.e. p->a = p->data), b to the second (p->b = p->data + strlen(p->a) + 1) and c to the third. Of course, you can make this process easier by doing it all at once:
size_t current = 0;
p->a = p->data;
p->b = p->c = NULL;
while(1)
{
int i = getc();
if(i == '\n' || i == EOF) break; // or whatever end conditions you expect
if(i == ':')
{
p->data[current] = '\0';
++current;
if(p->b == NULL) p->b = &p->data[current];
else if(p->c == NULL) p->c = &p->data[current];
else /* error */;
}
else
{
p->data[current] = i;
}
}
The type of each len_partN should be a type that can count up to the length of partN. E.g.:
typedef struct parsedPacket_struct {
char partA[300];unsigned short len_partA; // unsigned shorts have < 32k distinct values
char partB[300];unsigned short len_partB;
char partC[300];unsigned short len_partC;
}parsedPacket;
This seems like a design decision. If you want the struct to be easy to create, use the above approach, but beware its drawbacks (like "what if B has more than 300 chars?").