I have a data structure that acts as a buffer. It's two structures that have a char *color within them. I am able to store and retrieve the correct color out of each producer process by themselves, but the consumer does not seem to see anything in the color variable, though it knows it's not null. How can this be?
typedef struct {
char *color;
struct timeval time;
} data_struct;
typedef struct{
data_struct buf1;
data_struct buf2;
} buffer;
In the producer I store:
ptr->buf1.color = color;
gettimeofday (&tval, NULL);
ptr->buf1.time = tval;
And I am able to then print the color it just stored and it does work within the same process. But the consumer uses:
printf("%s\t", ptr->buf1.color);
struct timeval tval = ptr->buf1.time;
printf("%ld\n", (tval.tv_sec)*1000000L + tval.tv_usec);
And he just ends up printing a blank area, then a tab, then the time in seconds. How is it accessing the same shared memory but not seeing my string?? This approximately identical code works in the threaded version just fine. Thanks for your help!
Update with shared memory segments.
int shmem_id;
buffer *shmem_ptr;
key_t key = 7484;
int size = 2048;
int flag = 1023;
char keystr[10];
/* create a shared memory segment */
shmem_id = shmget (key, size, flag);
shmem_ptr = shmat (shmem_id, (void *) NULL, 1023);
shmem_ptr->buf1.color = NULL;
shmem_ptr->buf2.color = NULL;
sprintf (keystr, "%d", key);
Even though buf may be in shared memory, that does not guarantee that color is also pointing to something in shared memory. My guess is that color is pointing at some address that is only visible to the producer process and not the consumer process. To fix this, you could define the color field as a fixed length char array, and use
strncpy(ptr->buf1.color, color, BUFLEN);
ptr->buf1.color[BUFLEN-1] = '\0';
to set the color.
I think the reason that you are not malloc the struct and buffer properly as they are not protected and thus it would be over written by any thing. Try to do things like these:
typedef struct {
char *color;
struct timeval time;
} data_struct;
data_struct *temp=(data_struct*)malloc(sizeof(data_struct));
temp->color = (char *)malloc(strlen(color)+1);
strcpy(temp->color, color);
I think doing these would not generate problem. But remember to free the allocated memory first of string then structure.
free(temp->color);
free(temp);
It doesn't work because you have one process which stores a pointer, and a separate process which reads that pointer. But these two processes do not share the same address space, so the pointer is meaningless when read. You could solve this a few ways:
Use threads instead of processes; this way the producer and consumer will have a single address space and pointers from one will be valid in the other.
Store the text directly in the struct, without a pointer indirection (i.e. char color[30]).
Store an "offset pointer" in the struct, and store the character data somewhere else in the shared memory region (a sort of pool allocator).
Make an array of fixed strings in the producer and consumer and just store the index into that lookup table. This works if the strings are known in advance.
Related
This question already has answers here:
C - shared memory - dynamic array inside shared struct
(5 answers)
Closed 5 years ago.
I am working on a server with forking which stores a map(k->v) of strings in shared memory. I want to do it very simplistic but am lost in pointers and in what exactly I need to copy. So I extracted the relevant code which looks like this:
struct key_value {
char key[32];
char value[32];
};
struct key_value **map;
int *map_size;
int shmid = shmget(IPC_PRIVATE, sizeof(struct key_value**), 0600);
map = (struct key_value**) shmat(shmid, 0, 0);
int shmid_size = shmget(IPC_PRIVATE, sizeof(int), 0600);
map_size = (int*) shmat(shmid_size, 0, 0);
*map_size = 0;
//the above happens before fork()
char *c = "abc";
int shmid_struct = shmget(IPC_PRIVATE, sizeof(struct key_value*), 0600);
struct key_value *entry = (struct key_value*) shmat(shmid_struct, 0, 0);
*entry->key = *c;
printf("%s\n", map[0]->key);
//smhdt's & shmctl's
So what I want is to copy that string "abc" from *c into the map so into shared memory. Clearly, I do not yet fully understand pointers and structs so am hoping someone can clear it up. I currently get a segfault 'somewhere in main' (thanks gdb...).
Note that I am ok with the map having a fixed max_size for now (though would be great if dynamic).
EDIT: it's been pointed out in an answer that having a char* in the struct is difficult, so to use char[x] instead. Have updated the code to reflect that, but still not working.
Structures which contain pointers cannot be safely stored in shared memory, as the pointers are meaningless outside the process that created them. Even if the shared memory region is mapped at the same address in each process (which is true if the memory was mapped before a fork(), but may be false in other scenarios), pointers into non-shared memory will not behave properly, as each process may have different data at that address.
If you want to store strings in shared memory, you will need to store them as explicit character arrays, e.g.
struct key_value {
char key[32];
char value[32];
};
or use another scheme, such as storing an offset into a string table in the shared memory region.
Generally speaking, though, shared memory is not a good tool for inter-process communication. If your application depends on being able to share data in memory, threading is probably a better approach.
I'm looking at some example code and trying to figure this out but am stuck. I'm just trying to create a buffer of 10 ints and have my shared memory pointers point to that. Can someone help me understand what this code is actually doing and where I went wrong?
int shmem_id; /* shared memory identifier */
int *shmem_ptr[BUFSIZE]; /* pointer to shared segment */
key_t key = 4455; /* a key... */
int size = 2048; /* 2k memory */
int flag = 1023; /* permissions */
char keystr[10];
sprintf (keystr, "%d", key);
shmem_id = shmget (key, size, flag); /* create a shared memory segment */
shmem_ptr = shmat (shmem_id, (void *) NULL, 1023);
In reality I want it to a buffer of 10 struct items.
typedef struct widget{
char color[10];
};
Your declaration:
int *shmem_ptr[BUFSIZE];
declares shmem_ptr to be an array of pointers to integers. You just want a pointer to integers, so it should be:
int *shmem_ptr;
If the memory points to widget structures, you can do:
typedef struct widget {
char color[10];
} widget;
widget *shmem_ptr;
You don't need to declare the length when declaring a pointer. The length is specified when the shared memory block is created, not in the program that attaches to it.
I am working on a little project where I am trying to emulate a CPU declaring and assigning values to variables, so far so good. So I defined the following structures for my CPU and Memory.
CPU (Can only keep track of 3 variables)
typedef struct variableReference{
char *tag;
void *reference;
}variableReference;
typedef struct CPU{
int variableCounter;
int instructionPointer;
variableReference dataDictionary[3];
void *currentContext;
}CPU;
Memory (and it's creation function). By the way, CPU.currentContext points to the memory.base on startup, then it can change.
typedef struct Memory{
void *base;
int size;
}Memory;
Memory memory_create(int size){
Memory newMemory;
newMemory.base = malloc(size);
newMemory.size = size;
return newMemory;
}
So the first thing I do is allocate a block of memory and keep track of it using the CPU current context pointer. I already have a function that asks the cpu to declare a variable (variables can only be integer) and another one that asks the cpu to assign that variable a value (int). To do this, I keep track of a limited amount of variable names and references in cpu.dataDictionary.
The variable declaring seems to work just fine, but the problem occurs when I try to assign that variable a value, which I do like this (e.g. a = 4;):
cpu_assignVariable(&myCPU,"a",4,&myMemory);
At that time, in my cpu.dataDictionary I have this in the first record {"a",0x804b008}, so all I should have to do is look for that position in the allocated memory block, and copy the value (integer 4), like this:
void *reference = cpu_dereferenceVariable(*myCPU,tag); // gets a's address
memory_write(reference,1,sizeof(value),(void *) value); // writes the value
now the memory_write implementation, where I get the Segmentation Fault (I have an offset of 1 because on the "reference" position I previously wrote the variable name:
int memory_write(void *base, int offset,int size,void *value){
memcpy(base+offset,value,size);
return 0;
}
I'd expect the memory block to look like this |a|0|0|0|4|x|x|...|x|, but all I get is a segmentation fault error. Any ideas???
Thanks in Advance!
I have a packet from a server which is parsed in an embedded system. I need to parse it in a very efficient way, avoiding memory issues, like overlapping, corrupting my memory and others variables.
The packet has this structure "String A:String B:String C".
As example, here the packet received is compounded of three parts separated using a separator ":", all these parts must be accesibles from an structure.
Which is the most efficient and safe way to do this.
A.- Creating an structure with attributes (partA, PartB PartC) sized with a criteria based on avoid exceed this sized from the source of the packet, and attaching also an index with the length of each part in a way to avoid extracting garbage, this part length indicator could be less or equal to 300 (ie: part B).
typedef struct parsedPacket_struct {
char partA[2];int len_partA;
char partB[300];int len_partB;
char partC[2];int len_partC;
}parsedPacket;
The problem here is that I am wasting memory, because each structure should copy the packet content to each the structure, is there a way to only save the base address of each part and still using the len_partX.
How about replacing the (:) with a 0, and add a null to the end - then you have three char * to pass around. You will need to deal with 0 length strings, but that might solve it
To avoid corrupting memory and other variables, you generally declare large data buffers as statics and place them at file scope, then allocate a separate RAM segment for them. Having them sitting on the stack is a bad idea in any embedded system.
You need to consider whether there is an alignment requirement for the CPU and whether the code should be portable or not. The compiler is free to add any number of padding bytes anywhere in that struct, meaning you may not be able to do this:
parsedPacket pp;
memcpy(&pp, raw_data, sizeof(parsedPacket )) ;
For this reason, structs are generally a bad choise for storing data packages. The safest solution is this:
/* packet.h */
typedef struct parsedPacket_struct {
uint8_t* partA;
uint8_t* partB;
uint8_t* partC;
uint16_t len_partA;
uint16_t len_partB;
uint16_t len_partC;
}parsedPacket;
#define MAX_PART_A 2
#define MAX_PART_B 300
#define MAX_PART_C 2
void packet_receive (parsedPacket* packet);
/* packet.c */
static uint8 partA[MAX_PART_A];
static uint8 partB[MAX_PART_B];
static uint8 partC[MAX_PART_C];
void packet_receive (parsedPacket* packet)
{
/* receive data from server */
...
packet->len_partA = ...;
packet->len_partB = ...;
packet->len_partC = ...;
packet->partA = partA;
packet->partB = partB;
packet->partC = partC;
memcpy(partA, A_from_server, packet->len_partA);
memcpy(partB, B_from_server, packet->len_partB);
memcpy(partC, C_from_server, packet->len_partC);
}
This can be extended to contain several static buffers if needed, ie a static array of arrays for each buffer. As you are dealing with large amounts of data in an embedded system, you can never allow the program to stack the buffers at a whim. The maximum amount of copies of a received packet must be determined during program design.
I'm not sure why you think your approach is wasting memory, but here's what I would do if I were feeling especially hacky:
typedef struct {
char *a, *b, *c;
char data[1]; // or 0 if your compiler lets you, or nothing in C99
} parsedPacket;
This is called a flexible array member. Basically, when you allocate memory for your struct, you do this:
parsedPacket *p = malloc(offsetof(parsedPacket, data[N]));
N above becomes the amount of data your array needs, i.e. how long the string you read is. This allocates the struct so that the data member has enough size for your entire string of data. Then, copy the string you recieve into this member, replace ':' characters with '\0', and set a to the first string (i.e. p->a = p->data), b to the second (p->b = p->data + strlen(p->a) + 1) and c to the third. Of course, you can make this process easier by doing it all at once:
size_t current = 0;
p->a = p->data;
p->b = p->c = NULL;
while(1)
{
int i = getc();
if(i == '\n' || i == EOF) break; // or whatever end conditions you expect
if(i == ':')
{
p->data[current] = '\0';
++current;
if(p->b == NULL) p->b = &p->data[current];
else if(p->c == NULL) p->c = &p->data[current];
else /* error */;
}
else
{
p->data[current] = i;
}
}
The type of each len_partN should be a type that can count up to the length of partN. E.g.:
typedef struct parsedPacket_struct {
char partA[300];unsigned short len_partA; // unsigned shorts have < 32k distinct values
char partB[300];unsigned short len_partB;
char partC[300];unsigned short len_partC;
}parsedPacket;
This seems like a design decision. If you want the struct to be easy to create, use the above approach, but beware its drawbacks (like "what if B has more than 300 chars?").
I am writing a light weight serialization function and need to include two variable sized arrays within this.
How should I track the size of each?
How should I define the struct?
Am I going about this all wrong?
EDIT: the result must be a contiguous block of memory
This resolves to something like
typedef struct
{
size_t arr_size_1, arr_size_2;
char arr_1[0/*arr_size_1 + arr_size_2*/];
} ...;
The size(s) should be in the front of the dynamic sized data, so that it doesn't move when expanding your array.
You cannot have 2 unknown sized arrays in your struct, so you must collapse them into one and then access the data relative from the first pointer.
typedef struct MyStruct_s
{
int variable_one_size;
void* variable_one_buf;
int variable_two_size;
void* variable_two_buf;
} MyStruct;
MyStruct* CreateMyStruct (int size_one, int size_two)
{
MyStruct* s = (MyStruct*)malloc (sizeof (MyStruct));
s->variable_one_size = size_one;
s->variable_one_buf = malloc (size_one);
s->variable_two_size = size_two;
s->variable_two_buf = malloc (size_two);
}
void FreeMyStruct (MyStruct* s)
{
free (s->variable_one_buf);
free (s->variable_two_buf);
free (s);
}
Since the data should be continuous in memory it is necessary to malloc a chunk of memory of the right size and manage it's contents more or less manually. You probably best create a struct that contains the "static" information and related management functions that do the memory management and give access to the "dynamic" members of the struct:
typedef struct _serial {
size_t sz_a;
size_t sz_b;
char data[1]; // "dummy" array as pointer to space at end of the struct
} serial;
serial* malloc_serial(size_t a, size_t b) {
serial *result;
// malloc more memory than just sizeof(serial), so that there
// is enough space "in" the data member for both of the variable arrays
result = malloc(sizeof(serial) - 1 + a + b);
if (result) {
result->sz_a = a;
result->sz_b = b;
}
return result;
}
// access the "arrays" in the struct:
char* access_a(serial *s) {
return &s->data[0];
}
char* access_b(serial *s) {
return &s->data[s->sz_a];
}
Then you could do things like this:
serial *s = ...;
memcpy(access_a(s), "hallo", 6);
access_a(s)[1] = 'e';
Also note that you can't just assign one serial to another one, you need to make sure that the sizes are compatible and copy the data manually.
In order to serialize variably-sized data, you have to have a boundary tag of some sort. The boundary tag can be either a size written right before the data, or it can be a special value that is not allowed to appear in the data stream and is written right after the data.
Which you choose depends on how much data you are storing, and if you are optimizing for size in the output stream. It is often easier to store a size before-hand, because you know how big to make the receiving buffer. If you don't then you have to gradually resize your buffer on load.
In some ways, I'd do things like Dan Olson. However:
1) I'd create the final struct by having two instances of a simpler struct that has just one variable array.
2) I'd declare the array with byte* and use size_t for its length.
Having said this, I'm still not entirely clear on what you're trying to do.
edit
If you want it contiguous in memory, just define a struct with two lengths. Then allocate a block big enough for both blocks that you want to pass, plus the struct itself. Set the two lengths and copy the two blocks immediately after. I think it should be clear how the lengths suffice to make the struct self-describing.