I'm working a sort of "restaurant" implementation in C with client-server.
I am trying to send the following structure through a FIFO:
typedef struct {
int numtable; //table number to send answer
char timestamp[20]; //simple timestamp
int order[MENUSZ]; //array of int with dish IDs
} request;
About this struct, I basically send to the server the table number, to "build" the client FIFO name through a template, a timestamp, and order is a simple array filled with randomly chosen integers to "create" a sort of random menu request.
With this setup I didn't have problems, using
write(server_fd, &request, sizeof(request))
I had problems when I wanted to transform the array order[MENUSZ] in a pointer, to make a dynamic array, like this:
typedef struct {
int numtable;
char timestamp[20];
int *order;
} request;
After changing the struct, I used the malloc function to allocate enough space for the array:
request->order = malloc(sizeof(int)*numclients+1);
The array is fullfilled correctly, but for some reason the server can't read from the FIFO after I added this pointer, by doing
read(server_fd, &request, sizeof(request));
I can't figure out why it doesn't work with this pointer. Am I doing something wrong?
The array is fullfilled correctly, but for some reason the server can't read from the FIFO after I added this pointer, by doing
read(server_fd, &request, sizeof(request));
You are transferring your structure, which includes a pointer, and the value of the pointer will be transferred correctly, but it will not point to a valid address in the destination process, neither will there be memory allocated where the pointer points to.
Hence, you need to transfer the array separately and recreate the pointer in the destination process, something like:
read(server_fd, &request, sizeof(request));
/* allocate memory for request->order in the reader process */
request->order = malloc(sizeof(int)*numclients+1);
read(server_fd, request->order, sizeof(int)*numclients+1);
A yet better solution would be to also transfer the size of the array inside your structure.
On the sending side, you then need to send both, the structure and the array contents, something like
write(server_fd, &request, sizeof(request))
write(server_fd, request->order, sizeof(int)*numclients+1));
It is because sizeof(request) no longer tells you the size of the combined structure. Try this
typedef struct {
int numtable;
char timestamp[20];
int order[1];
} request;
When you have a new request
int reqsize = sizeof(request) + sizeof(int) * numclients;
request* req = malloc(reqsize);
This allows you to use req->order[1] to req->order[numclients - 1]. When you send it, use
write(server_fd, reqsize, sizeof(int))
write(server_fd, req, reqsize)
When reading
read(server_fd, &reqsize, sizeof(int))
Then allocate the request before reading
request* req = malloc(reqsize)
read(server_fd, req, reqsize)
This technique uses "the chumminess of C" http://c-faq.com/struct/structhack.html, which, as far as I know, works on all implementations of C.
Related
I'm writing an application which reads data from a UART interface. The data is sent in packets. Each packet has a channel associated with it. My application multiplexes received packets into virtual channels (threads) so that every channel can work independently of one another. When I receive a packet I have to do something depending on it's contents and produce a response. The response is sent back using the same UART interface.
The data sent is mostly binary. When I'm reading from the UART interface, I know the size of the packet beforehand, so I can preallocate memory with no problem.
The problem for me is producing a response. I know the maximum size of a packet, so I can create a static buffer when I'm constructing a response. If I we're to work with ASCII characters, instead of binary data, I could rely on NULL terminator to determine how long the data stored in the buffer is. However, I'm working with binary data, so using a NULL byte does not work. Instead, I have to keep a variable storing how many bytes of the buffer is used up already. I was thinking of using a custom data type for storing binary data:
typedef struct {
unsigned char buff[2048];
size_t buff_used;
} binary_data_t;
What would be a standart way of handling this?
Since you know the number of bytes you need to hold a packet, just use a flexible array member:
typedef struct
{
size_t bytes;
unsigned char data[];
} binary_data_t;
(Note that identifiers ending in _t are reserved by POSIX, and you really shouldn't be using them.)
Allocation and reading data (assumes you read() from a file descriptor):
binary_data_t *p = malloc( sizeof( *p ) + numDataBytes );
p->bytes = numDataBytes;
ssize_t bytes_read = read( uartFD, p->data, numDataBytes );
one way of doing it could be to store a pointer to where in your array next byte should be placed.
typedef struct {
unsigned char buff[2048];
char* pData;
} binary_data_t;
// at init
binary_data_t rspMsg;
rspMsg.pData = &rspMsg.buff[0];
// at entering data
*(rspMsg.pData) = data;
rspMsg.pData++;
// at sending data you know the length via
length = rspMsg.pData - &rspMsg.buff[0];
This is one way of solving this.
Can be done in many ways.
I have a data structure that acts as a buffer. It's two structures that have a char *color within them. I am able to store and retrieve the correct color out of each producer process by themselves, but the consumer does not seem to see anything in the color variable, though it knows it's not null. How can this be?
typedef struct {
char *color;
struct timeval time;
} data_struct;
typedef struct{
data_struct buf1;
data_struct buf2;
} buffer;
In the producer I store:
ptr->buf1.color = color;
gettimeofday (&tval, NULL);
ptr->buf1.time = tval;
And I am able to then print the color it just stored and it does work within the same process. But the consumer uses:
printf("%s\t", ptr->buf1.color);
struct timeval tval = ptr->buf1.time;
printf("%ld\n", (tval.tv_sec)*1000000L + tval.tv_usec);
And he just ends up printing a blank area, then a tab, then the time in seconds. How is it accessing the same shared memory but not seeing my string?? This approximately identical code works in the threaded version just fine. Thanks for your help!
Update with shared memory segments.
int shmem_id;
buffer *shmem_ptr;
key_t key = 7484;
int size = 2048;
int flag = 1023;
char keystr[10];
/* create a shared memory segment */
shmem_id = shmget (key, size, flag);
shmem_ptr = shmat (shmem_id, (void *) NULL, 1023);
shmem_ptr->buf1.color = NULL;
shmem_ptr->buf2.color = NULL;
sprintf (keystr, "%d", key);
Even though buf may be in shared memory, that does not guarantee that color is also pointing to something in shared memory. My guess is that color is pointing at some address that is only visible to the producer process and not the consumer process. To fix this, you could define the color field as a fixed length char array, and use
strncpy(ptr->buf1.color, color, BUFLEN);
ptr->buf1.color[BUFLEN-1] = '\0';
to set the color.
I think the reason that you are not malloc the struct and buffer properly as they are not protected and thus it would be over written by any thing. Try to do things like these:
typedef struct {
char *color;
struct timeval time;
} data_struct;
data_struct *temp=(data_struct*)malloc(sizeof(data_struct));
temp->color = (char *)malloc(strlen(color)+1);
strcpy(temp->color, color);
I think doing these would not generate problem. But remember to free the allocated memory first of string then structure.
free(temp->color);
free(temp);
It doesn't work because you have one process which stores a pointer, and a separate process which reads that pointer. But these two processes do not share the same address space, so the pointer is meaningless when read. You could solve this a few ways:
Use threads instead of processes; this way the producer and consumer will have a single address space and pointers from one will be valid in the other.
Store the text directly in the struct, without a pointer indirection (i.e. char color[30]).
Store an "offset pointer" in the struct, and store the character data somewhere else in the shared memory region (a sort of pool allocator).
Make an array of fixed strings in the producer and consumer and just store the index into that lookup table. This works if the strings are known in advance.
Hi I'm writing a program that sends a set of bytes through a message queue like so ...
#include <sys/msg.h>
#include <stddef.h>
key_t key;
int msqid;
struct pirate_msgbuf pmb = {2, { "L'Olonais", 'S', 80, 10, 12035 } };
key = ftok("/home/beej/somefile", 'b');
msqid = msgget(key, 0666 | IPC_CREAT);
/* stick him on the queue */
msgsnd(msqid, &pmb, sizeof(struct pirate_msgbuf) - sizeof(long), 0);
The above example is a simple program from beejs website that resembles mine.
What I'm doing however is sending a message with a struct like so ...
struct msg_queue{
long message_type;
char * buffer;
}
Now before I send my msg_queue, I created some alternative buffer that contains all sorts of information including null characters and such. Now when I do something like this ...
struct msg_queue my_queue;
my_queue.message_type = 1;
my_queue.buffer = "My message";
msgsnd(mysqid, &pmb, sizeof(struct msg_queue) - sizeof(long), 0);
I have no problems receiving the pointer and reading the values stored at that string. However if I were to do something similar like ...
struct msg_queue my_queue;
my_queue.message_type = 1;
my_queue.buffer = sum_buffer_with_lots_of_weird_values; // of type char *
msgsnd(mysqid, &pmb, sizeof(struct msg_queue) - sizeof(long), 0);
The pointer I pass through my queue to my other process will read garbage and not the values stored. I tried making my arbitrary array as a static char *, but that doesn't help either. How do I properly pass in my buffer through the queue? Thanks.
You shouldn't be sending a pointers to another process, they have no meaning (or point to something very different) in another process' address space.
Message queues aren't great for unbounded data like variable length strings. Change your pointer to a fixed length char array sufficiently big to hold the largest string and copy your string into the array before writing the queue. Or use another type of IPC such as domain socket.
Message Queue is used for inter-process communication.
When you malloc some memory in one process, it only exist in that process memory space not accessible by other process.
when you send that pointer over, you are sending a address space which is not accessible. It may even result in segmentation fault.
One way is to limit your buffer size, if applicable.
struct msg_queue{
long message_type;
char buffer[MAX_LEN];
}
Another way is to send it 2 times. The first msgsnd, sends the size of buffer to expect.
The next send, you send the char array over, using the size of the first send. :)
On receiving end, you first get the size, then receive the buffer.
Other way is to use pipes or socket.
"msgsend()" will only read the bytes in your buffer.
If one of those bytes happens to be a pointer (to some string or object somewhere else) ... guess what - the receiver will just get the binary pointer. Not the data being pointed to.
What you need to do is pack the entire contents of your message into a buffer, then send that linear buffer.
This question already has answers here:
Writing and reading (fwrite - fread) structures with pointers
(3 answers)
Closed 8 years ago.
gcc (GCC) 4.7.0
c89
Hello,
I have the following structure that I am trying to fwrite and fread.
However, because my device and resource are pointers. The fwrite will read the pointer values and not the data. I cannot use a array for the device or resource. Only pointers as they have to be dynamically allocated.
I allocate all memory for the structure elements before I write. Not shown here as I want to keep the snippet short. Nor is free'ing.
In my fread function, I allocate the memory for the device and resource so that the fread will read into these memory locations. However, this will not work.
What is the best way to do this?
Many thanks for any advice,
struct data {
int id;
int set;
char *device;
char *resource;
};
struct database {
struct data **db_data;
size_t database_rows;
size_t database_data_size;
};
int database_write(FILE *fp, const struct database *db)
{
rewind(fp);
if(fwrite(*db->db_data, sizeof(struct data), 1, fp) == -1) {
return DATABASE_ERROR;
}
return 0;
}
struct database* database_read(FILE *fp, size_t db_rows, size_t db_data_size)
{
struct database *db = NULL;
size_t i = 0;
db = malloc(sizeof(struct database));
db->database_rows = db_rows;
db->database_data_size = db_data_size;
db->db_data = malloc(sizeof(struct data) * db_rows);
for(i = 0; i < db_rows; i++) {
db->db_data[i] = malloc(sizeof(struct data));
db->db_data[i]->device = malloc(db_data_size);
db->db_data[i]->resource = malloc(db_data_size);
}
rewind(fp);
if(fread(*db->db_data, sizeof(struct data), 1, fp) == -1) {
return NULL;
}
return db;
}
You seem to have answered your own question, fread and fwrite just look at what's in memory and put that in the file. This works great if you're writing things that don't have pointers (e.g. big arrays of numbers). It's not designed to write structs with pointers.
If this file has a format, you need to do what the format says. If you're making up a format as you go, then you should write each member one by one into the file. You will need some sort of buffer to read into (you may need to resize this if you don't have a maximum length specification). Also, your database_write function will need to be changed quite a bit as well.
If device and resource can have variable length you should write down the size of device and then the data. Do the same for resource.
When you read them back you can read the size, then allocate memory and finally read the value.
You have yourself described you problem. fwrite will write the address and not the value.
May be you can use a field for the length of device and resource in your structure "struct data".
Create a wrapper for fread() and fwrite() which reads/writes this length.
In this wrapper you can memcpy devices, resource in a temporary buffer and use fwrite() on it.
This is a simple and very basic solution.
While sending packets in networks, you will generally see a structures containing char pointers. The first 4/8 bytes store the length of the data and the remaining bytes contain the actual data.
User reading the packet, first reads the beginning 4/8 bytes. Depending on this, read() call is issued to read the remaining data.
You may refer
Is the "struct hack" technically undefined behavior?
I am writing a light weight serialization function and need to include two variable sized arrays within this.
How should I track the size of each?
How should I define the struct?
Am I going about this all wrong?
EDIT: the result must be a contiguous block of memory
This resolves to something like
typedef struct
{
size_t arr_size_1, arr_size_2;
char arr_1[0/*arr_size_1 + arr_size_2*/];
} ...;
The size(s) should be in the front of the dynamic sized data, so that it doesn't move when expanding your array.
You cannot have 2 unknown sized arrays in your struct, so you must collapse them into one and then access the data relative from the first pointer.
typedef struct MyStruct_s
{
int variable_one_size;
void* variable_one_buf;
int variable_two_size;
void* variable_two_buf;
} MyStruct;
MyStruct* CreateMyStruct (int size_one, int size_two)
{
MyStruct* s = (MyStruct*)malloc (sizeof (MyStruct));
s->variable_one_size = size_one;
s->variable_one_buf = malloc (size_one);
s->variable_two_size = size_two;
s->variable_two_buf = malloc (size_two);
}
void FreeMyStruct (MyStruct* s)
{
free (s->variable_one_buf);
free (s->variable_two_buf);
free (s);
}
Since the data should be continuous in memory it is necessary to malloc a chunk of memory of the right size and manage it's contents more or less manually. You probably best create a struct that contains the "static" information and related management functions that do the memory management and give access to the "dynamic" members of the struct:
typedef struct _serial {
size_t sz_a;
size_t sz_b;
char data[1]; // "dummy" array as pointer to space at end of the struct
} serial;
serial* malloc_serial(size_t a, size_t b) {
serial *result;
// malloc more memory than just sizeof(serial), so that there
// is enough space "in" the data member for both of the variable arrays
result = malloc(sizeof(serial) - 1 + a + b);
if (result) {
result->sz_a = a;
result->sz_b = b;
}
return result;
}
// access the "arrays" in the struct:
char* access_a(serial *s) {
return &s->data[0];
}
char* access_b(serial *s) {
return &s->data[s->sz_a];
}
Then you could do things like this:
serial *s = ...;
memcpy(access_a(s), "hallo", 6);
access_a(s)[1] = 'e';
Also note that you can't just assign one serial to another one, you need to make sure that the sizes are compatible and copy the data manually.
In order to serialize variably-sized data, you have to have a boundary tag of some sort. The boundary tag can be either a size written right before the data, or it can be a special value that is not allowed to appear in the data stream and is written right after the data.
Which you choose depends on how much data you are storing, and if you are optimizing for size in the output stream. It is often easier to store a size before-hand, because you know how big to make the receiving buffer. If you don't then you have to gradually resize your buffer on load.
In some ways, I'd do things like Dan Olson. However:
1) I'd create the final struct by having two instances of a simpler struct that has just one variable array.
2) I'd declare the array with byte* and use size_t for its length.
Having said this, I'm still not entirely clear on what you're trying to do.
edit
If you want it contiguous in memory, just define a struct with two lengths. Then allocate a block big enough for both blocks that you want to pass, plus the struct itself. Set the two lengths and copy the two blocks immediately after. I think it should be clear how the lengths suffice to make the struct self-describing.