C - varying size text - c

Question

I have to write for my assignement a program that will consist of agents and a central server deamon. It will be a distributed shell - every command issued from a server will be also performed on every agent(the output will be sent back from every agent to central server).
I will have to deal with output commands (like ls -la /home/user/dir1) - on each agent the output may vary in size). The output of "find /" will also be BIG but I have to take somehow into account that something like that can happen. What is desired way of handling varying size outputs in C and operating on them? (saving to variable, sending it over a socket).

Answer 1

The way to deal with data of arbitrary size is to use dynamic allocation, i.e. the functions malloc(), realloc() and free(). You allocate and possibly grow the memory needed to store the command output.
Reading command output (assuming a Unix-like OS) is best done with popen().
Read the manuals of each of the mentioned functions for details.

Answer 2

Dynamic Memory Allocation
To hold your "variable length" strings, you should use dynamic memory allocation: the malloc family of functions.
#include <stdlib.h>
void *malloc(size_t size);
void free(void *ptr);
void *calloc(size_t nmemb, size_t size);
void *realloc(void *ptr, size_t size);
So, suppose you have your data stored in a variable char *ag_str. I suggest you malloc and then realloc the size of the buffer in blocks. Calling malloc and then realloc a thousand times to readjust the block size after each character is very costly.
So, you might do something like this:
#define BLOCK_SIZE 4096
struct mem_block {
size_t current_block_size;
size_t current_str_size;
char *ag_str;
};
struct mem_block *new_chunk(void)
{
struct mem_block *p = malloc(sizeof *p);
p->ag_str = malloc(BLOCK_SIZE);
p->current_block_size = BLOCK_SIZE;
p->current_str_size = 0;
return p;
}
void realloc_chunk(struct mem_block *chunk)
{
size_t ns = chunk->current_block_size + BLOCK_SIZE;
chunk->ag_str = realloc(chunk->ag_str, ns);
chunk->current_block_size = ns;
}
void cat_ag_str(struct mem_block *chunk, char *ag_str, size_t ag_len)
{
if (chunk->current_str_size + ag_len > chunk->current_block_size)
realloc_chunk(chunk);
strncat (chunk->ag_str, ag_str, ag_len);
chunk->current_str_size += ag_len;
}
void receive_from_agent(...)
{
struct mem_block *chunk = new_chunk();
ssize_t c; // Linux read/recv return
size_t count;
char buff[BLOCK_SIZE];
while((c = read(your_fd, buff, BLOCK_SIZE)) // or probably recv()
if (c < 0) ...
count = (size_t)c;
cat_ag_str(chunk, buff, count);
(...)
}
Note that this code was not tested and is just an idea for you. (Error checking was omitted)
struct mem_block: This will keep information about your current memory block.
new_chunk: function to create a new chunk handler for you.
realloc_chunk: anytime the amount of characters that must be written exceeds the amount of characters available in the chunk, we get one more block.
cat_ag_str: this will append what you just read to the memory block you have, effectively transforming chunks of data into one coherent big buffer.
receive_from_agent: this is the entry point of your receiving loop. You may use read or recv, I don't know which you use, but both return the amount of bytes read, which you'll use to pass to cat_ag_str.
It's important to note that you're reading in the same sized blocks as you realloc. (You can read in smaller chunks too, but never bigger).
You can do roughly the same for sending, but you don't need all that workaround for memory. You can just use a fixed sized buffer and copy data from your big string to it in fixed sizes, then you send the fixed-sized buffer.