I am currently writing a terminal based hex editor. and I have a few question regarding memory allocation.
To track changes the user has made I write them to an array of arrays like so, the [i][0] is the absolute offset of the change from the beginning of the file and [i][1] is the change itself:
unsigned long long writebuffer[10000][2];
but I have 2 problems with this. the first array (writebuffer[i][0]) NEEDS to be the sizeof unsigned long long but the second one ([i][1]) can be as small as sizeof unsigned char. Is it possible to do something like this??
also can I dynamically allocate the first index of writebuffer so I wouldn't initialize it like above but more like:
unsigned long long **writebuffer;
and then change the first index with malloc() and realloc(); while the second index would be 2 but have the size of a unsigned char.
Why not use a struct?
typedef struct {
long long offset;
int change; /* or unsigned short, or whatever you feel is right */
} t_change;
Be aware that the struct will likely get padded by the compiler to a different size if you choose to use unsigned char for the change element. What it gets padded to depends on your compiler, the compiler settings, and the target architecture.
You may define an array of type void:
void **writebuffer;
and then allocate and use each element as you like, for example:
*writebuffer[0] = (char*)malloc(sizeof(char));
Related
Hello everyone so i want to put a size_t variable of a number up to 4096 in order to keep track of how much "memory" i used in the array (yes it has to be a char array). So for example
#include <stdio.h>
#include <unistd.h>
int main(void)
{
char mem[4096] = {0};
size_t size = 4095;
mem[0] = size;
//then somehow be able to size = mem[0] to be able to recall the size later.
return 0;
}
I just want to be able to put a number in the first array elment in order to be able to keep track of how many elements ive used.
Im implementating a heap memory and i also have to implement malloc(), and free() i have a char array that simulates the heap memory, and i cant use global/ static local variables but i want to be able to keep track of the heap size. My best thought of action was to store the heap size in the "Heap" (char array) to be able to keep track of it.
If you must use a char array for storing the number of bytes used, you have (at least) two choices:
create a union of an uint16_t with the array. This will allow you to use the first two bytes to hold the number of bytes used.
essentially the same as #1 but do the math manually, and maintain the bytes yourself.
I'd recommend #1.
About the most straightforward way to do this that is defined by the C standard is with to copy the bytes from the size object to the memory:
memcpy(&mem[index], &size, sizeof size);
Good compilers with optimization enabled will optimize this to a store from size to memory if they can determine &mem[index] is suitably aligned (or an unaligned store instruction is available and a good method). The reverse, to retrieve the size, is memcpy(&size, &mem[index], sizeof size);.
It can be done with unions, but that may be cumbersome in this situation. The union would have to include the entire array.
Old C code would simply cast an address in mem to the desired type, as with * (size_t *) &mem[index] = size;. However, the C standard does not define the behavior when an array defined as an array of char is accessed through a size_t lvalue as the above does. This worked with old compilers, and modern compilers may still support it if an appropriate switch is given to ask the compiler to support it, but it is needless, so new code should avoid this in favor of more portability.
Note that you do not need to use a size_t if the size will only be 4096. For this, you can use uint16_t.
Another option is to calculate the two bytes:
static void RecordSize(unsigned char *memory, uint16_t size)
{
memory[0] = size >> 8; // Record high bits.
memory[1] = size; // Record low bits.
// This will automatically “chop off” high bits.
}
static uint16_t RetrieveSize(unsigned char *memory)
{
return (uint16_t) memory[0] << 8 | memory[1];
}
…
RecordSize(&mem[index], size);
…
size = RetrieveSize(&mem[index];
That would be used more rarely than the direct copy, but it could be useful if data is being recorded in memory to transmit to another computer system that may use a different byte ordering for its integers. This method explicitly stores the high bits first and the low bits second (or the code could be written to do the opposite, as desired).
Note that this code uses unsigned char. You should generally prefer to use unsigned char for manipulations of bytes as arbitrary data, as it avoids some problems that can occur with signed types. (The C standard leaves it implementation-defined whether the char type is signed or unsigned.)
I am learning flatbuffers and I wish to use them in C.
What I'm trying to do is writing a schema file for my structs.
this is one of my structs:
typedef struct
{
unsigned short request_number;
unsigned short length;
unsigned short height;
unsigned char *buffer;
} CASH_RECEIPT_REQUEST;
How can i write the pointer unsigned char *buffer in my schema file?
Don't store pointers, instead store the actual data they point to:
table CashReceiptRequest {
request_number:ushort;
length:ushort;
height:ushort;
buffer:[ubyte];
}
The buffer is a "vector of ubyte", as you can see. Note that if length is the size of the buffer, that field can be omitted, since the size is already stored as part of the vector.
Pointer sizes are platform dependent. If you decide use it on 64-bit processor with 8 bytes size pointers you can use ulong (as schema specifies in here) ulong size is 8 bytes. So at the parsing side you can typecast the ulong (In other words unsigned long) to char* type to obtain the correct pointer value.
Sample schema for your example looks like this
CASH_RECEIPT_REQUEST {
request_number:ushort;
length:ushort;
height:ushort;
buffer:ulong;
}
I'm trying to create a C client for dalmatinerdb but having trouble to understand how to combine the variables, write it to a buffer and send it to the database. The fact that dalmatinerdb is written in Erlang makes it more difficult. However, by looking at a python client for dalmatinerdb i have (probably) found the necessary variable sizes and order.
The erlang client has a function called "encode", see below:
encode({stream, Bucket, Delay}) when
is_binary(Bucket), byte_size(Bucket) > 0,
is_integer(Delay), Delay > 0, Delay < 256->
<<?STREAM,
Delay:?DELAY_SIZE/?SIZE_TYPE,
(byte_size(Bucket)):?BUCKET_SS/?SIZE_TYPE, Bucket/binary>>;
According to the official dalmatinerdb protocol we can see the following:
-define(STREAM, 4).
-define(DELAY_SIZE, 8). /bits
-define(BUCKET_SS, 8). /bits
Let's say i would like to create this kind of structure in C,
would it look something like the following:
struct package {
unsigned char[1] mode; // = "4"
unsigned char[1] delay; // = for example "5"
unsigned char[1] bucketNameSize; // = "5"
unsigned char[1] bucketName; // for example "Test1"
};
Update:
I realized that the dalmatinerdb frontend (web interface) only reacts and updates when values have been sent to the bucket. With other words just sending the first struct won't give me any clue if it's right or wrong. Therefore I will try to create a secondary struct with the actual values.
The erland code snippet which encodes values looks like this:
encode({stream, Metric, Time, Points}) when
is_binary(Metric), byte_size(Metric) > 0,
is_binary(Points), byte_size(Points) rem ?DATA_SIZE == 0,
is_integer(Time), Time >= 0->
<<?SENTRY,
Time:?TIME_SIZE/?SIZE_TYPE,
(byte_size(Metric)):?METRIC_SS/?SIZE_TYPE, Metric/binary,
(byte_size(Points)):?DATA_SS/?SIZE_TYPE, Points/binary>>;
The different sizes:
-define(SENTRY, 5)
-define(TIME_SIZE, 64)
-define(METRIC_SS, 16)
-define(DATA_SS, 32)
Which gives me this gives me:
<<?5,
Time:?64/?SIZE_TYPE,
(byte_size(Metric)):?16/?SIZE_TYPE, Metric/binary,
(byte_size(Points)):?32/?SIZE_TYPE, Points/binary>>;
My guess is that my struct containing a value should look like this:
struct Package {
unsigned char sentry;
uint64_t time;
unsigned char metricSize;
uint16_t metric;
unsigned char pointSize;
uint32_t point;
};
Any comments on this structure?
The binary created by the encode function has this form:
<<?STREAM, Delay:?DELAY_SIZE/?SIZE_TYPE,
(byte_size(Bucket)):?BUCKET_SS/?SIZE_TYPE, Bucket/binary>>
First let's replace all the preprocessor macros with their actual values:
<<4, Delay:8/unsigned-integer,
(byte_size(Bucket):8/unsigned-integer, Bucket/binary>>
Now we can more easily see that this binary contains:
a byte of value 4
the value of Delay as a byte
the size of the Bucket binary as a byte
the value of the Bucket binary
Because of the Bucket binary at the end, the overall binary is variable-sized.
A C99 struct that resembles this value can be defined as follows:
struct EncodedStream {
unsigned char mode;
unsigned char delay;
unsigned char bucket_size;
unsigned char bucket[];
};
This approach uses a C99 flexible array member for the bucket field, since its actual size depends on the value set in the bucket_size field, and you are presumably using this structure by allocating memory large enough to hold the fixed-size fields together with the variable-sized bucket field, where bucket itself is allocated to hold bucket_size bytes. You could also replace all uses of unsigned char with uint8_t if you #include <stdint.h>. In traditional C, bucket would be defined as a 0- or 1-sized array.
Update: the OP extended the question with another struct, so I've extended my answer below to cover it too.
The obvious-but-wrong way to write a struct corresponding to the metric/time/points binary is:
struct Wrong {
unsigned char sentry;
uint64_t time;
uint16_t metric_size;
unsigned char metric[];
uint32_t points_size;
unsigned char points[];
};
There are two problems with the Wrong struct:
Padding and alignment: Normally, fields are aligned on natural boundaries corresponding to their sizes. Here, the C compiler will align the time field on an 8-byte boundary, which means there will be padding of 7 bytes following the sentry field. But the Erlang binary contains no such padding.
Illegal flexible array field in the middle: The metric field size can vary, but we can't use the flexible array approach for it as we did in the earlier example because such arrays can only be used for the final field of a struct. The fact that the size of metric can vary means that it's impossible to write a single C struct that matches the Erlang binary.
Solving the padding and alignment issue requires using a packed struct, which you can achieve with compiler support such as the gcc and clang __packed__ attribute (other compilers might have other ways of achieving this). The variable-sized metric field in the middle of the struct can be solved by using two structs instead:
typedef struct __attribute((__packed__)) {
unsigned char sentry;
uint64_t time;
uint16_t size;
unsigned char metric[];
} Metric;
typedef struct __attribute((__packed__)) {
uint32_t size;
unsigned char points[];
} Points;
Packing both structs means their layouts will match the layouts of the corresponding data in the Erlang binary.
There's still a remaining problem, though: endianness. By default, fields in an Erlang binary are big-endian. If you happen to be running your C code on a big-endian machine, then things will just work, but if not — and it's likely you're not — the data values your C code reads and writes won't match Erlang.
Fortunately, endianness is easily handled: you can use byte swapping to write C code that can portably read and write big-endian data regardless of the endianness of the host.
To use the two structs together, you'd first have to allocate enough memory to hold both structs and both the metric and the points variable-length fields. Cast the pointer to the allocated memory — let's call it p — to a Metric*, then use the Metric pointer to store appropriate values in the struct fields. Just make sure you convert the time and size values to big-endian as you store them. You can then calculate a pointer to where the Points struct is in the allocated memory as shown below, assuming p is a pointer to char or unsigned char:
Points* points = (Points*)(p + sizeof(Metric) + <length of Metric.metric>);
Note that you can't just use the size field of your Metric instance for the final addend here since you stored its value as big-endian. Then, once you fill in the fields of the Points struct, again being sure to store the size value as big-endian, you can send p over to Erlang, where it should match what the Erlang system expects.
I need to define an array statically (in a *.h) file of size 12884901888 like.
unsigned char sram[12884901888]; //All of my code is C.
Above declaration gives error and does not work.
Because constants used in array declarations are unsigned int. But the constant i need to use (12884901888) is larger than the unsigned int limit.
How can i define the array as above, of size 12884901888 ?
Thank you.
-AD
P.S. I know many will say, optimize on that humongous array size, but i need to use same for some reason specific to my case.
Make the array dimension an unsigned long long.
unsigned char sram[12884901888ULL];
Is this for an embedded microcontroller? You can often get away with something like:
#define sram (*((unsigned char (*)[1]) 0))
Unless your compiler implements bounds checking, the array size does not matter. In any case you do not want the compiler to attempt to reserve 12884901888 bytes, because linking will fail.
Converting 12884901888 to hexadecimal gives : 0x3-0000-0000
(I separated each group of 16 bits)
In other words, this array of unsigned bytes needs 3 times 4 Gig
The compiler is supposed to generate 34 bits address pointer for this to work
I agree with finnw, you don't need to tell the compiler the size of the array. If you do specify the size, you will get a large OBJ file for the module and similarly large ELF/EXE for the final executable.
In C, in an Unix environment (Plan9), I have got an array as memory.
uchar mem[32*1024];
I need that array to contain different fields, such as an int (integer) to indicate the size of memory free and avaliable. So, I've tried this:
uchar* memp=mem;
*memp=(int)250; //An example of size I want to assign.
I know the size of an int is 4, so I have to force with casting or something like that, that the content of the four first slots of mem have the number 250 in this case, it's big endian.
But the problem is when I try to do what I've explained it doesn't work. I suppose there is a mistake with the conversion of types. I hopefully ask you, how could I force that mem[0] to mem[3] would have the size indicated, representated as an int and no as an uchar?
Thanks in advance
Like this:
*((int*) memp) = 250;
That says "Even though memp is a pointer to characters, I want you treat it as a pointer to integers, and put this integer where it points."
Have you considered using a union, as in:
union mem_with_size {
int size;
uchar mem[32*1024];
};
Then you don't have to worry about the casting. (You still have to worry about byte-ordering, of course, but that's a different issue.)
As others have pointed out, you need to cast to a pointer to int. You also need to make sure you take alignment of the pointer in consideration: on many architectures, an int needs to start at a memory location that is divisible by sizeof(int), and if you try to access an unaligned int, you get a SIGBUS. On other architectures, it works, but slowly. On yet others, it works quickly.
A portable way of doing this might be:
int x = 250;
memcpy(mem + offset, &x, sizeof(x));
Using unions may make this easier, though, so +1 to JamieH.
Cast pointer to int, not unsigned char again!
int * memp = (int *)mem;
* memp = 250; //An example of size I want to assign.