Portable C Implementation of Relative Pointers - c

I define relative pointer to mean what Ginger Bill describes as Self-Relative Pointers:
... define the base [to which an offset will be applied] to be the memory address of the offset itself
For example, consider this struct:
struct house {
int32_t weight;
}
struct person {
int32_t age;
struct house* residence;
}
int32_t getPersonsHousesWeight(struct person* p) {
return p->residence->weight;
}
The relative-pointer implementation of the same thing in C that I think might work is:
struct house { ... } // same as before
struct person {
int32_t age;
int64_t residence; // an offset from the person's address in memory
}
int32_t getPersonsHousesWeight(struct person* p) {
return ((struct residence*)((char*)p + (p->residence)))->weight;
}
Assuming that alignment of everything is good (all 8 bytes), is this free of undefined behavior?
EDIT
#tstanisl has provided an excellent answer (which I've accepted) that thoroughly explains UB in the context of stack allocations. I am curious how allocation into a large slab of contiguous heap would impact this analysis. For example:
int foo(void) {
char* base = mmap(NULL,4096,PROT_WRITE | PROT_READ,-1,MAP_PRIVATE | MAP_ANONYMOUS);
// Omitting mmap error checking
struct person* myPerson = (struct person*)(base + 128);
struct house* myHouse = (struct house*)(base + 256);
int32_t delta = (char*)myHouse - (char*)myPerson;
// Does the computation of delta invoke UB?
}

Usually it is going to be UB.
The first case is when person and house belong to separate object.
In such a case it will be UB because the pointer arithmetics is performed outside of the object.
int foo(void) {
struct person p;
struct house h;
p.residence = (char*)&h - (char*)&p; // already UB
getPersonsHousesWeight(&p); // UB again
}
In practice it means that the compiler is not obligated to notice that objects accessed from a pointers constructed from &p can alias with object h because p and h are separete memory regions (aka objects).
When both objects are placed inside a larger object then the situation is a bit better. Though it still would be technical UB.
int foo(void) {
struct ph {
struct person p;
struct house h;
} ph;
ph.p.residence = (char*)&ph.h - (char*)&ph.p; // still UB
getPersonsHousesWeight(&ph.p); // UB again
}
It UB because pointer arithmetic is done outside the member object.
(char*)&ph.h - 1 is a pointer outside of ph.h.
Note, that this code will likely work pretty much everywhere.
Otherwise, heavily used container_of-like macros would not work breaking a lot of existing code including the Linux kernel.
To avoid UB the pointer must be constructed in a special way to avoid moving outside of the originating object.
Rather using &ph.h one should use (char*)&ph + offsetof(struct ph, h).
Similarly &ph.p should be replaced with (char*)&ph + offsetof(struct ph, p).
Now this code should be portable:
int foo(void) {
struct ph {
struct person p;
struct house h;
} ph;
struct person *p_ptr = (struct person*)((char*)&ph + offsetof(struct ph, p));
struct house *h_ptr = (struct house*) ((char*)&ph + offsetof(struct ph, h));
ph.p.residence = (char*)h_ptr - (char*)p_ptr;
getPersonsHousesWeight(p_ptr);
}
Though it is very obscure.
The interesting discussion on this topic can be found at link

Related

How to correctly downcast in C?

I'm currently limited to coding in C and I want to do C object oriented programming.
One thing that comes to mind is how to correctly downcast a type in C without violating strict aliasing.
Imagine I have an animal struct with a vtable (meaning a struct of function pointers) and a dog like this:
typedef void (*sound_func)(const animal_t *animal);
struct animal_vtable {
sound_func sound;
};
typedef struct animal_vtable animal_vtable_t;
typedef struct animal {
animal_vtable_t * vtable;
int size;
} animal_t;
typedef struct dog {
animal_t animal;
} dog_t;
There will be cases when I want to know whether my animal is a dog, this is how I currently think of making an animal instance a dog, but I'm unsure if this will trigger undefined behavior or not.
dog_t *to_dog(animal_t *a) {
if (a->vtable != &dog_table) {
return NULL;
}
size_t offset = offsetof(dog_t, animal);
uintptr_t animal_offset = (uintptr_t) a;
return (dog_t *) (animal_offset - offset);
}
The key part here is that both the memory of dog_t * and animal_t * are on the same memory location for obvious reasons, but will this be a problem for optimizers? Currently I have -fno-strict-aliasing
enabled and thus I know it works, but is it safe to turn that off?
Below is the full working example which does not trigger errors when compiled with address and unefined behavior sanitizers.
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
/*
* Animal section
*/
struct animal_vtable;
typedef struct animal_vtable animal_vtable_t;
typedef struct animal {
animal_vtable_t * vtable;
int size;
} animal_t;
typedef void (*sound_func)(const animal_t *animal);
struct animal_vtable {
sound_func sound;
};
void animal_sound(const animal_t* animal) {
animal->vtable->sound(animal);
}
int animal_size(const animal_t* animal) {
return animal->size;
}
/*
* dog section
*/
void dog_bark(const animal_t *animal);
static animal_vtable_t dog_table = {
.sound = dog_bark
};
typedef struct dog {
animal_t animal;
} dog_t;
dog_t* make_dog(int size) {
dog_t* dog = malloc(sizeof(dog_t));
if (dog == NULL) {
return dog;
}
dog->animal = (animal_t) { .vtable = &dog_table, .size = size };
return dog;
}
void dog_bark(const animal_t *animal) {
printf("wuff!\n");
}
dog_t *to_dog(animal_t *a) {
if (a->vtable != &dog_table) {
return NULL;
}
size_t offset = offsetof(dog_t, animal);
uintptr_t animal_offset = (uintptr_t) a;
return (dog_t *) animal_offset - offset;
}
/*
* main tests
*/
int main(int argc, char** argv) {
dog_t *dog = make_dog(10);
if (dog == NULL) {
exit(-1);
}
animal_t *animal = &(dog->animal);
animal_sound(animal);
dog_t *d2 = to_dog(animal);
printf("dog addr: %p, d2 addr: %p\n", dog, d2);
printf("dog size: %d\n", animal_size(&d2->animal));
printf("dog size: %d\n", animal_size(&dog->animal));
free(dog);
}
I'm unsure if this will trigger undefined behavior or not.
dog_t *to_dog(animal_t *a) {
if (a->vtable != &dog_table) {
return NULL;
}
size_t offset = offsetof(dog_t, animal);
uintptr_t animal_offset = (uintptr_t) a;
return (dog_t *) animal_offset - offset;
}
The expression (dog_t *) animal_offset - offset does not mean what you think it means. It is equivalent to ((dog_t *) animal_offset) - offset, whereas what you appear to want is (dog_t *) (animal_offset - offset) (and these are different).
But more generally, you are making it harder than it needs to be. Supposing that you implement inheritance as you seem inclined to do, by making the first member of the child type an instance of the parent type, you can perform the kind of pointer conversion you demonstrate via a simple cast: (dog_t *) a. The language specification guarantees that this is valid under the conditions described, supposing that a is in fact a pointer to the animal member of a dog_t. This is specified in C17, paragraph 6.7.2.1/15 (emphasis added):
Within a structure object, the non-bit-field members and the units in
which bit-fields reside have addresses that increase in the order in
which they are declared. A pointer to a structure object, suitably
converted, points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa.
There may be unnamed padding within a structure object, but not at its
beginning.
Substantially the same wording appears in earlier versions of the standard, too.
As for
will this be a problem for optimizers? Currently I have
-fno-strict-aliasing enabled and thus I know it works, but is it safe to turn that off?
It should not be a problem for optimizers, provided that the definition of dog_t is visible in the translation unit. In that case, optimizers that are not deeply broken will know that pointers to dog_t and pointers to animal_t can alias each other.
However, the definition of dog_t being visible is a requirement for use of offsetof, but not a requirement for the pointer cast, so that may be something to watch out for. Also, it's not just this code where you need to watch out for aliasing issues. For safety relative to strict aliasing, every function that accesses pointers to both types will need to have the definition of dog_t visible.

Pack two objects using alignof

Is it conforming with the standard to pack two objects using the align of the second object to get the final size?
I'm using this approach for a doubly linked list, but extracting the relevant part:
#include <stdio.h>
#include <stdlib.h>
#include <stdalign.h>
struct node
{
struct node *prev;
struct node *next;
};
#define get_node(data, szof) ((struct node *)(((char *)data) + szof))
int main(void)
{
size_t align = alignof(struct node);
double *data;
// Round size up to nearest multiple of alignof(struct node)
size_t szof = (sizeof(*data) + (align - 1)) / align * align;
// Pack `data` + `struct node`
data = malloc(szof + sizeof(struct node));
// Get node using a generic pointer to calculate the offset
struct node *node = get_node(data, szof);
*data = 3.14;
node->prev = NULL;
node->next = NULL;
printf("%f\n", *data);
free(data);
return 0;
}
Where data can be a pointer to any primitive or composite type.
Is it conforming with the standard to pack two objects using the align of the second object to get the final size?
Sure, the code presented is valid.
There's nothing really to write here as it's harder to prove, rather then disprove something. The pointer values are properly aligned for the referenced types, there are no uninitialized memory accesses. If you remember about alignment yourself, then you can write whole programs without ever using struct.
In real code, I advise to make a structure and let the compiler figure it out[1]. We have offsetof.
struct double_and_node {
double data;
struct node node;
};
void *pnt = malloc(sizeof(double_and_node));
double *data = (struct node*)((char*)pnt + offsetof(struct double_and_node, data));
struct node *node = (struct node*)((char*)pnt + offsetof(struct double_and_node, data));
I guess you could research container_of and see C11 6.3.2.3p7.
[1] but really, if so, just use the structure anyway...:
struct double_and_node *pnt = malloc(sizeof(double_and_node));
double *data = &pnt->data;
struct node *node = &pnt->node;
Well, this is complicated. The ((char *)data) + szof line is arguably invoking undefined behavior depending on alignof(struct node) vs sizeof(double), but it isn't very obvious.
First of all, lets assume that double* data is actually pointing at a double. We would then be allowed to inspect this object through a character type pointer, as per 6.3.2.3/7:
When a pointer to an object is converted to a pointer to a character type,
the result points to the lowest addressed byte of the object. Successive increments of the
result, up to the size of the object, yield pointers to the remaining bytes of the object.
So we may do ((char *)data) + szof while we stick inside the actual double. Otherwise, if we go out of bounds of that double, the above quoted special rule doesn't apply.
Rather, we are supposedly left to the rules of pointer arithmetic, specified by the additive operators. Although these rules expect you to be using the pointed-at type double* and not a char*. Those rules don't really specify what happens when you inspect a double through a char* and go beyond sizeof(double) bytes.
So the ((char *)data) + szof going beyond sizeof(double) is questionable - I think it is undefined behavior no matter how you put it.
Then there's another aspect here... what if the char pointer is pointing at something without a type? The C standard doesn't specify what will happen then. And this is actually what the code is doing.
Because as it happens, data = malloc(szof + sizeof(struct node)); allocates a raw segment with no declared nor "effective type". The 6.5 rules then states that
If a value is stored into an object having no declared type through an
lvalue having a type that is not a character type, then the type of the lvalue becomes the
effective type of the object for that access and for subsequent accesses that do not modify
the stored value
And you don't lvalue access the actual memory until *data = 3.14;, in which case the memory gets the effective type double. This happens after the pointer arithmetic.
As an extended idea of previous answers, something generic could be done with macros using typeof() and offsetof() defining/using a structure defined on the fly to concatenate a data type with the node structure:
#include <stdio.h>
#include <stddef.h>
struct node
{
struct node *prev;
struct node *next;
};
#define LINKED_TYPE_SIZE(data) \
sizeof(struct { typeof(data) f; struct node node; })
#define LINKED_TYPE_NODE(datap) \
(struct node *)((char *)(datap) + offsetof(struct { typeof(*(datap)) f; struct node node; }, node))
int main(void)
{
double v1;
printf("size of linked double = %zu\n", LINKED_TYPE_SIZE(v1));
printf("%p, %p\n", &v1, LINKED_TYPE_NODE(&v1));
int v2;
printf("size of linked int = %zu\n", LINKED_TYPE_SIZE(v2));
printf("%p, %p\n", &v2, LINKED_TYPE_NODE(&v2));
short int v3;
printf("size of linked short int = %zu\n", LINKED_TYPE_SIZE(v3));
printf("%p, %p\n", &v3, LINKED_TYPE_NODE(&v3));
struct foo {
int f1;
char f2;
int f3;
} foo_struct;
printf("size of linked foo = %zu\n", LINKED_TYPE_SIZE(foo_struct));
printf("%p, %p\n", &foo_struct, LINKED_TYPE_NODE(&foo_struct));
return 0;
}
The execution of the preceding gives the following on a x86_64 Linux desktop:
$ gcc try.c -o try
$ ./try
size of linked double = 24
0x7ffdfbdf50f8, 0x7ffdfbdf5100
size of linked int = 24
0x7ffdfbdf50f4, 0x7ffdfbdf50fc
size of linked short int = 24
0x7ffdfbdf50f2, 0x7ffdfbdf50fa
size of linked foo = 32
0x7ffdfbdf5100, 0x7ffdfbdf5110
N.B.:
As typeof() is a non standard function, it is also possible to get rid of it by passing explicitly the type of the data as a parameter to the macros:
#define LINKED_TYPE_SIZE(type) \
sizeof(struct { type f; struct node node; })
#define LINKED_TYPE_NODE(type, datap) \
(struct node *)((char *)(datap) + offsetof(struct { type f; struct node node; }, node))

Type punning and malloc'ed memory

I originally asked this question: Type Punning with Unions and Heap
And not wanting the question to keep evolving to the point that anyone reading in the future had no idea what the original question was, I have a spin off question.
After reading this site:
https://kristerw.blogspot.com/2016/05/type-based-aliasing-in-c.html
Near the bottom it talks about malloc'd memory. Is it safe to say that casting from one pointer type to another pointer type is safe when memory is on the heap?
Example:
#include <stdio.h>
#include <stdlib.h>
struct test1
{
int a;
char b;
};
struct test2
{
int c;
char d;
};
void printer(const struct test2* value);
int main()
{
struct test1* aQuickTest = malloc(sizeof(struct test1));
aQuickTest->a = 42;
aQuickTest->b = 'a';
printer((struct test2*)aQuickTest); //safe because memory was malloc'd???
return 0;
}
void printer(const struct test2* value)
{
printf("Int: %i Char: %c",value->c, value->d);
}
And guessing it might not be safe. What would be the proper way to do this with memcpy? I will attempt to write an example with a function of what might hopefully work?
struct test2* converter(struct test1* original);
int main()
{
struct test1* aQuickTest = malloc(sizeof(struct test1));
aQuickTest->a = 42;
aQuickTest->b = 'a';
struct test2* newStruct = converter(aQuickTest);
printer(newStruct);
return 0;
}
struct test2* converter(struct test1* original)
{
struct test2* temp;
memcpy(&temp, &original, sizeof(struct test2));
return temp;
}
void *pnt = malloc(sizeof(struct test1));
What type has the memory behind pnt pointer? No type. It is uninitialized (it's value is "indeterminate"). There is just "memory".
Then you do:
struct test1* aQuickTest = malloc(sizeof(struct test1));
You only cast the pointer. Nothing happens here. No assembly is generated. Reading uninitialized memory is undefined behavior tho, so you can't read from aQuickTest->a (yet). But you can assign:
aQuickTest->a = 1;
This writes to an object struct test1 in the memory. This is assignment. You can now read aQuickTest->a, ie. print it.
But the following
printf("%d", ((struct test2*)aQuickTest)->a);
is undefined behavior (although it will/should work). You access the underlying object (ie. struct test1) using a not matching pointer type struct test2*. This is called "strict alias violation". Dereferencing an object (ie. doing -> or *) using a handle of not compatible type results in undefined behavior. It does not matter that struct test1 and struct test2 "look the same". They are different type. The rule is in C11 standard 6.5p7.
In the first code snipped undefined behavior happens on inside printf("Int: %i Char: %c",value->c. The access value-> accesses the underlying memory using incompatible handle.
In the second code snipped the variable temp is only a pointer. Also original is a pointer. Doing memcpy(&temp, &original, sizeof(struct test2)); is invalid, because &temp writes into the temp pointer and &original writes into the original pointer. No to the memory behind pointers. As you write out of bounds into&temppointer and read of bounds from&originalpointer (because most probablysizeof(temp) < sizeof(struct test2)andsizeof(original) < sizeof(struct test2)`), undefined behavior happens.
Anyway even if it were:
struct test1* original = &(some valid struct test1 object).
struct test2 temp;
memcpy(&temp, original, sizeof(struct test2));
printf("%d", temp.a); // undefined behavior
accessing the memory behind temp variable is still invalid. As the original didn't had struct test2 object, it is still invalid. memcpy doesn't change the type of the object in memory.

If only using the first element, do I have to allocate mem for the whole struct?

I have a structure where the first element is tested and dependent on its value the rest of the structure will or will not be read. In the cases where the first element's value dictates that the rest of the structure will not be read, do I have to allocate enough memory for the entire structure or just the first element?
struct element
{
int x;
int y;
};
int foo(struct element* e)
{
if(e->x > 3)
return e->y;
return e->x;
}
in main:
int i = 0;
int z = foo((struct element*)&i);
I assume that if only allocating for the first element is valid, then I will have to be wary of anything that may attempt to copy the structure. i.e. passing the struct to a function.
don't force your information into structs where it's not needed: don't use the struct as the parameter of your function.
either pass the member of your struct to the function or use inheritance:
typedef struct {
int foo;
} BaseA;
typedef struct {
int bar;
} BaseB;
typedef struct {
BaseA a;
BaseB b;
} Derived;
void foo(BaseB* info) { ... }
...
Derived d;
foo(&d.b);
BaseB b;
foo(&b);
if you're just curious (and seriously don't use this): you may.
typedef struct {
int foo, goo, hoo, joo;
} A;
typedef struct {
int unused, goo;
} B;
int foo(A* a) { return a->goo; }
...
B b;
int goo = foo((A*)&b);
In general you'll have to allocate a block of memory at least as many bytes as are required to fully read the accessed member with the largest offset in your structure. In addition when writing to this block you have to make sure to use the same member offsets as in the original structure.
The point being, a structure is only a block of memory with different areas assigned different interpretations (int, char, other structs etc...) and accessing a member of a struct (after reordering and alignment) boils down to simply reading from or writing to a bit of memory.
I do not think the code as given is legitimate. To understand why, consider:
struct CHAR_AND_INT { unsigned char c; int i; }
CHAR_AND_INT *p;
A compiler would be entitled to assume that p->c will be word-aligned and have whatever padding would be necessary for p->i to also be word-aligned. On some processors, writing a byte may be slower than writing a word. For example, a byte-store instruction may require the processor to read a word from memory, update one byte within it, and write the whole thing back, while a word-store instruction could simply store the new data without having to read anything first. A compiler that knew that p->c would be word-aligned and padded could implement p->c = 12; by using a word store to write the value 12. Such behavior wouldn't yield desired results, however, if the byte following p->c wasn't padding but instead held useful data.
While I would not expect a compiler to impose "special" alignment or padding requirements on any part of the structure shown in the original question (beyond those which apply to int) I don't think anything in the standard would forbid a compiler from doing so.
You need to only check that the structure itself is allocated; not the members (in that case at least)
int foo(struct element* e)
{
if ( e != 0) // check that the e pointer is valid
{
if(e->x != 0) // here you only check to see if x is different than zero (values, not pointers)
return e->y;
}
return 0;
}
In you edited change, I think this is poor coding
int i = 0;
int z = foo((struct element*)&i);
In that case, i will be allocation on the stack, so its address is valid; and will be valid in foo; but since you cast it into something different, the members will be garbage (at best)
Why do you want to cast an int into a structure?
What is your intent?

C inheritance through type punning, without containment?

I'm in a position where I need to get some object oriented features working in C, in particular inheritance. Luckily there are some good references on stack overflow, notably this Semi-inheritance in C: How does this snippet work? and this Object-orientation in C. The the idea is to contain an instance of the base class within the derived class and typecast it, like so:
struct base {
int x;
int y;
};
struct derived {
struct base super;
int z;
};
struct derived d;
d.super.x = 1;
d.super.y = 2;
d.z = 3;
struct base b = (struct base *)&d;
This is great, but it becomes cumbersome with deep inheritance trees - I'll have chains of about 5-6 "classes" and I'd really rather not type derived.super.super.super.super.super.super all the time. What I was hoping was that I could typecast to a struct of the first n elements, like this:
struct base {
int x;
int y;
};
struct derived {
int x;
int y;
int z;
};
struct derived d;
d.x = 1;
d.y = 2;
d.z = 3;
struct base b = (struct base *)&d;
I've tested this on the C compiler that comes with Visual Studio 2012 and it works, but I have no idea if the C standard actually guarantees it. Is there anyone that might know for sure if this is ok? I don't want to write mountains of code only to discover it's broken at such a fundamental level.
What you describe here is a construct that was fully portable and would have been essentially guaranteed to work by the design of the language, except that the authors of the Standard didn't think it was necessary to explicitly mandate that compilers support things that should obviously work. C89 specified the Common Initial Sequence rule for unions, rather than pointers to structures, because given:
struct s1 {int x; int y; ... other stuff... };
struct s2 {int x; int y; ... other stuff... };
union u { struct s1 v1; struct s2 v2; };
code which received a struct s1* to an outside object that was either
a union u* or a malloc'ed object could legally cast it to a union u*
if it was aligned for that type, and it could legally cast the resulting
pointer to struct s2*, and the effect of using accessing either struct s1* or struct s2* would have to be the same as accessing the union via either the v1 or v2 member. Consequently, the only way for a compiler to make all of the indicated rules work would be to say that converting a pointer of one structure type into a pointer of another type and using that pointer to inspect members of the Common Initial Sequence would work.
Unfortunately, compiler writers have said that the CIS rule is only applicable in cases where the underlying object has a union type, notwithstanding the fact that such a thing represents a very rare usage case (compared with situations where the union type exists for the purpose of letting the compiler know that pointers to the structures should be treated interchangeably for purposes of inspecting the CIS), and further since it would be rare for code to receive a struct s1* or struct s2* that identifies an object within a union u, they think they should be allowed to ignore that possibility. Thus, even if the above declarations are visible, gcc will assume that a struct s1* will never be used to access members of the CIS from a struct s2*.
By using pointers you can always create references to base classes at any level in the hierarchy. And if you use some kind of description of the inheritance structure, you can generate both the "class definitions" and factory functions needed as a build step.
#include <stdio.h>
#include <stdlib.h>
struct foo_class {
int a;
int b;
};
struct bar_class {
struct foo_class foo;
struct foo_class* base;
int c;
int d;
};
struct gazonk_class {
struct bar_class bar;
struct bar_class* base;
struct foo_class* Foo;
int e;
int f;
};
struct gazonk_class* gazonk_factory() {
struct gazonk_class* new_instance = malloc(sizeof(struct gazonk_class));
new_instance->bar.base = &new_instance->bar.foo;
new_instance->base = &new_instance->bar;
new_instance->Foo = &new_instance->bar.foo;
return new_instance;
}
int main(int argc, char* argv[]) {
struct gazonk_class* object = gazonk_factory();
object->Foo->a = 1;
object->Foo->b = 2;
object->base->c = 3;
object->base->d = 4;
object->e = 5;
object->f = 6;
fprintf(stdout, "%d %d %d %d %d %d\n",
object->base->base->a,
object->base->base->b,
object->base->c,
object->base->d,
object->e,
object->f);
return 0;
}
In this example you can either use base pointers to work your way back or directly reference a base class.
The address of a struct is the address of its first element, guaranteed.

Resources