When I was browsing the Linux kernel, I found a container_of macro which is defined as follows:
#define container_of(ptr, type, member) ({ \
const typeof( ((type *)0)->member ) *__mptr = (ptr); \
(type *)( (char *)__mptr - offsetof(type,member) );})
I understand what does container_of do, but what I do not understand is the last sentence, which is
(type *)( (char *)__mptr - offsetof(type,member) );})
If we use the macro as follows:
container_of(dev, struct wifi_device, dev);
The corresponding part of the last sentence would be:
(struct wifi_device *)( (char *)__mptr - offset(struct wifi_device, dev);
which looks like doing nothing.
Could anybody please fill the void here?
Your usage example container_of(dev, struct wifi_device, dev); might be a bit misleading as you are mixing two namespaces there.
While the first dev in your example refers to the name of pointer the second dev refers to the name of a structure member.
Most probably this mix up is provoking all that headache. In fact the member parameter in your quote refers to the name given to that member in the container structure.
Taking this container for example:
struct container {
int some_other_data;
int this_data;
}
And a pointer int *my_ptr to the this_data member you'd use the macro to get a pointer to struct container *my_container by using:
struct container *my_container;
my_container = container_of(my_ptr, struct container, this_data);
Taking the offset of this_data to the beginning of the struct into account is essential to getting the correct pointer location.
Effectively you just have to subtract the offset of the member this_data from your pointer my_ptr to get the correct location.
That's exactly what the last line of the macro does.
The last sentence cast:
(type *)(...)
a pointer to a given type. The pointer is calculated as offset from a given pointer dev:
( (char *)__mptr - offsetof(type,member) )
When you use the cointainer_of macro, you want to retrieve the structure that contains the pointer of a given field. For example:
struct numbers {
int one;
int two;
int three;
} n;
int *ptr = &n.two;
struct numbers *n_ptr;
n_ptr = container_of(ptr, struct numbers, two);
You have a pointer that points in the middle of a structure (and you know that is a pointer to the filed two [the field name in the structure]), but you want to retrieve the entire structure (numbers). So, you calculate the offset of the filed two in the structure:
offsetof(type,member)
and subtract this offset from the given pointer. The result is the pointer to the start of the structure. Finally, you cast this pointer to the structure type to have a valid variable.
conatainer_of() macro in Linux Kernel -
When it comes to managing several data structures in code, you'll almost always need to embed one structure into another and retrieve them at any moment without being asked questions about memory offsets or boundaries. Let's say you have a struct person, as defined here:
struct person {
int age;
int salary;
char *name;
} p;
By only having a pointer on age or salary, you can retrieve the whole structure wrapping (containing) that pointer. As the name says, the container_of macro is used to find the container of the given field of a structure. The macro is defined in include/linux/kernel.h and looks like the following:
#define container_of(ptr, type, member) ({ \
const typeof(((type *)0)->member) * __mptr = (ptr); \
(type *)((char *)__mptr - offsetof(type, member)); })
Don't be afraid of the pointers; just see them as follows:
container_of(pointer, container_type, container_field);
Here are the elements of the preceding code fragment:
pointer: This is the pointer to the field in the structure
container_type: This is the type of structure wrapping (containing) the pointer
container_field: This is the name of the field to which
pointer points inside the structure
Let's consider the following container:
struct person {
int age;
int salary;
char *name;
};
Now, let's consider one of its instances, along with a pointer to the age member:
struct person somebody;
[...]
int *age_ptr = &somebody.age;
Along with a pointer to the name member (age_ptr),you can use the container_of macro in order to get a pointer to the whole structure (container) that wraps this member by using the following:
struct person *the_person;
the_person = container_of(age_ptr, struct person, age);
container_of takes the offset of age at the beginning of the struct into account to get the correct pointer location. If you subtract the offset of the field age from the pointer age_ptr, you will get the correct location. This is what the macro's last line does:
(type *)( (char *)__mptr - offsetof(type,member) );
Applying this to a real example, gives the following:
struct family {
struct person *father;
struct person *mother;
int number_of_sons;
int family_id;
} f;
/*
* Fill and initialise f somewhere */ [...]
/*
* pointer to a field of the structure
* (could be any (non-pointer) member in the structure)
*/
int *fam_id_ptr = &f.family_id;
struct family *fam_ptr;
/* now let us retrieve back its family */
fam_ptr = container_of(fam_id_ptr, struct family, family_id);
The container_of macro is mainly used in generic containers in the kernel.
That's all about container_of macro in kernel.
It is an utilisation of a gcc extension, the statements expressions. If you see the macro as something returning a value, then the last line would be :
return (struct wifi_device *)( (char *)__mptr - offset(struct wifi_device, dev);
See the linked page for an explanation of compound statements. Here is an example :
int main(int argc, char**argv)
{
int b;
b = 5;
b = ({int a;
a = b*b;
a;});
printf("b %d\n", b);
}
The output is
b 25
Very useful link for understanding container_of macro in linux kernel.
https://linux-concepts.blogspot.com/2018/01/understanding-containerof-macro-in.html
A little real context says clearer, below use red-black tree as example, which is the
way that I understand container_of.
as Documentation/rbtree.txt states, in linux kernel code, it's not rb_node contain data
entry, rather
Data nodes in an rbtree tree are structures containing a struct
rb_node member.
struct vm_area_struct (in file include/linux/mm_types.h:284) is such a structure,
in the same
file, there is a macro rb_entry which is defined as
#define rb_entry(ptr, type, member) container_of(ptr, type, member)
clearly, rb_entry is same as container_of.
at mm/mmap.c:299 inside function definition browse_rb, there is a usage of rb_entry:
static int browse_rb(struct mm_struct *mm)
{
/* two line code not matter */
struct rb_node *nd, *pn = NULL; /*nd, first arg, i.e. ptr. */
unsigned long prev = 0, pend = 0;
for (nd = rb_first(root); nd; nd = rb_next(nd)) {
struct vm_area_struct *vma;
vma = rb_entry(nd, struct vm_area_struct, vm_rb);
/* -- usage of rb_entry (equivalent to container_of) */
/* more code not matter here */
now it is clear, in container_of(ptr, type, member),
type is the container struct, here struct vm_area_struct
member is name of a member of type instance, here vm_rb, which is of type rb_node,
ptr is a pointer pointing member of an type instance, here rb_node *nd.
what container_of do is, as in this example,
given address of obj.member (here obj.vm_rb), return the
address of obj.
since a struct is a block of contiguous memory, address of obj.vm_rb minus
offset between the struct and member will be the container's address.
include/linux/kernel.h:858 -- definition of container_of
include/linux/rbtree.h:51 -- definition of rb_entry
mm/mmap.c:299 -- usage of rb_entry
include/linux/mm_types.h:284 -- struct vm_area_struct
Documentation/rbtree.txt: -- Documentation of red-black tree
include/linux/rbtree.h:36 -- definition of struct rb_node
P.S.
Above files are in current develop version, i.e, 4.13.0-rc7.
file:k mean kth line in file.
Most Simplest Implementation of Container _of macro is below , It reduces all complex checking of type and works
#define offsetof(TYPE, MEMBER) ((size_t) &((TYPE *)0)->MEMBER)
#define container_of(ptr, type, member) ((type *)((char *)(ptr) - offsetof(type, member)))
ptr will give address of member and just subtract offset difference and you will
get start address.
Example usage
struct sample {
int mem1;
char mem2;
int mem3;
};
int main(void)
{
struct sample sample1;
printf("Address of Structure sample1 (Normal Method) = %p\n", &sample1);
printf("Address of Structure sample1 (container_of Method) = %p\n",
container_of(&sample1.mem3, struct sample, mem3));
return 0;
}
I saw an interesting design for a doubly-linked-list inside the Pintos operating system and I don't quite understand how it was able to achieve this behavior. I wrote a small adaptation to demo what I'm referring to.
#include <stdio.h>
#include <stdint.h>
#include "list.h"
struct foo {
int x;
struct list_node node;
};
int main()
{
struct list list;
list_initialize(&list);
struct foo n1 = {1, NULL};
list_push_back(&list, &n1.node);
struct list_node *e = list_get_front(&list);
struct foo *ptr = list_entry(e, struct foo, node);
printf("%d", ptr->x);
return 0;
}
In essence, list.h implements two structures; namely, list and list_node. A list can be made for any arbitrary struct so long as the struct includes a list_node field. A reference to the structure can be returned by passing a pointer to it's list_node using the following macro defined in list.h.
#define list_entry(LIST_NODE, STRUCT, MEMBER) \
((STRUCT *) ((uint8_t *) &(LIST_NODE)->next \
- offsetof (STRUCT, MEMBER.next)))
The list_node struct declares a known field name, that is used to develop the result.
The key is found in the expression:
(uint8_t *)&(LIST_NODE)->next - offsetof (STRUCT, MEMBER.next)
offsetof() gives the offset in bytes of the specified field in the specified struct.
(uint8_t *)&(LIST_NODE)->next returns a byte pointer, that points to (the beginning of) the specified member of the specified node.
When you subtract the byte offset of the field from the pointer to the field, you get a byte pointer to (the beginning of) the struct.
The macro then casts this to a pointer to the struct.
I have browsed lots of questions regarding to container_of(), but did not find a question asking how to retrieve a struct within a struct using container_of(). If this question is duplicate, please point it out. Sorry for the inconvenience.
I am currently implementing a PCI device driver remove() function combined with another kernel module, with the following setup (I eliminated some other fields to make this question easy to read):
struct my_device {
struct mutex lock;
struct my_dev {
struct cdev cdev;
} my_dev;
struct pci {
struct cdev cdev;
struct pci_dev *pdev;
} pci;
};
Here comes the question: When we have a pointer to pdev within struct pci of type struct pci_dev *, how could we get struct pci, then get struct my_device? Note that in probe(struct pci_dev *dev, const struct pci_device_id *id), I had assigned dev to a previously allocated struct my_device *dev_ptr, i.e. dev_ptr->pci.pdev = dev;
I tried the following:
struct pci *pci_t;
pci_t = container_of(ptr, struct pci, pdev)
but that did not work.
The error message is as follows:
error: call to '__compiletime_assert_243' declared with attribute error: pointer type mismatch in container_of()
Any ideas would be appreciated.
Updates:
I tried the approaches in the comments. Since the remove() function of pci driver has a parameter - struct pci_dev *dev, which represents the corresponding pci device, I did the following:
struct my_device *mydev;
mydev = container_of(&dev, struct my_device, pci.pdev);
But it did not return the pointer to the correct structure to mydev.
This is a simplified definition of the container_of() macro:
#define container_of(ptr, type, member) ((type *)((char *)(ptr) - offsetof(type, member)))
The offsetof(type, member) macro is as described in the C standard and gives the number of bytes from the start of some structure type type to the start of some member of that type member. ptr should be compatible with a pointer to the type of the member (but the simplified version above does not care). (char *)(ptr) converts the pointer to a byte pointer pointing to the member so that the pointer subtraction (char *)(ptr) - offsetof(type, member) produces a byte pointer to the start of the containing structure type. Finally, the cast (type *) converts that pointer to the proper type.
An example of usage. Given:
struct my_device {
struct mutex lock;
struct my_dev {
struct cdev cdev;
} my_dev;
struct pci {
struct cdev cdev;
struct pci_dev *pdev;
} pci;
};
and a pointer to an allocated struct my_device:
struct my_device *mydev = kzalloc(sizeof(*mydev), GFP_KERNEL);
and a pointer to the pci.pdev member:
struct pci_dev **ppdev = &mydev->pci.pdev;
then it is possible to convert that back to a pointer to the containing struct my_device using container_of():
struct my_device *myd = container_of(ppdev, struct my_device, pci.pdev);
OP has something like this:
struct my_device *mydev = kzalloc(sizeof(*mydev), GFP_KERNEL);
mydev->pci.pdev = pcidev;
and is trying to get back to the struct my_device from the pointer pcidev as follows:
struct my_device *mydev = container_of(&pcidev, struct my_device, pci.pdev); /* wrong! */
That does not work because &pcidev is the address of the pcidev variable, not the address of the original mydev->pci.pdev member.
The Linux kernel allows a PCI device driver to associate an arbitrary void * value with a PCI device. That is normally done in the PCI device driver's probe() handler, something like this:
pci_set_drvdata(pcidev, mydev);
Then until the PCI device is removed from the driver, or the driver calls pci_set_drvdata with some other value, the pointer can be retrieved by a call to pci_get_drvdata():
struct my_device *mydev = pci_get_drvdata(pcidev);
More generally, pci_set_drvdata() and pci_get_drvdata() are just wrappers around dev_set_drvdata() and dev_get_drvdata() that associate an arbitrary void * value with a struct device. The struct pci_device structure definition contains a member struct device dev;. pci_set_drvdata(pcidev, ptr) is equivalent to dev_set_drvdata(&pcidev->dev, ptr), and pci_get_drvdata(pcidev) is equivalent to dev_get_drvdata(&pcidev->dev).
Often, driver code has a pointer to a struct device that it knows is embedded within a struct pci_dev, and it can use container_of() to get a pointer to the containing struct pci_dev:
/* dptr is pointing the the struct device inside a struct pci_dev. */
static void foo(struct device *dptr)
{
struct pci_dev *pcidev = container_of(dptr, struct pci_dev, dev);
/* ... */
}
If the driver has previously used pci_set_drvdata() or dev_set_drvdata(), it can use pci_get_drvdata() or dev_get_drvdata() to retrieve the set value:
static void foo(struct device *dptr)
{
struct pci_dev *pcidev = container_of(dptr, struct pci_dev, dev);
struct my_device *mydev = pci_get_drvdata(pcidev);
struct my_device *mydev1 = dev_get_drvdata(dptr);
/* (mydev == mydev1) is true */
struct pci_dev *pcidev1 = mydev->pci.pdev;
/* (pcidev == pcidev1) is true assuming mydev->pci.pdev was previously set to pcidev */
/* ... */
}
Although it works, it is probably bad style to mix pci_set_drvdata() with a previous call to dev_get_drvdata() or to mix pci_get_drvdata() with a previous call to dev_set_drvdata().
in linux/include/linux/list.h I found:
/**
* list_entry - get the struct for this entry
* #ptr: the &struct list_head pointer.
* #type: the type of the struct this is embedded in.
* #member: the name of the list_head within the struct.
*/
#define list_entry(ptr, type, member) \
container_of(ptr, type, member)
What do they mean by "get the struct for this entry" can I see a usage example to understand in a better way?
This is a great example of a kind of Polymorphism in C. To borrow terminology from C++, the list_entry() macro allows you to downcast from a list_head type to any type that contains it.
Have a look at kthread.c for a simple fundamental example:
kernel/kthread.c:
struct kthread_create_info
{
/* Information passed to kthread() from kthreadd. */
int (*threadfn)(void *data);
void *data;
int node;
/* Result passed back to kthread_create() from kthreadd. */
struct task_struct *result;
struct completion *done;
struct list_head list;
};
...
int kthreadd(void *unused)
{
...
while (!list_empty(&kthread_create_list)) {
struct kthread_create_info *create;
create = list_entry(kthread_create_list.next,
struct kthread_create_info, list);
...
create_kthread(create);
By including a list_head object in the kthread_create_info struct, you can say that kthread_create_info "derives" from list_head. This allows kthread_create_info objects to be used as nodes in a list, meaning you can pass them to any of the functions declared in list.h by simply dereferencing the list member of the struct. The list_entry macro then gives you the mapping from a base class pointer to its derived start address.
In other words, given a list_head object that you know is contained within an outer kthread_create_info struct, you can recover a pointer to the kthread_create_info container.
This is an extremely common pattern in C programming, where object oriented constructs are desired, but a C++ compiler isn't available.
I was looking at Glibc codes. Some codes of glibc's queue caught my attention. I couldn't give a meaning to this struct definition. This struct doesn't have a name. Why? How does it work?
#define LIST_ENTRY(type) \
struct { \
struct type *le_next; /* next element */ \
struct type **le_prev; /* address of previous next element */ \
}
Source
That is actually a preprocessor macro, that could be expanded (most probably with trailing name) somewhere else.
In the comments at the start of that header file there is a reference to queue(3) man page that contains more details on that and other macros:
The macro LIST_ENTRY declares a structure that connects the elements
in the list.
And an example of use:
LIST_HEAD(listhead, entry) head = LIST_HEAD_INITIALIZER(head);
struct listhead *headp; /* List head. */
struct entry {
...
LIST_ENTRY(entry) entries; /* List. */
...
}
*n1, *n2, *n3, *np, *np_temp;
LIST_INIT(&head); /* Initialize the list. */
n1 = malloc(sizeof(struct entry)); /* Insert at the head. */
LIST_INSERT_HEAD(&head, n1, entries);
Being this C code (not C++), and C lacks templates, this preprocessor macro can be used to "simulate" templates (note the type parameter).
It's a macro that is used to declare a struct type, with next and prev pointers to instances of a second struct type. That second type can be a parent type, so you can make a "linkable struct" like this:
struct foo {
LIST_ENTRY(foo) list;
int value;
};
This creates a struct foo containing a member called list which in turn is the structure in the question, with the pointers pointing at struct foo.
We can now create a little linked list of struct foos like so:
struct foo fa, fb;
fa.value = 47;
fa.list.le_next = &fb;
fa.list.le_prev = NULL;
fb.value = 11;
fb.list.le_next = NULL;
fb.list.le_prev = &fa.list.le_next;
I'm not 100% sure about the last line, but I think it kind of makes sense.