I'm having this struct here (bool is an unsigned char)
typedef struct
{
bool a;
int b;
int c;
} test_t;
If I look at the memory of a variable using this type declared on stack I see that the aligned of member b is not correct and is not on 4 bytes boundary.
EDIT: This is caused by the inclusion of FreeRTOS_IP.h which includes a ported file for ti CCS pack_struct_start.h / pack_struct_end.h. The pack_struct_end.h file is not working correctly causing all remaining structs to stay packed.
Why the TI compiler packs my structure by default and then access the memory like they are not packed?
The compiler does what you tell it to do. In your case, you don't tell it very much, so the compiler does what others tell it to instead.
Specifically, your structure is probably in a header file that gets included in a random string of #includes in different files, but since you don't specify packing parameters (which by the way are #pragma pack, not the all-or-nothing gcc-specific attribute) it uses whatever leaked from the other header files you included.
Here's the lesson to learn from this: if you care at all, at any point, about the memory layout of your structure (ie, if you use it to lazily access memory from files or network data), always specify a packing attribute on your structure. No exceptions, ever. Also clean up after yourself (#pragma pack(pop)).
Related
I have MCU code (for AVR) with set of different drivers. Driver in use is selected at startup time (in init code) and only one of the drivers will be used at a time. Therefore, i can alias all of RAM segments into the same location. That is required, as RAM is very limited and all of drivers require 2-3 KB of storage for buffers etc.
Currently i have structures typedef'ed in all header files:
driver1.h:
typedef struct {
// Driver-specific variables for internal use.
// None of those and accessed from outside C-file
int internal_variable1_for_driver1;
int internal_variable2_for_driver1;
...
} st_driver1_t;
Other drivers have similar structures
In all_drivers.h:
#include "driver1.h"
#include "driver2.h"
#include "driver3.h"
...
union {
st_driver1_t DRV1;
st_driver2_t DRV2;
st_driver3_t DRV3;
...
} DRV;
DISCLAIMER: I understand that all RAM access from modules, other than selected one, should be disabled as it will alter whole union. I have a lot of ASM code in portfolio and that's obvious for me.
Now i have some conflicts because i have to include all_drivers.h, and therefore all driverN.h files into each driver. I want to hide all those headers from all of other drivers.
Is it ok to define all driver-specific structures with __attribute__((common)) in C-files and therefore hide all of structure from headers. As i understand, linker will merge all those structures into overlapping segment and create some analog of union.
driver1.c:
struct __attribute__((common)) {
int var1;
int var2;
...
} DRV;
driver2.c:
struct __attribute__((common)) {
long OtherVar1;
int ExtraOne;
...
} DRV;
Also, should I define those structs as static or not? Will this solution be stable or is it undocumented hack?
Is it ok to define all driver-specific structures with __attribute__((common))
With GCC and compatible compilers, you can use __attribute__((__common__)) for common variables (and __attribute__((__no_common__)) if some variables should not be common, but common is the default).
GCC switched its default behaviour from -fcommon to -fno-common some time ago (in 2020 with v10), so you can also compile all compilation units that need this with -fcommon.
should I define those structs as static [...]?
No. There are some conditions that objects must meet to be coalesced as common:
The objects are global (thus also in static storage).
The objects don't have initializers.
The objects have the same (assembly) name.
The compiler must be advised to put the objects in common as discussed above.
Sometimes it's preferred that the objects don't have the same name on C/C++ level, for example that the object for driver1 is named driver1 etc. That's fine if the objects have the same name on assembly level, which can be achieved by using the same assembly name for all such objects. This is a feature provided by GCC. The following example sets the assembly name to asm_name, so in the assembly code the name asm_name will be used, but in C code you use driver1:
typedef struct { ... } driver1_t;
__attribute__((__common__))
driver1_t driver1 __asm ("asm_name");
If the ABI requires leading underscores, you should use a name that fits that ABI and use an assembly name that starts with an _ like _asm_name.
There are use cases where common is much clearer and more convenient than putting all objects into a union, so don't be discouraged by these comments.
I would like to develop a device-driver on linux(written in C) and a user-space library wrapping all functions provided by my device-driver (also written in C). Just to make it more clear, my library wil provide the following methods:
int myOpen();
void myClose();
mySetConf(MyconfStruct conf)
etc.
The function will use the file associated to my device-driver, in particular:
myOpen will call the open() of my device-driver
myClose will call the close() of my device-driver
mySetConf will call the ioctl() of my device driver and pass the myConfStruct as a parameter to configure the device-driver using ioctl().
assume myConfStruct is a simple structure containing something like this:
typedef struct {
uint16_t var1;
uint8_t var2;
} myConfStruct;
I would like the myConfStruct to be a structure shared between both my user-application (library) and my kernel-driver using a single header.
Are there any best-practice while doing this?
I would like to have the structure defined into only one file, having it defined in multiple files seems to be quite error-prone if i plan on changing it in the future, but I understood that I should not include <linux/types.h> inside my user files and I shouldn't use <stdint.h> inside my device-driver.
So another question is also, how can I define the interface between a module and the user-application so that who is implementing the application is not forced to include any linux header?
What you are creating is a character device. The kernel documentation includes a specific section, the Linux driver implementer's guide, you should also read. Specifically, the ioctl based interfaces section, which also describes some of the considerations necessary (regarding alignment and 64-bit fields).
As to header files, see KernelHeaders article at kernelnewbies.org.
I would like to have the structure defined into only one file, having it defined in multiple files seems to be quite error-prone if i plan on changing it in the future.
No. You do specify the headers in two separate files: one for use in-kernel, and the other for use by userspace.
Kernel-userspace interface should be stable. You should take care to design your data structure so that you can extend it if necessary; preferably by adding some padding reserved for future use and required to be initialized to zero, and/or a version number at the beginning of the structure. Later versions must support all previous versions of the structure as well. Even if it is only a "toy" or "experimental" device driver, it is best to learn to do it right from the get go. This stuff is much, much harder to learn to "add afterwards"; I'm talking from deep experience here.
As a character device, you should also be prepared for the driver to be compiled on other architectures besides the one you are developing on. Even byte order ("endianness") can vary, although all Linux architectures are currently either ILP32 or LP64.
Also remember that there are several hardware architectures, including x86-64, that support both 64-bit and 32-bit userspace. So, even if you believe your driver will ever be used on x86-64, you cannot really assume the userspace is 64-bit (and not 32-bit). Look at existing code to see how it is done right; I recommend using e.g. bootlin's elixir to browse the Linux kernel sources.
Kernel-side header file should use __s8, __u8, __s16, __u16, __s32, __u32, __s64, or __u64. For pointers, use __u64, and u64_to_user_ptr().
Userspace-side header file should use <stdint.h> types (int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t) and uint64_t for pointers. Use a cast via (uintptr_t) for the conversion, i.e. ptr = (void *)(uintptr_t)u64; and u64 = (uintptr_t)ptr.
Ensure all members are naturally aligned. This means that an N-bit member is preceded by k×N bits of other members, where k is either zero or a positive integer. Thus, if your structure needs an unsigned and a signed 8-bit integer (one for version, one for foo), an 16-bit signed integer (bar), and a 64-bit unsigned integer (baz), considering the version should be first, you'll probably want
struct kernel_side {
__u8 version;
__s8 foo;
__u16 bar;
__u32 padding;
__u64 baz;
};
struct userspace_side {
uint8_t version;
int8_t foo;
uint16_t bar;
uint32_t padding;
uint64_t baz;
};
You can also have character arrays and such, but do note that a single ioctl data block is limited to 8191 bytes or less in length.
If you spend some time designing your interface structures, you'll find that careful design will avoid annoying issues like compat_ support (making them just simple wrappers). Personally, I end up creating a test version with a userspace test program to see what works best, and only then decide on the data structures.
I ran into some weird issues (at compile time) with my embedded C code when I increased the size of the array from size [2] to [27] which contained structured metadata. So it is obvious that this is a nested structured style program and NOT my original work!
I remember in assembly code instructions, there used to be a JMP FAR, JMP NEAR/SHORT.. Is there any such limitations in structures (obviously there is no jump in structures). Could there be a condition that the compiler cannot assign a contiguous memory block for a structure because there is a limit to it and fails during compilation.
SOME CLUE: The fix as i recall was to move the member (array member)from original structure to another nested structure. That's the reason why i deduced that there maybe a limitation.
//STRUCTURES
typedef UINT8 P_Name_t[5];
typedef UINT8 ChipSN_t[3];
typedef struct
{
ChipSN_t ChipSN;
<other members>
} ChipIdent_t;
typedef struct Data_t
{
ChipIdent_t ReadOnlyMemID;
<other members>
} Data_t;
typedef struct
{
P_Name_t NameOfPart;
<other members>
} Log_t;
Data_t Data_Src;
typedef struct
{
P_Name_t NameOfPart;
ChipSN_t ChipSN;
}PartNum_ID_t;
typedef struct
{
PartNum_ID_t PN_ChipID[27];
<other members>;
}
According to standard, there is a limitation; but the limitation is much more than the size you mentioned here. C89 and C99 standard has the limit of 1023 structure members. See below text from the standard.
5.2.4.1 Translation limits
— 1023 members in a single structure or union
So looks like if your code is correct and limited to 1023 structure members the compiler you are using may have the limitation.
There may indeed be a limit to the number of structure elements your compiler can handle for a single structure, but as MCG quoted from the C Standard, this limit must be at least 1023 for compliant compilers and most compilers have higher limits or no practical limits at all.
I suspect the problem is elsewhere:
The Makefile for your project may have incomplete dependencies on the header file with the structure definitions: modules compiled with different definitions will be incompatible and produce undefined behavior. Remove all object files and recompile the project.
Some assembly modules may rely on the original structure layout: changing the definition will only affect the recompiled C modules.
Some library modules or OS system calls may rely on the original layout. Recompiling all libraries or even patching the OS may be required.
Some functions may rely on the original layout and use casts to access some of the structures via another type. Changing the layout may break such code, which is brittle anyway as even changing compilers may produce similar problems.
It is also possible that some of the type be multiply defined in different modules. This is poor and very error prone coding style as changing only one of the definitions will lead to undefined behavior when the object modules are linked together.
if you want to cut to the chase, please skip down to the last two paragraphs. If you're interested in my predicament and the steps I've taken to solve it, continue reading directly below.
I am currently developing portions of a C library as part of my internship. So naturally, there are some parts of code which should not be accessible to the user while others should be. I am basically developing several architecture-optimized random number generators (RNG's)(uniform, Gaussian, and exponential distributed numbers). The latter two RNG's depend on the uniform generator , which is in a different kernel (project). So, in the case that the user wants to use more than one RNG, I want to make sure I'm not duplicating code needlessly since we are constrained with memory (no point in having the same function defined multiple times at different addresses in the code segment).
Now here's where the problem arises. The convention for all other kernels in the library is that we have a two header files and two C files (one each for the natural C implementation and the optimized C version (which may use some intrinsic functions and assembly and/or have some restrictions to make it faster and better for our architecture). This is followed by another C file (a testbench) where our main function is located and it tests both implementations and compares the results. With that said, we cannot really add an additional header file for private or protected items nor can we add a global header file for all these generators.
To combat this restriction, I used extern functions and extern const int's in the C files which depend on the uniform RNG rather than #define's at the top of each C file in order to make the code more portable and easily modified in one place. This worked for the most part.
However, the tricky bit is that we are using an internal type within these kernels (which should not be seen by the user and should not be placed in the header file). Again, for portability, I would like to be able to change the definition of this typedef in one place rather than in multiple places in multiple kernels since the library may be used for another platform later on and for the algorithms to work it is critical that I use 32-bit types.
So basically I'm wondering if there's any way I can make a typedef "protected" in C. That is, I need it to be visible among all C files which need it, but invisible to the user. It can be in one of the header files, but must not be visible to the user who will be including that header file in his/her project, whatever that may be.
============================Edit================================
I should also note that the typedef I am using is an unsigned int. so
typedef unsigned int myType
No structures involved.
============================Super Edit==========================
The use of stdint.h is also forbidden :(
I am expanding on Jens Gustedt’s answer since the OP still has questions.
First, it is unclear why you have separate header files for the two implementations (“natural C” and “optimized C”). If they implement the same API, one header should serve for either.
Jens Gustedt’s recommendation is that you declare a struct foo in the header but define it only in the C source file for the implementation and not in the header. A struct declared in this way is an incomplete type, and source code that can only see the declaration, and not the definition, cannot see what is in the type. It can, however, use pointers to the type.
The declaration of an incomplete struct may be as simple as struct foo. You can also define a type, such as typedef struct foo foo; or typedef struct foo Mytype;, and you can define a type that is a pointer to the struct, such as typedef struct foo *FooPointer;. However, these are merely for convenience. They do not alter the basic notion, that there is a struct foo that API users cannot see into but that they can have pointers to.
Inside the implementation, you would fully define the struct. If you want an unsigned int in the struct, you would use:
struct foo
{
unsigned int x;
};
In general, you define the struct foo to contain whatever data you like.
Since the API user cannot define struct foo, you must provide functions to create and destroy objects of this type as necessary. Thus, you would likely have a function declared as extern struct foo *FooAlloc(some parameters);. The function creates a struct foo object (likely by calling malloc or a related function), initializes it with data from the parameters, and returns a pointer to the object (or NULL if the creation or initialization fails). You would also have a function extern void FooFree(struct foo *p); that frees a struct foo object. You might also have functions to reset, set, or alter the state of a foo object, functions to copy foo objects, and functions to report about foo objects.
Your implementations could also define some global struct foo objects that could be visible (essentially by address only) to API users. As a matter of good design, this should be done only for certain special purposes, such as to provide instances of struct foo objects with special meanings, such as a constant object with a permanent “initial state” for copying.
Your two implementations, the “natural C” and the “optimized C” implementations may have different definitions for the struct foo, provided they are not both used in a program together. (That is, each entire program is compiled with one implementation or the other, not both. If necessary, you could mangle both into a program by using a union, but it is preferable to avoid that.)
This is not a singleton approach.
Just do
typedef struct foo foo;
These are two declarations, a forward declaration of a struct and a type alias with the same name. Forward declared struct can be used to nothing else than to define pointers to them. This should give you enough abstraction and type safety.
In all your interfaces you'd have
extern void proc(foo* a);
and you'd have to provide functions
extern foo* foo_alloc(size_t n);
extern void foo_free(foo* a);
This would bind your users as well as your library to always use the same struct. Thereby the implementation of foo is completely hidden to the API users. You could even one day to decide to use something different than a struct since users should use foo without the struct keyword.
Edit: Just a typedef to some kind of integer wouldn't help you much, because these are only aliases for types. All your types aliased to unsigned could be used interchangeably. One way around this would be to encapsulate them inside a struct. This would make your internal code a bit ugly, but the generated object code should be exactly the same with a good modern compiler.
Can a typedef struct be used without knowing its type?
e.g. There is a module on another embedded microcontroller that expects a struct and the struct is sent from another board and the struct is a typedef struct. Can the expected struct be accessed. Can its data be read?
Another question that arises, is how are structs usually sent around systems and the developer using them needs to know the structs fields.
Are the modules that declared them just included and the developer needs to find out the fields?
Can structs data be accessed without knowing its fields?
If you have an incomplete struct type, you should not be accessing its data. However, you can pass around pointers to that type just fine, and code that knows the complete type of the struct can access the data the pointer points to.
If you want to manipulate the data of the struct in two different modules, you will need to have the complete type declaration in both of them. This is usually put into a header file.
mystruct.h
#ifndef _MYSTRUCT_H
#define _MYSTRUCT_H
typedef struct mystruct{
int a;
int b;
} mystruct;
#endif
foo.c
#include "mystruct.h"
int foo(mystruct m){
return m.a;
}
bar.c
#include "mystruct.h"
int bar(mystruct m){
return m.b;
}
To access any of the fields of a struct (whether or not it is typedefed) a complete declaration of the struct must be visible at the point where the code attempts to access the field. Which physical board produces the data is entirely irrelevant.
"Complete declaration" and "visible" are technical terms with precise definitions that are too lengthy to get into here. For what you're asking, this approximation should be good enough: a struct declaration is complete if and only if it has this form
struct foo {
/* list of fields */
};
And it's visible if it appears at top level, textually above the function(s) that attempt to access fields of the struct. Usually, the declaration would come from a header file, but there's no requirement that it do so (remember that #include operates on text, not on the symbol table, unlike say Java import).
By contrast, if all you have is a declaration like this
struct foo;
then the type is incomplete and the only thing you can do with the struct (to first order) is pass around pointers to it.
Can a typedef struct be used without
knowing its type?
Well, both yes and no. You only need to include a header file declaring the struct if you need to access the fields, but you don't need to include if you are just passing a pointer forward, i.e. relaying some parameter as you are are moving between the abstraction layers.
Another question that arises, is how
are structs usually sent around
systems and the developer using them
needs to know the structs fields.
When the struct has been declared and seen by the compiler, the compiler knows that struct of type X takes up so and so number of bytes in memory and how the data is ordered. If there are four 32 bits integers declared after each other, they will be aligned next to each other in memory for 128 bits or 16 bytes. The header file defines this like a contract. "If you include me, here's how many bytes I take up in memory and here are the different types that belong to me".
Are the modules that declared them
just included and the developer needs
to find out the fields?
I'm not really sure of what you mean here. The developer can also take a look at the header file (just like the compiler does) to see the SAME contract, but obviously explained through a higher abstraction layer, i.e. the human readable code. So he/she can know that the first field in the struct is called fooField. The developer then knows that he can access that field through that name or identifier, e.g.
NumberStruct someNumberStruct;
getSomeNumbers(&someNumberStruct);
int number = someNumberStruct.fooField;
Can structs data be accessed without
knowing its fields?
Here's the yes from the first question. A pointer is just pointing to some address in memory, as long as you have access to write and read that memory, you can do anything. You could in fact pass around stuff as a void* (i.e. type less pointer) and manually read bytes from that same contract, you "know" that the struct is so and so large in memory and the order of the fields, because you have taken a look at the code :) It's obviously a bit dangerous since you must be sure that the other side of that contract hasn't changed, then fun stuff could happen :) So as soon as ANYTHING in some struct has changed, you must update all code that utilize that contract without including the header file.
Hope this could shed some light onto your structs :)