I would like to develop a device-driver on linux(written in C) and a user-space library wrapping all functions provided by my device-driver (also written in C). Just to make it more clear, my library wil provide the following methods:
int myOpen();
void myClose();
mySetConf(MyconfStruct conf)
etc.
The function will use the file associated to my device-driver, in particular:
myOpen will call the open() of my device-driver
myClose will call the close() of my device-driver
mySetConf will call the ioctl() of my device driver and pass the myConfStruct as a parameter to configure the device-driver using ioctl().
assume myConfStruct is a simple structure containing something like this:
typedef struct {
uint16_t var1;
uint8_t var2;
} myConfStruct;
I would like the myConfStruct to be a structure shared between both my user-application (library) and my kernel-driver using a single header.
Are there any best-practice while doing this?
I would like to have the structure defined into only one file, having it defined in multiple files seems to be quite error-prone if i plan on changing it in the future, but I understood that I should not include <linux/types.h> inside my user files and I shouldn't use <stdint.h> inside my device-driver.
So another question is also, how can I define the interface between a module and the user-application so that who is implementing the application is not forced to include any linux header?
What you are creating is a character device. The kernel documentation includes a specific section, the Linux driver implementer's guide, you should also read. Specifically, the ioctl based interfaces section, which also describes some of the considerations necessary (regarding alignment and 64-bit fields).
As to header files, see KernelHeaders article at kernelnewbies.org.
I would like to have the structure defined into only one file, having it defined in multiple files seems to be quite error-prone if i plan on changing it in the future.
No. You do specify the headers in two separate files: one for use in-kernel, and the other for use by userspace.
Kernel-userspace interface should be stable. You should take care to design your data structure so that you can extend it if necessary; preferably by adding some padding reserved for future use and required to be initialized to zero, and/or a version number at the beginning of the structure. Later versions must support all previous versions of the structure as well. Even if it is only a "toy" or "experimental" device driver, it is best to learn to do it right from the get go. This stuff is much, much harder to learn to "add afterwards"; I'm talking from deep experience here.
As a character device, you should also be prepared for the driver to be compiled on other architectures besides the one you are developing on. Even byte order ("endianness") can vary, although all Linux architectures are currently either ILP32 or LP64.
Also remember that there are several hardware architectures, including x86-64, that support both 64-bit and 32-bit userspace. So, even if you believe your driver will ever be used on x86-64, you cannot really assume the userspace is 64-bit (and not 32-bit). Look at existing code to see how it is done right; I recommend using e.g. bootlin's elixir to browse the Linux kernel sources.
Kernel-side header file should use __s8, __u8, __s16, __u16, __s32, __u32, __s64, or __u64. For pointers, use __u64, and u64_to_user_ptr().
Userspace-side header file should use <stdint.h> types (int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t) and uint64_t for pointers. Use a cast via (uintptr_t) for the conversion, i.e. ptr = (void *)(uintptr_t)u64; and u64 = (uintptr_t)ptr.
Ensure all members are naturally aligned. This means that an N-bit member is preceded by k×N bits of other members, where k is either zero or a positive integer. Thus, if your structure needs an unsigned and a signed 8-bit integer (one for version, one for foo), an 16-bit signed integer (bar), and a 64-bit unsigned integer (baz), considering the version should be first, you'll probably want
struct kernel_side {
__u8 version;
__s8 foo;
__u16 bar;
__u32 padding;
__u64 baz;
};
struct userspace_side {
uint8_t version;
int8_t foo;
uint16_t bar;
uint32_t padding;
uint64_t baz;
};
You can also have character arrays and such, but do note that a single ioctl data block is limited to 8191 bytes or less in length.
If you spend some time designing your interface structures, you'll find that careful design will avoid annoying issues like compat_ support (making them just simple wrappers). Personally, I end up creating a test version with a userspace test program to see what works best, and only then decide on the data structures.
Related
Given a CPU architecture, is the exact binary form of a struct determined exactly?
For example, struct stat64 is used by glibc and the Linux kernel. I see glibc define it in sysdeps/unix/sysv/linux/x86/bits/stat.h as:
struct stat64 {
__dev_t st_dev; /* Device. */
# ifdef __x86_64__
__ino64_t st_ino; /* File serial number. */
__nlink_t st_nlink; /* Link count. */
/* ... et cetera ... */
}
My kernel was compiled already. Now when I compile new code using this definition, they have binary compatibility. Where is this guaranteed? The only guarantees I know of are:
The first element has offset 0
Elements declared later have higher offsets
So if the kernel code declares struct stat64 in the exact same way (in the C code), then I know that the binary form has:
st_dev # offset 0
st_ino # offset at least sizeof(__dev_t)
But I'm not currently aware of any way to determine the offset of st_ino. Kernighan & Ritchie give the simple example
struct X {
char c;
int i;
}
where on my x86-64 machine, offsetof(struct X, i) == 4. Perhaps there are some general alignment rules that determine the exact binary form of a struct for each CPU architecture?
Given a CPU architecture, is the exact binary form of a struct determined exactly?
No, the representation or layout (“binary form”) of a structure is ultimately determined by the C implementation, not by the CPU architecture. Most C implementations intended for normal purposes follow recommendations provided by the manufacturer and/or the operating system. However, there may be circumstances where, for example, a certain alignment for a particular type might give slightly better performance but is not required, and so one C implementation might choose to require that alignment while another does not, and this can result in different structure layout.
In addition, a C implementation might be designed for special purposes, such as providing compatibility with legacy code, in which case it might choose to replicate the alignment of some old compiler for another architecture rather than to use the alignment required by the target processor.
However, let’s consider structures in separate compilations using one C implementation. Then C 2018 6.2.7 1 says:
… Moreover, two structure, union, or enumerated types declared in separate translation units are compatible if their tags and members satisfy the following requirements: If one is declared with a tag, the other shall be declared with the same tag. If both are completed anywhere within their respective translation units, then the following additional requirements apply: there shall be a one-to-one correspondence between their members such that each pair of corresponding members are declared with compatible types; if one member of the pair is declared with an alignment specifier, the other is declared with an equivalent alignment specifier; and if one member of the pair is declared with a name, the other is declared with the same name. For two structures, corresponding members shall be declared in the same order. For two structures or unions, corresponding bit-fields shall have the same widths…
Therefore, if two structures are declared identically in separate translation units, or with the minor variations permitted in that passage, then they are compatible, which effectively means they have the same layout or representation.
Technically, that passage applies only to separate translation units of the same program. The C standard defines behaviors for one program; it does not explicitly define interactions between programs (or fragments of programs, such as kernel extensions) and the operating system, although to some extent you might consider the operating system and everything running in it as one program. However, for practical purposes, it applies to everything compiled with that C implementation.
This means that as long as you use the same C implementation as the kernel is compiled with, identically declared structures will have the same representation.
Another consideration is that we might use different compilers for compiling the kernel and compiling programs. The kernel might be compiled with Clang while a user prefers to use GCC. In this case, it is a matter for the compilers to document their behaviors. The C standard does not guarantee compatibility, but the compilers can, if they choose to, perhaps by both documenting that they adhere to a particular Application Binary Interface (ABI).
Also note that a “C implementation” as discussed above is not just a particular compiler but a particular compiler with particular switches. Various switches may change how a compiler behaves in ways that cause to be effectively a different C implementation, such as switches to conform to one version of the C standard or another, switches affecting whether structures are packed, switches affecting sizes of integer types, and so on.
I am searching for a good way to document the following enum / typedef design in a shared library that I have inherited. There are many instances of the following pattern:
enum {
VAL1,
VAL2,
VAL3
};
typedef u_int16_t val_t;
int16_t myfunc(val_t foo);
This obviously causes problems with Doxygen's link generation.
Unfortunately, I cannot change this to the proper typedef enum { ... } val_t; structure that one would expect as this library runs on many different embedded systems ranging from native 16bit to 64bit. Per the C standard, an enum is guaranteed to be large enough to hold an int. Thus, changing these into proper typedefs promotes their size from sizeof(u_int16_t) (2 bytes) to sizeof(int) (usually 4 bytes but not always). Chaos ensues from storage size differences; nevermind the coordinated release nightmare required to rebuild all dependent software.
Given the above, is there any good way to mark in the comment block preceding the enum that the below is actually an alias for the typedef?
Failing that, I am looking for a solution that will concisely allow me to find these values from an arbitrary function signature. I can name the enums to something arbitrary (ex. val_enum_t) so I have considered both #see mentions as well as direct links using the # prefix. In both cases, another layer of links is generated.
My ideal solution would consist of the following:
One click from function signature to enumerated value list. Failing one click, as few clicks as possible.
Inline documentation for the enums using //!<.
If a direct association cannot be made, a single comment block to describe functionality which can be displayed on both the typedef as well as the enum itself.
Which of these items can safely be assumed to be defined in any practically-usable platform ABI?
Value of CHAR_BIT
Size, alignment requirements and object representation of:
void*, size_t, ptrdiff_t
unsigned char and signed char
intptr_t and uintptr_t
float, double and long double
short and long long
int and long (but here I expect a "no")
Pointer to an object type for which the platform ABI specifies these properties
Pointer to function whose type only involves types for which the platform ABI specifies these properties
Object representation of a null object pointer
Object representation of a null function pointer
For example, if I have a library (compiled by an unknown, but ABI-conforming compiler) which publishes this function:
void* foo(void *bar, size_t baz, void* (*qux)());
can I assume to be able to safely call it in my program regardless of the compiler I use?
Or, taken the other way round, if I am writing a library, is there a set of types such that if I limit the library's public interface to this set, it will be guaranteed to be usable on all platforms where it builds?
I don't see how you can expect any library to be universally compatible. If that were possible, there would not be so many compiled variations of libraries.
For example, you could call a 64-bit library from a 16-bit program as long as you set up the call correctly. But you would have to know you're calling a 64-bit based library.
Portability is a much-talked about goal, but few truly achieve it. After 30+ years of system-level, firmware and application programming, I think of it as more of a fantasy versus a goal. Unfortunately, hardware forces us to optimize for the hardware. Therefore, when I write a library, I use the following:
Compile for ABI
Use a pointer to a structure for input and output for all function calls:
int lib_func(struct *input, struct *output);
Where the returning int indicates errors only. I make all error codes unique. I require the user to call an init function prior to any use of the library. The user calls it as:
lib_init(sizeof(int), sizeof(char *), sizeof(long), sizeof(long long));
So that I can decide if there will be any trouble or modify any assumptions if needed. I also add a function allowing the user to learn my data sizes and alignment in addition to version numbers.
This is not to say the user or I am expected to "on-the-fly" modify code or spend lots of CPU power reworking structures. But this allows the application to make absolutely sure it's compatible with me and vice-versa.
The other option which I have employed in the past, is to simply include several entry-point functions with my library. For example:
int lib_func32();
int lib_func16();
int lib_func64();
It makes a bit of a mess for you, but you can then fix it up using the preprocessor:
#ifdef LIB_USE32
#define lib_function lib_func32
#endif
You can do the same with data structures but I'd recommend using the same size data structure regardless of CPU size -- unless performance is a top-priority. Again, back to the hardware!
The final option I explore is whether to have entry functions of all sizes and styles which convert the input to my library's expectations, as well as my library's output.
For example, your lib_func32(&input, &output) can be compiled to expect a 32-bit aligned, 32-bit pointer but it converts the 32-bit struct into your internal 64-bit struct then calls your 64 bit function. When that returns, it reformats the 64-bit struct to its 32-bit equivalent as pointed to by the caller.
int lib_func32(struct *input32, struct *output32)
{
struct input64;
struct output64;
int retval;
lib_convert32_to_64(input32, &input64);
retval = lib_func64(&input64, &output64);
lib_convert64_to_32(&output64, output32);
return(retval);
}
In summary, a totally portable solution is not viable. Even if you begin with total portability, eventually you will have to deviate. This is when things truly get messy. You break your style for deviations which then breaks your documentation and confuses users. I think it's better to just plan it from the start.
Hardware will always cause you to have deviations. Just consider how much trouble 'endianness' causes -- not to mention the number of CPU cycles which are used each day swapping byte orders.
The C standard contains an entire section in the appendix summarizing just that:
J.3 Implementation-defined behavior
A completely random subset:
The number of bits in a byte
Which of signed char and unsigned char is the same as char
The text encodings for multibyte and wide strings
Signed integer representation
The result of converting a pointer to an integer and vice versa (6.3.2.3). Note that this means any pointer, not just object pointers.
Update: To address your question about ABIs: An ABI (application binary interface) is not a standardized concept, and it isn't said anywhere that an implementation must even specify an ABI. The ingredients of an ABI are partly the implementation-defined behaviour of the language (though not all of it; e.g. signed-to-unsigned conversion is implementation defined, but not part of an ABI), and most of the implementation-defined aspects of the language are dictated by the hardware (e.g. signed integer representation, floating point representation, size of pointers).
However, more important aspects of an ABI are things like how function calls work, i.e. where the arguments are stored, who's responsible for cleaning up the memory, etc. It is crucial for two compilers to agree on those conventions in order for their code to be binarily compatible.
In practice, an ABI is usually the result of an implementation. Once the compiler is complete, it determines -- by virtue of its implementation -- an ABI. It may document this ABI, and other compilers, and future versions of the same compiler, may like to stick to those conventions. For C implementations on x86, this has worked rather well and there are only a few, usually well documented, free parameters that need to be communicated for code to be interoperable. But for other languages, most notably C++, you have a completely different picture: There is nothing coming near a standard ABI for C++ at all. Microsoft's compiler breaks the C++ ABI with every release. GCC tries hard to maintain ABI compatibility across versions and uses the published Itanium ABI (ironically for a now dead architecture). Other compilers may do their own, completely different thing. (And then you have of course issues with C++ standard library implementations, e.g. does your string contain one, two, or three pointers, and in which order?)
To summarize: many aspects of a compiler's ABI, especially pertaining to C, are dictated by the hardware architecture. Different C compilers for the same hardware ought to produce compatible binary code as long as certain aspects like function calling conventions are communicated properly. However, for higher-level languages all bets are off, and whether two different compilers can produce interoperable code has to be decided on a case-by-case basis.
If I understand your needs correctly, uint style ones are the only ones that will give you binary compatibility guarantee and of cause int, char will but others tend to differ. i.e long on Windows and Linux, Windows considers it 4byte and Linux as 8byte. If you are really dependent on ABI, you have to plan for the platforms you are going to deliver and may be use typedefs to make things standardized and readable.
gcc 4.7.2
c89
Hello,
I am using the Apache Portable Runtime and looking at their typedef's
typedef short apr_int16_t
typedef int apr_int16_t
typedef size_t apr_size_t /* This is basically the same, so what's the point */
etc.
So what is the point of all this?
When should you decided to use C's built-in standard data types or typedef's data types?
I just gave a example using the APR. However, I am also speaking generally as well. There is also the stdint.h header file that typedef's data types.
Many thanks for any suggestions,
In my opinion, it is better to have custom defined data types for native data types of the system as it helps in clearly distingushing the size of the types.
For Ex: A long may be 32 bit or 64 bit depending on the machine in which your code runs and the way it has been built. But, if your code specifically needs a 64 bit variable, then naming it as uint_64_t or something similar will always help in associating the size clearly.
In such cases, the code be written as:
#if _64BIT_
typedef long uint_64_t
#else
typedef long long uint_64_t
#endif
But as suggested by Mehrdad, don't use it "just for kicks". : )
Great question.
So what is the point of all this?
It's meant to be for abstraction, but like anything else, it is sometimes misused/overused.
Sometimes it's necessary for backwards compatibility (e.g. typedef VOID void in Windows), sometimes it's necessary for proper abstraction (e.g. typedef unsigned int size_t), and sometimes it's completely pointless logically, but makes typing easier (e.g. typedef char const *LPCSTR in Windows).
When should you decided to use C's built-in standard data types or typedef's data types?
If it makes something easier, or if it implements a proper abstraction barrier, use it.
What exactly that means is something you'll just have to learn over time.
But don't use it "just for kicks"!
Is there any alternative to non-ISO gcc specific extension __attribute__ on 64-bit kernels ?
Three types that i've noticed are: function attributes, type attributes and variable attributes.
eg.
i'd like to avoid using __attribute__((__packed__)) for structures passed over the network, even though some gcc based code do use it.
Any suggestions or pointers on how to entirely avoid __attribute__ usage in C systems/kernel code ?
thanks
Saifi.
Any suggestions or pointers on how to entirely avoid attribute usage in C systems/kernel code?
You can build your network packets piece by piece, copying each data element into the correct place in a char* buffer.
Pros: you don't have any alignment issues and it's generally portable, especially if you use the exact-width integer types from <stdint.h>
Cons: it's tedious and potentially error-prone
I'm assuming based on your comments that its your code you want to change, not the whole Linux kernel (etc).
Not sure about function attributes etc but specifically for attribute packed, I have done the following with no issues so far.
Basically instead of relying on the compiler to pack, you could use manual pad fields coupled with compile-time asserts.
struct foo {
u32 field1;
u16 field2;
u16 pad; // manual padding
// continue for other fields that the compiler would automatically pad for you with attribute packed
u32 field3;
};
To check your structure you can use a compile time assert, something like this:
#define CASSERT(cond, name) typedef cassert__##name[cond ? 1 : -1]
CASSERT(offsetof(foo, field1) == 0, field1_wrong);
CASSERT(offsetof(foo, field2) == 4, field2_wrong);
When your assertions are wrong, the build will fail with a helpful error and line number