I would like to understand the innards of the Python import system, including the rough spots. In the Python C API documentation, there's this terse reference to one such rough spot:
This is so important that we’re going to pick the top of it apart
still further:
PyObject_HEAD_INIT(NULL)
This line is a bit of a wart; what we’d like
to write is:
PyObject_HEAD_INIT(&PyType_Type)
as the type of a type object is
“type”, but this isn’t strictly conforming C and some compilers
complain.
Why is this not strictly conforming C? Why do some compilers accept this without complaint and others do not?
I now think the following is misleading, skip down to "SUBSTANTIAL EDIT"
Scrolling about a page down there is what I believe is a clue. This quote regards initializing another member of the struct but it sounds like the same issue and this time it is explained.
We’d like to just assign this to the tp_new slot, but we can’t, for
portability sake, On some platforms or compilers, we can’t statically
initialize a structure member with a function defined in another C
module
This still leaves me a bit confused, in part due to the odd word choice of "module". I think the second quote meant to say that static initialization that relies on calls to functions in separate compilation units is a non-standard extension. I still don't understand why that would be so. Is that what's going on in the first quote?
SUBSTANTIAL EDIT:
The use of PyObject_HEAD_INIT(NULL) is advised to go at the very top of the initialization of an instance of PyTypeObject.
The definition of PyTypeObject looks like this:
typedef struct _typeobject {
PyObject_VAR_HEAD
const char *tp_name; /* For printing, in format "<module>.<name>" */
Py_ssize_t tp_basicsize, tp_itemsize; /* For allocation */
/* Methods to implement standard operations */
destructor tp_dealloc;
/*... lots more ... */
} PyTypeObject;
The PyObject_HEAD_INIT(NULL) macro is used to initialize the top of PyTypeObject instances. The top of the PyTypeObject definition is created by the macro PyObject_VAR_HEAD. PyObject_VAR_HEAD is:
/* PyObject_VAR_HEAD defines the initial segment of all variable-size
* container objects. These end with a declaration of an array with 1
* element, but enough space is malloc'ed so that the array actually
* has room for ob_size elements. Note that ob_size is an element count,
* not necessarily a byte count.
*/
#define PyObject_VAR_HEAD \
PyObject_HEAD \
Py_ssize_t ob_size; /* Number of items in variable part */
#define Py_INVALID_SIZE (Py_ssize_t)-1
In turn, PyObject_HEAD expands to:
/* PyObject_HEAD defines the initial segment of every PyObject. */
#define PyObject_HEAD \
_PyObject_HEAD_EXTRA \
Py_ssize_t ob_refcnt; \
struct _typeobject *ob_type;
_PyObject_HEAD_EXTRA is only used in debugging builds and normally expands to nothing. The members being initialized by the PyObject_HEAD_INIT macro are ob_refcnt and ob_type. ob_type is the one that we would like to initialize with &PyType_Type but we're told that would violate the C Standard. ob_type points to a _typeobject, which is typedef'd as a PyTypeObject (the same struct that we're trying to initialize). We use the PyObject_HEAD_INIT macro, which initializes those two values, expands as so:
#define PyObject_HEAD_INIT(type) \
_PyObject_EXTRA_INIT \
1, type,
So we're starting a reference count at 1 and setting a member pointer to whatever is in the type parameter. The Python documentation says we can't set the type parameter it to the address of PyType_Type because that is not strictly standard C so we settle for NULL.
PyType_Type is declared in the same translation unit a few lines below.
PyAPI_DATA(PyTypeObject) PyType_Type; /* built-in 'type' */
PyAPI_DATA is defined elsewhere. It has a few different conditional definitions.
#define PyAPI_DATA(RTYPE) extern __declspec(dllexport) RTYPE
#define PyAPI_DATA(RTYPE) extern RTYPE
So the Python API documentation is saying that we'd like to initialize an instance of a PyTypeObject with a pointer to previously declared PyTypeObject that was declared with the extern qualifier. What in the C Standard would that violate?
The initialization of PyType_Type occurs in a .c file. A typical Python extension that initializes a PyTypeObject, as described above, will be dynamically loaded by code that was compiled with this initialization:
PyTypeObject PyType_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"type", /* tp_name */
sizeof(PyHeapTypeObject), /* tp_basicsize */
sizeof(PyMemberDef), /* tp_itemsize */
(destructor)type_dealloc, /* tp_dealloc */
/* ... lots more ... */
}
PyObject_HEAD_INIT(&PyType_Type)
Produces
1, &PyType_Type
which initializes fields
Py_ssize_t ob_refcnt;
struct _typeobject *ob_type;
PyType_Type is defined with PyAPI_DATA(PyTypeObject) PyType_Type which produces
extern PyTypeObject PyType_Type;
possibly with a __declspec qualifier. PyTypeObject is a typedef for struct _typeobject, so we have
extern struct _typeobject PyType_Type;
so PyObject_HEAD_INIT(&PyType_Type) would initialize the struct _typeobject* ob_type field with a struct _typeobject* ... which is certainly valid C, so I don't see why they say it isn't.
I came upon an explanation of this elsewhere in the Python source code.
/* We link this module statically for convenience. If compiled as a shared
library instead, some compilers don't allow addresses of Python objects
defined in other libraries to be used in static initializers here. The
DEFERRED_ADDRESS macro is used to tag the slots where such addresses
appear; the module init function must fill in the tagged slots at runtime.
The argument is for documentation -- the macro ignores it.
*/
#define DEFERRED_ADDRESS(ADDR) 0
And then the macro is used where NULL appears at the top of the OP.
PyVarObject_HEAD_INIT(DEFERRED_ADDRESS(&PyType_Type), 0)
Related
In the magenta kernel there is a paragraph in which struct got not only a type and name but one option more. I found in the references nothing to explain that syntax. So what is __CPU_ALIGN as argument in struct for and where do I find the syntax for it?
struct type name ???
#if WITH_SMP
/* a global state structure, aligned on cpu cache line to minimize aliasing */
struct mp_state mp __CPU_ALIGN = {
.hotplug_lock = MUTEX_INITIAL_VALUE(mp.hotplug_lock),
.ipi_task_lock = SPIN_LOCK_INITIAL_VALUE,
};
I know that __CPU_ALIGN itself is used to have aligned bytes for the CPU memory size.
It's a macro shorthand for the aligned attribute, which is a GCC extension.
The macro is defined as follows:
#define __CPU_ALIGN __ALIGNED(CACHE_LINE)
The macro __ALIGNED in turn is defined like this:
#define __ALIGNED(x) __attribute__((aligned(x)))
...which matches the syntax in the GCC documentation. (The value of CACHE_LINE depends on the architecture.)
Question
I try to find static (compile time) asserts, to ensure (as good as possible) things below. As I use them in an auto code generation context (see “Background” below) they do not have to be neat, the only have to break compilation, at best with zero overhead. Elegant variants are welcomed though.
The following things shall be checked:
A Type Identity
typedef T T1;
typedef T T2;
typedef X T3;
T1 a;
T2 b;
T3 c;
SA_M1(T1,T2); /* compilation */
SA_M1(T1,T3); /* compilation error */
SA_M2(a,b); /* compilation */
SA_M2(a,c); /* compilation error */
where X and T are C Types (including structured, aggregated, object pointer, not so important function pointer). Note again, that a set of partly successful solutions also helps.
Some solutions that I assume will partly work:
comparing the sizes
checking if the type is a pointer as claimed by trying to dereference it.
for unsigned integers: Compare a casted slightly to big value with the expected wrap around value.
for floats, compare double precision exact representable value with the casted one (hoping the best for platform specific rounding operations)
B A Variable has global Scope
My solution here is at the momement simply to generate a static function, that tries to get a reference to the global variable Assume that X is a global variable:
static void SA_IsGlobal_X() {(void) (&X == NULL); /* Dummy Operation */}
C A Function has the correct number of parameters
I have no idea yet.
D If the prototype of a functions is as it is expected
I have no idea yet.
E If a function or macro parameters are compile time constants (
This question is discussed here for macros:
Macro for use in expression while enforcing its arguments to be compile time constants
For functions, an wrapper macro could do.
Z Other things you might like to check considering the “background” part below
Preferred are answers that can be done with C89, have zero costs in runtime, stack and (with most compilers) code size. As the checks will be auto generated, readability is not so important, but I like to place the checks in static functions, whenever possible.
Background:
I want to provide C functions as well as an interface generator to allow them to smoothly being integrated in different C frameworks (with C++ on the horizon). The user of the interface generator then only specifies where the inputs come from, and which of the outputs shall go where. Options are at least:
RAW (as it is implemented - and should be used)
from the interface functions parameter, which is of a type said to be the same as my input/output (and perhaps is a field of a structure or an array element)
from a getter/setter function
from a global variable
using a compile time constant
I will:
ask for a very detailed interface specification (including specification errors)
use parsers to check typedefs and declarations (including tool bugs and my tool usage errors)
But this happens at generation time. Besides everything else: if the user change either the environment or takes a new major version of my function (this can be solved by macros checking versions), without running the interface generator again, I would like to have a last defense line at compile time.
The resulting code of the generations might near worst case be something like:
#include "IFMyFunc.h" /* contains all user headers for the target framework(s) */
#include "MyFunc.h"
RetType IFMYFunc(const T1 a, const struct T2 * const s, T3 * const c)
{
/* CHECK INTERFACE */
CheckIFMyFunc();
/* get d over a worst case parametrized getter function */
const MyD_type d = getD(s->dInfo);
/* do horrible call by value and reference stuff, f and g are global vars */
c.c1 = MyFunc(a,s->b,c.c1,d,f,&(c->c2), &e,&g);
set(e);
/* return something by return value */
return e;
}
(I am pretty sure I will restrict the combos though).
static void CheckIFMyFunc(void)
{
/* many many compile time checks of types and specifications */
}
or I will provide a piece of code (local block) to be directly infused - which is horrible architecture, but might be necessary if we can't abandon some of the frame work fast enough, supported by some legacy scripts.
for A, would propose:
#define SA_M1(A, B) \
do { \
A ___a; \
B ___b = ___a; \
(void)___b; \
} while (0)
for D (and I would say that C is already done by D)
typedef int (*myproto)(int a, char **c);
#define FN_SA(Ref, Challenger) \
do { \
Ref ___f = Challenger; \
(void) ___f; \
} while (0)
void test(int argc, char **argv);
int main(int argc, char **argv)
{
FN_SA(myproto, main);
FN_SA(myproto, test); /* Does not compile */
return 0;
}
Nevertheless, there are some remaining problems with void *:
any pointer may be casted to/from void * in C, which will probably make the solution for A fail in some cases....
BTW, if you plan to use C++ in the meanterm, you could just use C++ templates and so on to have this stests done here. Would be far more clean and reliable IMHO.
We will write functional/unit tests for C code. This C program will be run as embedded software. However we need to run tests on a Linux environment.
The problem is that parts of the code under test looks like this:
my_addresses.h:
#define MY_BASE_ADDRESS (0x00600000)
#define MY_OFFSET_ADDRESS (0x108)
my_code.c
#include "my_addresses.h"
static const My_Type* my_ptr =
(My_Type*)(MY_BASE_ADDRESS + MY_OFFSET_ADDRESS);
/* my_ptr is dereferenced and used ... */
Obviously, this will not run so well on Linux host environment.
Is there some way we can work around this issue during testing? Can we somehow "redirect" the program to use other addresses, that are valid addresses to memory allocated during test procedures?
Our first attempt was to replace "my_addresses.h" with another header file during tests, which (extern) declares variables instead of hard defines - then assign malloc'd memory to MY_BASE_ADDRESS, etc. The problem with that is the "static const" declaration in the c file. Of course you cannot assign a variable to a static const type.
Preferably, we should not modify the code under test (although in the worst case it may come to that).
You could check for e.g. the __linux__ macro and use conditional compilation. When on Linux use an array as base, and make it big enough to keep all the data needed in it.
Something like e.g.
#ifdef __linux__
int8_t array[1024];
# define MY_BASE_ADDRESS array
#else
# define MY_BASE_ADDRESS 0x00600000
#endif
In the Linux environment you can define a global array and then use the address of this as your base pointer.
const char my_buffer[1024];
#define my_base_addr (&my_buff)
Assuming
your embedded memory layout is small enough to model unchanged in its entirety (or in a few chunks) on Linux,
your compiler is happy with the constant memory address expressions below, and
My_Type in your example was defined as typedef My_Type1_t * My_Type;
you could (1) separate the definition of the embedded memory layout from deciding how it is placed and perhaps (2) gain some type-safety if you declare a struct for the layout:
#pragma (essential: stuff to force structs to contain no extra padding)
typedef struct {
char pad0[0x108];
My_Type1_t foo;
char pad1[0x210];
My_Type2_t bar;
...
} Memory_Layout_t;
#pragma (preferably: something to revert to previous struct layout options)
(If you don’t like calculating the size of pad1, use a union.)
Then make the variants:
#ifdef __linux__
Memory_Layout_t Embedded_Memory;
# define Embedded_Memory_P (& Embedded_Memory)
#else
# define Embedded_Memory_P ((Memory_Layout_t *) (0x00600000))
#endif
and reference it with
static const My_Type my_ptr = & Embedded_Memory_P->foo;
I found this code in a header file for a device that I need to use, and although I've been doing C for years, I've never run into this:
struct device {
};
struct spi_device {
struct device dev;
};
and it used as in:
int spi_write_then_read(struct spi_device *spi,
const unsigned char *txbuf, unsigned n_tx,
unsigned char *rxbuf, unsigned n_rx);
and also here:
struct spi_device *spi = phy->spi;
where it is defined the same.
I'm not sure what the point is with this definition. It is in a header file for a linux application of the board, but am baffled by it use. Any explanations, ideas? Anyone seen this before (I'm sure some of you have :).
Thanks!
:bp:
This is not C as C structures have to contain at least one named member:
(C11, 6.7.2.1 Structure and union specifiers p8) "If the struct-declaration-list does not contain any named members, either directly or via an anonymous structure or anonymous union, the behavior is undefined."
but a GNU C extension:
GCC permits a C structure to have no members:
struct empty {
};
The structure has size zero
https://gcc.gnu.org/onlinedocs/gcc/Empty-Structures.html
I don't know what is the purpose of this construct in your example but in general I think it may be used as a forward declaration of the structure type. Note that in C++ it is allowed to have a class with no member.
In Linux 2.4 there is an example of an empty structure type with conditional compilation in the definition of spin_lock_t type alias in Linux kernel 2.4 (in include/linux/spinlock.h):
#if (DEBUG_SPINLOCKS < 1)
/* ... */
typedef struct { } spinlock_t;
#elif (DEBUG_SPINLOCKS < 2)
/* ... */
typedef struct {
volatile unsigned long lock;
} spinlock_t;
#else /* (DEBUG_SPINLOCKS >= 2) */
/* ... */
typedef struct {
volatile unsigned long lock;
volatile unsigned int babble;
const char *module;
} spinlock_t;
#endif
The purpose is to save some space without having to change the functions API in case DEBUG_SPINLOCKS < 1. It also allows to define dummy (zero-sized) objects of type spinlock_t.
Another example in the (recent) Linux kernel of an empty structure hack used with conditional compilation in include/linux/device.h:
struct acpi_dev_node {
#ifdef CONFIG_ACPI
void *handle;
#endif
};
See the discussion with Greg Kroah-Hartman for this last example here:
https://lkml.org/lkml/2012/11/19/453
This is not standard C.
C11: 6.2.5-20:
— A structure type describes a sequentially allocated nonempty set of member objects (and, in certain circumstances, an incomplete array), each of which has an optionally specified name and possibly distinct type.
J.2 Undefined behavior:
The behavior is undefined in the following circumstances:
....
— A structure or union is defined without any named members (including those
specified indirectly via anonymous structures and unions) (6.7.2.1).
GCC uses it as an extension (no more detailed is given there about when/where should it be used). Using this in any program will make it compiler specific.
One reason might to do this for a library is that the library developers do not want you to know or interfere with the internals of these struct. It these cases they may provide an "interface" version of the structs spi_device/device (which is what you may see) and have a second type definition that defines another version of said structs for use inside the library with the actual members.
Since you cannot access struct members or even create compatible structs of that type yourself with that approach (since even your compiler would not know the size actual size of this struct), this only works if the library itself creates the structs, only ever passes you pointers to it, and does not need you to modify any members.
If you add an empty struct as the first member of another struct, the empty
struct can serve as a "marker interface", i.e. when you cast a pointer to that
outer struct to a pointer of the inner struct and the cast succeeds you know
that the outer struct is "marked" as something.
Also it might just be a place holder for future development, not to sure. Hope this helps
This is valid C
struct empty;
struct empty *empty;
and facilitates use of addresses of opaque regions of memory.
Such addresses are usually obtained from and passed to library subroutines.
For example, something like this is done in stdio.h
So today's exercise wants me to use this header.h obviously to give me the function corresponding to the operator.
#ifndef __HEADER__
#define __HEADER__
operator operator_table[] = {{"-", &function_sub}, \
{"+", &function_add}, \
{"*", &function_mul}, \
{"/", &function_div}, \
{"%", &function_mod}};
#endif
First thing I noticed is that operator type isn't defined so maybe I should typedef it to an int ?
Then the real problem start, I've read both K&R and C Primer Plus from beginning and haven't encountered this syntax, or at least I don't recognize it, is it some kind of dictionary ? How can I use it ?
It seems to be an array of structures, the structure (named operator) containing a string and a function pointer. There is no special syntax, just a normal array definition and initialization.
The \ is part of the preprocessor, and is a line-continuation "operator". It simply means that the preprocessor will create a single line out of this for the compiler to see.
operator here is a structure. and it could be defined in this way:
typedef struct {
char *op;
int (*func)(int, int);
} operator;
func here is a pointer to a function
function_sub, function_add, function_mul, function_div and function_mod should be a functions defined in your c code
In C, operator is not a keyword (which is not the case in C++ for example), and is here used as a symbol representing a type. So either it is typedef'd somewhere or it needs defining.
Looking at the array, this is an array of structs, the structure corresponding to operator being made of a char * and a function pointer. "+" is associated to function_add(type, type), and so on. type is not specified here as it cannot be inferred from this piece of code. The same applies to the number of arguments, I assumed 2 but this is arbitrary.
So, to use header.h, you have either to:
include in your .c file the header file that defines operator, if it exists.
or define it your own way without forgetting to define the functions in charge of the actual processing.
For example:
#ifndef __OPERATOR_HEADER__
#define __OPERATOR_HEADER__
float function_add(float, float);
float function_sub(float, float);
/* etc, the body of these function being defined in your .c file */
typedef struct operator {
char *operator_name;
float (*operator_function)(float, float);
};
#endif