Related
Considering a dynamic library with this native function that returns the sum of all even (32-bit unsigned) numbers in an array:
uint32_t sum_of_even(const uint32_t *numbers, size_t length);
The implementation of the function above was written in Rust as below, and packaged into a C dynamic library.
use libc::size_t;
use std::slice;
#[no_mangle]
pub extern "C" fn sum_of_even(n: *const u32, len: size_t) -> u32 {
let numbers = unsafe {
assert!(!n.is_null());
slice::from_raw_parts(n, len as usize)
};
numbers
.iter()
.filter(|&v| v % 2 == 0)
.sum()
}
I wrote the following Julia (v1.0.1) wrapper function:
lib = Libdl.dlopen(libname)
sumofeven_sym = Libdl.dlsym(lib, :sum_of_even)
sumofeven(a) = ccall(
sumofeven_sym,
UInt32,
(Ptr{UInt32}, Csize_t),
a, length(a)
)
The documentation states multiple times that arguments in ccall are converted to become compatible with the C function prototype (emphasis mine):
Each argvalue to the ccall will be converted to the corresponding argtype, by automatic insertion of calls to unsafe_convert(argtype, cconvert(argtype, argvalue)). (See also the documentation for unsafe_convert and cconvert for further details.) In most cases, this simply results in a call to convert(argtype, argvalue).
And moreover, that when passing an Array{T} by Ptr{U} to a C function, the call is invalidated if the two types T and U are different, since no reinterpret cast is added (section Bits Types):
When an array is passed to C as a Ptr{T} argument, it is not reinterpret-cast: Julia requires that the element type of the array matches T, and the address of the first element is passed.
Therefore, if an Array contains data in the wrong format, it will have to be explicitly converted using a call such as trunc(Int32, a).
However, this is seemingly not the case. If I deliberately pass an array with another type element:
println(sumofeven(Float32[1, 2, 3, 4, 5, 6]))
The program calls the C function with the array passed directly, without converting the values nor complaining about the different element types, resulting in either senseless output or a segmentation fault.
If I redefine the function to accept a Ref{UInt32} instead of a Ptr{UInt32}, I am prevented from calling it with the array of floats:
ERROR: LoadError: MethodError: Cannot `convert` an object of type Array{Float32,1} to an object of type UInt32
Closest candidates are:
convert(::Type{T<:Number}, !Matched::T<:Number) where T<:Number at number.jl:6
convert(::Type{T<:Number}, !Matched::Number) where T<:Number at number.jl:7
convert(::Type{T<:Integer}, !Matched::Ptr) where T<:Integer at pointer.jl:23
...
However, Ref was not designed for arrays.
Making the example work with Ptr{UInt32} requires me to either specify Array{UInt32} as the type of input a (static enforcement), or convert the array first for a more flexible function.
sumofeven(a:: Array{UInt32}) = ccall( # ← either this
sumofeven_sym,
UInt32,
(Ptr{UInt32}, Csize_t),
convert(Array{UInt32}, a), # ← or this
length(a))
With that, I still feel that there is a gap in my reasoning. What is the documentation really suggesting when it says that an array passed to C as a Ptr{T} is not reinterpret-cast? Why is Julia letting me pass an array of different element types without any explicit conversion?
This turned out to be either a bug in the core library or a very misguided documentation, depending on the perspective (issue #29850). The behavior of the function unsafe_convert changed from version 0.4 to 0.5, in a way that makes it more flexible than what is currently suggested.
According to this commit, unsafe_convert changed from this:
unsafe_convert(::Type{Ptr{Void}}, a::Array) = ccall(:jl_array_ptr, Ptr{Void}, (Any,), a)
To this:
unsafe_convert{S,T}(::Type{Ptr{S}}, a::AbstractArray{T}) = convert(Ptr{S}, unsafe_convert(Ptr{T}, a))
For arrays, this relaxed implementation will enable a transformation from an array of T to a pointer of another type S. In practice, unsafe_convert(cconvert(array)) will reinterpret-cast the array's base pointer, as in the C++ nomenclature. We are left with a dangerously reinterpreted array across the FFI boundary.
The key takeaway is that one needs to take extra care when passing arrays to C functions, as the element type of an array in a C-call function parameter is not statically enforced. Use type signatures and/or explicit conversions where applicable.
I am coming from a C++ background, and have recently taken up C. I am am having trouble assigning a number to a type (or vice versa); what I need is some way to assign a unique ID to a type, preferably starting from 0. My goal is to have a function (or macro) that indexes an array based on a passed-in type, which I believe to only be achievable through macros.
Also, since I use the sizeof() the type which I need to be passed in, it makes using enums as an alternative difficult. If I were to pass the enumerator to the function/macro instead, then I would have to get the type from the number, the exact opposite (but maybe easier) problem.
Is this even possible in C? I have tried researching this question, but have not found any answer to this problem particularly, which I was able to do in C++ with templates, like so:
int curTypeIdx = 0;
template <typename T>
struct TypeHandle {
static int const val;
}
template <typename T>
int const TypeHandle<T>::val = curTypeIdx++;
The reason for this is that I am building an ECS. I have an EntityManager struct, which is supposed to contain arrays of components. Since I plan for this to be general-purpose, I defined an upper limit of components (MAX_COMPONENTS) and have an array of char*s of length MAX_COMPONENTS. At a basic level, The goal is to give the user of the EntityManager the ability to define their own components, and store them in these generic arrays.
If there is any other way to
Thank you all for any advice.
If you are OK with enumerating ALL supported types once( and update the list if language comes up with new types), then you can use the stringize functionality of C macros and an array of strings to achieve what you want.
#define GET_TYPE_ID(type) get_type_id(#type)
const char *type_strings[] = { "char", "unsigned char", "short" /* so on.. */};
int get_type_id(const char* type_string) {
for( int i = 0; i < sizeof(type_strings)/sizeof(const char*); i++) {
if ( strcmp(type_string, type_strings[i]) == 0 ) return i;
}
// Should never reach here if you have taken care of all types and
// don't pass in illegal types.
}
Now you can get an integer ID for each type with GET_TYPE_ID(int), GET_TYPE_ID(char) and so on.
In my D program, I have a read-only array of fixed length and I wish to index the array by an enumerated type.
If I do something like
static const my_struct_t aray[ my_enum_t ] = ... whatever ...;
my_enum_t index;
result = aray[ index ];
then the code produced by GDC is huge, full of calls to the runtime when the array is indexed. So it looks as if either the array is being treated as variable-length or as an associative array (hash table) or something, anyway far from a lightweight C-style array of fixed length with straightforward indexing. Since enums have a fixed cardinality and can't grow, and I have a modest sparse range of values (I'm not misusing the keyword enum just to defined a load of random constants) then I don't know why this happened.
I fixed the issue by changing the line to
static const my_struct_t aray[ my_enum_t.max + 1 ]
and as I understand it that will mean the value in the square brackets is just a known constant of integral type. Since the index is now not an enum at all, I now have an array indexed by an integer, so I have lost type checking, I could index it with any random integer typed variable rather than ensuring that only the correct (strong) type is used.
What should I be doing?
In the more general case, (silly example)
static const value_t aray[ bool ] = blah
for example, where I have an index type that is perfectly sensible semantically, but not just a typeless size_t/int/uint I presume I would get the same problem.
I wouldn't want to say that this is a compiler design problem. It's certainly a case of sub-optimal behaviour. But to be fair to the compiler what exactly is telling it whether the array is fixed-length or variable, and sparse or dense? I want two things; type checking of the index and non-variable length. Actually, in this particular case the array is const (I could have put immutable just as well) so it clearly can't be variable-length any way. But with an array that has modifiable content but is of fixed length you need to be able to declare that it is fixed-length.
V[K] name is the syntax for an associative array which does indeed do runtime calls and such, even when the type is limited to a small number of values like bool or an enum. The compiler probably could optimize that, making it act to the program like an AA while implementing it as a simple fixed-length array, but it doesn't; it treats all key types the same.
I would suggest going with what you started: T[enum.max + 1], but then doing a wrapper if you want to force type safety. You can make the index overloads static if you only want one instance of it:
enum Foo {
one,
two
}
struct struct_t {}
struct array {
static private struct_t[Foo.max + 1] content;
static struct_t opIndex(Foo idx) { return content[cast(int) idx]; }
}
void main() {
struct_t a = array[Foo.one];
}
Then, you can just genericize that if you want simpler reuse.
struct enum_array(Key, Value) {
static private struct_t[Key.max + 1] content;
static Value opIndex(Key idx) { return content[cast(int) idx]; }
}
alias array = enum_array!(Foo, struct_t);
Or, of course, you don't need to make it static, you could do a regular instance too, and initialize the contents inside and such.
In D, both static and dynamic arrays are indexed by size_t, just like they would be in C and C++. And you can't change the type of the index in D any more than you can in C or C++. So, in D, if you put a type between the brackets in the array declaration, you're defining an associative array and not a static array. If you want a static array, you must provide an integer literal or compile-time constant, and there is no way to require that a naked, static array be indexed by an enum type that has a base type of size_t or a type that implicitly converts to size_t.
If you want to require that your static array be indexed by a type other than size_t, then you need to wrap it in a struct or class and control the access to the static array via the member functions. You could overload opIndex to take your enum type and treat your struct type as if it were a static array. So, the effect should then be effectively what you were trying to do with putting the enum type in the static array declaration, but it would be the member function that took the enum value and called the static array with it rather than doing anything to the static array itself.
I have tried:
const ascii = "abcdefghijklmnopqrstuvwxyz"
const letter_goodness []float32 = { .0817,.0149,.0278,.0425,.1270,.0223,.0202, .0609,.0697,.0015,.0077,.0402,.0241,.0675, .0751,.0193,.0009,.0599,.0633,.0906,.0276, .0098,.0236,.0015,.0197,.0007 }
const letter_goodness = { .0817,.0149,.0278,.0425,.1270,.0223,.0202, .0609,.0697,.0015,.0077,.0402,.0241,.0675, .0751,.0193,.0009,.0599,.0633,.0906,.0276, .0098,.0236,.0015,.0197,.0007 }
const letter_goodness = []float32 { .0817,.0149,.0278,.0425,.1270,.0223,.0202, .0609,.0697,.0015,.0077,.0402,.0241,.0675, .0751,.0193,.0009,.0599,.0633,.0906,.0276, .0098,.0236,.0015,.0197,.0007 }
The first declaration and initialization works fine, but the second, third and fourth don't work.
How can I declare and initialize a const array of floats?
An array isn't immutable by nature; you can't make it constant.
The nearest you can get is:
var letter_goodness = [...]float32 {.0817, .0149, .0278, .0425, .1270, .0223, .0202, .0609, .0697, .0015, .0077, .0402, .0241, .0675, .0751, .0193, .0009, .0599, .0633, .0906, .0276, .0098, .0236, .0015, .0197, .0007 }
Note the [...] instead of []: it ensures you get a (fixed size) array instead of a slice. So the values aren't fixed but the size is.
As pointed out by #jimt, the [...]T syntax is sugar for [123]T. It creates a fixed size array, but lets the compiler figure out how many elements are in it.
From Effective Go:
Constants in Go are just that—constant. They are created at compile time, even when defined as locals in functions, and can only be numbers, characters (runes), strings or booleans. Because of the compile-time restriction, the expressions that define them must be constant expressions, evaluatable by the compiler. For instance, 1<<3 is a constant expression, while math.Sin(math.Pi/4) is not because the function call to math.Sin needs to happen at run time.
Slices and arrays are always evaluated during runtime:
var TestSlice = []float32 {.03, .02}
var TestArray = [2]float32 {.03, .02}
var TestArray2 = [...]float32 {.03, .02}
[...] tells the compiler to figure out the length of the array itself. Slices wrap arrays and are easier to work with in most cases. Instead of using constants, just make the variables unaccessible to other packages by using a lower case first letter:
var ThisIsPublic = [2]float32 {.03, .02}
var thisIsPrivate = [2]float32 {.03, .02}
thisIsPrivate is available only in the package it is defined. If you need read access from outside, you can write a simple getter function (see Getters in golang).
There is no such thing as array constant in Go.
Quoting from the Go Language Specification: Constants:
There are boolean constants, rune constants, integer constants, floating-point constants, complex constants, and string constants. Rune, integer, floating-point, and complex constants are collectively called numeric constants.
A Constant expression (which is used to initialize a constant) may contain only constant operands and are evaluated at compile time.
The specification lists the different types of constants. Note that you can create and initialize constants with constant expressions of types having one of the allowed types as the underlying type. For example this is valid:
func main() {
type Myint int
const i1 Myint = 1
const i2 = Myint(2)
fmt.Printf("%T %v\n", i1, i1)
fmt.Printf("%T %v\n", i2, i2)
}
Output (try it on the Go Playground):
main.Myint 1
main.Myint 2
If you need an array, it can only be a variable, but not a constant.
I recommend this great blog article about constants: Constants
As others have mentioned, there is no official Go construct for this. The closest I can imagine would be a function that returns a slice. In this way, you can guarantee that no one will manipulate the elements of the original slice (as it is "hard-coded" into the array).
I have shortened your slice to make it...shorter...:
func GetLetterGoodness() []float32 {
return []float32 { .0817,.0149,.0278,.0425,.1270,.0223 }
}
In addition to #Paul's answer above, you can also do the following if you only need access to individual elements of the array (i.e. if you don't need to iterate on the array, get its length, or create slices out of it).
Instead of
var myArray [...]string{ /* ... */ }
you can do
func myConstArray(n int) string {
return [...]string{ /* ... */ }[n]
}
and then instead of extracting elements as
str := myArray[i]
you extract them as
str := myConstArray(i)
Link on Godbolt: https://godbolt.org/z/8hz7E45eW (note how in the assembly of main no copy of the array is done, and how the compiler is able to even extract the corresponding element if n is known at compile time - something that is not possible with normal non-const arrays).
If instead, you need to iterate on the array or create slices out of it, #Paul's answer is still the way to go¹ (even though it will likely have a significant runtime impact, as a copy of the array needs to be created every time the function is called).
This is unfortunately the closest thing to const arrays we can get until https://github.com/golang/go/issues/6386 is solved.
¹ Technically speaking you can also do it with the const array as described in my answer, but it's quite ugly and definitely not very efficient at runtime: https://go.dev/play/p/rQEWQhufGyK
How do I figure out the type of an object in C? My goal with this is to create a linked-list container in C.
Let's assume I have a function that takes a void pointer:
...
Foo *f;
f = Allocate(f);
...
void *Allocate(void *item)
{
return malloc(sizeof(f.GetType));
}
How do I make the above syntax possible?
That's not possible. A void * pointer, is just a pointer to the memory, nothing more, no more information is attached to it. It's impossible to do what you are asking, you can't know how many bytes to malloc.
That's why, the qsort function from stdlib.h library takes as a parameter
the size in bytes of each array element. If what you suggested was possible, then
qsort wouldn't need such a parameter.
Perhaps you could do something like this:
...
Foo *f;
f = Allocate(f, sizeof(Foo));
...
void *Allocate(void *item, size_t size)
{
return malloc(size);
}
In C, there's no cross platform way to find out the underlying type of a void*, since the type information is not stored in the object. In C++ this feature has been added for objects.
You could of course implement it yourself for your own objects by - for example - storing the size of every object in the object itself, but for system objects there is no built in way.
It is just not possible to get the type using a void * to that object. If you can calculate the size of Foo before calling Allocate(), then get the size and pass it as a parameter to Allocate(), else you should make Foo visible inside Allocate() to find the size.
The only way to do this in C is to create your own type system where you include typeinfo to the types that you create. That is what has been done for instance in Objective-C. In C, all types are just a series of bytes, when you allocate memory you are allocating just bytes, what you use those bytes for is of no concern to the C compiler.
The (arguable) beauty of C is that it doesn't do this. A void pointer is just an address that could point to anything. You could either create your own object system (structures with a "type" field and a "data" field is a simple way). From your implementation, it might also be possible to stick an allocate function into the Foo structure (assuming that it is) which knows how to allocate itself (the type is known here).
void *Allocate(void *item)
{
return malloc(sizeof(f.GetType));
}
It is impossible. ISO C language does't supply any facility to get the type info through variable. Gcc's extensions contains an operator called 'typeof', it could get the type info from a real type. such as:
typeof (x[0](1))
int a; typeof (a) b; b = a;
but even though typeof also could not get the real type info from a void pointer. the code below could not be compiled succesfully:
void *Allocate(typeof(*item) *item)
{
return malloc(sizeof(*item));
}