How to allocate arrays on the heap in Rust 1.0? - arrays

There is already a question for this but related to Rust 0.13 and the syntax seems to have changed. From the current documentation I understood that creating an array on the heap would be something like this:
fn main() {
const SIZE: usize = 1024 * 1024;
Box::new([10.0; SIZE]);
}
But when I run this program I get the following error:
thread '<main>' has overflowed its stack
What am I doing wrong?

The problem is that the array is being passed to the Box::new function as an argument, which means it has to be created first, which means it has to be created on the stack.
You're asking the compiler to create 8 megabytes of data on the stack: that's what's overflowing it.
The solution is to not use a fixed-size array at all, but a Vec. The simplest way I can think of to make a Vec of 8 million 10.0 is this:
fn main() {
const SIZE: usize = 1024 * 1024;
let v = vec![10.0; SIZE];
}
Or, if for some reason you'd rather use iterators:
use std::iter::repeat;
fn main() {
const SIZE: usize = 1024 * 1024;
let v: Vec<_> = repeat(10.0).take(SIZE).collect();
}
This should only perform a single heap allocation.
Note that you can subsequently take a Vec and turn it into a Box<[_]> by using the into_boxed_slice method.
See also:
Performance comparison of a Vec and a boxed slice

I ran into this too, and ended up first at Creating a fixed-size array on heap in Rust where you find the full answer.
The gist of the answer is this macro:
/// A macro similar to `vec![$elem; $size]` which returns a boxed array.
///
/// ```rustc
/// let _: Box<[u8; 1024]> = box_array![0; 1024];
/// ```
macro_rules! box_array {
($val:expr ; $len:expr) => {{
// Use a generic function so that the pointer cast remains type-safe
fn vec_to_boxed_array<T>(vec: Vec<T>) -> Box<[T; $len]> {
let boxed_slice = vec.into_boxed_slice();
let ptr = ::std::boxed::Box::into_raw(boxed_slice) as *mut [T; $len];
unsafe { Box::from_raw(ptr) }
}
vec_to_boxed_array(vec![$val; $len])
}};
}
which I like most, as it simply gives you what the OP wanted:
An array, and it works with stable rust.

The accepted answer is very unsatisfying as sometimes we do want arrays allocated on the heap to preserve the size information at the type level. Johannes's answer can now be slightly improved using const generics. Instead of a macro we can use a function like:
fn vec_to_boxed_array<T: Copy, const N: usize>(val: T) -> Box<[T; N]> {
let boxed_slice = vec![val; N].into_boxed_slice();
let ptr = Box::into_raw(boxed_slice) as *mut [T; N];
unsafe { Box::from_raw(ptr) }
}

Related

Can't initialize array of not copyable structs [duplicate]

I’m trying to initialize a fixed-size array of some nullable, non-copyable type, like an Option<Box<Thing>> for some kind of Thing. I’d like to pack two of them into a struct without any extra indirection. I’d like to write something like this:
let array: [Option<Box<Thing>>; SIZE] = [None; SIZE];
But it doesn’t work because the [e; n] syntax requires that e implements Copy. Of course, I could expand it into SIZE Nones, but that can be unwieldy when SIZE is large. I don’t believe this can be done with a macro without an unnatural encoding of SIZE. Is there a good way to do it?
Yes, this is easy with unsafe; is there a way to do it without unsafe?
As of Rust 1.38 (released in September 2019), a cleaner alternative to previously posted answers is possible using an intermediate const initializer. This approach works for arrays of any size:
const SIZE: usize = 100;
const INIT: Option<Box<Thing>> = None;
let array: [Option<Box<Thing>>; SIZE] = [INIT; SIZE];
(It works with or without the Box; the example uses Box because it was used in the question.)
One limitation is that the array item must have a default representation that can be evaluated at compile time - a constant, enum variant, or a primitive container composed of those. None or a tuple of numbers will work, but a non-empty Vec or String won't.
You could use the Default trait to initialize the array with default values:
let array: [Option<Box<Thing>>; SIZE] = Default::default();
See this playground for a working example.
Note that this will only work for arrays with up to 32 elements, because Default::default is only implemented for up to [T; 32]. See https://doc.rust-lang.org/std/default/trait.Default.html#impl-Default-for-%5BT%3B%2032%5D.
As of Rust 1.55.0 (which introduced [T]::map()), the following will work:
const SIZE: usize = 100;
#[derive(Debug)]
struct THING { data: i64 }
let array = [(); SIZE].map(|_| Option::<THING>::default());
for x in array {
println!("x: {:?}", x);
}
Rust Playground
I'm copying the answer by chris-morgan and adapting it to match the question better, to follow the recommendation by dbaupp downthread, and to match recent syntax changes:
use std::mem;
use std::ptr;
#[derive(Debug)]
struct Thing {
number: usize,
}
macro_rules! make_array {
($n:expr, $constructor:expr) => {{
let mut items: [_; $n] = mem::uninitialized();
for (i, place) in items.iter_mut().enumerate() {
ptr::write(place, $constructor(i));
}
items
}}
}
const SIZE: usize = 50;
fn main() {
let items = unsafe { make_array!(SIZE, |i| Box::new(Some(Thing { number: i }))) };
println!("{:?}", &items[..]);
}
Note the need to use unsafe here: The problem is that if the constructor function panic!s, this would lead to undefined behavior.
Go through the heap
If you can create a Vec of your type, you can convert it into an array:
use std::convert::TryInto;
#[derive(Clone)]
struct Thing;
const SIZE: usize = 100;
fn main() {
let v: Vec<Option<Thing>> = vec![None; SIZE];
let v: Box<[Option<Thing>; SIZE]> = match v.into_boxed_slice().try_into() {
Ok(v) => v,
Err(_) => unreachable!(),
};
let v: [Option<Thing>; SIZE] = *v;
}
In many cases, you actually want to leave it as a Vec<T>, Box<[T]>, or Box<[T; N]> as these types all put the data in the heap. Large arrays tend to be... large... and you don't want all that data on the stack.
See also:
What is the use of into_boxed_slice() methods?
How to get a slice as an array in Rust?
How do I get an owned value out of a `Box`?
Keep it simple
Type out all the values:
struct Thing;
const SIZE: usize = 5;
fn main() {
let array: [Option<Box<Thing>>; SIZE] = [None, None, None, None, None];
}
You could use a build script to generate this code for you. For an example of this, see:
How to create a static string at compile time
An alternative approach using the arrayvec crate that generalizes easily to situations other than initializing everything with a fixed value:
use arrayvec::ArrayVec;
let array = std::iter::repeat(None)
.take(SIZE)
.collect::<ArrayVec<Option<Box<Thing>>, SIZE>>()
.into_inner()
.unwrap();
(playground)
On nightly Rust, you can use inline const. This is a variant of the answer by #user4815162342, but one that doesn't require you to declare a separate constant and repeat the type:
#![feature(inline_const)]
let array: [Option<Box<Thing>>; SIZE] = [const { None }; SIZE];
Until this is stabilized (hopefully soon), you can also use the inline-const crate, but this does require you to repeat the type.
let stackoverflow: [Option<&mut ()>;0xDEADBEEF] = std::array::from_fn(|_| None);
dbg!(stackoverflow);
playground

Can't convert Iterators into js_sys::Array

I'm trying to implement a method to send an array of u32 (eventually an array of arrays of usize, if possible), since you can't just declare a public array field on a wasm_bindgen type. However, using the example outlined in the wasm_bindgen PR 1749, I can't seem to convert arrays or slices to a js_sys::Array; it only works for Vecs. My question is, why? See below
pub fn test() -> js_sys::Array {
let arr: [u32; 5] = [0,1,2,3,4];
let slice = &arr[0..2];
let vec: Vec<u32> = vec![0,1,2];
arr.into_iter().map(JsValue::from).collect() // This doesn't work
slice.into_iter().map(JsValue::from).collect() // Also doesn't work
vec.into_iter().map(JsValue::from).collect() // Works as expected!
}
The specific error is: the trait 'wasm_bindgen::cast::JsCast' is not implemented for 'u32'
The array and slice examples don't seem to work for any number type, ints or floats. My only thought is because the implementation in PR 1749 seems to expect a ref, and arrays are allocated on the stack that the FromIterator is not valid for items in an array?
Is there some other way to achieve what I'm trying to do with the array (passing across the boundary to JS through wasm_bindgen), or if not, why? I'd be very interested to know.
Although Rust arrays and slices have an into_iter method it returns the same Iterator as the iter method does which iterates over references to values instead of the values themselves. Yes, this is confusing. Since JsValue::from is implemented for u32 but not for &u32 you can take your Iterator<Item = &u32> and convert it to a Iterator<Item = u32> using the copied method. Fixed working examples:
use wasm_bindgen::JsValue;
use js_sys::Array;
fn array_to_js_array(array: [u32; 5]) -> Array {
array.iter().copied().map(JsValue::from).collect()
}
fn slice_to_js_array(slice: &[u32]) -> Array {
slice.iter().copied().map(JsValue::from).collect()
}
fn vec_to_js_array(vec: Vec<u32>) -> Array {
vec.into_iter().map(JsValue::from).collect()
}

How do I return an array from a Rust function?

I want to create a function that generates an array of X size with random values.
use rand::prelude::*;
fn generateArray(howManyValues: u32)->[f64]
{
let mut rng = rand::thread_rng();
let array: [f64, howManyValues];
for i in 0..howManyValues
{
array[i] = rng.gen()*100;
}
println!("{:?}", array);
return array;
}
fn main() {
generateArray(10);
}
I want to create a function that generates an array of X size with random values.
The type [f64] is a slice, not an array. An array needs a length, like [f64; 25].
Rust's Vec is probably better than a array or slice for this job, unless you're using some sort of foreign function interface (FFI), or a Rust API requires an array (it probably requires Deref<Target = [T]>, basically generic over all array-like types including Vec and arrays). Vec is analogous to C++'s std::vector, and Java's ArrayList.
use rand::prelude::*;
fn generate_vec(len: usize) -> Vec<f64> {
let mut rng = rand::thread_rng();
let mut vec = Vec::with_capacity(len);
for _ in 0..len {
vec.push(rng.gen::<f64>() * 100.);
}
return vec;
}
fn main() {
let generated_vec = generate_vec(10);
println!("{:?}", generated_vec);
}
I made some style changes[1]. If you really want an array, read on.
If you know the size of the array at compile time, you can use an array. If it is too big to fit on the stack, you'll get a stack overflow. Rust 1.51 made this slightly more ergonomic with const generics.
use rand::prelude::*;
fn generate_array<const LEN: usize>() -> [f64; LEN] {
let mut rng = rand::thread_rng();
let mut arr = [0.; LEN];
for item in arr.iter_mut() {
*item = rng.gen::<f64>() * 100.;
}
arr
}
fn main() {
// generate_array can make an array of any length (within stack size limits)
let generated_array1:[f64; 5] = generate_array();
let generated_array2:[f64; 10] = generate_array();
println!("{:?}", generated_array2);
}
If you don't know the length at compile time, or it is too big to fit on the stack, the closest you can get to an array is Box<[T]> (Box of slice of T) using into_boxed_slice().
use rand::prelude::*;
fn generate_array(len: usize) -> Box<[f64]> {
let mut rng = rand::thread_rng();
let mut vec = Vec::with_capacity(len);
for _ in 0..len {
vec.push(rng.gen::<f64>() * 100.);
}
vec.into_boxed_slice()
}
fn main() {
let generated_array = generate_array(10);
println!("{:?}", generated_array);
}
[1] words in function names separated with underscores instead of camelCase, moved println to main() and changed the type of len to usize, since this is how arrays are typically indexed in Rust.
First, like other programming languages array in Rust cannot grow or shrink. You can either use a vector.
let v : Vec<i32> = Vec::new();
you can use .push() function to push values in vector.
Adapting your function, and still returning an array, requires specifying the number of elements in the return type i.e. fn generateArray()->[f64; 100]. The rust type system accounts for the size of an array, so [f64; 100] (an array of length 100 containing f64s) is a different type from [f64; 99]. Specifying the array's length in the function makes is less reusable, as it always returns the same size array.
If you simply must create stack-allocated arrays full of random floats, you could use a macro, but this limits you to array sizes known at compile time (constexpr in C++ lingo, const in Rust lingo). I consider this macro approach slightly unidiomatic rust. The macro could look like
use rand::prelude::*;
macro_rules! make_array {
($size:literal) => {{
let mut rng = rand::thread_rng();
let mut arr = [0_f64; $size];
for item in arr.iter_mut() {
*item = rng.gen::<f64>()*100.;
}
arr
}}
}
fn main() {
let arr = make_array!(32);
println!("{:?}", arr);
let arr = make_array!(16);
println!("{:?}", arr);
}
This is more reusable than the modified function, as you can specify different numbers of elements.

Can I get a Rust array's length with only a type, not a concrete variable?

I want to rewrite the following C++ code into Rust:
using storage = array<int, 3>;
const size_t storage_len = sizeof(storage) / sizeof(storage::value_type);
How can I get that constant length value without a concrete variable?
As motivation, although it may seem trivial, I want to print the array's element count without declaring a variable. I know I could use a constant value or declare a dummy variable, but I wonder how Rust can preserve C++ code.
I admit without a concrete variable is not clear. I want to achieve the above C++ feature, but this explanation can be misleading. I'm curious if there is any way to get the array's element type.
Rust now has support for const generics. I left the old answer so people have an idea why someone might have asked this in the first place.
New Answer since Rust 1.51:
pub trait Length {
const LEN: usize;
}
impl<T, const LENGTH: usize> Length for [T; LENGTH] {
const LEN: usize = LENGTH;
}
Old Answer:
I understand that you want to retrieve the array length from the type information alone. Rust does not have built-in PI types (a.k.a. const generics). This means generic parameters which are not types (like an integer for an array length) are currently not supported by the language.
There is an issue tracking this and we are likely to see support for it in the future, though not the near future.
If you have to, you can work around that limitation by implementing a trait for each type:
trait GetLength {
fn len() -> usize;
}
impl<T> GetLength for [T; 0] {
fn len() -> usize {
0
}
}
impl<T> GetLength for [T; 1] {
fn len() -> usize {
1
}
}
// ...
fn main() {
println!("{}", <[String; 1]>::len());
}
Macros can help prevent repetitive typing:
trait GetLength {
fn len() -> usize;
}
macro_rules! impl_get_length {
($v:expr) => {
impl<T> GetLength for [T; $v] {
fn len() -> usize {
$v
}
}
};
}
impl_get_length!{ 0 }
impl_get_length!{ 1 }
// ...
fn main() {
println!("{}", <[String; 1]>::len());
}
Crates like typenum also help to provide some support for const generics within the existing language.
In Rust, you can get the size of a type with std::mem::size_of, so you can get the length of your array type the same way as in C++:
use std::mem::size_of;
type Storage = [i32; 3];
fn main() {
println!("Length: {}", size_of::<Storage>() / size_of::<i32>());
}
playground
However this requires knowing the type of items stored in the array. I don't know of a way to get that without instantiating a variable.
Just for fun:
use std::mem;
use std::ops::Deref;
fn main() {
assert_eq!(5, num_elems::<[i32; 5]>());
}
fn num_elems<T>() -> usize
where
T: 'static,
&'static T: IntoIterator,
<&'static T as IntoIterator>::Item: Deref,
<<&'static T as IntoIterator>::Item as Deref>::Target: Sized,
{
fn inner<S, I>() -> usize
where
I: Deref,
<I as Deref>::Target: Sized,
{
mem::size_of::<S>() / mem::size_of::<I::Target>()
}
inner::<T, <&'static T as IntoIterator>::Item>()
}
This will work for any array up to 32 elements, and will panic if the array element type is zero-sized. Also, you can use other things, besides array types, and I have no idea what it will do.
You can use mem::size_of:
let storage_len = std::mem::size_of::<[i32; 3]>() / std::mem::size_of::<i32>();
Arrays coerce to slices, so any method available on slices are also available on arrays. Like len():
let v = [0u32; 128];
assert_eq!(128, v.len());

How do I create and initialize an immutable array?

I want to create an array. I don't need the array to be mutable, and at the time of creation, I have all the information I need to calculate the i-th member of the array. However, can't figure out how to create an immutable array in Rust.
Here's what I have now:
let mut my_array: [f32; 4] = [0.0; 4];
for i in 0..4 {
// some calculation, doesn't matter what exactly
my_array[i] = some_function(i);
}
And here's what I want:
let my_array: [f32; 4] = array_factory!(4, some_function);
How can I achieve that in Rust?
Here's the macro definition with sample usage:
macro_rules! array_factory(
($size: expr, $factory: expr) => ({
unsafe fn get_item_ptr<T>(slice: *mut [T], index: usize) -> *mut T {
(slice as *mut T).offset(index as isize)
}
let mut arr = ::std::mem::MaybeUninit::<[_; $size]>::uninit();
unsafe {
for i in 0..$size {
::std::ptr::write(get_item_ptr(arr.as_mut_ptr(), i), $factory(i));
}
arr.assume_init()
}
});
);
fn some_function(i: usize) -> f32 {
i as f32 * 3.125
}
fn main() {
let my_array: [f32; 4] = array_factory!(4, some_function);
println!("{} {} {} {}", my_array[0], my_array[1], my_array[2], my_array[3]);
}
The macro's body is essentially your first version, but with a few changes:
The type annotation on the array variable is omitted, because it can be inferred.
The array is created uninitialized, because we're going to overwrite all values immediately anyway. Messing with uninitialized memory is unsafe, so we must operate on it from within an unsafe block. Here, we're using MaybeUninit, which was introduced in Rust 1.36 to replace mem::uninitialized1.
Items are assigned using std::ptr::write() due to the fact that the array is uninitialized. Assignment would try to drop an uninitialized value in the array; the effects depend on the array item type (for types that implement Copy, like f32, it has no effect; for other types, it could crash).
The macro body is a block expression (i.e. it's wrapped in braces), and that block ends with an expression that is not followed by a semicolon, arr.assume_init(). The result of that block expression is therefore arr.assume_init().
Instead of using unsafe features, we can make a safe version of this macro; however, it requires that the array item type implements the Default trait. Note that we must use normal assignment here, to ensure that the default values in the array are properly dropped.
macro_rules! array_factory(
($size: expr, $factory: expr) => ({
let mut arr = [::std::default::Default::default(), ..$size];
for i in 0..$size {
arr[i] = $factory(i);
}
arr
});
)
1 And for a good reason. The previous version of this answer, which used mem::uninitialized, was not memory-safe: if a panic occurred while initializing the array (because the factory function panicked), and the array's item type had a destructor, the compiler would insert code to call the destructor on every item in the array; even the items that were not initialized yet! MaybeUninit avoids this problem because it wraps the value being initialized in ManuallyDrop, which is a magic type in Rust that prevents the destructor from running automatically.
Now, there is a (pretty popular) crate to do that exact thing: array_init
use array_init::array_init;
let my_array: [f32; 4] = array_init(some_function);
PS:
There is a lot of discussion and evolution around creating abstractions around arrays inside the rust team.
For example, the map function for arrays is already available, and it will become stable in rust 1.55.
If you wanted to, you could implement your function with map:
#![feature(array_map)]
let mut i = 0usize;
result = [(); 4].map(|_| {v = some_function(i);i = i+1; v})
And there are even discussions around your particular problem, you can look here
Try to make your macro expand to this:
let my_array = {
let mut tmp: [f32, ..4u] = [0.0, ..4u];
for i in range(0u, 4u) {
tmp[i] = somefunction(i);
}
tmp
};
What I don't know is whether this is properly optimized to avoid moving tmp to my_array. But for 4 f32 values (128 bits) it probably does not make a significant difference.

Resources