Related
I've just completed a CodeWars kata on vowel-counting in C & Rust. The sample code is easy & obvious. Arrays can be used as fast mappings. Here, I map characters to logical values (0 or 1).
C implementation:
#include <stddef.h>
#include <stdint.h>
// :) This const array design looks smart & IMO-readable. Use of C99 feature.
const uint8_t areVowels[256]= {['a']=1, ['e']=1, ['i']=1, ['o']=1, ['u']=1};
size_t get_count(const unsigned char *s)
{
auto size_t count= 0;
for (;*s!='\0';s++){
count+= areVowels[*s];
}
return count;
}
Rust implementation:
// :( Here is me pain. Unreadable, python3-generated array content.
const ARE_VOWELS:[u8;256]= [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0];
fn get_count(s: &str) -> usize {
let mut vowels_count: usize = 0;
for c in s.bytes(){
vowels_count+= ARE_VOWELS[c as usize] as usize;
}
return vowels_count;
}
I wish I knew an alternative for array designators in Rust, which are a useful C99 feature. Initialization of the same byte array is much more awkward in my Rust code.
static mut is unnecessary here (and also bad practice). You can assign the value of a constant to the value of a code block which is capable of being evaluated in a const context
const ARE_VOWELS: [u8;256] = {
let mut data = [0u8; 256];
data['a' as usize] = 1;
data['e' as usize] = 1;
data['i' as usize] = 1;
data['o' as usize] = 1;
data['u' as usize] = 1;
data
};
See this answer for prettying this up with a macro.
An alternative is a macro that provides the syntax you want, its actually not too hard to make a general purpose implementation:
macro_rules! array {
($def:expr; $len:expr; $([$idx:expr]=$val:expr),* $(,)?) => { {
let mut a = [$def; $len];
$(a[$idx] = $val;)*
a
} }
}
let data = array![0; 256; ['a' as usize]=1, ['e' as usize]=1];
You can of course move the as usize into the macro if you like.
I found some (unsafe ⇒ amusing) work around with "mut static value runtime initialization".
const VOWELS:&[u8;5]= b"aeiou";
static mut ARE_VOWELS:[u8;256]= [0;256];
fn init(){
unsafe{
let ptr: *mut u8= &mut ARE_VOWELS[0];
for b in VOWELS{
*(ptr.offset(*b as isize))= 1;
}
}
}
fn get_count(s: &str) -> usize {
init();
let mut vowels_count: usize = 0;
for c in s.bytes(){
unsafe{
vowels_count+= ARE_VOWELS[c as usize] as usize;
}
}
return vowels_count;
}
Although I have serious doubts about the performance & maintainability of such piece of code.
Alright I am very new to Rust and I'm making a modding tool port for my friend. I need to read in a gazitillion u32s from a file and I haven't found a simple solution for that. The function I came up with looks like this:
pub fn read_u32(filename: &String, offset: usize) -> u32{
let data: Vec<u8> = std::fs::read(filename).unwrap();
let sliced_data = &data[offset..(offset+4)];
let mut value: u32 = 0;
for i in 0..4{
value += (sliced_data[3-i] as u32) * u32::pow(256, i as u32);
}
return value;
}
Now, this seems to be very, very slow. Quite obviously. How could I improve this function, or more like, with what can I replace it?
(Note: The u32s all are big endian)
Solution:
pub fn read_u32(filename: &String, offset: usize) -> u32{
let mut file = std::fs::File::open(filename).unwrap();
let mut buffer = [0; 4];
file.seek(SeekFrom::Start(offset as u64)).unwrap();
file.read(&mut buffer[..]).unwrap() as u32;
return u32::from_be_bytes(buffer);
}
Thanks to #user1937198!
I am working on a project that involves reading different information from a file at different offsets.
Currently, I am using the following code:
// ------------------------ SECTORS PER CLUSTER ------------------------
// starts at 13
opened_file.seek(SeekFrom::Start(13)).unwrap();
let aux: &mut [u8] = &mut [0; 1];
let _buf = opened_file.read_exact(aux);
// ------------------------ RESERVED SECTORS ------------------------
// starts at 14
opened_file.seek(SeekFrom::Start(14)).unwrap();
let aux: &mut [u8] = &mut [0; 2];
let _buf = opened_file.read_exact(aux);
But as you can see, I need to create a new buffer of the size I want to read every time. I can't specify it directly as a parameter of the function.
I created a struct but I could not make a struct of all the different pieces of data I wanted. For example:
struct FileStruct {
a1: &mut [u8] &mut [0; 1],
a2: &mut [u8] &mut [0; 2],
}
Which are the types that are required for the read_exact method to work?
Is there a more effective way to read information from different offsets of a file without having to repeatedly copy-paste these lines of code for every piece of information I want to read from the file? Some sort of function, Cursor, or Vector to easily move around the offset? And a way to write this info into struct fields?
The easiest way is to have a struct of owned arrays, then seek and read into the struct.
use std::io::{self, prelude::*, SeekFrom};
#[derive(Debug, Clone, Default)]
struct FileStruct {
a1: [u8; 1],
a2: [u8; 2],
}
fn main() -> io::Result<()> {
let mut file_struct: FileStruct = Default::default();
let mut opened_file = unimplemented!(); // open file somehow
opened_file.seek(SeekFrom::Start(13))?;
opened_file.read_exact(&mut file_struct.a1)?;
opened_file.seek(SeekFrom::Start(14))?;
opened_file.read_exact(&mut file_struct.a2)?;
println!("{:?}", file_struct);
Ok(())
}
Playground link
This is still decently repetitive, so you can make a seek_read function to reduce the repetition:
use std::io::{self, prelude::*, SeekFrom};
#[derive(Debug, Clone, Default)]
struct FileStruct {
a1: [u8; 1],
a2: [u8; 2],
}
fn seek_read(mut reader: impl Read + Seek, offset: u64, buf: &mut [u8]) -> io::Result<()> {
reader.seek(SeekFrom::Start(offset))?;
reader.read_exact(buf)?;
Ok(())
}
fn main() -> io::Result<()> {
let mut file_struct: FileStruct = Default::default();
let mut opened_file = unimplemented!(); // open file somehow
seek_read(&mut opened_file, 13, &mut file_struct.a1)?;
seek_read(&mut opened_file, 14, &mut file_struct.a2)?;
println!("{:?}", file_struct);
Ok(())
}
Playground link
The repetition can be lowered even more by using a macro:
use std::io::{self, prelude::*, SeekFrom};
#[derive(Debug, Clone, Default)]
struct FileStruct {
a1: [u8; 1],
a2: [u8; 2],
}
macro_rules! read_offsets {
($file: ident, $file_struct: ident, []) => {};
($file: ident, $file_struct: ident, [$offset: expr => $field: ident $(, $offsets: expr => $fields: ident)*]) => {
$file.seek(SeekFrom::Start($offset))?;
$file.read_exact(&mut $file_struct.$field)?;
read_offsets!($file, $file_struct, [$($offsets => $fields),*]);
}
}
fn main() -> io::Result<()> {
let mut file_struct: FileStruct = Default::default();
let mut opened_file = unimplemented!(); // open file somehow
read_offsets!(opened_file, file_struct, [13 => a1, 14 => a2]);
println!("{:?}", file_struct);
Ok(())
}
Playground link
This is a complementary answer to Aplet123's: it's not quite clear that you must store the bytes as is into a structure, so you can also allocate one buffer (as a fixed-size array) and reuse it with the correctly sized slice e.g.
let mut buf = [0u8;16];
opened_file.read_exact(&mut buf[..4])?; // will read 4 bytes
// do thing with the first 4 bytes
opened_file.read_exact(&mut buf[..8])?; // will read 8 bytes this time
// etc...
You could also use the byteorder crate, which lets you directly read numbers or sequences of numbers. It basically just does the unrelying "create stack buffer of the right size; read; decode" for you.
That's especially useful because it looks a lot like "SECTORS PER CLUSTER" should be a u8 and "RESERVED SECTORS" should be a u16. With byteorder you can straight read_16() or read_u8().
Also building on Aplet123's answer, the following function seek_read doesn't require to know how many bytes to read at compile time, since it uses a Vector instead of a byte slice:
// Starting at `offset`, reads the `amount_to_read` from `reader`.
// Returns the bytes as a vector.
fn seek_read(
reader: &mut (impl Read + Seek),
offset: u64,
amount_to_read: usize,
) -> Result<Vec<u8>> {
// A buffer filled with as many zeros as we'll read with read_exact
let mut buf = vec![0; amount_to_read];
reader.seek(SeekFrom::Start(offset))?;
reader.read_exact(&mut buf)?;
Ok(buf)
}
Here are some tests to demonstrate how seek_read behaves:
use std::io::Cursor;
#[test]
fn seek_read_works() {
let bytes = b"Hello world!";
let mut reader = Cursor::new(bytes);
assert_eq!(seek_read(&mut reader, 0, 2).unwrap(), b"He");
assert_eq!(seek_read(&mut reader, 1, 4).unwrap(), b"ello");
assert_eq!(seek_read(&mut reader, 6, 5).unwrap(), b"world");
assert_eq!(seek_read(&mut reader, 2, 0).unwrap(), b"");
}
#[test]
#[should_panic(expected = "failed to fill whole buffer")]
fn seek_read_beyond_buffer_fails() {
let mut reader = Cursor::new(b"Hello world!");
seek_read(&mut reader, 6, 99).unwrap();
}
#[test]
#[should_panic(expected = "failed to fill whole buffer")]
fn start_seek_reading_beyond_buffer_fails() {
let mut reader = Cursor::new(b"Hello world!");
seek_read(&mut reader, 99, 1).unwrap();
}
fn change(a: &mut i32, b: &mut i32) {
let c = *a;
*a = *b;
*b = c;
}
fn main() {
let mut v = vec![1, 2, 3];
change(&mut v[0], &mut v[1]);
}
When I compile the code above, it has the error:
error[E0499]: cannot borrow `v` as mutable more than once at a time
--> src/main.rs:9:32
|
9 | change(&mut v[0], &mut v[1]);
| - ^ - first borrow ends here
| | |
| | second mutable borrow occurs here
| first mutable borrow occurs here
Why does the compiler prohibit it? v[0] and v[1] occupy different memory positions, so it's not dangerous to use these together. And what should I do if I come across this problem?
You can solve this with split_at_mut():
let mut v = vec![1, 2, 3];
let (a, b) = v.split_at_mut(1); // Returns (&mut [1], &mut [2, 3])
change(&mut a[0], &mut b[0]);
There are uncountably many safe things to do that the compiler unfortunately does not recognize yet. split_at_mut() is just like that, a safe abstraction implemented with an unsafe block internally.
We can do that too, for this problem. The following is something I use in code where I need to separate all three cases anyway (I: Index out of bounds, II: Indices equal, III: Separate indices).
enum Pair<T> {
Both(T, T),
One(T),
None,
}
fn index_twice<T>(slc: &mut [T], a: usize, b: usize) -> Pair<&mut T> {
if a == b {
slc.get_mut(a).map_or(Pair::None, Pair::One)
} else {
if a >= slc.len() || b >= slc.len() {
Pair::None
} else {
// safe because a, b are in bounds and distinct
unsafe {
let ar = &mut *(slc.get_unchecked_mut(a) as *mut _);
let br = &mut *(slc.get_unchecked_mut(b) as *mut _);
Pair::Both(ar, br)
}
}
}
}
Since Rust 1.26, pattern matching can be done on slices. You can use that as long as you don't have huge indices and your indices are known at compile-time.
fn change(a: &mut i32, b: &mut i32) {
let c = *a;
*a = *b;
*b = c;
}
fn main() {
let mut arr = [5, 6, 7, 8];
{
let [ref mut a, _, ref mut b, ..] = arr;
change(a, b);
}
assert_eq!(arr, [7, 6, 5, 8]);
}
The borrow rules of Rust need to be checked at compilation time, that is why something like mutably borrowing a part of a Vec is a very hard problem to solve (if not impossible), and why it is not possible with Rust.
Thus, when you do something like &mut v[i], it will mutably borrow the entire vector.
Imagine I did something like
let guard = something(&mut v[i]);
do_something_else(&mut v[j]);
guard.do_job();
Here, I create an object guard that internally stores a mutable reference to v[i], and will do something with it when I call do_job().
In the meantime, I did something that changed v[j]. guard holds a mutable reference that is supposed to guarantee nothing else can modify v[i]. In this case, all is good, as long as i is different from j; if the two values are equal it is a huge violation of the borrow rules.
As the compiler cannot guarantee that i != j, it is thus forbidden.
This was a simple example, but similar cases are legions, and are why such access mutably borrows the whole container. Plus the fact that the compiler actually does not know enough about the internals of Vec to ensure that this operation is safe even if i != j.
In your precise case, you can have a look at the swap(..) method available on Vec that does the swap you are manually implementing.
On a more generic case, you'll probably need an other container. Possibilities are wrapping all the values of your Vec into a type with interior mutability, such as Cell or RefCell, or even using a completely different container, as #llogiq suggested in his answer with par-vec.
The method [T]::iter_mut() returns an iterator that can yield a mutable reference for each element in the slice. Other collections have an iter_mut method too. These methods often encapsulate unsafe code, but their interface is totally safe.
Here's a general purpose extension trait that adds a method on slices that returns mutable references to two distinct items by index:
pub trait SliceExt {
type Item;
fn get_two_mut(&mut self, index0: usize, index1: usize) -> (&mut Self::Item, &mut Self::Item);
}
impl<T> SliceExt for [T] {
type Item = T;
fn get_two_mut(&mut self, index0: usize, index1: usize) -> (&mut Self::Item, &mut Self::Item) {
match index0.cmp(&index1) {
Ordering::Less => {
let mut iter = self.iter_mut();
let item0 = iter.nth(index0).unwrap();
let item1 = iter.nth(index1 - index0 - 1).unwrap();
(item0, item1)
}
Ordering::Equal => panic!("[T]::get_two_mut(): received same index twice ({})", index0),
Ordering::Greater => {
let mut iter = self.iter_mut();
let item1 = iter.nth(index1).unwrap();
let item0 = iter.nth(index0 - index1 - 1).unwrap();
(item0, item1)
}
}
}
}
On recent nightlies, there is get_many_mut():
#![feature(get_many_mut)]
fn main() {
let mut v = vec![1, 2, 3];
let [a, b] = v
.get_many_mut([0, 1])
.expect("out of bounds or overlapping indices");
change(a, b);
}
Building up on the answer by #bluss you can use split_at_mut() to create a function that can turn mutable borrow of a vector into a vector of mutable borrows of vector elements:
fn borrow_mut_elementwise<'a, T>(v:&'a mut Vec<T>) -> Vec<&'a mut T> {
let mut result:Vec<&mut T> = Vec::new();
let mut current: &mut [T];
let mut rest = &mut v[..];
while rest.len() > 0 {
(current, rest) = rest.split_at_mut(1);
result.push(&mut current[0]);
}
result
}
Then you can use it to get a binding that lets you mutate many items of original Vec at once, even while you are iterating over them (if you access them by index in your loop, not through any iterator):
let mut items = vec![1,2,3];
let mut items_mut = borrow_mut_elementwise(&mut items);
for i in 1..items_mut.len() {
*items_mut[i-1] = *items_mut[i];
}
println!("{:?}", items); // [2, 3, 3]
The problem is that &mut v[…] first mutably borrows v and then gives the mutable reference to the element to the change-function.
This reddit comment has a solution to your problem.
Edit: Thanks for the heads-up, Shepmaster. par-vec is a library that allows to mutably borrow disjunct partitions of a vec.
I publish my daily utils for this to crate.io. Link to the doc.
You may use it like
use arref::array_mut_ref;
let mut arr = vec![1, 2, 3, 4];
let (a, b) = array_mut_ref!(&mut arr, [1, 2]);
assert_eq!(*a, 2);
assert_eq!(*b, 3);
let (a, b, c) = array_mut_ref!(&mut arr, [1, 2, 0]);
assert_eq!(*c, 1);
// ⚠️ The following code will panic. Because we borrow the same element twice.
// let (a, b) = array_mut_ref!(&mut arr, [1, 1]);
It's a simple wrapper around the following code, which is sound. But it requires that the two indexes are different at runtime.
pub fn array_mut_ref<T>(arr: &mut [T], a0: usize, a1: usize) -> (&mut T, &mut T) {
assert!(a0 != a1);
// SAFETY: this is safe because we know a0 != a1
unsafe {
(
&mut *(&mut arr[a0] as *mut _),
&mut *(&mut arr[a1] as *mut _),
)
}
}
Alternatively, you may use a method that won't panic with mut_twice
#[inline]
pub fn mut_twice<T>(arr: &mut [T], a0: usize, a1: usize) -> Result<(&mut T, &mut T), &mut T> {
if a0 == a1 {
Err(&mut arr[a0])
} else {
unsafe {
Ok((
&mut *(&mut arr[a0] as *mut _),
&mut *(&mut arr[a1] as *mut _),
))
}
}
}
in the code below the creation of the string buffer
is a quickest way i have found as there's no allocation deallocation done
if i understand correctly
pub extern fn rust_print_file() -> *mut PackChar {
//set min size to 50 - avoid expanding when line count is 50 or less
let mut out_vec = Vec::with_capacity(50);
let mut curdr = env::current_dir().unwrap();//get path to file dir
let fl_str = "file_test.txt";
curdr.push(fl_str);//created full path to be used
let file = BufReader::new(File::open(curdr).unwrap());
//here i try to accommodate each line in a struct
let mut line_index = 0;
for line in file.lines() {
let cur_line = line.unwrap();
let loclbuf_size = cur_line.len();
let mut loclbuf = String::with_capacity(buffer_size);
//i tried two ways
loclbuf.push_str(cur_line.unwrap()); // can't be done
loclbuf.push_str(line.unwrap()); // can't be done too
let pack_char = PackChar {
int_val: line_index,
buffer_size: loclbuf_size as i32,
buffer: loclbuf.as_ptr() as *mut _,
};
line_index+=1;
mem::forget(buffer);
out_vec.push(pack_char);
}
Box::into_raw(out_vec.into_boxed_slice()) as *mut _
}
this is the struct i am using to pass to C#
#[repr(C)]
pub struct PackChar {
pub int_val: c_int,
pub buffer: *mut c_char,
pub buffer_size: c_int,
}
when generating some dummy text
i have checked, and it passes data correctly to "the other side" to use it.
but not with the read line task, produced text as coded above.
and this is another way i have tried, although i prefer the above code but this one throws a compile error.
error: use of moved value: buffer [E0382] on forget(buffer)
#[no_mangle]
pub extern fn rust_return_file_read_lines() -> *mut PackChar {
let mut out_vec = Vec::with_capacity(50);
let mut curdr = env::current_dir().unwrap();
let fl_str = "file_test.txt";
curdr.push(fl_str);
let file = BufReader::new(File::open(curdr).unwrap());
let mut lindex = 0;
for line in file.lines() {
let tmpbuffer = line.unwrap().into_bytes();
let tmpbuffer_size = buffer.len();
let pack_char = PackChar {
int_val: lindex,
buffer_size: tmpbuffer_size as i32,
buffer: Box::into_raw(tmpbuffer.into_boxed_slice()) as *mut _
};
lindex+=1;
mem::forget(buffer);
out_vec.push(pack_char);
}
Box::into_raw(out_vec.into_boxed_slice()) as *mut _
}
Edit
as long as the type of buffer :
buffer: loclbuf.as_ptr() as *mut _,
i could pass the the data correctly to c#.
so how could i read lines that way so each will be stored into buffer as described ?
as it appear now it seem that there was a bug in my visual studio, it's not the first time it ever happen but as i am new to rust i was sure that the code is wrong .
this is what is working for me, i will be happy to have comments and suggestions
extern crate libc;
use std::env;
use libc::c_char;
use libc::c_int;
use std::mem;
use std::io::{BufReader, BufRead};
use std::fs::File;
#[repr(C)]
pub struct PackChar {
pub int_val: c_int,
pub buffer: *mut c_char, // changed
pub dbuffer_size: c_int, // added
}
#[no_mangle]
pub extern fn rust_print_file() -> *mut PackChar {
let mut out_vec = Vec::with_capacity(50 as usize);
let mut cwd = env::current_dir().unwrap();
let fl_str = "file_test.txt";
cwd.push(fl_str);
let file = BufReader::new(File::open(cwd).unwrap());
for (index, line) in file.lines().enumerate() {
let buffer = line.unwrap();
let buffer_size = buffer.len();
let pack_char = PackChar {
int_val: index as i32,
dbuffer_size: buffer_size as i32,
buffer: buffer.as_ptr() as *mut _,
};
mem::forget(buffer); // don't deallocate memory
out_vec.push(pack_char);
}
Box::into_raw(out_vec.into_boxed_slice()) as *mut _ // changed
}