I'm trying to create a memoization function in Rust. The problem comes when getting a mutable reference for the cache HashMap. I'm not still confident with the type system and I'm struggling a bit.
use std::collections::HashMap;
use std::hash::Hash;
fn memoize<A, B, F>(f: F, cache: &'static HashMap<A, B>) -> impl Fn(A) -> B
where
A: Eq + Hash + Copy,
B: Clone,
F: Fn(A) -> B,
{
move |value: A| {
if !cache.contains_key(&value) {
cache.insert(value, f(value.clone()));
}
let res = cache.get(&value).unwrap();
res.clone()
}
}
The error is:
error[E0596]: cannot borrow immutable borrowed content `**cache` as mutable
--> src/lib.rs:12:13
|
12 | cache.insert(value, f(value.clone()));
| ^^^^^ cannot borrow as mutable
Why cannot a static lifetime parameter be mutable?
A variable is immutable by default in Rust, therefore you cannot mutate a variable that is not declared as mut. The 'static lifetime does not influence the mutability, but only how long the variable lives.
A Fn "[...] can be called repeatedly without mutating state.". And exactly here is the problem. You want to mutate the environment (in this case your HashMap).
You have to use a FnMut to be able to mutate the environment.
If you use the Entry API, you can simplify your code:
use std::collections::HashMap;
use std::hash::Hash;
fn memoize<A, B, F>(f: F, cache: &'static mut HashMap<A, B>) -> impl FnMut(A) -> B
where
A: Eq + Hash + Copy,
B: Clone,
F: Fn(A) -> B,
{
move |value: A| {
let res = cache.entry(value).or_insert_with(|| f(value));
res.clone()
}
}
As a sidenote, if you compile your code with #[feature(nll)] the error message is actually very good.
error[E0596]: cannot borrow `*cache` as mutable, as `Fn` closures cannot mutate their captured variables
--> src/lib.rs:14:13
|
14 | cache.insert(value, f(value.clone()));
| ^^^^^ cannot borrow as mutable
|
help: consider changing this to accept closures that implement `FnMut`
Related
I have an iterator in Rust that loops over a Vec<u8> and applies the same function at two different stages. I do this by chaining a couple of map functions together. Here is the relevant code (where example, example_function_1, and example_function_2 are stand-in variables and functions respectively):
NOTE: example.chunks() is a custom function! Not the default one on slices!
let example = vec![0, 1, 2, 3];
let mut hashers = Cycler::new([example_function_1, example_function_2].iter());
let ret: Vec<u8> = example
//...
.chunks(hashers.len())
.map(|buf| hashers.call(buf))
//...
.map(|chunk| hashers.call(chunk))
.collect();
Here is the code for Cycler:
pub struct Cycler<I> {
orig: I,
iter: I,
len: usize,
}
impl<I> Cycler<I>
where
I: Clone + Iterator,
I::Item: Fn(Vec<u8>) -> Vec<u8>,
{
pub fn new(iter: I) -> Self {
Self {
orig: iter.clone(),
len: iter.clone().count(),
iter,
}
}
pub fn len(&self) -> usize {
self.len
}
pub fn reset(&mut self) {
self.iter = self.orig.clone();
}
pub fn call(&mut self, buf: Bytes) -> Bytes {
// It is safe to unwrap because it should indefinietly continue without stopping
self.next().unwrap()(buf)
}
}
impl<I> Iterator for Cycler<I>
where
I: Clone + Iterator,
I::Item: Fn(Vec<u8>) -> Vec<u8>,
{
type Item = I::Item;
fn next(&mut self) -> Option<I::Item> {
match self.iter.next() {
next => next,
None => {
self.reset();
self.iter.next()
}
}
}
// No size_hint, try_fold, or fold methods
}
What confuses me is that the second time I reference hashers it says this:
error[E0499]: cannot borrow `hashers` as mutable more than once at a time
--> libpressurize/src/password/password.rs:28:14
|
21 | .map(|buf| hashers.call(buf))
| ----- ------- first borrow occurs due to use of `hashers` in closure
| |
| first mutable borrow occurs here
...
28 | .map(|chunk| hashers.call(chunk))
| --- ^^^^^^^ ------- second borrow occurs due to use of `hashers` in closure
| | |
| | second mutable borrow occurs here
| first borrow later used by call
Shouldn't this work because the mutable reference is not used at the same time?
Please let me know if more info/code is needed to answer this.
.map(|buf| hashers.call(buf))
You're probably thinking that in the above line, hashers is mutably borrowed to call it. That's true (since Cycler::call takes &mut self) but it's not what the compiler error is about. In this line, hashers is mutably borrowed to construct the closure |buf| hashers.call(buf), and that borrow lasts as long as the closure does.
Thus, when you write
.map(|buf| hashers.call(buf))
//...
.map(|chunk| hashers.call(chunk))
you are constructing two closures which live at the same time (assuming this is std::iter::Iterator::map) and mutably borrowing hashers for each of them, which is not allowed.
This error is actually protecting you against a side-effect hazard: it's not obvious (in a purely local analysis) what order the side effects of the two call()s will be performed in, because the map()s could do anything they like with the closures. Given the code you wrote, I assume you're doing this on purpose, but the compiler doesn't know that you know what you're doing.
(We can't even predict what the interleaving will be just because they're iterators. Inside of your //... there could be, say a .filter() step which leads to hashers.call(buf) being called several times between each call to hashers.call(chunk), or something else that produces a different number of outputs than inputs.)
If you know that you want the interleaving of side-effects that is “whenever either map() decides to call it”, then you can gain that freedom with a RefCell or other interior mutability, as dianhenglau's answer demonstrates.
Shouldn't this work because the mutable reference is not used at the same time?
No. The rules of references stated that "At any given time, you can have either one mutable reference or any number of immutable references", no matter if it is or isn't used at the same time. See this answer for the reason behind the rules.
As for workaround, since you're sure that the mutations do not occur simultaneously, you can use std::cell::RefCell as explained in this chapter. Modify the code into:
use std::cell::RefCell;
let example = vec![0, 1, 2, 3];
// Remove the "mut", wrap Cycler in RefCell.
let hashers = RefCell::new(Cycler::new([example_function_1, example_function_2].iter()));
let ret: Vec<u8> = example
//...
.chunks(hashers.borrow().len())
// Borrow hashers as immutable inside the closure, then borrow the Cycler as mutable.
.map(|buf| hashers.borrow_mut().call(buf))
//...
.map(|chunk| hashers.borrow_mut().call(chunk))
.collect();
But for example in the following code, I want to insert a value into a mutable BTreeMap/Hashmap in each loop, my simple solution triggered an error, what is the elegant way to do that in Rust?
Do I need clone the BTreeMap at the beginning of each loop?
pub fn build_string(word: &str, trie: &mut BTreeMap<&str, &Node>, word_index: usize) {
for char in word.chars() {
let char_label = char.to_string();
let insert_node = Node::new(&char_label, word_index);
let insert_result = trie.insert(&char_label, &insert_node);
println!("char: {}, at index", char);
}
}
error is
error[E0597]: `insert_node` does not live long enough
--> src/lib.rs:69:54
|
65 | pub fn build_string(word: &str, trie: &mut BTreeMap<&str, &Node>, word_index: usize) {
| - let's call the lifetime of this reference `'1`
...
69 | let insert_result = trie.insert(&char_label, &insert_node);
| -------------------------^^^^^^^^^^^^-
| | |
| | borrowed value does not live long enough
| argument requires that `insert_node` is borrowed for `'1`
70 | println!("char: {}, at index", char);
71 | }
| - `insert_node` dropped here while still borrowed
The issue is not with your BTreeMap but rather with the lifetime of your Node. You are creating a node inside the method build_string and trying to insert a reference to it into the map. However, once the method build_string goes out of scope, the local insert_node gets dropped but the reference would still exist in the tree. This is not allowed.
Therefore, you could change the BTreeMap to have values of Node instead of &Node. Then you would simply insert the insert_node into the tree which would take ownership of the node thus avoiding any reference leaks
I have a for loop that iterates over a slice of Point structs. The Points will have some fields modified in the loop, so the function containing the loop requires a mutable reference to the slice.
The problem arises when I need to pass a (immutable) reference pointing to the slice to a function within the for loop that iterates over the mutable reference:
#[derive(Debug)]
struct Point {
x: i32,
y: i32,
}
fn main() {
let mut grid = vec![];
grid.push(Point { x: 10, y: 10 });
grid.push(Point { x: -1, y: 7 });
calculate_neighbors(&mut grid);
}
fn calculate_neighbors(grid: &mut [Point]) {
for pt in grid.iter_mut() {
pt.x = nonsense_calc(grid);
}
}
#[allow(unused_variables)]
fn nonsense_calc(grid: &[Point]) -> i32 {
unimplemented!();
}
Playground
error[E0502]: cannot borrow `*grid` as immutable because it is also borrowed as mutable
--> src/main.rs:18:30
|
17 | for pt in grid.iter_mut() {
| ---------------
| |
| mutable borrow occurs here
| mutable borrow used here, in later iteration of loop
18 | pt.x = nonsense_calc(grid);
| ^^^^ immutable borrow occurs here
The compiler complains that grid cannot be borrowed as immutable, because a mutable borrow already exists. This is correct, and I can see the problem it is trying to prevent, but how do I achieve what I need to do? Ideally, I do not have to create a copy of the grid, as this can be expensive.
A solution to avoid borrow the array for the iteration would be to use indexes:
fn calculate_neighbors(grid: &mut [Point]) {
for i in 0..grid.len() {
grid[i].x = nonsense_calc(grid);
}
}
I am trying to extract a .tar.bz file (or .tar.whatever actually) and also be able to have a xx% progress report. So far I have this:
pub fn extract_file_with_progress<P: AsRef<Path>>(&self, path: P) -> Result<()> {
let path = path.as_ref();
let size = fs::metadata(path)?;
let mut f = File::open(path)?;
let decoder = BzDecoder::new(&f);
let mut archive = Archive::new(decoder);
for entry in archive.entries()? {
entry?.unpack_in(".")?;
let pos = f.seek(SeekFrom::Current(0))?;
}
Ok(())
}
The idea is to use pos/size to get the percentage, but compiling the above function gets me the error cannot borrow f as mutable because it is also borrowed as immutable.
I understand what the error means, but I don't really use f as mutable; I only use the seek function to get the current position.
Is there a way to work-around this, either by forcing the compiler to ignore the mutable borrow or by getting the position in some immutable way?
Files are a bit special. The usual read() and seek() and write() methods (defined on the Read, Seek and Write traits) take self by mutable reference:
fn read(&mut self, buf: &mut [u8]) -> Result<usize>
fn seek(&mut self, pos: SeekFrom) -> Result<u64>
fn write(&mut self, buf: &[u8]) -> Result<usize>
However, all mentioned traits are also implemented for &File, i.e. for immutable references to a file:
impl<'a> Read for &'a File
impl<'a> Seek for &'a File
impl<'a> Write for &'a File
So you can modify a file even if you only have a read-only reference to the file. For these implementations, the Self type is &File, so accepting self by mutable reference in fact means accepting a &mut &File, a mutable reference to a reference to a file.
Your code passes &f to BzDecoder::new(), creating an immutable borrow. Later you call f.seek(SeekFrom::Current(0)), which passes f to seek by mutable reference. However, this is not allowed, since you already have an immutable borrow of the file. The solution is to use the Seek implementation on &File instead:
(&mut &f).seek(SeekFrom::Current(0))
or slightly simpler
(&f).seek(SeekFrom::Current(0))
This only creates a second immutable borrow, which is allowed by Rust's rules for references.
I created a playground example demonstrating that this works. If you replace (&f) with f you get the error you originally got.
Where is the 'static lifetime requirement coming from when using a trait type in std::rc::Rc and how can I prevent it? E.g. when trying to compile this code
trait Handler{}
fn add(a: std::rc::Rc<Handler>) {
}
fn main() {
add(0);
}
rust reports
error[E0308]: mismatched types
--> test.rs:7:9
...
= note: expected type `std::rc::Rc<Handler + 'static>`
found type `{integer}`
NOTE: the error itself is expected; I am just interested in the Handler + 'static diagnostic output. The Real program creates instances of a trait type, stores them into a HashMap and runs a type specific function on it. It fails to compile with
| - borrowed value only lives until here
|
= note: borrowed value must be valid for the static lifetime...
at this place.
Second example
The following code is more real-world and demonstrates the issue perhaps better:
trait Handler {
}
struct SomeHandler<'a>(&'a u32);
impl <'a> SomeHandler<'a> {
fn new(a: &'a u32) -> std::rc::Rc<Handler> {
std::rc::Rc::new(SomeHandler(a))
}
}
impl <'a> Handler for SomeHandler<'a> {
}
fn main() {
let a: u32;
SomeHandler::new(&a);
}
It fails with
8 | std::rc::Rc::new(SomeHandler(a))
| ^
= note: but, the lifetime must be valid for the static lifetime...
= note: ...so that the expression is assignable:
expected std::rc::Rc<Handler + 'static>
found std::rc::Rc<Handler>
Why explicit lifetimes do not work
The simple demo might be fixed by adding an explicit lifetime (e.g. Rc<Handler + 'a>). Unfortunately, this is not an option (nor trying to make anything 'static) because real code is intended to look like
struct PollRegistry {
...
handlers: std::collections::HashMap<mio::Token, std::rc::Weak<PollHandler>>,
}
impl PollRegistry {
fn register<'a>(&mut self, handler: &std::rc::Rc<PollHandler>,
interest: mio::Ready, opts: mio::PollOpt)
-> std::io::Result<()> {
....
self.handlers.insert(token, std::rc::Rc::downgrade(handler));
}
}
and methods in PollHandler specializations create and own other PollHandler specializations which are registered in the registry by these methods.
rustc 1.27.1