Just a question so I can better understand Rust.
Why does this fail:
for event in rulesengine::RuleEvent::into_enum_iter()
{
let mut event_stats: payloads::EventStats;
event_stats.name = rulesengine::RuleEvent::to_string(&event);
event_stats.event_count = event_counters.event_count[event as usize];
event_stats.event_participant_count = event_counters.event_participant_count[event as usize];
event_stats.event_trigger_count = event_counters.event_trigger_count[event as usize];
rules_stats.event_stats.push(event_stats);
}
With:
error[E0382]: assign to part of moved value: `event_stats`
--> src/motion.rs:317:37
|
316 | ... let mut event_stats: payloads::EventStats;
| --------------- move occurs because `event_stats` has type `payloads::EventStats`, which does not implement the `Copy` trait
317 | ... event_stats.name = rulesengine::RuleEvent::to_string(&event).clone();
| ^^^^^^^^^^^^^^^^ value partially assigned here after move
...
323 | ... },
| - value moved here, in previous iteration of loop
error: aborting due to previous error
When this works:
for event in rulesengine::RuleEvent::into_enum_iter()
{
let name = rulesengine::RuleEvent::to_string(&event);
let event_count = event_counters.event_count[event as usize];
let event_participant_count = event_counters.event_participant_count[event as usize];
let event_trigger_count = event_counters.event_trigger_count[event as usize];
let event_stats = payloads::EventStats { name, event_count, event_participant_count, event_trigger_count };
rules_stats.event_stats.push(event_stats);
}
or even better
for event in rulesengine::RuleEvent::into_enum_iter()
{
let event_stats = payloads::EventStats
{
name: rulesengine::RuleEvent::to_string(&event),
event_count: event_counters.event_count[event as usize],
event_participant_count: event_counters.event_participant_count[event as usize],
event_trigger_count: event_counters.event_trigger_count[event as usize],
};
rules_stats.event_stats.push(event_stats);
}
The variable event_stats is mutable yes, but local. It doesn't exist in the "previous iteration of loop" to be moved.
I believe the error is a mis-diagnosis of the problem. There have been compiler bugs in the past where the message incorrectly blames the loop for uninitialized variables. Perhaps this is another one of those cases.
The core issue is you are trying to assign a field to an uninitialized value:
let mut event_stats: payloads::EventStats;
event_stats.name = ...;
You have declared event_stats but have not initialized it. Rust does not allow you to piece-wise initialize a struct, at least not without a lot of extra juggling.
Both your working solutions create event_stats from a struct literal:
let event_stats = payloads::EventStats { name: ..., ... };
which is the correct way to initialize a struct.
Related
I have an iterator in Rust that loops over a Vec<u8> and applies the same function at two different stages. I do this by chaining a couple of map functions together. Here is the relevant code (where example, example_function_1, and example_function_2 are stand-in variables and functions respectively):
NOTE: example.chunks() is a custom function! Not the default one on slices!
let example = vec![0, 1, 2, 3];
let mut hashers = Cycler::new([example_function_1, example_function_2].iter());
let ret: Vec<u8> = example
//...
.chunks(hashers.len())
.map(|buf| hashers.call(buf))
//...
.map(|chunk| hashers.call(chunk))
.collect();
Here is the code for Cycler:
pub struct Cycler<I> {
orig: I,
iter: I,
len: usize,
}
impl<I> Cycler<I>
where
I: Clone + Iterator,
I::Item: Fn(Vec<u8>) -> Vec<u8>,
{
pub fn new(iter: I) -> Self {
Self {
orig: iter.clone(),
len: iter.clone().count(),
iter,
}
}
pub fn len(&self) -> usize {
self.len
}
pub fn reset(&mut self) {
self.iter = self.orig.clone();
}
pub fn call(&mut self, buf: Bytes) -> Bytes {
// It is safe to unwrap because it should indefinietly continue without stopping
self.next().unwrap()(buf)
}
}
impl<I> Iterator for Cycler<I>
where
I: Clone + Iterator,
I::Item: Fn(Vec<u8>) -> Vec<u8>,
{
type Item = I::Item;
fn next(&mut self) -> Option<I::Item> {
match self.iter.next() {
next => next,
None => {
self.reset();
self.iter.next()
}
}
}
// No size_hint, try_fold, or fold methods
}
What confuses me is that the second time I reference hashers it says this:
error[E0499]: cannot borrow `hashers` as mutable more than once at a time
--> libpressurize/src/password/password.rs:28:14
|
21 | .map(|buf| hashers.call(buf))
| ----- ------- first borrow occurs due to use of `hashers` in closure
| |
| first mutable borrow occurs here
...
28 | .map(|chunk| hashers.call(chunk))
| --- ^^^^^^^ ------- second borrow occurs due to use of `hashers` in closure
| | |
| | second mutable borrow occurs here
| first borrow later used by call
Shouldn't this work because the mutable reference is not used at the same time?
Please let me know if more info/code is needed to answer this.
.map(|buf| hashers.call(buf))
You're probably thinking that in the above line, hashers is mutably borrowed to call it. That's true (since Cycler::call takes &mut self) but it's not what the compiler error is about. In this line, hashers is mutably borrowed to construct the closure |buf| hashers.call(buf), and that borrow lasts as long as the closure does.
Thus, when you write
.map(|buf| hashers.call(buf))
//...
.map(|chunk| hashers.call(chunk))
you are constructing two closures which live at the same time (assuming this is std::iter::Iterator::map) and mutably borrowing hashers for each of them, which is not allowed.
This error is actually protecting you against a side-effect hazard: it's not obvious (in a purely local analysis) what order the side effects of the two call()s will be performed in, because the map()s could do anything they like with the closures. Given the code you wrote, I assume you're doing this on purpose, but the compiler doesn't know that you know what you're doing.
(We can't even predict what the interleaving will be just because they're iterators. Inside of your //... there could be, say a .filter() step which leads to hashers.call(buf) being called several times between each call to hashers.call(chunk), or something else that produces a different number of outputs than inputs.)
If you know that you want the interleaving of side-effects that is “whenever either map() decides to call it”, then you can gain that freedom with a RefCell or other interior mutability, as dianhenglau's answer demonstrates.
Shouldn't this work because the mutable reference is not used at the same time?
No. The rules of references stated that "At any given time, you can have either one mutable reference or any number of immutable references", no matter if it is or isn't used at the same time. See this answer for the reason behind the rules.
As for workaround, since you're sure that the mutations do not occur simultaneously, you can use std::cell::RefCell as explained in this chapter. Modify the code into:
use std::cell::RefCell;
let example = vec![0, 1, 2, 3];
// Remove the "mut", wrap Cycler in RefCell.
let hashers = RefCell::new(Cycler::new([example_function_1, example_function_2].iter()));
let ret: Vec<u8> = example
//...
.chunks(hashers.borrow().len())
// Borrow hashers as immutable inside the closure, then borrow the Cycler as mutable.
.map(|buf| hashers.borrow_mut().call(buf))
//...
.map(|chunk| hashers.borrow_mut().call(chunk))
.collect();
I am trying to remove all the parentheses in a string. Not thinking about it too hard, I just do a simple regexp replace (i.e. the problem in question is not particularly about getting rid of arbitrary levels of nested parentheses, but feel free to suggest a better way of doing that in a comment if you want).
use regex::Regex;
fn main() -> Result<(), Box<dyn std::error::Error>> {
let input = "Text (with some (nested) parentheses)!";
let re = Regex::new(r"\([^()]*\)")?;
let output = re.replace_all(&input, "");
let output = re.replace_all(&output, "");
// let output = re.replace_all(&output, "");
// let output = re.replace_all(&output, "");
// let output = re.replace_all(&output, "");
// let output = re.replace_all(&output, "");
// ...
assert_eq!("Text !", output);
println!("Works!");
Ok(())
}
Because I do not know how nested the parentheses will be, I need to do the replacement in a loop rather than repeating it "just enough times". Creating a loop, however, creates a new scope and that's where I'm hitting a dead point in the discussion with the borrow checker.
The simplest case that shows what I am trying to do in the loop would be:
let mut output = re.replace_all(&input, "");
while re.is_match(&output) {
output = re.replace_all(&output, "");
}
However that cannot be done because I am assigning to a borrowed variable:
error[E0506]: cannot assign to `output` because it is borrowed
--> src/main.rs:9:9
|
9 | output = re.replace_all(&output, "");
| ^^^^^^ ------- borrow of `output` occurs here
| |
| assignment to borrowed `output` occurs here
| borrow later used here
What I would like to do, ideally, is to create new variable binding with the same name, but using let output = will shadow the outer variable binding, so the loop would cycle infinitely.
No matter what inner or outer temporary variable I create I cannot make it do what I want. I also tried using the fact that re.replace_all() returns Cow and tried using .to_owned() and .to_string() in a couple of places, but that didn't help either.
Here's a link to a playground.
re.replace_all() returns Cow
This is the root of the problem. The compiler knows that the return value might reference output, but it will also replace output, causing output to be dropped right away. If it allowed this, the reference would point to unallocated memory, leading to memory unsafety.
The solution is to avoid borrowing at all.
tried using .to_owned()
to_owned on a Cow just returns the same Cow. Perhaps you meant into_owned?
let mut output = re.replace_all(&input, "").into_owned();
while re.is_match(&output) {
output = re.replace_all(&output, "").into_owned();
}
and .to_string() in a couple of places
This works as well:
let mut output = re.replace_all(&input, "").to_string();
while re.is_match(&output) {
output = re.replace_all(&output, "").to_string();
}
Shepmaster's answer works, but it's not as efficient as it could be. A subtle property of the Cow type is that by inspecting it, we can determine whether the string was modified, and skip work if it wasn't.
Due to constraints of the Rust type system, if the value was not modified then Cow::into_owned() makes a copy. (Cow::into_owned() of a modified value does not copy). (into_owned documentation)
In your use case, we can detect unmodified Cow -- Cow::Borrowed -- and skip into_owned().
let mut output = /* mutable String */;
while re.is_match(&output).unwrap() {
match re.replace_all(&output, "") {
// Unmodified -- skip copy
Cow::Borrowed(_) => {}
// replace_all() returned a new value that we already own
Cow::Owned(new) => output = new,
}
}
But we can go further. Calling both is_match() and replace_all() means the pattern is matched twice. With our new knowledge of Cows, we can optimize that away:
let mut output = /* mutable String */;
// Cow::Owned is returned when the string was modified.
while let Cow::Owned(new) = re.replace_all(&output, "") {
output = new;
}
Edit: If your input value is immutable, you can avoid the .to_string() copy by making it Cow as well:
let input = "value";
let mut output = Cow::from(input);
while let Cow::Owned(new) = re.replace_all(&output, "") {
output = Cow::Owned(new);
}
I'm trying to implement a commonly used pattern - using the result of a previous loop iteration in the next loop iteration. For example, to implement pagination where you need to give the id of the last value on the previous page.
struct Result {
str: String,
}
fn main() {
let times = 10;
let mut last: Option<&str> = None;
for i in 0..times {
let current = do_something(last);
last = match current {
Some(r) => Some(&r.str.to_owned()),
None => None,
};
}
}
fn do_something(o: Option<&str>) -> Option<Result> {
Some(Result {
str: "whatever string".to_string(),
})
}
However, I'm not sure how to actually get the value out of the loop. Currently, the compiler error is temporary value dropped while borrowed (at &r.str.to_owned()), though I made many other attempts, but to no avail.
The only way I found to actually get it working is to create some sort of local tmp_str variable and do a hack like this:
match current {
Some(r) => {
tmp_str.clone_from(&r.str);
last = Some(&tmp_str);
}
None => {
last = None;
}
}
But that doesn't feel like it's the way it's supposed to be done.
In your code, it remains unclear who the owner of the String referenced in last: Option<&str> is supposed to be. You could introduce an extra mutable local variable that owns the string. But then you would have two variables: the owner and the reference, which seems redundant. It would be much simpler to just make last the owner:
struct MyRes {
str: String,
}
fn main() {
let times = 10;
let mut last: Option<String> = None;
for _i in 0..times {
last = do_something(&last).map(|r| r.str);
}
}
fn do_something(_o: &Option<String>) -> Option<MyRes> {
Some(MyRes {
str: "whatever string".to_string(),
})
}
In do_something, you can just pass the whole argument by reference, this seems more likely to be what you wanted. Also note that naming your own struct Result is a bad idea, because Result is such a pervasive trait built deeply into the compiler (?-operator etc).
Follow-up question: Option<&str> or Option<String>?
Both Option<&str> and Option<String> have different trade-offs. One is better for passing string literals, other is better for passing owned Strings. I'd actually propose to use neither, and instead make the function generic over type S that implements AsRef<str>. Here is a comparison of various methods:
fn do_something(o: &Option<String>) {
let _a: Option<&str> = o.as_ref().map(|r| &**r);
let _b: Option<String> = o.clone();
}
fn do_something2(o: &Option<&str>) {
let _a: Option<&str> = o.clone(); // do you need it?
let _b: Option<String> = o.map(|r| r.to_string());
}
fn do_something3<S: AsRef<str>>(o: &Option<S>) {
let _a: Option<&str> = o.as_ref().map(|s| s.as_ref());
let _b: Option<String> = o.as_ref().map(|r| r.as_ref().to_string());
}
fn main() {
let x: Option<String> = None;
let y: Option<&str> = None;
do_something(&x); // nice
do_something(&y.map(|r| r.to_string())); // awkward & expensive
do_something2(&x.as_ref().map(|x| &**x)); // cheap but awkward
do_something2(&y); // nice
do_something3(&x); // nice
do_something3(&y); // nice, in both cases
}
Note that not all of the above combinations are very idiomatic, some are added just for completeness (e.g. asking for AsRef<str> and then building an owned String out of seems a bit strange).
r.str.to_owned() is a temporary value. You can take a reference to a temporary, but because the temporary value will usually be dropped (destroyed) at the end of the innermost enclosing statement, the reference becomes dangling at that point. In this case the "innermost enclosing statement" is either the last line of the loop, or the loop body itself -- I'm not sure exactly which one applies here, but it doesn't matter, because either way, you're trying to make last contain a reference to a String that will soon be dropped, making last unusable. The compiler is right to stop you from using it again in the next iteration of the loop.
The easiest fix is just to not make last a reference at all -- in the example, it's not necessary or desirable. Just use Option<String>:
fn main() {
let times = 10;
let mut last = None;
for _ in 0..times {
last = match do_something(last) {
Some(r) => Some(r.str),
None => None,
};
}
}
fn do_something(_: Option<String>) -> Option<Result> {
// ...
}
There are also ways to make the reference version work; here is one:
let mut current; // lift this declaration out of the loop so `current` will have
// a lifetime longer than one iteration
for _ in 0..times {
current = do_something(last);
last = match current {
Some(ref r) => Some(&r.str), // borrow from `current` in the loop instead
// of from a newly created String
None => None,
};
}
You might want to do this if your code is more complicated than the example and using String would mean a lot of potentially expensive .clone()s.
I have a for loop that iterates over a slice of Point structs. The Points will have some fields modified in the loop, so the function containing the loop requires a mutable reference to the slice.
The problem arises when I need to pass a (immutable) reference pointing to the slice to a function within the for loop that iterates over the mutable reference:
#[derive(Debug)]
struct Point {
x: i32,
y: i32,
}
fn main() {
let mut grid = vec![];
grid.push(Point { x: 10, y: 10 });
grid.push(Point { x: -1, y: 7 });
calculate_neighbors(&mut grid);
}
fn calculate_neighbors(grid: &mut [Point]) {
for pt in grid.iter_mut() {
pt.x = nonsense_calc(grid);
}
}
#[allow(unused_variables)]
fn nonsense_calc(grid: &[Point]) -> i32 {
unimplemented!();
}
Playground
error[E0502]: cannot borrow `*grid` as immutable because it is also borrowed as mutable
--> src/main.rs:18:30
|
17 | for pt in grid.iter_mut() {
| ---------------
| |
| mutable borrow occurs here
| mutable borrow used here, in later iteration of loop
18 | pt.x = nonsense_calc(grid);
| ^^^^ immutable borrow occurs here
The compiler complains that grid cannot be borrowed as immutable, because a mutable borrow already exists. This is correct, and I can see the problem it is trying to prevent, but how do I achieve what I need to do? Ideally, I do not have to create a copy of the grid, as this can be expensive.
A solution to avoid borrow the array for the iteration would be to use indexes:
fn calculate_neighbors(grid: &mut [Point]) {
for i in 0..grid.len() {
grid[i].x = nonsense_calc(grid);
}
}
This question is motivated by this CodinGame puzzle.
I am implementing a basic pathfinding algorithm using Dijkstra's method. It uses a boundary HashMap and a finished HashMap to hold pathfinding-related node info. In a particular loop, I find the highest-valued node in boundary, remove the node, add the node to finished, and add/update the node's neighbors' info in boundary.
Attempting to mutate boundary while looping over it is making Rust's borrow checker queasy, but the logic of the loop seems sound to me. How do I rewrite it so that the compiler shares my confidence? (Or fix the errors I'm missing, if that's the issue.)
Code:
On Rust Playground here
use std::io;
use std::collections::{HashSet, HashMap};
use std::cmp::Ordering;
use std::cell::RefCell;
struct NodeInfo {
nbrs: HashSet<i32>,
gwlinks: i32,
}
#[derive(PartialEq,PartialOrd)]
struct PFInfo {
avg: f32,
cum: i32,
dist: i32,
prev: Option<i32>,
}
impl Eq for PFInfo {}
impl Ord for PFInfo {
fn cmp(&self, other: &PFInfo) -> Ordering {
match self.partial_cmp(other) {
Some(ord) => ord,
None => Ordering::Equal
}
}
}
type Graph = HashMap<i32, RefCell<NodeInfo>>;
type PFGraph = HashMap<i32, PFInfo>;
// Find the path that passes the most gateway links per distance traveled,
// starting at a given node. This is meant to simulate the behavior of an
// "agent" which traverses the graph in the puzzle mentioned above.
fn generate_path(si: &i32, graph: &Graph) -> Vec<i32> {
let n = graph.len();
let mut boundary = PFGraph::with_capacity(n);
let mut finished = PFGraph::with_capacity(n);
boundary.insert( si.clone(),
PFInfo {
avg: 0.,
cum: graph.get(&si).unwrap().borrow().gwlinks,
dist: 0,
prev: None } );
// Keep grabbing the key corresponding the highest value until `boundary` is
// empty
while let Some( (currid, _) ) = boundary.iter().max_by_key(|x| x.1) {
// Move the node from `boundary` to `finished`
let val = boundary.remove(&currid).unwrap();
finished.insert(currid.clone(), val);
// Add or update all adjacent nodes that are not in `finished`
for nbrid in graph.get(&currid).unwrap()
.borrow()
.nbrs.iter()
.filter(|x| !finished.contains_key(x)) {
let currval = finished.get(&currid).unwrap();
let prev = Some(currid.clone());
let dist = currval.dist + 1;
let cum = currval.cum + graph.get(nbrid).unwrap().borrow().gwlinks;
let avg = cum as f32 / dist as f32;
boundary.insert(
nbrid.clone(),
PFInfo {
avg: avg,
cum: cum,
dist: dist,
prev: prev,
}
);
}
}
let mut path = Vec::new();
let mut currid = finished.iter().max_by_key(|x| x.1).unwrap().0.clone();
path.push(currid.clone());
while let Some(previd) = finished.get(&currid).unwrap().prev {
path.push(previd.clone());
currid = previd.clone();
}
path.reverse();
path
}
macro_rules! parse_input {
($x:expr, $t:ident) => ($x.trim().parse::<$t>().unwrap())
}
#[test]
fn test_generate_path() {
let mut inputs = "8 13 2
6 2
7 3
6 3
5 3
3 4
7 1
2 0
0 1
0 3
1 3
2 3
7 4
6 5
4
5".lines();
let header = inputs.next().unwrap().split_whitespace().collect::<Vec<_>>();
let n = parse_input!(header[0], i32); // the total number of nodes in the level, including the gateways
let l = parse_input!(header[1], i32); // the number of links
let e = parse_input!(header[2], i32); // the number of exit gateways
let mut graph = Graph::with_capacity(n as usize);
for node in 0..n {
graph.insert(node, RefCell::new(NodeInfo{ nbrs: HashSet::new(), gwlinks: 0 }));
}
let graph = graph;
for _ in 0..l as usize {
let link = inputs.next().unwrap();
let nodes = link.split(" ").collect::<Vec<_>>();
let n1 = parse_input!(nodes[0], i32); // N1 and N2 defines a link between these nodes
let n2 = parse_input!(nodes[1], i32);
graph.get(&n1).unwrap().borrow_mut().nbrs.insert(n2);
graph.get(&n2).unwrap().borrow_mut().nbrs.insert(n1);
}
let mut gateways = HashSet::new();
for _ in 0..e as usize {
let ei = parse_input!(inputs.next().unwrap(), i32); // the index of a gateway node
gateways.insert(ei);
}
let gateways = gateways;
for gwid in &gateways {
for gwnbr in &graph.get(gwid).unwrap().borrow().nbrs {
(&graph).get(&gwnbr).unwrap().borrow_mut().gwlinks += 1;
}
}
assert_eq!(generate_path(&0, &graph), vec![0, 3]);
}
Errors:
rustc 1.18.0 (03fc9d622 2017-06-06)
error[E0502]: cannot borrow `boundary` as mutable because it is also borrowed as immutable
--> <anon>:53:19
|
50 | while let Some( (currid, _) ) = boundary.iter().max_by_key(|x| x.1) {
| -------- immutable borrow occurs here
...
53 | let val = boundary.remove(&currid).unwrap();
| ^^^^^^^^ mutable borrow occurs here
...
76 | }
| - immutable borrow ends here
error[E0502]: cannot borrow `boundary` as mutable because it is also borrowed as immutable
--> <anon>:66:13
|
50 | while let Some( (currid, _) ) = boundary.iter().max_by_key(|x| x.1) {
| -------- immutable borrow occurs here
...
66 | boundary.insert(
| ^^^^^^^^ mutable borrow occurs here
...
76 | }
| - immutable borrow ends here
error: aborting due to 2 previous errors
I found a solution to my issue, and it's somewhat generalizable, which is what I was hoping for. The problem was that an implicit reference created in the while let statement was living to the end of the loop even though it was only needed on that one line. The borrow begins at .iter() and is no longer needed once the referenced value is cloned at the end of the expression.
while let Some( (currid, _) ) = boundary.iter().max_by_key(|x| x.1).clone() {
// ^---where borrow begins ^---where borrow could end
// Move the node from `boundary` to `finished`
let val = boundary.remove(&currid).unwrap();
finished.insert(currid.clone(), val);
...
} // <--- where borrow does end
The trick was moving the binding of currid into the loop. When the value is borrowed in the while let statement, the borrow checker apparently thinks the borrow needs to last throughout the loop. If, instead, the implicit borrow is made in a regular let binding, the borrow checker is smart enough to realize the borrow can be safely discarded at the end of the line.
while !boundary.is_empty() {
let currid = boundary.iter().max_by_key(|x| x.1).unwrap().0.clone();
// ^---where borrow begins ^---where borrow ends
// Move the node from `boundary` to `finished`
let val = boundary.remove(&currid).unwrap();
finished.insert(currid.clone(), val);
...
}
I guess the take away here is that if you need to mutate a structure in a loop that depends on it, put any borrows of the structure inside the loop and keep those borrows as short as possible – for example, by using clone.
This might be one of the situations eventually mitigated by the proposed non-lexical lifetimes.