I have a HashMap with different hashing algorithms which implement digest::DynDigest, but when they are looped over and update()d the final hash is always the same (Sha224 d14a...) (you can change the file PathBuf parameter to anything and get the exact same hash every time). So the update() and/or finalize() is not updating the instance in the HashMap.
What is the correct way to loop over the different hashers so that HashMap keeps correct updated state?
use digest::{Digest, DynDigest}; // https://crates.io/crates/digest/reverse_dependencies
use sha2; // implements digest::DynDigest
use std::collections::HashMap;
use std::fs::File;
use std::io;
use std::io::{BufReader, Read};
use std::path::PathBuf;
fn checksum_to_hex(bytes: &[u8]) -> String {
let mut s: String = String::new();
for b in bytes {
s.push_str(format!("{:02x}", b).as_str());
}
s
}
pub fn hash_file(
fpath: PathBuf,
hashers: HashMap<&str, Box<dyn DynDigest>>,
) -> io::Result<HashMap<&str, String>> {
let f = File::open(fpath).expect("failed to open file for hashing");
let mut buffer = [0u8; 1048576];
let mut reader = BufReader::new(f);
loop {
let count = reader.read(&mut buffer)?;
if count == 0 {
break;
}
for (_, mut hasher) in hashers.to_owned() {
hasher.update(&buffer[..count]);
}
}
// Hash results per hasher
let mut hashes: HashMap<&str, String> = HashMap::new();
for (k, hasher) in hashers.to_owned() {
let res = checksum_to_hex(hasher.finalize().to_vec().as_slice());
hashes.insert(k, res);
}
Ok(hashes)
}
fn main() {
let mut hashers: HashMap<&str, Box<dyn DynDigest>> = HashMap::new();
hashers.insert("Sha224", Box::new(sha2::Sha224::new()));
hashers.insert("Sha256", Box::new(sha2::Sha256::new()));
hashers.insert("Sha512", Box::new(sha2::Sha512::new()));
for (htype, hashres) in hash_file(PathBuf::from("/bin/ls"), hashers).expect("") {
println!(" {} {}", htype, hashres);
}
}
Rust Playground
You need to replace the first hashers.to_owned() with hashers.iter_mut():
pub fn hash_file(
fpath: PathBuf,
mut hashers: HashMap<&str, Box<dyn DynDigest>>,
) -> io::Result<HashMap<&str, String>> {
let f = File::open(fpath).expect("failed to open file for hashing");
let mut buffer = [0u8; 1048576];
let mut reader = BufReader::new(f);
loop {
let count = reader.read(&mut buffer)?;
if count == 0 {
break;
}
for (_, hasher) in hashers.iter_mut() {
// or `for hasher in hashers.values_mut()`
hasher.update(&buffer[..count]);
}
}
// Hash results per hasher
let mut hashes: HashMap<&str, String> = HashMap::new();
for (k, hasher) in hashers {
let res = checksum_to_hex(hasher.finalize().to_vec().as_slice());
hashes.insert(k, res);
}
Ok(hashes)
}
playground
.to_owned() will clone (create an independent deep copy) the map. When you iterate over that, you're iterating through a different map than the one you eventually return.
Instead, you want to iterate over references to the elements: this is what .iter_mut() (or .values_mut() if you only need the values) will give you.
This way, you don't need the second .to_owned(), but you do need to mark the hashers argument as mutable.
Related
This question already has an answer here:
Multiple return types in rust [duplicate]
(1 answer)
Closed 10 months ago.
I'm trying to implement a function in rust that iterates either on the StdIn Lines or a BuffReader Lines.
Both types have lines.next() method. After that, the code is just String manipulation that works.
I originally made it worked (with repetitive and implicitly-typed branches) but the two branches had similar code. This gave me the impression I can DRY/refactor my code better.
The problem is (I think) that my two match branches, in main, do not return exactly the same type. Is it possible to make them "stick together" because I'm only using one common method? (It reminds me of Python duck typing)
Here's my refacto try:
use clap::Parser;
use exitcode;
use std::any::Any;
use std::fs::File;
use std::io::{self, BufRead, BufReader, Lines, Write};
fn colorist(lines: &mut dyn Any, mut writer: impl std::io::Write) {
while let Some(line_wrap) = lines.next() {
match line_wrap {
Ok(line) => {
// ...
let buffer = "Some value base on line";
match writeln!(writer, "{}", buffer){
Ok(()) => (),
Err(e) => {eprintln!("{:?}", e);}
};
},
Err(e) => {eprintln!("{:?}", e);}
}
}
}
fn main() {
let args = Args::parse(); // have one '--file' arg being a filepath or is a empty str ("")
let mut writer = io::stdout();
let mut lines: Lines<dyn Any> = match args.file.is_empty() { // either read from stdin or read the file
false => {
let file = File::open(args.file).unwrap();
let reader = BufReader::new(file);
reader.lines() // type `std::io::Lines<BufReader<File>>`
},
true => {
let stdin = io::stdin();
stdin.lock().lines() // type `std::io::Lines<StdinLock<'_>>`
}
};
colorist(&mut lines, &mut writer);
}
This is a toy project, and I'm still learning rust on my free time. I might have some bad designs there and here.
As #PitaJ & #Chayim-friedman suggested, Either solved my problem.
I ended up with:
use either::*;
fn colorist(lines: &mut Either<Lines<BufReader<File>>, Lines<StdinLock>>, mut writer: impl std::io::Write) {...}
fn main() {
let args = Args::parse();
let mut writer = io::stdout();
let stdin = io::stdin();
let mut lines = match args.file.is_empty(){
false => {
let file = File::open(args.file).unwrap();
let reader = BufReader::new(file);
either::Left(reader.lines())
},
true => {
either::Right(stdin.lock().lines())
}
};
colorist(&mut lines, &mut writer);
match write!(writer, "{}", RESET){
_ => (),
};
let _ = writer.flush();
std::process::exit(exitcode::OK);
}
Following is the code I'm working on, I explore a directory path with a queue and I want to store the filesystem tree in my data structure (the enum Entry):
use failure::Error;
use std::collections::VecDeque;
use std::fs;
use std::io;
use std::path::{Path, PathBuf};
fn main() -> Result<(), Error> {
let paths = visit_dir(Path::new(".")).map_err(Error::from)?;
Ok(())
}
#[derive(Debug)]
enum Entry {
Dir(PathBuf, Vec<Entry>),
File(PathBuf),
}
impl Entry {
fn new_dir(path: &Path) -> Entry {
Entry::Dir(path.to_path_buf(), Vec::new())
}
fn new_file(path: &Path) -> Entry {
Entry::File(path.to_path_buf())
}
/// Append a new Entry to self if self is a directory.
fn push(&mut self, path: &Path) -> Option<&mut Entry> {
if let Entry::Dir(_, ref mut content) = self {
let entry = if path.is_dir() {
Entry::new_dir(path)
} else {
Entry::new_file(path)
};
content.push(entry);
return content.last_mut();
}
None
}
fn path(&self) -> &Path {
match self {
Entry::Dir(path, _) => path,
Entry::File(path) => path,
}
}
}
fn visit_dir(root: &Path) -> io::Result<Entry> {
let mut dir = Entry::new_dir(root);
let mut queue = VecDeque::new();
queue.push_back(&mut dir);
while !queue.is_empty() {
let parent = queue.pop_front().unwrap();
let path = parent.path();
if path.is_dir() {
for entry in fs::read_dir(path)? {
let entry = entry?;
let path = entry.path();
let entry = parent.push(&path).unwrap();
if path.is_dir() {
queue.push_back(entry);
}
}
}
}
Ok(dir)
}
Link to Playground
The error I am getting is:
error[E0499]: cannot borrow `*parent` as mutable more than once at a time
--> src/main.rs:61:29
|
61 | let entry = parent.push(&path).unwrap();
| ^^^^^^ mutable borrow starts here in previous iteration of loop
My questions are:
Why exactly do I get this error?
Are there any workaround to solve it?
entry borrows parent mutably in the line:
let entry = parent.push(&path).unwrap();
Then, you do not release entry, because you store it in queue. So, parent is still borrowed mutably when this is the 2nd iteration in the loop. This is unsafe to do that.
What you are trying to do is a tree of mutable references. That won't work in Rust, and this is generally a bad idea. You should modify the way you want to implement that.
I would implement an array v to mark the current vertex as visited. However, iter.next() cannot be an index of LinkedList or an index of an array.
The output of iter.next() is not an integer, it is an option Some.
How to use the value in Some? If I would use value in Some as an index of array. Should I convert the value into an integer?
fn bfs(s: usize) {
let mut cells = vec![LinkedList::<usize>::new(); 4];
for (i, cell) in cells.iter_mut().enumerate() {
cell.push_front(i);
}
cells[0].push_back(1);
cells[0].push_back(2);
cells[1].push_back(2);
cells[2].push_back(0);
cells[2].push_back(3);
cells[3].push_back(3);
let mut iter = cells[s].iter();
let mut v = vec![0; 4];
// Entry point
for i in 0..cells[s].len() {
// It is incorrect, "iter.next()" it cannot be a index of LinkedList
// vector "v" to keep track of which vertex has been traversed
//v[iter.next()] = 1;
}
}
fn main() {
bfs(2);
}
iter.next() return an Option, you just need to deconstruct it:
for i in 0..cells[s].len() {
let x = match iter.next() {
Some(x) => *x,
None => return,
};
v[x] = 1;
}
Rust's for loops can iterate for you, so you don't need to explicitly create an iterator or call next(). The Option will already be unwrapped inside the loop, because the loop automatically ends when next() returns None.
for &i in &cells[s] {
v[i] = 1;
}
This saves you having to keep track of index variables just to create a loop. And the code is a lot shorter and more readable.
This is the final version
fn bfs(s: usize) {
let mut cells = vec![LinkedList::<usize>::new(); 4];
let mut traversal = LinkedList::new();
let mut queue = LinkedList::new();
for (i, cell) in cells.iter_mut().enumerate() {
cell.push_front(i);
}
cells[0].push_back(1);
cells[0].push_back(2);
cells[1].push_back(2);
cells[2].push_back(0);
cells[2].push_back(3);
cells[3].push_back(3);
let mut v = vec![0; 4];
let mut done = false;
let c = s;
while !done {
for &x in &cells[c] {
if v[x] == 0 {
v[x] = 1;
queue.push_back(x);
traversal.push_back(x)
}
}
let c = queue.pop_back();
if queue.is_empty() {
// walkthrough
for i in 0..cells.len() {
if v[i] == 0 {
v[i] = 1;
queue.push_back(i);
traversal.push_back(i)
}
}
if queue.is_empty() {
done = true;
}
}
}
println!("{:?}", traversal);
}
You can use the reference of the integer value in Some as the index of the array.
let mut v = vec![0; 4];
while !done {
let mut iter = cells[c].iter().peekable();
// Entry point
while let Some(&x) = iter.next_if(|&&n| v[n] == 0) {
v[x] = 1;
queue.push_back(x);
traversal.push_back(x)
}
…
This is the shortest I came up with, after extensive googling and studying the sources:
let mut buf = [0u8; 200];
for elem in buf.iter_mut() {
*elem = 0;
}
Is there really no way to make this a one-liner, like buf.set_all(0)?
Is there really no way to make this a one-liner, like buf.set_all(0)?
Sure you can make it a one-liner...
for elem in buf.iter_mut() { *elem = 0; }
Okay, okay... if you do this a lot, you can define an extension trait that provides a set_all method.
trait SetAll {
type Elem;
fn set_all(&mut self, value: Self::Elem);
}
impl<T> SetAll for [T] where T: Clone {
type Elem = T;
fn set_all(&mut self, value: T) {
for e in self {
*e = value.clone();
}
}
}
But in terms of just using what's in the standard library, there's for_each (as noted by Sven Marnach):
buf.iter_mut().for_each(|x| *x = 0)
You could simply construct a new array and move it into the array you want to change.
buf = [0u8; 200];
If the compiler is decent, it can optimize out the temporary array and write directly to the target.
in your scene, this is my solution:
let mut buf = [1u8; 200];
unsafe {
std::ptr::write_volatile(&mut buf, [0u8;200]);
}
Is there really no way to make this a one-liner, like buf.set_all(0)?
and you also can do like this:
struct Buf([u8;200]);
impl Buf {
fn clear(&mut self) {
unsafe {
std::ptr::write_volatile(&mut self.0, [0u8;200]);
}
}
}
fn main() {
let mut buf1 = Buf([1u8; 200]);
let mut buf2 = Buf([2u8; 200]);
buf1.clear();
buf2.clear();
println!("{}, {}", buf1.0[199], buf2.0[0]);
}
I am trying to have a struct which has a field which I assume should be of type Result<TempDir>. When I initialise an implementation of the field with new(), I would like that particular field to be initialised by the creation of a new temp directory. Later, I want to implement a method to read from that directory.
Here's the code, I am more worried about the syntax and proper use of libraries (why exactly are there over four libraries for read/write buffering in Rust, this is insane) as the logic should be right. Dont worry too much about the trait implementations, I just need directions in the syntax. Please don't be too harsh, as I know it doesn't compile, but with just two changes it should.
extern crate rustc_back;
use std::path::Path;
use std::fs::File;
use rustc_back::tempdir::TempDir as TempDir;
pub struct MyStorage {
temp_dir : Result<TempDir>
}
impl MyStorage {
pub fn new() -> MyStorage {
//tempo = match TempDir::new("encrypt_storage");
let store = match TempDir::new("encrypt_storage") {
Ok(dir) => dir,
Err(e) => panic!("couldn't create temporary directory: {}", e)
};
MyStorage { temp_dir: store }
//MyStorage { temp_dir: TempDir::new("encrypt_storage") }
}
}
impl Storage for MyStorage {
fn get(&self, name: Vec<u8>) -> Vec<u8> {
//let mut f = std::fs::File::open(self.temp_dir.path() / name);
let mut f = std::fs::File::open(&self.temp_dir){
// The `desc` field of `IoError` is a string that describes the error
Err(why) => panic!("couldn't open: {}", why.description()),
Ok(file) => file,
};
let mut s = String::new();
//f.read_to_string(&mut s);
match f.read_to_string(&mut s){
Err(why) => panic!("couldn't read: {}", why.description()),
Ok(_) => print!("contains:\n{}", s),
}
s.to_vec()
}
fn put(&mut self, name: Vec<u8>, data: Vec<u8>) {
// self.entries.push(Entry { name : name, data : data })
let mut f = File::create(self.temp_dir.path() / name);
f.write_all(data);
}
fn put(&mut self, name: Vec<u8>, data: Vec<u8>) {
// self.entries.push(Entry { name : name, data : data })
let mut f = File::create(self.temp_dir.path() / name);
f.write_all(data);
}
}
After fixing the indentation (Rust uses 4 spaces per level), removing the Storage for since you didn't provide that trait, removing commented-out code, and adding a main, you are left with this:
extern crate rustc_back;
use std::path::Path;
use std::fs::File;
use rustc_back::tempdir::TempDir as TempDir;
pub struct MyStorage {
temp_dir : Result<TempDir>
}
impl MyStorage {
pub fn new() -> MyStorage {
let store = match TempDir::new("encrypt_storage") {
Ok(dir) => dir,
Err(e) => panic!("couldn't create temporary directory: {}", e)
};
MyStorage { temp_dir: store }
}
}
impl MyStorage {
fn get(&self, name: Vec<u8>) -> Vec<u8> {
let mut f = std::fs::File::open(self.temp_dir.path() / name);
let mut s = String::new();
f.read_to_string(&mut s);
s.to_vec()
}
fn put(&mut self, name: Vec<u8>, data: Vec<u8>) {
let mut f = File::create(self.temp_dir.path() / name);
f.write_all(data);
}
}
fn main() {}
Compiling that has this error:
error: wrong number of type arguments: expected 2, found 1 [E0243]
temp_dir : Result<TempDir>
^~~~~~~~~~~~~~~
Which nicely points to the problematic type. Let's look at the docs for Result, which includes the definition:
pub enum Result<T, E> {
Ok(T),
Err(E),
}
So Result has two type parameters - T is used for the success case, and E is used for the failure case. Your code is only specifying one of them. My guess is that you looked at the docs for TempDir and copy-and-pasted:
fn new(prefix: &str) -> Result<TempDir>
However, if you click on the Result there, you'll see it goes to io::Result, which is simply a type alias that binds E to io::Error:
type Result<T> = Result<T, Error>;
With all that exposition out of the way, you can "fix" your problem by changing your MyStorage struct:
pub struct MyStorage {
temp_dir: std::io::Result<TempDir>,
}
And then you will get another compiler error, as you are already dealing with the Result via the match in MyStorage::new. You aren't storing a io::Result<TempDir>, you are just storing a TempDir! Changing your struct further:
pub struct MyStorage {
temp_dir: TempDir,
}
unlocks a whole new set of errors for you to figure out; but now you have gotten past that first hurdle!