Go find files in directory recursively - file

Question

I want to find all files matching a specific pattern in a directory recursively (including subdirectories). I wrote the code to do this:
libRegEx, e := regexp.Compile("^.+\\.(dylib)$")
if e != nil {
log.Fatal(e)
}
files, err := ioutil.ReadDir("/usr/lib")
if err != nil {
log.Fatal(err)
}
for _, f := range files {
if libRegEx.MatchString(f.Name()) {
println(f.Name())
}
}
Unfortunately, it only searches in /usr/bin, but I also want to search for matches in its subdirectories. How can I achieve this? Thanks.

Answer 1

The standard library's filepath package includes Walk for exactly this purpose: "Walk walks the file tree rooted at root, calling walkFn for each file or directory in the tree, including root." For example:
libRegEx, e := regexp.Compile("^.+\\.(dylib)$")
if e != nil {
log.Fatal(e)
}
e = filepath.Walk("/usr/lib", func(path string, info os.FileInfo, err error) error {
if err == nil && libRegEx.MatchString(info.Name()) {
println(info.Name())
}
return nil
})
if e != nil {
log.Fatal(e)
}

Answer 2

Starting with Go 1.16 (Feb 2021), you can use filepath.WalkDir:
package main
import (
"io/fs"
"path/filepath"
)
func walk(s string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if ! d.IsDir() {
println(s)
}
return nil
}
func main() {
filepath.WalkDir("..", walk)
}

Answer 3

If you are looking for something that doesn't use walk, I found this project
The main recursive algorithm seems effective despite using strings. It basically amounts to the below code and kinda reminds me of merge sort and other recursive algorithms:
func processed(fileName string, processedDirectories []string) bool {
for i := 0; i < len(processedDirectories); i++ {
if processedDirectories[i] != fileName {
continue
}
return true
}
return false
}
func listDirContents(path string, dirs []string) {
files, _ := ioutil.ReadDir(path)
for _, f := range files {
var newPath string
if path != "/" {
newPath = fmt.Sprintf("%s/%s", path, f.Name())
} else {
newPath = fmt.Sprintf("%s%s", path, f.Name())
}
if f.IsDir() {
if !processed(newPath, dirs) {
dirs = append(dirs, newPath)
listDirContents(newPath, dirs)
}
} else {
fmt.Println(newPath)
}
}
}
That actually prints all found paths starting from the provided directory and includes all sub-directories. Therefor you would have to check if the path contains your target string instead of just printing the path with fmt.Println() statements.
After trying it out vs the find command, it scanned my /home directory in about .8s... the find command found the same files but did it in about .3s (a full .5s faster than the above algorithm).

Answer 4

You can use all the files in the directory using following code:
files, err := ioutil.ReadDir(dirPath)
check(err)
for _, file := range files {
fmt.Println(dirPath + file.Name())
}
The code is using the io/ioutil package to read all the files in the given directory and then looping through them to print there names.