I don't really know how to formulate the question, but here it is.
I'm using fsnotify to watch some directories for changes and when a file changes, I sync the change to another directory. But I want to add newly created directories to the watch too and it's not really working.
Here's my code:
func Watcher() {
watcher, err := fsnotify.NewWatcher()
defer watcher.Close()
done := make(chan bool)
go func() {
for {
select {
case event := <-watcher.Events:
if file.Mode().IsDir() {
err = os.Mkdir(dest, 0755)
err = watcher.Add(dest)
}
case err := <-watcher.Errors:
log.Println("error:", err)
}
}
}()
dirs, err := readLines("dirs")
for _, el := range dirs {
err = watcher.Add(el)
}
check(err)
<-done
}
The function is much longer, but I've deleted the non-important parts. Everything works, except the err = watcher.Add(dest).
How can I make it watch more directories?
It was working just fine, but I got some variables wrong. Should have been watcher.Add(event.Name) instead of watcher.Add(dest).
Related
I'm coding in Go, and I created a file handler and a program that prints the value of that file.
However, the file that should be created with file.Filename is deleted when I run it.
I don't know what the reason is, even if I try to debug, the answer doesn't come out, and even if I google it, I don't get the answer.
(64bit windows 10 (WSL2))
package main
import (
"fmt"
"io"
"io/ioutil"
"os"
"github.com/labstack/echo"
)
func checkErr(err error) {
if err != nil {
panic(err)
}
}
func readFile(filename string) string {
data, err := ioutil.ReadFile(filename)
checkErr(err)
return string(data)
}
func main() {
e := echo.New()
e.POST("/file", func(c echo.Context) error {
file, err := c.FormFile("file")
checkErr(err)
src, err := file.Open()
checkErr(err)
defer src.Close()
dst, err := os.Create(file.Filename)
checkErr(err)
defer dst.Close()
_, err = io.Copy(dst, src)
checkErr(err)
data := readFile(file.Filename)
fmt.Println(data)
return c.String(200, "sd")
})
e.Logger.Fatal(e.Start(":5000"))
}
I'm guessing that your file exists, but the code that you wrote is reading the file before the changes are "flushed to disk".
Right here:
defer dst.Close()
_, err = io.Copy(dst, src)
Should Close() or Sync() your writer as soon as possible, otherwise you may read before the write is finished. And since your readFile() function isn't re-using the file, you might as well just close (not Sync()) it immediately, not deferred
Try this:
_, err = io.Copy(dst, src)
dst.Close()
if err != nil {
}
There could be an error while copying, but we still want to Close() the file (if there wasn't an error during the os.Create, os.Open, or os.OpenFile...
I have some .log files and want to monitor if any data is appended to any of them to collect that data into a DB.
How do I open an opened-for-writing file and how do I monitor for new lines/changes?
Thank you.
I wrote this small piece of code that will monitor for file change and if detected, run the function you passed to it.
func watchFileAndRun(filePath string, fn func()) error {
defer func() { r := recover(); if r != nil { logCore("ERROR","Error:watching file:", r) } }()
fn()
initialStat, err := os.Stat(filePath)
checkErr(err)
for {
stat, err := os.Stat(filePath)
checkErr(err)
if stat.Size() != initialStat.Size() || stat.ModTime() != initialStat.ModTime() {
fn()
initialStat, err = os.Stat(filePath)
checkErr(err)
}
time.Sleep(10 * time.Second)
}
return nil
}
To call this function where loadEnv is a function:
go watchFileAndRun("config.json", loadEnv)
Some Notes:
go watchFileAndRun() will run the routing asynchronously. IE in the background.
checkErr is my own function you can simply put a standard
if err != nil {fmt.printf("Something went wrong",err)} in its place.
and the defer first line is to prevent the program crashing the main if something went wrong. You could nest it deeper if you wanted to make this function more resilient.
Hope it helps someone. and its smaller than the nsnotify plug in which to be frank I could get working either.
I'm making a function like ioutil.ReadDir() but recursively due I want all the files in folders and subfolders and ioutil.ReadDir() just do it in the specified folder but I don't know how to append items to an array of []os.FileInfo that I've created.
This is what I have:
func GetFilesRecursively(searchDirectory string) (foundFileList []os.FileInfo, errorGenerated error){
fileList := []os.FileInfo{}
allFilesAndFolders := []string{}
//Get all the files and directories
err := filepath.Walk(searchDirectory, func(path string, f os.FileInfo, err error) error {
allFilesAndFolders = append(allFilesAndFolders, path)
return nil
})
// Remove directories due those are also added into the array and we don't need them
for _, file := range allFilesAndFolders{
fileInfo, _ := os.Stat(file)
if (!fileInfo.Mode().IsDir()){
fileList = append(fileList, file) //error here!!
}
}
return fileList, err
}
The error is in the comments in the code snippet above
How could I do that?
You try to append a string to an array of os.FileInfo. That is a type error.
Here is a version of your code that does what you want and looks more Go-like, with shorter variable names, unnamed return types (their types perfectly decribe what they do) and using var to create an empty slice.
func GetFilesRecursively(root string) ([]os.FileInfo, error) {
var files []os.FileInfo
// walk all directories but only collect files
err := filepath.Walk(root, func(path string, f os.FileInfo, err error) error {
if !f.IsDir() {
files = append(files, f)
}
return nil
})
return files, err
}
I am trying to use a file instead of a DB to get a prototype up and running. I have a program that (1) reads existing content from the file to a map, (2) takes JSON POSTs that add content to the map, (3) on exit, writes to the file.
First, the file is not being created. Then I created an empty file. It is not being written to.
I am trying to read the file, determine if there is existing content. If there is not existing content, create a blank map. If there is existing content, unmarshal it into a new map.
func writeDB() {
eventDBJSON, err := json.Marshal(eventDB)
if err != nil {
panic(err)
}
err2 := ioutil.WriteFile("/Users/sarah/go/dat.txt", eventDBJSON, 0777)
if err2 != nil {
panic(err2)
}
}
func main() {
dat, err := ioutil.ReadFile("/Users/sarah/go/dat.txt")
if err != nil {
panic(err)
}
if dat == nil {
eventDB = DB{
events: map[string]event{},
}
} else {
if err2 := json.Unmarshal(dat, &eventDB); err2 != nil {
panic(err2)
}
}
router := httprouter.New()
router.POST("/join", JoinEvent)
router.POST("/create", CreateEvent)
log.Fatal(http.ListenAndServe(":8080", router))
defer writeDB()
}
There is no way for the server to ever reach defer writeDB().
http.ListenAndServe blocks, and if it did return anything, you log.Fatal that, which exits your app at that point.
You can't intercept all ways an app can exit, getting SIGKILL, machine loss of power, etc.
I'm assuming you really just want to write some code, bounce the server, repeat
If that's the case, then Ctrl-C is good enough.
If you want to write your file on Ctrl-C, look at the signal package.
Also, defer on the last line of a function really has no purpose as defer basically means "do this last".
you can use (*os.File).Stat() to get a file's FileInfo which contain its size
file, err := os.Open( filepath )
if err != nil {
// handle error
}
fi, err := file.Stat()
if err != nil {
// handle error
}
s := fi.Size()
I'm currently learning how to develop with Go (or golang) and I have a strange issue:
I try to create a script looking inside an HTML file in order to get all the sources of each tags.
The goal of the script is to merge all the retrieved files.
So, that's for the story: for now, I'm able to get the content of each JavaScript files but... I can't concatenate them...
You can see below my script:
//Open main file
mainFilePath := "/path/to/my/file.html"
mainFileDir := path.Dir(mainFilePath)+"/"
mainFileContent, err := ioutil.ReadFile(mainFilePath)
if err == nil {
mainFileContent := string(mainFileContent)
var finalFileContent bytes.Buffer
//Start RegExp searching for JavaScript src
scriptReg, _ := regexp.Compile("<script src=\"(.*)\">")
scripts := scriptReg.FindAllStringSubmatch(mainFileContent,-1)
//For each SRC found...
for _, path := range scripts {
//We open the corresponding file
subFileContent, err := ioutil.ReadFile(mainFileDir+path[1])
if err == nil {
//And we add its content to the "final" variable
fmt.Println(finalFileContent.Write(subFileContent))
} else {
fmt.Println(err)
}
}
//Try to display the final result
// fmt.Println(finalFileContent.String())
fmt.Printf(">>> %#v", finalFileContent)
fmt.Println("Y U NO WORKS? :'(")
} else {
fmt.Println(err)
}
So, each fmt.Println(finalFileContent.Write(subFileContent)) display something like 6161 , so I assume the Write() method is correctly executed.
But fmt.Printf(">>> %#v", finalFileContent) displays nothing. Absolutely nothing (even the ">>>" are not displayed!) And it's the same for the commented line just above.
The funny part is that the string "Y U NO WORK ? :'(" is correctly displayed...
Do you know why?
And do you know how to solve this issue?
Thanks in advance!
You are ignoring some errors. What are your results when you run the following version of your code?
package main
import (
"bytes"
"fmt"
"io/ioutil"
"path"
"regexp"
)
func main() {
//Open main file
mainFilePath := "/path/to/my/file.html"
mainFileDir := path.Dir(mainFilePath) + "/"
mainFileContent, err := ioutil.ReadFile(mainFilePath)
if err == nil {
mainFileContent := string(mainFileContent)
var finalFileContent bytes.Buffer
//Start RegExp searching for JavaScript src
scriptReg, _ := regexp.Compile("<script src=\"(.*)\">")
scripts := scriptReg.FindAllStringSubmatch(mainFileContent, -1)
//For each SRC found...
for _, path := range scripts {
//We open the corresponding file
subFileContent, err := ioutil.ReadFile(mainFileDir + path[1])
if err == nil {
//And we add its content to the "final" variable
// fmt.Println(finalFileContent.Write(subFileContent))
n, err := finalFileContent.Write(subFileContent)
fmt.Println("finalFileContent Write:", n, err)
} else {
fmt.Println(err)
}
}
//Try to display the final result
// fmt.Println(finalFileContent.String())
// fmt.Printf(">>> %#v", finalFileContent)
n, err := fmt.Printf(">>> %#v", finalFileContent)
fmt.Println()
fmt.Println("finalFileContent Printf:", n, err)
fmt.Println("Y U NO WORKS? :'(")
} else {
fmt.Println(err)
}
}
UPDATE:
The statement:
fmt.Println("finalFileContent Printf:", n, err)
Outputs:
finalFileContent Printf: 0 write /dev/stdout: winapi error #8
or
finalFileContent Printf: 0 write /dev/stdout: Not enough storage is available to process this command.
From MSDN:
ERROR_NOT_ENOUGH_MEMORY
8 (0x8)
Not enough storage is available to process this command.
The formatted output to the Windows console overflows the buffer (circa 64KB).
There is a related Go open issue:
Issue 3376: windows: detect + handle console in os.File.Write