Golang How to read input filename in Go - file

I would like to run my go file on my input.txt where my go program will read the input.txt file when I type in go run command ie:
go run goFile.go input.txt
I don't want to put input.txt in my goFile.go code since my go file should run on any input name not just input.txt.
I try ioutil.ReadAll(os.Stdin) but I need to change my command to
go run goFile.go < input.txt
I only use package fmt, os, bufio and io/ioutil. Is it possible to do it without any other packages?

Please take a look at the package documentation of io/ioutil which you are already using.
It has a function exactly for this: ReadFile()
func ReadFile(filename string) ([]byte, error)
Example usage:
func main() {
// First element in os.Args is always the program name,
// So we need at least 2 arguments to have a file name argument.
if len(os.Args) < 2 {
fmt.Println("Missing parameter, provide file name!")
return
}
data, err := ioutil.ReadFile(os.Args[1])
if err != nil {
fmt.Println("Can't read file:", os.Args[1])
panic(err)
}
// data is the file content, you can use it
fmt.Println("File content is:")
fmt.Println(string(data))
}

Firs you check for the provided argument. If the first argument satisfy the condition of an input file, then you use the ioutil.ReadFile method, providing parameter the os.Args result.
package main
import (
"fmt"
"os"
"io/ioutil"
)
func main() {
if len(os.Args) < 1 {
fmt.Println("Usage : " + os.Args[0] + " file name")
os.Exit(1)
}
file, err := ioutil.ReadFile(os.Args[1])
if err != nil {
fmt.Println("Cannot read the file")
os.Exit(1)
}
// do something with the file
fmt.Print(string(file))
}
Another possibility is to use:
f, err := os.Open(os.Args[0])
but for this you need to provide the bytes lenght to read:
b := make([]byte, 5) // 5 is the length
n, err := f.Read(b)
fmt.Printf("%d bytes: %s\n", n, string(b))

For running .go file from command-line by input parameter like file (for example abc.txt).We need use mainly os, io/ioutil, fmt packages. Additionally for reading command line parameters we use
os.Args Here is example code
package main
import (
"fmt"
"os"
"io/ioutil"
)
func main() {
fmt.Println(" Hi guys ('-') ")
input_files := os.Args[1:]
//input_files2 := os.Args[0];
//fmt.Println("if2 : ",input_files2)
if len(input_files) < 1{
fmt.Println("Not detected files.")
}else{
fmt.Println("File_name is : ",input_files[0])
content, err := ioutil.ReadFile(input_files[0])
if err != nil {
fmt.Println("Can't read file :", input_files[0],"Error : ",err)
}else {
fmt.Println("Output file content is(like string type) : \n",string(content))//string Output
fmt.Println("Output file content is(like byte type) : \n",content)//bytes Output
}
}
}
Args holds command line arguments, including the command as Args[0].
If the Args field is empty or nil, Run uses {Path}.
In typical use, both Path and Args are set by calling Command.
Args []string
function. This function return back array on string type https://golang.org/pkg/os/exec/.Args hold the command-line arguments, starting with the program name. In this case short way to take filename from command-line is this functions os.Args[1:] . And here is output
elshan_abd$ go run main.go abc.txt
Hi guys ('-')
File_name is : abc.txt
Output file content is(like string type) :
aaa
bbb
ccc
1234
Output file content is(like byte type) :
[97 97 97 10 98 98 98 10 99 99 99 10 49 50 51 52 10]
Finally we need for reading content file this function
func ReadFile(filename string) ([]byte, error) source is https://golang.org/pkg/io/ioutil/#ReadFile

Related

How to get file length in Go dynamically?

I have the following code snippet:
func main() {
// Some text we want to compress.
original := "bird and frog"
// Open a file for writing.
f, _ := os.Create("C:\\programs\\file.gz")
// Create gzip writer.
w := gzip.NewWriter(f)
// Write bytes in compressed form to the file.
while ( looping over database cursor) {
w.Write([]byte(/* the row from the database as obtained from cursor */))
}
// Close the file.
w.Close()
fmt.Println("DONE")
}
However, I wish to know a small modification. When the size of file reaches a certain threshold I want to close it and open a new file. And that too in compressed format.
For example:
Assume a database has 10 rows each row is 50 bytes.
Assume compression factor is 2, ie 1 row of 50 bytes is compressed to 25 bytes.
Assume a file size limit is 50 bytes.
Which means after every 2 records I should close the file and open a new file.
How to keep track of the file size while its still open and still writing compressed documents to it ?
gzip.NewWriter takes a io.Writer. It is easy to implement custom io.Writer that does what you want.
E.g. Playground
type MultiFileWriter struct {
maxLimit int
currentSize int
currentWriter io.Writer
}
func (m *MultiFileWriter) Write(data []byte) (n int, err error) {
if len(data)+m.currentSize > m.maxLimit {
m.currentWriter = createNextFile()
}
m.currentSize += len(data)
return m.currentWriter.Write(data)
}
Note: You will need to handle few edge cases like what if len(data) is greater than the maxLimit. And may be you don't want to split a record across files.
You can use the os.File.Seek method to get your current position in the file, which as you're writing the file will be the current file size in bytes.
For example:
package main
import (
"compress/gzip"
"fmt"
"os"
)
func main() {
// Some text we want to compress.
lines := []string{
"this is a test",
"the quick brown fox",
"jumped over the lazy dog",
"the end",
}
// Open a file for writing.
f, err := os.Create("file.gz")
if err != nil {
panic(err)
}
// Create gzip writer.
w := gzip.NewWriter(f)
// Write bytes in compressed form to the file.
for _, line := range lines {
w.Write([]byte(line))
w.Flush()
pos, err := f.Seek(0, os.SEEK_CUR)
if err != nil {
panic(err)
}
fmt.Printf("pos: %d\n", pos)
}
// Close the file.
w.Close()
// The call to w.Close() will write out any remaining data
// and the final checksum.
pos, err := f.Seek(0, os.SEEK_CUR)
if err != nil {
panic(err)
}
fmt.Printf("pos: %d\n", pos)
fmt.Println("DONE")
}
Which outputs:
pos: 30
pos: 55
pos: 83
pos: 94
pos: 107
DONE
And we can confirm with wc:
$ wc -c file.gz
107 file.gz

Max size of slice of string in Go

I want to input 200000 space separated strings of arbitary numbers. When taking the input using bufio.Reader it only take a few of them. Here is the code:
package main
import (
"bufio"
"fmt"
"io"
"os"
"strings"
)
func main() {
reader := bufio.NewReaderSize(os.Stdin, 1024*1024)
scoresTemp := strings.Split(readLine(reader), " ")
fmt.Println(scoresTemp)
fmt.Println("---")
fmt.Println(len(scoresTemp))
}
func readLine(reader *bufio.Reader) string {
str, _, err := reader.ReadLine()
if err == io.EOF {
return ""
}
return strings.TrimRight(string(str), "\r\n")
}
The slice length should be 200000, but it only take 410 items. If I increase the size of reader, it would be the same. What is the max size of slice of strings in Go?, How to work on it?
I believe you have an issue with your input rather than on your Go code. I've tried your code on my local machine and got this result:
$ for((i=0;i<200000;i++)) do echo -n "x "; done | go run main.go
...
---
200001
Your code doesn't iterate on the input very well. The problem isn't with slices.
Try the following code if it does what you want.
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
inputScanner := bufio.NewScanner(os.Stdin)
inputScanner.Split(bufio.ScanWords)
scoresTemp := make([]string, 0, 200000)
for inputScanner.Scan() {
scoresTemp = append(scoresTemp, inputScanner.Text())
}
fmt.Println(scoresTemp)
fmt.Println("---")
fmt.Println(len(scoresTemp))
}
For the explanation:
bufio.Scanner helps "scanning" a certain input and splitting it in whichever way you like. By default it splits be new lines. Which brings us to number 2.
bufio.ScanWords is a function that splits bytes on spaces (including new-lines). This defines the behaviour of inputScanner.
Next comes the slice of strings where we store our data. It's initialized with 0 elements, and capacity of 200,000 strings. This optimizes allocation times.
PRINT !!

Is it possible to get filename where code is called in Golang? [duplicate]

This question already has answers here:
Is it possible get information about caller function in Golang?
(2 answers)
Closed 5 years ago.
I was able to get the full path of the current directory, Now I would like to create a function that will read or get the filename where the code is executed. I am able to get the filename but it is return the original filename where the code is written :
func GetFileName() string {
_, fpath, _, ok := runtime.Caller(0)
if !ok {
err := errors.New("failed to get filename")
panic(err)
}
filename := filepath.Base(fpath)
// remove extension
filename = removeExtension(filename)
return filename + ".log"
}
What I want to do is getting the current fileName where the code is executed like :
I created app.go :
package my
function getCurrentFileName() string {
// some code here that will return the filename where this code is executed.
}
and then when I call getCurrentFileName() in a different file like hello.go in a different location. it will return hello.go.
I have been stuck here for a while and looking for an answer.
Basically this is what you tell / pass to runtime.Caller(): the number of stack entries to skip before returning an entry.
If you pass 0 as in your code, that means return the stack entry where runtime.Caller() is called (where you called runtime.Caller()). Passing 1 will skip your function, and return the function that called your function:
pc, file, line, ok := runtime.Caller(1)
if ok {
fmt.Printf("Called from %s, line #%d, func: %v\n",
file, line, runtime.FuncForPC(pc).Name())
}
Example calling a function that contains this (subplay.A() in my example):
7 func main() {
8 // Comment line
9 subplay.A()
10 }
Output:
Called from /home/icza/gows/src/play/play.go, line #9, func: main.main
We see that the code prints that play.go called our function at line #9, from the function main() of the main package.
This helper function can give those information:
func Here(skip ...int) (string, string, int, error) {
sk := 1
if len(skip) > 0 && skip[0] > 1 {
sk = skip[0]
}
var pc uintptr
var ok bool
pc, fileName, fileLine, ok := runtime.Caller(sk)
if !ok {
return "", "", 0, fmt.Errorf("N/A")
}
fn := runtime.FuncForPC(pc)
name := fn.Name()
ix := strings.LastIndex(name, ".")
if ix > 0 && (ix+1) < len(name) {
name = name[ix+1:]
}
funcName := name
nd, nf := filepath.Split(fileName)
fileName = filepath.Join(filepath.Base(nd), nf)
return funcName, fileName, fileLine, nil
}
The skip parameter is the number of caller frames that we need to go up (the chain of callers).

Jump to specific line in file in Go

In Go is it possible to jump to particular line number in a file and delete it? Something like linecache in python.
I'm trying to match some substrings in a file and remove the corresponding lines. The matching part I've taken care of and I have an array with line numbers I need to delete but I'm stuck on how to delete the matching lines in the file.
This is an old question, but if anyone is looking for a solution I wrote a package that handles going to any line in a file. Link here. It can open a file and seek to any line position without reading the whole file into memory and splitting.
import "github.com/stoicperlman/fls"
// This is just a wrapper around os.OpenFile. Alternatively
// you could open from os.File and use fls.LineFile(file) to get f
f, err := fls.OpenFile("test.log", os.O_CREATE|os.O_WRONLY, 0600)
defer f.Close()
// return begining line 1/begining of file
// equivalent to f.Seek(0, io.SeekStart)
pos, err := f.SeekLine(0, io.SeekStart)
// return begining line 2
pos, err := f.SeekLine(1, io.SeekStart)
// return begining of last line
pos, err := f.SeekLine(0, io.SeekEnd)
// return begining of second to last line
pos, err := f.SeekLine(-1, io.SeekEnd)
Unfortunately I'm not sure how you would delete, this just handles getting you to the correct position in the file. For your case you could use it to go to the line you want to delete and save the position. Then seek to the next line and save that as well. You now have the bookends of the line to delete.
// might want lineToDelete - 1
// this acts like 0 based array
pos1, err := f.SeekLine(lineToDelete, io.SeekStart)
// skip ahead 1 line
pos2, err := f.SeekLine(1, io.SeekCurrent)
// pos2 will be the position of the first character in next line
// might want pos2 - 1 depending on how the function works
DeleteBytesFromFileFunction(f, pos1, pos2)
Based on my read of the linecache module it takes a file and explodes it into an array based on '\n' line endings. You could replicate the same behavior in Go by using strings or bytes. You could also use the bufio library to read a file a line by line and only store or save the lines you want.
package main
import (
"bytes"
"fmt"
)
import "io/ioutil"
func main() {
b, e := ioutil.ReadFile("filename.txt")
if e != nil {
panic(e)
}
array := bytes.Split(b, []byte("\n"))
fmt.Printf("%v", array)
}
I wrote a small function that allowing you remove from a file a specific line.
package main
import (
"io/ioutil"
"os"
"strings"
)
func main() {
path := "path/to/file.txt"
removeLine(path, 2)
}
func removeLine(path string, lineNumber int) {
file, err := ioutil.ReadFile(path)
if err != nil {
panic(err)
}
info, _ := os.Stat(path)
mode := info.Mode()
array := strings.Split(string(file), "\n")
array = append(array[:lineNumber], array[lineNumber+1:]...)
ioutil.WriteFile(path, []byte(strings.Join(array, "\n")), mode)
}

Go : concatenate file contents

I'm currently learning how to develop with Go (or golang) and I have a strange issue:
I try to create a script looking inside an HTML file in order to get all the sources of each tags.
The goal of the script is to merge all the retrieved files.
So, that's for the story: for now, I'm able to get the content of each JavaScript files but... I can't concatenate them...
You can see below my script:
//Open main file
mainFilePath := "/path/to/my/file.html"
mainFileDir := path.Dir(mainFilePath)+"/"
mainFileContent, err := ioutil.ReadFile(mainFilePath)
if err == nil {
mainFileContent := string(mainFileContent)
var finalFileContent bytes.Buffer
//Start RegExp searching for JavaScript src
scriptReg, _ := regexp.Compile("<script src=\"(.*)\">")
scripts := scriptReg.FindAllStringSubmatch(mainFileContent,-1)
//For each SRC found...
for _, path := range scripts {
//We open the corresponding file
subFileContent, err := ioutil.ReadFile(mainFileDir+path[1])
if err == nil {
//And we add its content to the "final" variable
fmt.Println(finalFileContent.Write(subFileContent))
} else {
fmt.Println(err)
}
}
//Try to display the final result
// fmt.Println(finalFileContent.String())
fmt.Printf(">>> %#v", finalFileContent)
fmt.Println("Y U NO WORKS? :'(")
} else {
fmt.Println(err)
}
So, each fmt.Println(finalFileContent.Write(subFileContent)) display something like 6161 , so I assume the Write() method is correctly executed.
But fmt.Printf(">>> %#v", finalFileContent) displays nothing. Absolutely nothing (even the ">>>" are not displayed!) And it's the same for the commented line just above.
The funny part is that the string "Y U NO WORK ? :'(" is correctly displayed...
Do you know why?
And do you know how to solve this issue?
Thanks in advance!
You are ignoring some errors. What are your results when you run the following version of your code?
package main
import (
"bytes"
"fmt"
"io/ioutil"
"path"
"regexp"
)
func main() {
//Open main file
mainFilePath := "/path/to/my/file.html"
mainFileDir := path.Dir(mainFilePath) + "/"
mainFileContent, err := ioutil.ReadFile(mainFilePath)
if err == nil {
mainFileContent := string(mainFileContent)
var finalFileContent bytes.Buffer
//Start RegExp searching for JavaScript src
scriptReg, _ := regexp.Compile("<script src=\"(.*)\">")
scripts := scriptReg.FindAllStringSubmatch(mainFileContent, -1)
//For each SRC found...
for _, path := range scripts {
//We open the corresponding file
subFileContent, err := ioutil.ReadFile(mainFileDir + path[1])
if err == nil {
//And we add its content to the "final" variable
// fmt.Println(finalFileContent.Write(subFileContent))
n, err := finalFileContent.Write(subFileContent)
fmt.Println("finalFileContent Write:", n, err)
} else {
fmt.Println(err)
}
}
//Try to display the final result
// fmt.Println(finalFileContent.String())
// fmt.Printf(">>> %#v", finalFileContent)
n, err := fmt.Printf(">>> %#v", finalFileContent)
fmt.Println()
fmt.Println("finalFileContent Printf:", n, err)
fmt.Println("Y U NO WORKS? :'(")
} else {
fmt.Println(err)
}
}
UPDATE:
The statement:
fmt.Println("finalFileContent Printf:", n, err)
Outputs:
finalFileContent Printf: 0 write /dev/stdout: winapi error #8
or
finalFileContent Printf: 0 write /dev/stdout: Not enough storage is available to process this command.
From MSDN:
ERROR_NOT_ENOUGH_MEMORY
8 (0x8)
Not enough storage is available to process this command.
The formatted output to the Windows console overflows the buffer (circa 64KB).
There is a related Go open issue:
Issue 3376: windows: detect + handle console in os.File.Write

Resources