I am using flag package to read all the parameters I am passing to my Golang program. The problem is that if I pass an argument such as "\x41BC", it is not read as a 3 byte array (with chars 'A', 'B' and 'C'), but as a 6 byte array ('\', 'x', '4', '1', 'B', 'C').
If it could be useful to answer, I am reading that string using:
flag.StringVar(¶m, "param", "", "the param with hex chars")
Is there a way to avoid this?
Thanks in advance!
"\x41BC" is a quoted string. The flag package does not do any unquoting, it will just hand you over the arguments that were specified when starting your application. You can use the strconv.Unquote() and strconv.UnquoteChar() functions to unquote them.
One thing you should be aware of is that strconv.Unquote() can only unquote strings that are in quotes (e.g. start and end with a quote char " or a back quote char `), so we have to manually append that.
See this example:
s := `\x41BC`
fmt.Println(s)
s2, err := strconv.Unquote(`"` + s + `"`)
if err != nil {
panic(err)
}
fmt.Println(s2)
Output (try it on the Go Playground):
\x41BC
ABC
So if you want to be able to provide quoted strings as command line arguments and still have the unquoted values, you have to unquote them with strconv.Unquote() after calling flag.Parse(), for example:
var param string
flag.StringVar(¶m, "param", "", "the param with hex chars")
flag.Parse()
var err error
param, err = strconv.Unquote(`"` + param + `"`)
if err != nil {
panic(err) // handle error
}
// param now contains the unquoted argument value
Related
I am using Zap logger and I want to limit the information that gets logged. For example , I have below code snippet
func (m *mountCommand) Execute(args []string) error {
filelogger.Info("First log", zap.Strings("input args", args))
And the log output is as below
{"level":"info","ts":"2017-11-16T10:04:40.225Z","msg":"First log","input args":["/var/lib/kubelet/pods/74785895-cab5-11e7-88ed-ce1c8b57856c/volumes/xyz-xandndnd",
"{\"kubernetes.io/secret/access-key\":\"Qdfnnfbbdnsjnxni8ehh=\",\"kubernetes.io/secret/secret-key\":\"GGHNHwsffUIJMNBNBVV==\",
\"parallel-count\":\"5\",\"region\":\"iam-standard\"}"]}
How do I just prevent the access-key and secret-key values from getting added to the log due to sensitivity of the data.
I tried many string manipulation functions in Go so that I can just remove those contents before using the zap object.There seems no simple function to get this done in a straight forward simple way.
Your current args is a slice of the following strings:
/var/lib/kubelet/pods/74785895-cab5-11e7-88ed-ce1c8b57856c/volumes/xyz-xandndnd
{"kubernetes.io/secret/access-key":"Qdfnnfbbdnsjnxni8ehh=","kubernetes.io/secret/secret-key":"GGHNHwsffUIJMNBNBVV==","parallel-count":"5","region":"iam-standard"}
The input argument that contains the security-sensitive data is at index 1, and it's a JSON text.
You should not "string-manipulate" a JSON text. First you should unmarshal it into a Go value, then manipulate it, then marshal it back.
This is how this can be done:
// Make a copy of args:
args2 := append([]string{}, args...)
// Unmarshal:
var m map[string]interface{}
if err := json.Unmarshal([]byte(args2[1]), &m); err != nil {
panic(err)
}
// Modify:
m["kubernetes.io/secret/access-key"] = "XXX"
m["kubernetes.io/secret/secret-key"] = "YYY"
// Marshal:
s2, err := json.Marshal(m)
if err != nil {
panic(err)
}
args2[1] = string(s2)
// Verify:
fmt.Println(args2[1])
// Now use args2 to log
filelogger.Info("First log", zap.Strings("(masked) input args", args2))
The "Verify:" is obviously not needed, it's just for us to see the result. Output on the Go Playground:
{"kubernetes.io/secret/access-key":"XXX","kubernetes.io/secret/secret-key":"YYY","parallel-count":"5","region":"iam-standard"}
In your solution you should also add slice index checks to avoid run-time panic.
Structs and JSON are not super fun in Go.
I have a simple example of some JSON, and a struct. Everything seems to get parsed Okay, but for some reason the array does not get picked up.
Can anyone tell me what I might be missing.
Code---
package main
import (
"encoding/base64"
"fmt"
"encoding/json"
)
type Oauth struct {
Aud string `json:"aud"`
Cid string `json:"cid"`
Exp int `json:"exp"`
Iat int `json:"iat"`
Iss string `json:"iss"`
Jti string `json:"jti"`
Scp []string `json:"scp"`
Sub string `json:"sub"`
UID string `json:"uid"`
Ver int `json:"ver"`
}
func main () {
// This is the String {"ver":1,"jti":"AT.zgv9oQpw-7l3BCg6Xb5NCG2Pf8zxgiQa1EUBXycmaDk","iss":"https://companyx.okta.com/oauth2/aus1a4ibdat0JYw5s1d8","aud":"http://localhost","iat":1484538606,"exp":1484542206,"cid":"3jmNvVCFZ5F6lWOzIONO","uid":"00uy74c0h7NGTLBSXQOC","scp":["read","remove","reserve"],"sub":"oktaadmin#okta.com"}
encoded := "eyJ2ZXIiOjEsImp0aSI6IkFULnpndjlvUXB3LTdsM0JDZzZYYjVOQ0cyUGY4enhnaVFhMUVVQlh5Y21hRGsiLCJpc3MiOiJodHRwczovL2NvbXBhbnl4Lm9rdGEuY29tL29hdXRoMi9hdXMxYTRpYmRhdDBKWXc1czFkOCIsImF1ZCI6Imh0dHA6Ly9sb2NhbGhvc3QiLCJpYXQiOjE0ODQ1Mzg2MDYsImV4cCI6MTQ4NDU0MjIwNiwiY2lkIjoiM2ptTnZWQ0ZaNUY2bFdPeklPTk8iLCJ1aWQiOiIwMHV5NzRjMGg3TkdUTEJTWFFPQyIsInNjcCI6WyJyZWFkIiwicmVtb3ZlIiwicmVzZXJ2ZSJdLCJzdWIiOiJva3RhYWRtaW5Ab2t0YS5jb20ifQ"
data, _ := base64.StdEncoding.DecodeString(encoded)
fmt.Println(string(data))
fmt.Println ("")
var x Oauth
json.Unmarshal([]byte(data), &x)
fmt.Printf ("%+v",x.Scp);
}
The result is always an Empty Array []
The encoded string isn't a valid JSON, easy to detect because you're ignoring an important error on Unmarshal, try this:
err := json.Unmarshal(data, &x)
fmt.Println(err)
It looks like you missed the last '}' of your JSON.
Your base64-encoded string is not valid:
illegal base64 data at input byte 404
I have a string as: - ["a","b","c"]. How to parse / convert it into a Go array? I can do string parsing but is there any out of the box function in Go for the same.
How about using json.Unmarshal()?
s := `["a","b","c"]`
var arr []string
if err := json.Unmarshal([]byte(s), &arr); err != nil {
fmt.Println("Error:", err)
}
fmt.Printf("%q", arr)
Output (try it on the Go Playground):
["a" "b" "c"]
But know that package json does a lot of reflection kung-fu under the hood, it's faster if you write the parsing yourself. On the other hand, package json will also handle random white-spaces in the input – even newline characters and Unicode sequences, like this one (it's equivalent to ["a","b","c"]):
s := `[ "a" , "b"
,"\u0063" ] `
I would like to run my go file on my input.txt where my go program will read the input.txt file when I type in go run command ie:
go run goFile.go input.txt
I don't want to put input.txt in my goFile.go code since my go file should run on any input name not just input.txt.
I try ioutil.ReadAll(os.Stdin) but I need to change my command to
go run goFile.go < input.txt
I only use package fmt, os, bufio and io/ioutil. Is it possible to do it without any other packages?
Please take a look at the package documentation of io/ioutil which you are already using.
It has a function exactly for this: ReadFile()
func ReadFile(filename string) ([]byte, error)
Example usage:
func main() {
// First element in os.Args is always the program name,
// So we need at least 2 arguments to have a file name argument.
if len(os.Args) < 2 {
fmt.Println("Missing parameter, provide file name!")
return
}
data, err := ioutil.ReadFile(os.Args[1])
if err != nil {
fmt.Println("Can't read file:", os.Args[1])
panic(err)
}
// data is the file content, you can use it
fmt.Println("File content is:")
fmt.Println(string(data))
}
Firs you check for the provided argument. If the first argument satisfy the condition of an input file, then you use the ioutil.ReadFile method, providing parameter the os.Args result.
package main
import (
"fmt"
"os"
"io/ioutil"
)
func main() {
if len(os.Args) < 1 {
fmt.Println("Usage : " + os.Args[0] + " file name")
os.Exit(1)
}
file, err := ioutil.ReadFile(os.Args[1])
if err != nil {
fmt.Println("Cannot read the file")
os.Exit(1)
}
// do something with the file
fmt.Print(string(file))
}
Another possibility is to use:
f, err := os.Open(os.Args[0])
but for this you need to provide the bytes lenght to read:
b := make([]byte, 5) // 5 is the length
n, err := f.Read(b)
fmt.Printf("%d bytes: %s\n", n, string(b))
For running .go file from command-line by input parameter like file (for example abc.txt).We need use mainly os, io/ioutil, fmt packages. Additionally for reading command line parameters we use
os.Args Here is example code
package main
import (
"fmt"
"os"
"io/ioutil"
)
func main() {
fmt.Println(" Hi guys ('-') ")
input_files := os.Args[1:]
//input_files2 := os.Args[0];
//fmt.Println("if2 : ",input_files2)
if len(input_files) < 1{
fmt.Println("Not detected files.")
}else{
fmt.Println("File_name is : ",input_files[0])
content, err := ioutil.ReadFile(input_files[0])
if err != nil {
fmt.Println("Can't read file :", input_files[0],"Error : ",err)
}else {
fmt.Println("Output file content is(like string type) : \n",string(content))//string Output
fmt.Println("Output file content is(like byte type) : \n",content)//bytes Output
}
}
}
Args holds command line arguments, including the command as Args[0].
If the Args field is empty or nil, Run uses {Path}.
In typical use, both Path and Args are set by calling Command.
Args []string
function. This function return back array on string type https://golang.org/pkg/os/exec/.Args hold the command-line arguments, starting with the program name. In this case short way to take filename from command-line is this functions os.Args[1:] . And here is output
elshan_abd$ go run main.go abc.txt
Hi guys ('-')
File_name is : abc.txt
Output file content is(like string type) :
aaa
bbb
ccc
1234
Output file content is(like byte type) :
[97 97 97 10 98 98 98 10 99 99 99 10 49 50 51 52 10]
Finally we need for reading content file this function
func ReadFile(filename string) ([]byte, error) source is https://golang.org/pkg/io/ioutil/#ReadFile
There is something wrong when I unmarshal the json array.
How do I correct it ? the code is:http://play.golang.org/p/AtU9q8Hlye
package main
import (
"encoding/json"
"fmt"
)
type Server struct {
ServerName string
ServerIP string
}
type Serverslice struct {
Name string
Servers []Server
}
func main() {
var s []Serverslice
str := `{"name":"dxh","servers":[{"serverName":"VPN0","serverIP":"127.0.0.1"},{"serverName":"Beijing_VPN","serverIP":"127.0.0.2"}],
"name":"dxh1,"servers":[{"serverName":"VPN1","serverIP":"127.0.0.1"},{"serverName":"Beijing_VPN","serverIP":"127.0.0.2"}]}`
json.Unmarshal([]byte(str), &s) //the wrong line.....................
fmt.Println(len(s))
}
First of all, you're ignoring the error return value from json.Unmarshal. You probably want something like:
if err := json.Unmarshal([]byte(str), &s); err != nil {
log.Fatalln(err)
}
With that change, we can see that your JSON data isn't valid: invalid character 's' after object key:value pair. There is a missing quote at the end of "dxh1 on the second line.
Fixing that error and rerunning the program you'll get a different error: json: cannot unmarshal object into Go value of type []main.Serverslice. There are two possible problems here:
You meant to decode into an object. In this case, just declare s as a Serverslice. Here is a version of your program that makes that change: http://play.golang.org/p/zgyr_vnn-_
Your JSON is supposed to be an array (possible, since it seems to have duplicate keys). Here's an updated version with the JSON changed to provide an array: http://play.golang.org/p/Wl6kUaivEm