Go: Deserialize array string - arrays

I have a string as: - ["a","b","c"]. How to parse / convert it into a Go array? I can do string parsing but is there any out of the box function in Go for the same.

How about using json.Unmarshal()?
s := `["a","b","c"]`
var arr []string
if err := json.Unmarshal([]byte(s), &arr); err != nil {
fmt.Println("Error:", err)
}
fmt.Printf("%q", arr)
Output (try it on the Go Playground):
["a" "b" "c"]
But know that package json does a lot of reflection kung-fu under the hood, it's faster if you write the parsing yourself. On the other hand, package json will also handle random white-spaces in the input – even newline characters and Unicode sequences, like this one (it's equivalent to ["a","b","c"]):
s := `[ "a" , "b"
,"\u0063" ] `

Related

Dont print secrets in log

I am using Zap logger and I want to limit the information that gets logged. For example , I have below code snippet
func (m *mountCommand) Execute(args []string) error {
filelogger.Info("First log", zap.Strings("input args", args))
And the log output is as below
{"level":"info","ts":"2017-11-16T10:04:40.225Z","msg":"First log","input args":["/var/lib/kubelet/pods/74785895-cab5-11e7-88ed-ce1c8b57856c/volumes/xyz-xandndnd",
"{\"kubernetes.io/secret/access-key\":\"Qdfnnfbbdnsjnxni8ehh=\",\"kubernetes.io/secret/secret-key\":\"GGHNHwsffUIJMNBNBVV==\",
\"parallel-count\":\"5\",\"region\":\"iam-standard\"}"]}
How do I just prevent the access-key and secret-key values from getting added to the log due to sensitivity of the data.
I tried many string manipulation functions in Go so that I can just remove those contents before using the zap object.There seems no simple function to get this done in a straight forward simple way.
Your current args is a slice of the following strings:
/var/lib/kubelet/pods/74785895-cab5-11e7-88ed-ce1c8b57856c/volumes/xyz-xandndnd
{"kubernetes.io/secret/access-key":"Qdfnnfbbdnsjnxni8ehh=","kubernetes.io/secret/secret-key":"GGHNHwsffUIJMNBNBVV==","parallel-count":"5","region":"iam-standard"}
The input argument that contains the security-sensitive data is at index 1, and it's a JSON text.
You should not "string-manipulate" a JSON text. First you should unmarshal it into a Go value, then manipulate it, then marshal it back.
This is how this can be done:
// Make a copy of args:
args2 := append([]string{}, args...)
// Unmarshal:
var m map[string]interface{}
if err := json.Unmarshal([]byte(args2[1]), &m); err != nil {
panic(err)
}
// Modify:
m["kubernetes.io/secret/access-key"] = "XXX"
m["kubernetes.io/secret/secret-key"] = "YYY"
// Marshal:
s2, err := json.Marshal(m)
if err != nil {
panic(err)
}
args2[1] = string(s2)
// Verify:
fmt.Println(args2[1])
// Now use args2 to log
filelogger.Info("First log", zap.Strings("(masked) input args", args2))
The "Verify:" is obviously not needed, it's just for us to see the result. Output on the Go Playground:
{"kubernetes.io/secret/access-key":"XXX","kubernetes.io/secret/secret-key":"YYY","parallel-count":"5","region":"iam-standard"}
In your solution you should also add slice index checks to avoid run-time panic.

GO Lang decode JSON (simple array not being picked up)

Structs and JSON are not super fun in Go.
I have a simple example of some JSON, and a struct. Everything seems to get parsed Okay, but for some reason the array does not get picked up.
Can anyone tell me what I might be missing.
Code---
package main
import (
"encoding/base64"
"fmt"
"encoding/json"
)
type Oauth struct {
Aud string `json:"aud"`
Cid string `json:"cid"`
Exp int `json:"exp"`
Iat int `json:"iat"`
Iss string `json:"iss"`
Jti string `json:"jti"`
Scp []string `json:"scp"`
Sub string `json:"sub"`
UID string `json:"uid"`
Ver int `json:"ver"`
}
func main () {
// This is the String {"ver":1,"jti":"AT.zgv9oQpw-7l3BCg6Xb5NCG2Pf8zxgiQa1EUBXycmaDk","iss":"https://companyx.okta.com/oauth2/aus1a4ibdat0JYw5s1d8","aud":"http://localhost","iat":1484538606,"exp":1484542206,"cid":"3jmNvVCFZ5F6lWOzIONO","uid":"00uy74c0h7NGTLBSXQOC","scp":["read","remove","reserve"],"sub":"oktaadmin#okta.com"}
encoded := "eyJ2ZXIiOjEsImp0aSI6IkFULnpndjlvUXB3LTdsM0JDZzZYYjVOQ0cyUGY4enhnaVFhMUVVQlh5Y21hRGsiLCJpc3MiOiJodHRwczovL2NvbXBhbnl4Lm9rdGEuY29tL29hdXRoMi9hdXMxYTRpYmRhdDBKWXc1czFkOCIsImF1ZCI6Imh0dHA6Ly9sb2NhbGhvc3QiLCJpYXQiOjE0ODQ1Mzg2MDYsImV4cCI6MTQ4NDU0MjIwNiwiY2lkIjoiM2ptTnZWQ0ZaNUY2bFdPeklPTk8iLCJ1aWQiOiIwMHV5NzRjMGg3TkdUTEJTWFFPQyIsInNjcCI6WyJyZWFkIiwicmVtb3ZlIiwicmVzZXJ2ZSJdLCJzdWIiOiJva3RhYWRtaW5Ab2t0YS5jb20ifQ"
data, _ := base64.StdEncoding.DecodeString(encoded)
fmt.Println(string(data))
fmt.Println ("")
var x Oauth
json.Unmarshal([]byte(data), &x)
fmt.Printf ("%+v",x.Scp);
}
The result is always an Empty Array []
The encoded string isn't a valid JSON, easy to detect because you're ignoring an important error on Unmarshal, try this:
err := json.Unmarshal(data, &x)
fmt.Println(err)
It looks like you missed the last '}' of your JSON.
Your base64-encoded string is not valid:
illegal base64 data at input byte 404

Get Golang flags as byte array

I am using flag package to read all the parameters I am passing to my Golang program. The problem is that if I pass an argument such as "\x41BC", it is not read as a 3 byte array (with chars 'A', 'B' and 'C'), but as a 6 byte array ('\', 'x', '4', '1', 'B', 'C').
If it could be useful to answer, I am reading that string using:
flag.StringVar(&param, "param", "", "the param with hex chars")
Is there a way to avoid this?
Thanks in advance!
"\x41BC" is a quoted string. The flag package does not do any unquoting, it will just hand you over the arguments that were specified when starting your application. You can use the strconv.Unquote() and strconv.UnquoteChar() functions to unquote them.
One thing you should be aware of is that strconv.Unquote() can only unquote strings that are in quotes (e.g. start and end with a quote char " or a back quote char `), so we have to manually append that.
See this example:
s := `\x41BC`
fmt.Println(s)
s2, err := strconv.Unquote(`"` + s + `"`)
if err != nil {
panic(err)
}
fmt.Println(s2)
Output (try it on the Go Playground):
\x41BC
ABC
So if you want to be able to provide quoted strings as command line arguments and still have the unquoted values, you have to unquote them with strconv.Unquote() after calling flag.Parse(), for example:
var param string
flag.StringVar(&param, "param", "", "the param with hex chars")
flag.Parse()
var err error
param, err = strconv.Unquote(`"` + param + `"`)
if err != nil {
panic(err) // handle error
}
// param now contains the unquoted argument value

Swift - Write an Array to a Text File

I read into myArray (native Swift) from a file containing a few thousand lines of plain text..
myData = String.stringWithContentsOfFile(myPath, encoding: NSUTF8StringEncoding, error: nil)
var myArray = myData.componentsSeparatedByString("\n")
I change some of the text in myArray (no point pasting any of this code).
Now I want to write the updated contents of myArray to a new file.
I've tried this ..
let myArray2 = myArray as NSArray
myArray2.writeToFile(myPath, atomically: false)
but the file content is then in the plist format.
Is there any way to write an array of text strings to a file (or loop through an array and append each array item to a file) in Swift (or bridged Swift)?
As drewag points out in the accepted post, you can build a string from the array and then use the writeToFile method on the string.
However, you can simply use Swift's Array.joinWithSeparator to accomplish the same with less code and likely better performance.
For example:
// swift 2.0
let array = [ "hello", "goodbye" ]
let joined = array.joinWithSeparator("\n")
do {
try joined.writeToFile(saveToPath, atomically: true, encoding: NSUTF8StringEncoding)
} catch {
// handle error
}
// swift 1.x
let array = [ "hello", "goodbye" ]
let joined = "\n".join(array)
joined.writeToFile(...)
With Swift 5 and I guess with Swift 4 you can use code snippet which works fine to me.
let array = ["hello", "world"]
let joinedStrings = array.joined(separator: "\n")
do {
try joinedStrings.write(toFile: outputURL.path, atomically: true, encoding: .utf8)
} catch let error {
// handle error
print("Error on writing strings to file: \(error)")
}
You need to reduce your array back down to a string:
var output = reduce(array, "") { (existing, toAppend) in
if existing.isEmpty {
return toAppend
}
else {
return "\(existing)\n\(toAppend)"
}
}
output.writeToFile(...)
The reduce method takes a collection and merges it all into a single instance. It takes an initial instance and closure to merge all elements of the collection into that original instance.
My example takes an empty string as its initial instance. The closure then checks if the existing output is empty. If it is, it only has to return the text to append, otherwise, it uses String Interpolation to return the existing output and the new element with a newline in between.
Using various syntactic sugar features from Swift, the whole reduction can be reduced to:
var output = reduce(array, "") { $0.isEmpty ? $1 : "\($0)\n\($1)" }
Swift offers numerous ways to loop through an array. You can loop through the strings and print to a text file one by one. Something like so:
for theString in myArray {
theString.writeToFile(myPath, atomically: false, encoding: NSUTF8StringEncoding, error: nil);
}

How to Unmarshal the json array?

There is something wrong when I unmarshal the json array.
How do I correct it ? the code is:http://play.golang.org/p/AtU9q8Hlye
package main
import (
"encoding/json"
"fmt"
)
type Server struct {
ServerName string
ServerIP string
}
type Serverslice struct {
Name string
Servers []Server
}
func main() {
var s []Serverslice
str := `{"name":"dxh","servers":[{"serverName":"VPN0","serverIP":"127.0.0.1"},{"serverName":"Beijing_VPN","serverIP":"127.0.0.2"}],
"name":"dxh1,"servers":[{"serverName":"VPN1","serverIP":"127.0.0.1"},{"serverName":"Beijing_VPN","serverIP":"127.0.0.2"}]}`
json.Unmarshal([]byte(str), &s) //the wrong line.....................
fmt.Println(len(s))
}
First of all, you're ignoring the error return value from json.Unmarshal. You probably want something like:
if err := json.Unmarshal([]byte(str), &s); err != nil {
log.Fatalln(err)
}
With that change, we can see that your JSON data isn't valid: invalid character 's' after object key:value pair. There is a missing quote at the end of "dxh1 on the second line.
Fixing that error and rerunning the program you'll get a different error: json: cannot unmarshal object into Go value of type []main.Serverslice. There are two possible problems here:
You meant to decode into an object. In this case, just declare s as a Serverslice. Here is a version of your program that makes that change: http://play.golang.org/p/zgyr_vnn-_
Your JSON is supposed to be an array (possible, since it seems to have duplicate keys). Here's an updated version with the JSON changed to provide an array: http://play.golang.org/p/Wl6kUaivEm

Resources