How to check whether two slices are backed up by the same array?
For example:
a := []int{1, 2, 3}
b := a[0:1]
c := a[2:3]
alias(b, c) == true
How should alias look like?
In general you can't tell if the backing array is shared between 2 slices, because using a full slice expression, one might control the capacity of the resulting slice, and then there will be no overlap even when checking the capacity.
As an example, if you have a backing array with 10 elements, a slice may be created that only contains the first 2 elements, and its capacity might be 2. And another slice may be create that only holds its last 2 elements, its capacity again being 2.
See this example:
a := [10]int{}
x := a[0:2:2]
y := a[8:10:10]
fmt.Println("len(x) = ", len(x), ", cap(x) = ", cap(x))
fmt.Println("len(y) = ", len(y), ", cap(y) = ", cap(y))
The above will print that both lengths and capcities of x and y are 2. They obviously have the same backing array, but you won't have any means to tell that.
Edit: I've misunderstood the question, and the following describes how to tell if (elements of) 2 slices overlap.
There is no language support for this, but since slices have a contiguous section of some backing array, we can check if the address range of their elements overlap.
Unfortunately pointers are not ordered in the sense that we can't apply the < and > operators on them (there are pointers in Go, but there is no pointer arithmetic). And checking if all the addresses of the elements of the first slice matches any from the second, that's not feasible.
But we can obtain a pointer value (an address) as a type of uintptr using the reflect package, more specifically the Value.Pointer() method (or we could also do that using package unsafe, but reflect is "safer"), and uintptr values are integers, they are ordered, so we can compare them.
So what we can do is obtain the addresses of the first and last elements of the slices, and by comparing them, we can tell if they overlap.
Here's a simple implementation:
func overlap(a, b []int) bool {
if len(a) == 0 || len(b) == 0 {
return false
}
amin := reflect.ValueOf(&a[0]).Pointer()
amax := reflect.ValueOf(&a[len(a)-1]).Pointer()
bmin := reflect.ValueOf(&b[0]).Pointer()
bmax := reflect.ValueOf(&b[len(b)-1]).Pointer()
return !(amax < bmin || amin > bmax)
}
Testing it:
a := []int{0, 1, 2, 3}
b := a[0:2]
c := a[2:4]
d := a[0:3]
fmt.Println(overlap(a, b)) // true
fmt.Println(overlap(b, c)) // false
fmt.Println(overlap(c, d)) // true
Try it on the Go Playground.
Found one way of this here. The idea is that while I don't think there's a way of finding the beginning of the backing array, ptr + cap of a slice should[*] point to the end of it. So then one compares the last pointer for equality, like:
func alias(x, y nat) bool {
return cap(x) > 0 && cap(y) > 0 && &x[0:cap(x)][cap(x)-1] == &y[0:cap(y)][cap(y)-1]
}
[*] The code includes the following note:
Note: alias assumes that the capacity of underlying arrays is never changed for nat values; i.e. that there are no 3-operand slice expressions in this code (or worse, reflect-based operations to the same effect).
Related
A basic question that I'm struggling to find an answer for as there are a lot of answers about how to join two slices using the append function and the spread operator which erroneously use the word 'array'.
I am new to Go and have made the assumption that using sized arrays is good practice where the size is known. However I am struggling to work with arrays as I can't figure out how to do simple operations such as concatenation. Here is some code.
var seven [7]int
five := [5]int{1,2,3,4,5}
two := [2]int{6,7}
//this doesn't work as both the inputs and assignment are the wrong type
seven = append(five,two)
//this doesn't work as the assignment is still the wrong type
seven = append(five[:],two[:])
//this works but I'm not using arrays anymore so may as well use slices everywhere and forget sizing
seven2 := append(five[:],two[:])
As far as I can see I can either just give up on arrays and use slices exclusively or I could write a loop to explicitly construct the new array. Is there a third option?
append() can only be used to append elements to a slice. If you have an array, you can't pass that directly to append().
What you may do is slice the array, so you get a slice (which will use the array as its backing store), and you can use that slice as the target and source of elements.
For example:
s := seven[:0]
s = append(s, five[:]...)
s = append(s, two[:]...)
fmt.Println(seven)
This will print (try it on the Go Playground):
[1 2 3 4 5 6 7]
Also note that since append() returns the resulting slice, it's possible to write all this in one line:
_ = append(append(seven[:0], five[:]...), two[:]...)
(Storing the result is not needed here because we have and want to use only the backing array, but in general that is not the case.)
This outputs the same, try it on the Go Playground. Although this isn't very readable, so it's not worth compacting it into a single line.
Although when you have the target array, "appending" arrays is nothing more than copying them to the target, to the proper position. For that, you may use the builtin copy() function too. Note that the copy() function also accepts only slices, so you have to slice the arrays here too.
copy(seven[:], five[:])
copy(seven[len(five):], two[:])
fmt.Println(seven)
This will output the same. Try this one on the Go Playground.
You can use copy
copy(seven[:], five[:])
copy(seven[5:], two[:])
fmt.Printf("%v\n", seven)
> [1 2 3 4 5 6 7]
You can concatenate two arrays in go using copy function
package main
import "fmt"
func main() {
five := [5]int{1, 2, 3, 4, 5}
two := [2]int{6, 7}
var n [len(five) + len(two)]int
copy(n[:], five[:])
copy(n[len(five):], two[:])
fmt.Println(n)
}
https://blog.golang.org/go-slices-usage-and-internals
Golang runtime used to check whether current index exceeds the maximum possible.
On the side of array, it look ups its type (which contain its len and reference to the element type), because that's type, that can be registered only at compile time.
// each array mention with unique size creates new type
array := [5]byte{1,2,3,4,5}
On the side of slice, it look ups their header which looks like:
type slice {
data *byte
len int
cap int // capacity, the maximum possible index
}
As you can see, any slice is a single structure with data and len, cap fields, meanwhile array is just single pointer to data (*byte).
When you trying to convert array to slice, it just creates slice header and fills fields with:
slice := array[:]
==
slice := Slice{}
slice.data = array
slice.len = type_of(array).len
slice.cap = type_of(array).len
you can do that simply by converting array into slice:
arr1 := [...]int {1,2,3,}
arr2 := [...]int {4,5,6, }
//arr3 = arr1 + arr2 // not allowed
// converting arrays into slice
slc_arr1, slc_arr2 := arr1[:], arr2[:]
slc_arr3 := make([]int, 0)
slc_arr3 = append(slc_arr1, slc_arr2...)
fmt.Println(slc_arr3) // [1 2 3 4 5 6]
There is a more general way of appending an array of any type(once Golang has generics, but for now this solution is specific to strings. Just change the type as appropriate). The notion of Fold comes from Functional Programming. Note I have also included a filter function which also uses Fold. The solution is not stack safe but in many cases that does not matter. It can be made stack safe with trampolining. At the end is an example of its usage.
func FoldRightStrings(as, z []string, f func(string, []string) []string) []string {
if len(as) > 1 { //Slice has a head and a tail.
h, t := as[0], as[1:len(as)]
return f(h, FoldRightStrings(t, z, f))
} else if len(as) == 1 { //Slice has a head and an empty tail.
h := as[0]
return f(h, FoldRightStrings([]string{}, z, f))
}
return z
}
func FilterStrings(as []string, p func(string) bool) []string {
var g = func(h string, accum []string) []string {
if p(h) {
return append(accum, h)
} else {
return accum
}
}
return FoldRightStrings(as, []string{}, g)
}
func AppendStrings(as1, as2 []string) []string {
var g = func(h string, accum []string) []string {
return append(accum, h)
}
return FoldRightStrings(as1, as2, g)
}
func TestAppendStringArrays(t *testing.T) {
strings := []string{"a","b","c"}
bigarray := AppendStrings(AppendStrings(strings, strings),AppendStrings(strings, strings))
if diff := deep.Equal(bigarray, []string{"a","b","c","c","b","a","a","b","c","c","b","a"}); diff != nil {
t.Error(diff)
}
}
Correct me if I'm wrong, there are only 3 types of loops in Go.
Type1 (The most basic type, with a single condition):
for i <= 3 {...}
Type2 (Classical for-loop)
for j := 7; j <= 9; j++ {...}
Type3 (infinite loop rely on break)
for {...break}
Then I come across this for loop that sums the value from array
nums := []int{2, 3, 4}
sum := 0
for _, num := range nums {
sum += num
}
fmt.Println("sum:", sum)//"sum: 9"
Is the above for-loop to be considered Type1 where it automatically applies <= and range of nums as max value? Can I in any way change the value? maybe I need two extra loops? Can we apply something like range + 2?
From Effective Go:
The Go for loop is similar to—but not the same as—C's. It unifies for and while and there is no do-while. There are three forms, only one of which has semicolons.
// Like a C for
for init; condition; post { }
// Like a C while
for condition { }
// Like a C for(;;)
for { }
It continues:
If you're looping over an array, slice, string, or map, or reading from a channel, a range clause can manage the loop.
for key, value := range oldMap {
newMap[key] = value
}
From this I think of range loops as a for condition { } loop, where the condition (such as it is) is that the variables being designated as the values of the array/slice/string/map/chan are not nil, though in practice even explicitly nil values work
for _, v := range []interface{}{nil, nil, nil, nil} {
// will still iterate four times
}
In reality it might be more useful to think of Go's for loop as a contextual combination of a C-style for init; condition; post loop, a classical while loop, and a more modern foreach loop.
Type1 will work almost like while in other languages. Since there is no while in go language it uses for in that case.
Type2 is the classic for loop like in other languages as you have also stated.
Type3 is used to range over various data structures like arrays, slice, maps
range on arrays and slices provides both the index and value for each
entry like the example you have given
nums := []int{2, 3, 4}
sum := 0
for _, num := range nums {
sum += num
}
fmt.Println("sum:", sum)//"sum: 9"
range on map iterates over key/value pairs.
kvs := map[string]string{"a": "apple", "b": "banana"}
for k, v := range kvs {
fmt.Printf("%s -> %s\n", k, v)
}
range on strings iterates over Unicode code points. The first value is
the starting byte index of the rune and the second the rune itself.
for i, c := range "go" {
fmt.Println(i, c)
}
For more information, Check Go by example website for usage of range
I am trying to create a 2d array in Go:
board := make([][]string, m)
for i := range board {
board[i] = make([]string, n)
}
However, given the verbosity of that, I am wondering if there is a better or more succinct way to handle this problem (either to generate dynamic arrays, or a different/idiomatic data-structure to handle such board-game like data)?
Background:
this is for a board game
the dimensions of the board are not known until a user starts playing (so, say, MxN).
I want to store an arbitrary character (or a single char string) in each cell. In my TicTacToe game that will be the 'X' or an 'O' (or any other character the user chooses).
What you are building in your sample code is not a 2D array, but rather a slice of slices: each of the sub-slices could be of a different length with this type, which is why you have separate allocations for each.
If you want to represent the board with a single allocation though, one option would be to allocate a single slice, and then use simple arithmetic to determine where elements are. For example:
board := make([]string, m*n)
board[i*m + j] = "abc" // like board[i][j] = "abc"
The way you described creates a slice of slices, which looks similar to a 2d array that you want. I would suggest you to change the type to uint8, as you only care about 3 states nothing / first / second player.
This allocates each row separately (you will see at least m + 1 allocs/op in your benchmarks). This is not really nice because there is no guarantee that the separate allocations would be localized close to each other.
To maintain locality you can do something like this:
M := make([][]uint8, row)
e := make([]uint8, row * col)
for i := range M {
a[i] = e[i * col:(i + 1) * col]
}
This will end up with only 2 allocations and the slice of slices will maintain data locality. Note that you will still be able to access your M in 2d format M[2][6].
A good video which explains how to do this even faster.
For a multi dimensional array, we can have any of the 2 uses cases,
You know the dimensions of array while compiling
You get to know the array dimension only at runtime, ie may from the user
input or so
For use case 1
matr := [5][5]int{}
For use case 2
var m, n int
fmt.Scan(&m, &n)
var mat = make([][]int, m)
for i := range mat {
mat[i] = make([]int, n)
fmt.Printf("Row %d: %v\n", i, mat[i])
}
In short, we have to rely on make for creating dynamic arrays
I am trying to write an application that reads RPM files. The start of each block has a Magic char of [4]byte.
Here is my struct
type Lead struct {
Magic [4]byte
Major, Minor byte
Type uint16
Arch uint16
Name string
OS uint16
SigType uint16
}
I am trying to do the following:
lead := Lead{}
lead.Magic = buffer[0:4]
I am searching online and not sure how to go from a slice to an array (without copying). I can always make the Magic []byte (or even uint64), but I was more curious on how would I go from type []byte to [4]byte if needed to?
The built in method copy will only copy a slice to a slice NOT a slice to an array.
You must trick copy into thinking the array is a slice
copy(varLead.Magic[:], someSlice[0:4])
Or use a for loop to do the copy:
for index, b := range someSlice {
varLead.Magic[index] = b
}
Or do as zupa has done using literals. I have added onto their working example.
Go Playground
You have allocated four bytes inside that struct and want to assign a value to that four byte section. There is no conceptual way to do that without copying.
Look at the copy built-in for how to do that.
Try this:
copy(lead.Magic[:], buf[0:4])
Tapir Liui (auteur de Go101) twitte:
Go 1.18 1.19 1.20 will support conversions from slice to array: golang/go issues 46505.
So, since Go 1.18,the slice copy2 implementation could be written as:
*(*[N]T)(d) = [N]T(s)
or, even simpler if the conversion is allowed to present as L-values:
[N]T(d) = [N]T(s)
Without copy, you can convert, with the next Go 1.17 (Q3 2021) a slice to an array pointer.
This is called "un-slicing", giving you back a pointer to the underlying array of a slice, again, without any copy/allocation needed:
See golang/go issue 395: spec: convert slice x into array pointer, now implemented with CL 216424/, and commit 1c26843
Converting a slice to an array pointer yields a pointer to the underlying array of the slice.
If the length of the slice is less than the length of the array,
a run-time panic occurs.
s := make([]byte, 2, 4)
s0 := (*[0]byte)(s) // s0 != nil
s2 := (*[2]byte)(s) // &s2[0] == &s[0]
s4 := (*[4]byte)(s) // panics: len([4]byte) > len(s)
var t []string
t0 := (*[0]string)(t) // t0 == nil
t1 := (*[1]string)(t) // panics: len([1]string) > len(s)
So in your case, provided Magic type is *[4]byte:
lead.Magic = (*[4]byte)(buffer)
Note: type aliasing will work too:
type A [4]int
var s = (*A)([]int{1, 2, 3, 4})
Why convert to an array pointer? As explained in issue 395:
One motivation for doing this is that using an array pointer allows the compiler to range check constant indices at compile time.
A function like this:
func foo(a []int) int
{
return a[0] + a[1] + a[2] + a[3];
}
could be turned into:
func foo(a []int) int
{
b := (*[4]int)(a)
return b[0] + b[1] + b[2] + b[3];
}
allowing the compiler to check all the bounds once only and give compile-time errors about out of range indices.
Also:
One well-used example is making classes as small as possible for tree nodes or linked list nodes so you can cram as many of them into L1 cache lines as possible.
This is done by each node having a single pointer to a left sub-node, and the right sub-node being accessed by the pointer to the left sub-node + 1.
This saves the 8-bytes for the right-node pointer.
To do this you have to pre-allocate all the nodes in a vector or array so they're laid out in memory sequentially, but it's worth it when you need it for performance.
(This also has the added benefit of the prefetchers being able to help things along performance-wise - at least in the linked list case)
You can almost do this in Go with:
type node struct {
value int
children *[2]node
}
except that there's no way of getting a *[2]node from the underlying slice.
Go 1.20 (Q1 2023): this is addressed with CL 430415, 428938 (type), 430475 (reflect) and 429315 (spec).
Go 1.20
You can convert from a slice to an array directly with the usual conversion syntax T(x). The array's length can't be greater than the slice's length:
func main() {
slice := []int64{10, 20, 30, 40}
array := [4]int64(slice)
fmt.Printf("%T\n", array) // [4]int64
}
Go 1.17
Starting from Go 1.17 you can directly convert a slice to an array pointer. With Go's type conversion syntax T(x) you can do this:
slice := make([]byte, 4)
arrptr := (*[4]byte)(slice)
Keep in mind that the length of the array must not be greater than the length of the slice, otherwise the conversion will panic.
bad := (*[5]byte)(slice) // panics: slice len < array len
This conversion has the advantage of not making any copy, because it simply yields a pointer to the underlying array.
Of course you can dereference the array pointer to obtain a non-pointer array variable, so the following also works:
slice := make([]byte, 4)
var arr [4]byte = *(*[4]byte)(slice)
However dereferencing and assigning will subtly make a copy, since the arr variable is now initialized to the value that results from the conversion expression. To be clear (using ints for simplicity):
v := []int{10,20}
a := (*[2]int)(v)
a[0] = 500
fmt.Println(v) // [500 20] (changed, both point to the same backing array)
w := []int{10,20}
b := *(*[2]int)(w)
b[0] = 500
fmt.Println(w) // [10 20] (unchanged, b holds a copy)
One might wonder why the conversion checks the slice length and not the capacity (I did). Consider the following program:
func main() {
a := []int{1,2,3,4,5,6}
fmt.Println(cap(a)) // 6
b := a[:3]
fmt.Println(cap(a)) // still 6
c := (*[3]int)(b)
ptr := uintptr(unsafe.Pointer(&c[0]))
ptr += 3 * unsafe.Sizeof(int(0))
i := (*int)(unsafe.Pointer(ptr))
fmt.Println(*i) // 4
}
The program shows that the conversion might happen after reslicing. The original backing array with six elements is still there, so one might wonder why a runtime panic occurs with (*[6]int)(b) where cap(b) == 6.
This has actually been brought up. It's worth to remember that, unlike slices, an array has fixed size, therefore it needs no notion of capacity, only length:
a := [4]int{1,2,3,4}
fmt.Println(len(a) == cap(a)) // true
You might be able to do the whole thing with one read, instead of reading individually into each field. If the fields are fixed-length, then you can do:
lead := Lead{}
// make a reader to dispense bytes so you don't have to keep track of where you are in buffer
reader := bytes.NewReader(buffer)
// read into each field in Lead, so Magic becomes buffer[0:4],
// Major becomes buffer[5], Minor is buffer[6], and so on...
binary.Read(reader, binary.LittleEndian, &lead)
Don't. Slice itself is suffice for all purpose. Array in go lang should be regarded as the underlying structure of slice. In every single case, use only slice. You don't have to array yourself. You just do everything by slice syntax. Array is only for computer. In most cases, slice is better, clear in code. Even in other cases, slice still is sufficient to reflex your idea.
I try to write a small application in go that takes 'x' numbers of integers from standard input, calculates the mean and gives it back. I have only gotten so far:
func main() {
var elems, mean int
sum := 0
fmt.Print("Number of elements? ")
fmt.Scan(&elems)
var array = new([elems]int)
for i := 0; i < elems; i++ {
fmt.Printf("%d . Number? ", i+1)
fmt.Scan(&array[i])
sum += array[i];
}............
When trying to compile this I get the following error message:
invalid array bound elems
What is wrong here?
You should use a slice instead of an array:
//var array = new([elems]int) - no, arrays are not dynamic
var slice = make([]int,elems) // or slice := make([]int, elems)
See "go slices usage and internals". Also you may want to consider using range for your loop:
// for i := 0; i < elems; i++ { - correct but less idiomatic
for i, v := range slice {
In my opinion, this results from confusion over the usage of the new and make functions. This is a known issue/feature in the Go language, as evidenced by several discussions about new vs make at golang-nuts.
The difference between new and make may become clearer by letting Go print out the type of the value created by new and make:
package main
import "fmt"
func main() {
fmt.Printf("%T %v\n", new([10]int), new([10]int))
fmt.Printf("%T %v\n", make([]int, 10), make([]int, 10))
}
The output:
*[10]int &[0 0 0 0 0 0 0 0 0 0]
[]int [0 0 0 0 0 0 0 0 0 0]
As can be seen from the type, to access an array element of new([10]int) we would first need to dereference the pointer.
Both new and make require a Go type as their 1st argument. However, the expression [elems]int is not a Go type (unless elems is a Go constant, which isn't the case here).
For further reference, see http://golang.org/doc/go_spec.html#Allocation and http://golang.org/doc/go_spec.html#The_zero_value.
To get a better understanding of whether the result of new is usable, it may be helpful to lookup whether len and cap work with zero (nil) values: http://golang.org/doc/go_spec.html#Length_and_capacity
See The Go Programming Language Specification
http://golang.org/ref/spec#Array_types
http://golang.org/ref/spec#Constants
It says:"The length is part of the array's type; it must evaluate to a non- negative constant representable by a value of type int. "
Constants by no means vary.