Is there any way I can keep a strong reference in Go?
Given the following convoluted piece of code:
package main
import (
"fmt"
)
func main() {
slice := make([]int, 5)
slice[3] = 25 // whatever index between 0 and 4 included I don't care
slicesArray := make([]*[]int, 2)
slicesArray[0] = &slice
fmt.Println((*(slicesArray[0]))[3])
slice = nil
fmt.Println((*(slicesArray[0]))[3])
}
This program of course crashes because once setting the slice to nil the garbage collector marks the memory area as dirty.
But is there a way to tell Go that my slice of pointers to slices should keep a strong reference to those slices?
Also, is there any memory usage gain in keeping references to slices rather than declaring slicesArray as [][]int? Is there any doc clearly stating how this is supposed to work?
TL;DR: (summary)
You just need to make a copy of the slice value. A value of slice type is a descriptor to an underlying array:
slice2 := slice
Now slice2 will refer to the same, shared underlying array and so it will be reachable after slice is zerored.
In long:
slice := make([]int, 5)
This creates a local variable named slice of type []int (and will be initialized with a descriptor referring to an array of size 5 created in the background by make). It's a slice which is a descriptor to a contiguous part of an underlying array which is created automatically in the background.
slicesArray[0] = &slice
This stores the address of the local variable (named slice) into the 0th element of slicesArray. Note that slicesArray is just a slice of pointers (which pointers may point to values of type []int but that doesn't matter now).
So still no copy of the original slice is created.
And so when you zero the only value of type []int (which is the local variable named slice), you zero the only slice value (and you'll lose the only reference to its backing array).
You want to keep the slice value after slice has been zeroed? Just make a copy of it (of the slice value). Copying a value of slice type only makes a copy of the descriptor, the backing array is not copied; and the copy will refer to the same backing array (it's shared):
slice2 := slice // makes a copy of the slice value
slice = nil
After this *(slicesArray[0]) will still point to a slice value being nil, but we have a copy of the original slice (and the shared backing array).
So doing:
slicesArray[0] = &slice2
fmt.Println((*(slicesArray[0]))[3])
Will print again 25. Go Playground
Should you keep slice values or pointers to slices?
Since slices are just descriptors which are relatively small, you should keep and work with slice values. A slice value already contains a "reference" to a backing array. Adding another indirection by using a pointer to a slice just complicates and slightly slows things down. A slice is already designed to be small, efficient and flexible (compared to "real" arrays in Go).
Of course a pointer to a slice value may also be useful in some cases, for example if you would ever want to create a function similar to the builtin append() function without returning the new slice. Or when you create a custom type whose underlying type is a slice type, and you define methods for this type which modify the slice value (in this case a pointer receiver is necessary).
Further readings: (the docs explaining everything in detail)
Go Slices: usage and internals
Arrays, slices (and strings): The mechanics of 'append'
Related
On Go's slice tricks wiki and Go libraries (e.g., this example), you sometimes see code like the following to copy a slice into a new backing array.
// In a library at the end of a function perhaps...
return append(whateverSlice[:0:0], whateverSlice...)
// In an assignment, as in the wiki example...
b = append(a[:0:0], a...)
Here's what I think I understand:
All of the items in the slice that is the second parameter to append are copied over to a new backing array.
In the first parameter to append, the code uses a full slice expression. (We can rewrite the first parameter as a[0:0:0], but the first 0 will be supplied if omitted. I assume that's not relevant to the larger meaning here.)
Based on the spec, the resulting slice should have the same type as the original, and it should have a length and capacity of zero.
(Again, not directly relevant, but I know that you can use copy instead of append, and it's a lot clearer to read.)
However, I still can't fully understand why the syntax append(someSlice[:0:0], someSlice...) creates a new backing array. I was also initially confused why the append operation didn't mess with (or truncate) the original slice.
Now for my guesses:
I'm assuming that all of this is necessary and useful because if you just assign newSlice := oldSlice, then changes to the one will be reflected in the other. Often, you won't want that.
Because we don't assign the result of the append to the original slice (as is normal in Go), nothing happens to the original slice. It isn't truncated or changed in any way.
Because the length and capacity of anySlice[:0:0] are both zero, Go must create a new backing array if it's going to assign the elements of anySlice to the result. Is this why a new backing array is created?
What would happen if anySlice... had no elements? A snippet on the Go Playground suggests that if you use this append trick on an empty slice, the copy and the original initially have the same backing array. (Edit: as a commenter explains, I misunderstood this snippet. The snippet shows that the two items are initially the same, but neither has a backing array yet. They both point initially to a generic zero value.) Since the two slices both have a length and capacity of zero, the minute you add anything to one of them, that one gets a new backing array. Therefore, I guess, the effect is still the same. Namely, the two slices cannot affect each other after the copy is made by append.
This other playground snippet suggests that if a slice has more than zero elements, the append copy method leads immediately to a new backing array. In this case, the two resulting slices come apart, so to speak, immediately.
I am probably worrying way too much about this, but I'd love a fuller explanation of why the append(a[:0:0], a...) trick works the way it does.
Because the length and capacity of anySlice[:0:0] are both zero, Go must create a new backing array if it's going to assign the elements of anySlice to the result. Is this why a new backing array is created?
Because capacity is 0, yes.
https://pkg.go.dev/builtin#go1.19.3#append
If it has sufficient capacity, the destination is resliced to accommodate the new elements. If it does not, a new underlying array will be allocated.
cap=0 is NOT sufficient for non-empty slice, allocating a new array is necessary.
I have a fixed-size array I want to get the last element:
let array = [1, 2, 3];
According to the documentation page of the fixed-size array, there is no last method directly implemented for that type.
After a quick research, I have noticed the slice type implements the last method. All I have to do now is to find a way to convert my array to a slice. I can do that using the iter method:
let last = array.iter().last();
But I also have noted that I can omit the call to iter and get the exactly same behavior:
let last = array.last();
Why is this possible? How can the fixed-size array type call a method it doesn't implement? I have taken a look to all the traits implemented for that type and none of them have a last method.
While I was writing this question, I also noticed the iter function is not defined for the fixed-size array type. Is the documentation partial? Or am I bad at reading it?
This is a special case for the array-types, which coerce into slices of the same type automatically. The docs have (only) a small sentence about this:
Arrays coerce to slices ([T]), so a slice method may be called on an
array.
I both cases the array is coerced into a slice, so you end up calling .iter() and .last() on an implicit slice, not on the array.
I am learning GO. According to documentation, slices are richer than arrays.
However, I am failing to grasp hypothetical use cases for slices.
What would be use case where one would use a slice instead of array?
Thanks!
This is really pretty elementary and probably should already have been covered in whatever documentation you're reading (unless it's just the language spec), but: A Go array always has a fixed size. If you always need 10 things of type T, [10]T is fine. But what if you need a variable number of things n, where n is determined at runtime?
A Go slice—which consists of two parts, a slice header and an underlying backing array—is pretty ideal for holding information needed to access a variable-sized array. Note that just declaring a slice-header variable:
var x []T
doesn't actually allocate any array of T yet: the slice header will be initialized to hold nil (converted to the right type) as the (missing) backing array, 0 as the current size, and 0 as the capacity of this array. As a result of this, the test x == nil will say that yes, x is nil. To get an actual array, you will need either:
an actual array, or
a call to make, or
use of the built-in append or similar (e.g., copy, append hidden behind some function, etc).
Since the call to make happens at runtime, it can make an array of whatever size is needed at this point. A series of calls to append can build up an array. Note that each call to append may have to allocate a new backing array, or may be able to extend the existing array in-place, depending on what's in the capacity. That's why you need x = append(x, elem) or x = append(x, elems...) and not just append(x, elem) or append(x, elems...).
The Go blog entry on slices has a lot more to say on this. I like this page more than the sequence of pages in the Go Tour starting here, but opinions vary.
I'm doing a micro-optimisation of my LRU cache solution in Golang where I'm using https://golang.org/pkg/container/list/. My solution works by having a map[int]*list.Element, where each list.List list.Element is []int, with [0] being key, and [1] being value.
I'm trying to move from []int to [2]int for my optimisation, but I'm then running into the issue that modifying the fixed-size array, after ee := e.Value.([2]int) (note the [2]int type for fixed-size array), is no longer modifying the underlying values in the list.Element, unlike was the case w/ ee := e.Value.([]int) (note the []int type), which I guess makes perfect sense, since slices are based on references, whereas fixed-size arrays are based on copied values.
I've tried stuff like e.Value.([2]int)[1] = …, as well as various combinations with := &e.Value…, and casting from [2]int to []int, but it all results in complier errors.
Q: Is there no way to use the container/list with an embedded array, and perform modifications of said fixed-size array in-place?
As you already noted:
I guess makes perfect sense, since slices are based on references, whereas fixed-size arrays are based on copied values
So if you want to make this work, you'll need to use references to your fixed-size arrays by storing pointers to the arrays instead of array values.
That way, you'll be able to modify the underlying array through the list element.
See here for a simple example:
package main
import (
"container/list"
"fmt"
)
func main() {
l := list.New()
// create a fixed size array and initialize it
var arr [2]int
arr[0] = 1
arr[1] = 2
// push a pointer to the array into the list
elem := l.PushFront(&arr)
// modify the stored array
elem.Value.(*[2]int)[0] = 3
// print the element from iterating the list
for e := l.Front(); e != nil; e = e.Next() {
fmt.Println(e.Value)
}
// print the underlying array, both are modified
fmt.Println(arr)
}
EDIT
Note that this behaviour is not something specific to this list implementation, but rather related how type assertions work in the language itself.
See here:
https://golang.org/doc/effective_go.html#interface_conversions
Quoting from that section (and adding emphasis of my own):
The syntax borrows from the clause opening a type switch, but with an explicit type rather than the type keyword: value.(typeName) and the result is a new value with the static type typeName.
When using reference types, copying the value does not affect you cause copying a pointer value ends up allowing you to change the same underlying reference.
But when the array is a value itself, it does not make sense to assign to the copy. When you try to modify the array directly (without assigning the type assertion to a variable) Go would even catch this at compile time so that you don't assign to this "temporary" copy which would obviously be a mistake. That's how you got all your syntax errors when trying to do this.
To overcome this (if you don't want to use pointers) one possibility might be for you to implement your own list, borrowing from the implementation you are using but making your Element's value an explicit [2]int instead of an interface.
That would remove the need to make the type assertion and you'll be able to modify the underlying array.
Q: Is there a way, in golang, to define a function which accepts an array of arbitrary length as argument?
e.g.,
function demoArrayMagic(arr [magic]int){
....
}
I have understood that in golang, array length is part of the variable type, for this reason the following function is not going to accept one arbitrary array as input
function demoArray(arr [2]int){
....
}
This function is not going to compile with arrInput [6]int as input--i.e., demoArray(arrInput) will fail to compile.
Also the following function, which accepts a slice argument, does not accept arrays as arguments (different types, as expected):
function demoSlice(arr []int){
....
}
i.e., demoSlice(arrInput) does not compile, expects a slice not an array.
The question is, is there a way to define a function that takes arrays of arbitrary length (arrays, NOT slice)? It looks quite strange and limiting for a language to impose this constraint.
The question makes sense independently from the motivation, but, in my case, the reason behind is the following. I have a set of functions which takes as arguments data structures of type [][]int.
I noticed that GOB serialization for them is 10x slower (MB/s) than other data structures I have. I suppose that may be related to the chain of derefencing operations in slices. Moving from slices to array--i.e.,defining objects of type [10000][128]int--may improve situation (I hope).
Regards
P.s: I remind now that Go, passes/uses things 'by value', using arrays may be overkill cause golang is going to copy them lot of times. I think I'll stay with slices and I'll try to understand GOB internals a bit.
There is not. Go does not support generics.
The only way would be to use interface{}, but that would allow to pass a value of any type, not just arrays of your desired type.
Arrays in Go are "secondary". The solution is to use slices for your requirement.
One thing to note here is that you may continue to use arrays, and only slice them when you want to pass them to this function, e.g.:
func main() {
a1 := [1]int{1}
demo(a1[:])
a2 := [2]int{1, 2}
demo(a2[:])
}
func demo(s []int) {
fmt.Println("Passed:", s)
}
Output of the above (try it on the Go Playground):
Passed: [1]
Passed: [1 2]