I know one can use SetWindowLongPtr + GWLP_USERDATA to store a pointer which points to some data.
But could one store the data directly, for example "a handle", "a bool, an "int" or other larger data.
From http://msdn.microsoft.com/zh-tw/library/windows/desktop/ms644898%28v=vs.85%29.aspx, it says:
Sets new extra information that is private to the application, such as handles or pointers.
, so I guess to store a handle is OK. I also used this method to store an RGB value without problem.
But I don't know if this is a good idea to do things like this. And can we store other data which is large (for example, a structure)?
p.s: The motivation of this question is: When I create a dialog window, I want to store data for each of its controls. Of course I can use static variables in the window procedure and pass pointer (to them) to SetWindowLongPtr function. But this is not "perfect" in theory, because when the dialog window is closed, I don't need these data anymore. Of course, in practice, the data I need to use is very small, and I should not care about the usage of memory. But I still like to know if there is a better way.
You only need one pointer to store anything you want. Declare a struct with the data you want to store. Allocate it before the CreateWindowEx() call and pass the pointer as the last argument. You get it back in your window procedure for the WM_CREATE message, CREATESTRUCT.lpCreateParams field. Now call SetWindowsLongPtr to store that pointer.
Anytime you need it back, use GetWindowlongPtr to recover the pointer to the struct. You'll need to cleanup again, use the WM_NCDESTROY message to release the pointer.
Note that this is a standard technique used in C++ class libraries that wrap the winapi. Do consider using one of them instead of spinning this yourself.
The SetWindowLongPtr function can store a piece of data which has the same size as LONG_PTR (most likely 32bit or 64bit). If your data can be stored in that size, you're fine. I.e. a bool would be fine, so would most handles (since handles tend to be pointers, too).
A typical RGB value would work as well since it's stored as three bytes (one byte per color component) or four bytes (an extra byte for the alpha channel).
If you need more space than this, you should allocate a structure somewhere else and store a pointer to that structure.
Related
In Go there are various ways to return a struct value or slice thereof. For individual ones I've seen:
type MyStruct struct {
Val int
}
func myfunc() MyStruct {
return MyStruct{Val: 1}
}
func myfunc() *MyStruct {
return &MyStruct{}
}
func myfunc(s *MyStruct) {
s.Val = 1
}
I understand the differences between these. The first returns a copy of the struct, the second a pointer to the struct value created within the function, the third expects an existing struct to be passed in and overrides the value.
I've seen all of these patterns be used in various contexts, I'm wondering what the best practices are regarding these. When would you use which? For instance, the first one could be ok for small structs (because the overhead is minimal), the second for bigger ones. And the third if you want to be extremely memory efficient, because you can easily reuse a single struct instance between calls. Are there any best practices for when to use which?
Similarly, the same question regarding slices:
func myfunc() []MyStruct {
return []MyStruct{ MyStruct{Val: 1} }
}
func myfunc() []*MyStruct {
return []MyStruct{ &MyStruct{Val: 1} }
}
func myfunc(s *[]MyStruct) {
*s = []MyStruct{ MyStruct{Val: 1} }
}
func myfunc(s *[]*MyStruct) {
*s = []MyStruct{ &MyStruct{Val: 1} }
}
Again: what are best practices here. I know slices are always pointers, so returning a pointer to a slice isn't useful. However, should I return a slice of struct values, a slice of pointers to structs, should I pass in a pointer to a slice as argument (a pattern used in the Go App Engine API)?
tl;dr:
Methods using receiver pointers are common; the rule of thumb for receivers is, "If in doubt, use a pointer."
Slices, maps, channels, strings, function values, and interface values are implemented with pointers internally, and a pointer to them is often redundant.
Elsewhere, use pointers for big structs or structs you'll have to change, and otherwise pass values, because getting things changed by surprise via a pointer is confusing.
One case where you should often use a pointer:
Receivers are pointers more often than other arguments. It's not unusual for methods to modify the thing they're called on, or for named types to be large structs, so the guidance is to default to pointers except in rare cases.
Jeff Hodges' copyfighter tool automatically searches for non-tiny receivers passed by value.
Some situations where you don't need pointers:
Code review guidelines suggest passing small structs like type Point struct { latitude, longitude float64 }, and maybe even things a bit bigger, as values, unless the function you're calling needs to be able to modify them in place.
Value semantics avoid aliasing situations where an assignment over here changes a value over there by surprise.
Passing small structs by value can be more efficient by avoiding cache misses or heap allocations. In any case, when pointers and values perform similarly, the Go-y approach is to choose whatever provides the more natural semantics rather than squeeze out every last bit of speed.
So, Go Wiki's code review comments page suggests passing by value when structs are small and likely to stay that way.
If the "large" cutoff seems vague, it is; arguably many structs are in a range where either a pointer or a value is OK. As a lower bound, the code review comments suggest slices (three machine words) are reasonable to use as value receivers. As something nearer an upper bound, bytes.Replace takes 10 words' worth of args (three slices and an int). You can find situations where copying even large structs turns out a performance win, but the rule of thumb is not to.
For slices, you don't need to pass a pointer to change elements of the array. io.Reader.Read(p []byte) changes the bytes of p, for instance. It's arguably a special case of "treat little structs like values," since internally you're passing around a little structure called a slice header (see Russ Cox (rsc)'s explanation). Similarly, you don't need a pointer to modify a map or communicate on a channel.
For slices you'll reslice (change the start/length/capacity of), built-in functions like append accept a slice value and return a new one. I'd imitate that; it avoids aliasing, returning a new slice helps call attention to the fact that a new array might be allocated, and it's familiar to callers.
It's not always practical follow that pattern. Some tools like database interfaces or serializers need to append to a slice whose type isn't known at compile time. They sometimes accept a pointer to a slice in an interface{} parameter.
Maps, channels, strings, and function and interface values, like slices, are internally references or structures that contain references already, so if you're just trying to avoid getting the underlying data copied, you don't need to pass pointers to them. (rsc wrote a separate post on how interface values are stored).
You still may need to pass pointers in the rarer case that you want to modify the caller's struct: flag.StringVar takes a *string for that reason, for example.
Where you use pointers:
Consider whether your function should be a method on whichever struct you need a pointer to. People expect a lot of methods on x to modify x, so making the modified struct the receiver may help to minimize surprise. There are guidelines on when receivers should be pointers.
Functions that have effects on their non-receiver params should make that clear in the godoc, or better yet, the godoc and the name (like reader.WriteTo(writer)).
You mention accepting a pointer to avoid allocations by allowing reuse; changing APIs for the sake of memory reuse is an optimization I'd delay until it's clear the allocations have a nontrivial cost, and then I'd look for a way that doesn't force the trickier API on all users:
For avoiding allocations, Go's escape analysis is your friend. You can sometimes help it avoid heap allocations by making types that can be initialized with a trivial constructor, a plain literal, or a useful zero value like bytes.Buffer.
Consider a Reset() method to put an object back in a blank state, like some stdlib types offer. Users who don't care or can't save an allocation don't have to call it.
Consider writing modify-in-place methods and create-from-scratch functions as matching pairs, for convenience: existingUser.LoadFromJSON(json []byte) error could be wrapped by NewUserFromJSON(json []byte) (*User, error). Again, it pushes the choice between laziness and pinching allocations to the individual caller.
Callers seeking to recycle memory can let sync.Pool handle some details. If a particular allocation creates a lot of memory pressure, you're confident you know when the alloc is no longer used, and you don't have a better optimization available, sync.Pool can help. (CloudFlare published a useful (pre-sync.Pool) blog post about recycling.)
Finally, on whether your slices should be of pointers: slices of values can be useful, and save you allocations and cache misses. There can be blockers:
The API to create your items might force pointers on you, e.g. you have to call NewFoo() *Foo rather than let Go initialize with the zero value.
The desired lifetimes of the items might not all be the same. The whole slice is freed at once; if 99% of the items are no longer useful but you have pointers to the other 1%, all of the array remains allocated.
Copying or moving the values might cause you performance or correctness problems, making pointers more attractive. Notably, append copies items when it grows the underlying array. Pointers to slice items from before the append may not point to where the item was copied after, copying can be slower for huge structs, and for e.g. sync.Mutex copying isn't allowed. Insert/delete in the middle and sorting also move items around so similar considerations can apply.
Broadly, value slices can make sense if either you get all of your items in place up front and don't move them (e.g., no more appends after initial setup), or if you do keep moving them around but you're confident that's OK (no/careful use of pointers to items, and items are small or you've measured the perf impact). Sometimes it comes down to something more specific to your situation, but that's a rough guide.
If you can (e.g. a non-shared resource that does not need to be passed as reference), use a value. By the following reasons:
Your code will be nicer and more readable, avoiding pointer operators and null checks.
Your code will be safer against Null Pointer panics.
Your code will be often faster: yes, faster! Why?
Reason 1: you will allocate less items in the heap. Allocating/deallocating from stack is immediate, but allocating/deallocating on Heap may be very expensive (allocation time + garbage collection). You can see some basic numbers here: http://www.macias.info/entry/201802102230_go_values_vs_references.md
Reason 2: especially if you store returned values in slices, your memory objects will be more compacted in memory: looping a slice where all the items are contiguous is much faster than iterating a slice where all the items are pointers to other parts of the memory. Not for the indirection step but for the increase of cache misses.
Myth breaker: a typical x86 cache line are 64 bytes. Most structs are smaller than that. The time of copying a cache line in memory is similar to copying a pointer.
Only if a critical part of your code is slow I would try some micro-optimization and check if using pointers improves somewhat the speed, at the cost of less readability and mantainability.
Three main reasons when you would want to use method receivers as pointers:
"First, and most important, does the method need to modify the receiver? If it does, the receiver must be a pointer."
"Second is the consideration of efficiency. If the receiver is large, a big struct for instance, it will be much cheaper to use a pointer receiver."
"Next is consistency. If some of the methods of the type must have pointer receivers, the rest should too, so the method set is consistent regardless of how the type is used"
Reference : https://golang.org/doc/faq#methods_on_values_or_pointers
Edit : Another important thing is to know the actual "type" that you are sending to function. The type can either be a 'value type' or 'reference type'.
Even as slices and maps acts as references, we might want to pass them as pointers in scenarios like changing the length of the slice in the function.
A case where you generally need to return a pointer is when constructing an instance of some stateful or shareable resource. This is often done by functions prefixed with New.
Because they represent a specific instance of something and they may need to coordinate some activity, it doesn't make a lot of sense to generate duplicated/copied structures representing the same resource -- so the returned pointer acts as the handle to the resource itself.
Some examples:
func NewTLSServer(handler http.Handler) *Server -- instantiate a web server for testing
func Open(name string) (*File, error) -- return a file access handle
In other cases, pointers are returned just because the structure may be too large to copy by default:
func NewRGBA(r Rectangle) *RGBA -- allocate an image in memory
Alternatively, returning pointers directly could be avoided by instead returning a copy of a structure that contains the pointer internally, but maybe this isn't considered idiomatic:
No such examples found in the standard libraries...
Related question: Embedding in Go with pointer or with value
Regarding to struct vs. pointer return value, I got confused after reading many highly stared open source projects on github, as there are many examples for both cases, util I found this amazing article:
https://www.ardanlabs.com/blog/2014/12/using-pointers-in-go.html
"In general, share struct type values with a pointer unless the struct type has been implemented to behave like a primitive data value.
If you are still not sure, this is another way to think about. Think of every struct as having a nature. If the nature of the struct is something that should not be changed, like a time, a color or a coordinate, then implement the struct as a primitive data value. If the nature of the struct is something that can be changed, even if it never is in your program, it is not a primitive data value and should be implemented to be shared with a pointer. Don’t create structs that have a duality of nature."
Completedly convinced.
So I have a struct that looks something like this (more or less):
typedef struct AST_STRUCT
{
enum {
AST_OBJECT,
AST_REFERENCE,
AST_VARIABLE,
AST_VARIABLE_DEFINITION,
AST_VARIABLE_ASSIGNMENT,
AST_VARIABLE_MODIFIER,
AST_FUNCTION_DEFINITION,
AST_FUNCTION_CALL,
AST_NULL,
AST_STRING,
AST_CHAR,
AST_FLOAT,
AST_LIST,
AST_BOOLEAN,
AST_INTEGER,
AST_COMPOUND,
AST_TYPE,
AST_BINOP,
AST_NOOP,
AST_BREAK,
AST_RETURN,
AST_IF,
AST_ELSE,
AST_WHILE,
AST_ATTRIBUTE_ACCESS,
AST_LIST_ACCESS,
AST_NEW
} type;
struct AST_STRUCT* variable_value;
}
Now I would like to write this struct, serialized to the disk into a .dat file.
The problem is that as you can see, it has a field called variable_value .
I am using this function to write it to disk:
https://www.geeksforgeeks.org/readwrite-structure-file-c/
I am also using the other function in that article to read it from the disk.
It appears as if the variable_value field on the struct is not loaded properly.
How would I write the entire struct to disk and maintain the data of the variable_value field?
I was first thinking about dumping the variable_value field into a separate file and then sort of "link it back" into the struct once I load it, but maybe there is another way of doing this?
You are trying to serialize a linked structure (I assume a tree, from the name "AST").
If we imagine for a second that you successfully did that and wrote it to disk, when you load it back, you'll allocate memory for the links in the tree, but the addresses (values of pointers) of these memory chunks are not guaranteed to be the same as the old ones. You will not be able to reconstruct the tree.
So you can't use the value of the addresses as links on the disk. You'll need to use some other method. These days, the popular method is the JSON format, which would work for you assuming that you don't have cross-links or back-links in the tree.
So what you need is a JSON C library. I've never used one, but here are a couple I found in 2 seconds:
json-c
cJSON
JSON and xml are perfectly good formats to use if you want to save your data as text. They do have issues with restoring multiple references to the same object, but even they can be resolved.
If the data is a simple link-list, you can simply restore the objects in order and repair the link-list yourself as you restore the objects. If the data structure is more complex than that you need a more generic solution.
If you really want to save the data as binary, then you need to fix up pointers when you reload. The main way to do this is to keep a map of saved addresses vs newly allocated addresses. If you really don't like the idea of emitting the saved addresses, you can use a serial number to represent each unique address you find.
For each object you save you have to record it's scalar address, and the type of object, before you save the object itself.
For each pointer you save, you need to save the scalar address, and "remember" to later save that referenced object.
On restoring, when you restore an object you need to load the object based on its type, and create a mapping entry that shows how the saved address has turned into a restored address.
If an object contains any pointers, the address stored in those pointers is found by applying that mapping. However, you will often find that the object has not yet been loaded for that pointer, so you will also need to record a mapping from the saved scalar address to the address of the pointer. You can then either fix up all these unfinished pointers after you finish loading, or you can do it for a particular referenced object after that object is loaded.
You need to put in a little extra care to handle objects that support multiple inheritance, by noting that the pointer does not point to the root of the object.
But otherwise, this is just about all you need, plus considerations of versioning, endianness, padding - if you care about the longevity of the data you are saving.
In Go source I have
type T struct {
// some data
}
func (t *T)M(arg0 SomeType1) {
// some computations
}
var Obj *T
In C sources I have
// SomeType1C is equivalent to SomeType1.
typedef void (*CallbackFunc)(SomeType1C);
// callback will be called !after! register_callback function returns.
void register_callback(CallbackFunc callback);
I would like to use Obj.M as callback for register_callback in C.
On MS Windows for winapi I pass smth like C.CallbackFunc(unsafe.Pointer(syscall.NewCallback(Obj.M))) to register_callback for this (not sure is it fully correct, but at least this works). But where is no NewCallback for non-Windows systems.
PS:
I'm sure that callback is registered after T is initialised and removed before T is removed.
I may have multiple instances of T and some of them may be used to callback's 'source' at same time (so T is not some kind of singltone).
Function pointer callbacks in GoLang's wiki uses gateway function, but I don't see how to adequate use it with struct's method.
Base idea:
Use exported callback as a proxy between C and Go:
//export callback
func callback(data0 SomeType1C, data1 Data){ // data1 - data passed to register_callback_with_data
obj := convertDataToObj(data1)
obj.M(data0)
}
and register it like this:
register_callback_with_data(callback, convertObjToData(obj));
Where are 3 ways: wrong (and easy), limited (medium) and right (hard).
Wrong (and easy) way:
Pass pointer to Go struct into C (as in original answer). This is totally wrong because Go runtime can move struct in memory. Usually this operation is transparent (all Go pointers will be updated automatically). But pointers in C memory to this struct will not be updated and program may crash/UB/... when tries to use it. Do not use this way.
Limited (medium) way:
Similar to previous, but with Go struct allocated in C memory:
Obj = (*T)(C.calloc(C.size_t(unsafe.Sizeof(T{}))))
In this case Obj can not be moved by Go runtime because it is in C memory. But now if Obj has pointers to Go memory (fields with *-variables, maps, slices, channels, function-pointers, ...) then this also may cause crash/UB/... This is because:
if there are no (other) Go pointers to the same variable (memory), then Go runtime thinks that this memory is free and can be reused,
or, if there is other Go pointer to same variable (memory), then Go can move this variable in memory.
So, use this way only if struct has no pointers to Go memory. Usually this means that struct contains only primitive fields (ints, floats, bool).
Right (and hard) way:
Assign id (of integer type for example) for each object of type T and pass this id into C. In exported callback you should convert id back to object. This is right way with no limitation, so this way may be used always. But this way requires to maintain some array/slice/map to convert between objects and ids. Moreover, this convertation may require some synchronization for thread-safe (so see sync.Mutex and sync.RWMutex).
Original answer:
Not best answer and has restrictions, but no other suggested. In my case I can pass additional data to register_callback. This data will be passed back to callback on each call. So I pass unsafe.Pointer(Obj) as data and use gateway function:
//export callback
func callback(data SomeType1C, additionalData unsafe.Pointer){
obj := (*T)(additionalData) // Get original Obj (pointer to instance of T)
dataGo := *(*SomeType1)(unsafe.Pointer(&data)) // Cast data from C to Go type
obj.M(dataGo)
}
and register it like this:
register_callback_with_data(callback, unsafe.Pointer(Obj));
PS: but still want to know how to do this better in general case (without additional data).
is there a way to create a fixed size array in LabView?
I know that I can do some check on the array size, then discard values when an array size become greater than a specific value. But, I think that is a common problem, so there is some built in function in LabView to have a fixed size array?
As far as I know this is impossible, unless they changed something in one of their latest releases but I doubt it: it would probably require a serious rewrite of the core array code.
The closest you can get is writing your own (possibly polymorphic) array class in which you encapsulate an actual array, that you initialize once with a certain size. For the rest your class only exposes methods to get/set by index. No resize etc.
Or, if you are talking about arrays of controls etc on the front panel, you can probably do this at the UI level by hide the indexing control from it and making sure it cannot be resized graphically. Or probably it's also doable to create a custom control and strip lots of array functionality from it.
If the array size is fixed at design time, then you might consider using a cluster instead. There is even a primitive to convert an array to a cluster of fixed size, provided the length is less then 257. (Array To Cluster function.)
There is also a primitive to go the other way if you need to index the array.
One implementation that you could do is a queue with a fixed size. You can use preview queue and flush queue to implement the functionality you want. However a specific custom class is probably a better idea.
In regular desktop LabVIEW, fixed-sized arrays would be something you'd have to code as per the answers you've already gotten here. However, in LabVIEW FPGA with, say, cRIO, all arrays must be fixed-size.
When calling the Call Library Function Node to a WINAPI DLL, there are times where a structure element may be officially be defined as BYTE[130]. So how do you absolutely, positively make sure your cluster has exactly the space for 130 bytes?
You can't do it with arrays no matter what, because LabVIEW arrays are pointers to a structure (the first element being the length), meaning any array you insert will only allocate enough space for a pointer, 4 bytes.
The work-around I came up with is to insert a cluster that includes sixteen U64 and one U16, pass that through an unflatten to string and you'll find it's exactly 130 bytes long.
When the cluster returns from the call, merely type cast the flattened into string results into a U8 array
I'm changing some codes in a database library. The way it works I send a void pointer, to get the size of it I call a query and using the query I calculate the size of the structure. Now the problem is I receive the struct as params but the function fails before/in the middle of the first fetch. After that I need to clear the structure, but I dont even have the size.
I know the best way is send the size of the structure as a param, but I have thousands and thousands programs already compiled, the library is from 1996, so I need to find a way to calculate the structure size even if the type is void.
One idea I had was to calculate the position of the next element that is not in the structure
0x000010 0x000042
[int|char[30]|int|int][int]
So the size is 32, because the 0x00042-0x000010 is 32.
Is there a way to know when I got out of the structure.
the prototype of the function is
int getData(char* fields, void* myStruct)
I need to find out the structure size.
Sorry if I missed some information, the code is HUGE and unfortunately I cannot post it here.
No, in general there's no way, given a void *, to figure out what you're after. The only thing you can do is compare it against NULL, which of course doesn't help here.
Note that there's nothing in the void * that even says it points at a struct, it could just as well be pointing into the middle of an array.
If you have some global means of recording the pointers before they're passed to getData(), you might be able to implement a look-up function that simply compares the pointer value against those previously recorded, but that's just using the pointer value as a key.