I am being passed an array from a C program that does not include the size of the array; that is, it just passes a pointer to the array. The array is a generic type <Item>. How can I determine the end of the array in order to detect a buffer overflow?
I tried iterating through the array until I received something that wasn't an <Item>. That worked most of the time but sometimes the nonsense at the end Would be of type <Item>. I am using C and calling a function from an external class I had no deal in developing. <Item> is a struct with multiple references to other arrays (sort of like a linked list).
EDIT:
The api stated that the array was intended to be a read-only version. The problem is I cannot read it if I do not know the size. It doesn't appear there is a sentinel value. There is a random comment stating that if the size is needed use sizeOf (array)/sizeOf (Item) which doesn't work. It was developed by a team that no longer works here. The problem is other code already relies on this C code and I cannot change it without fear of ruining other code.
It is not possible to determine the end of an array based on just a pointer to an element of that array.
I tried iterating through the array until I received something that wasn't an <Item>
It's also not possible to determine whether particular memory location contains an object of particular type - or whether it contains any object. Even if you could, how would you determine if the object that you find is really part of the array and not just a separate <Item> object that happens to be there?
A possible solution is to use a sentinel value to represent the end of an array. For example, you could define the interface such that <Item>.member == 0 if and only if that is the last element of the array. This is similar to how null-terminated strings work.
If all you have is a pointer and no size or known "end-of-array" marker (sentinel) in the data, then you have an impossible situation. There is no way in that case to determine the size/end of the passed array.
Related
On Go's slice tricks wiki and Go libraries (e.g., this example), you sometimes see code like the following to copy a slice into a new backing array.
// In a library at the end of a function perhaps...
return append(whateverSlice[:0:0], whateverSlice...)
// In an assignment, as in the wiki example...
b = append(a[:0:0], a...)
Here's what I think I understand:
All of the items in the slice that is the second parameter to append are copied over to a new backing array.
In the first parameter to append, the code uses a full slice expression. (We can rewrite the first parameter as a[0:0:0], but the first 0 will be supplied if omitted. I assume that's not relevant to the larger meaning here.)
Based on the spec, the resulting slice should have the same type as the original, and it should have a length and capacity of zero.
(Again, not directly relevant, but I know that you can use copy instead of append, and it's a lot clearer to read.)
However, I still can't fully understand why the syntax append(someSlice[:0:0], someSlice...) creates a new backing array. I was also initially confused why the append operation didn't mess with (or truncate) the original slice.
Now for my guesses:
I'm assuming that all of this is necessary and useful because if you just assign newSlice := oldSlice, then changes to the one will be reflected in the other. Often, you won't want that.
Because we don't assign the result of the append to the original slice (as is normal in Go), nothing happens to the original slice. It isn't truncated or changed in any way.
Because the length and capacity of anySlice[:0:0] are both zero, Go must create a new backing array if it's going to assign the elements of anySlice to the result. Is this why a new backing array is created?
What would happen if anySlice... had no elements? A snippet on the Go Playground suggests that if you use this append trick on an empty slice, the copy and the original initially have the same backing array. (Edit: as a commenter explains, I misunderstood this snippet. The snippet shows that the two items are initially the same, but neither has a backing array yet. They both point initially to a generic zero value.) Since the two slices both have a length and capacity of zero, the minute you add anything to one of them, that one gets a new backing array. Therefore, I guess, the effect is still the same. Namely, the two slices cannot affect each other after the copy is made by append.
This other playground snippet suggests that if a slice has more than zero elements, the append copy method leads immediately to a new backing array. In this case, the two resulting slices come apart, so to speak, immediately.
I am probably worrying way too much about this, but I'd love a fuller explanation of why the append(a[:0:0], a...) trick works the way it does.
Because the length and capacity of anySlice[:0:0] are both zero, Go must create a new backing array if it's going to assign the elements of anySlice to the result. Is this why a new backing array is created?
Because capacity is 0, yes.
https://pkg.go.dev/builtin#go1.19.3#append
If it has sufficient capacity, the destination is resliced to accommodate the new elements. If it does not, a new underlying array will be allocated.
cap=0 is NOT sufficient for non-empty slice, allocating a new array is necessary.
I am learning GO. According to documentation, slices are richer than arrays.
However, I am failing to grasp hypothetical use cases for slices.
What would be use case where one would use a slice instead of array?
Thanks!
This is really pretty elementary and probably should already have been covered in whatever documentation you're reading (unless it's just the language spec), but: A Go array always has a fixed size. If you always need 10 things of type T, [10]T is fine. But what if you need a variable number of things n, where n is determined at runtime?
A Go slice—which consists of two parts, a slice header and an underlying backing array—is pretty ideal for holding information needed to access a variable-sized array. Note that just declaring a slice-header variable:
var x []T
doesn't actually allocate any array of T yet: the slice header will be initialized to hold nil (converted to the right type) as the (missing) backing array, 0 as the current size, and 0 as the capacity of this array. As a result of this, the test x == nil will say that yes, x is nil. To get an actual array, you will need either:
an actual array, or
a call to make, or
use of the built-in append or similar (e.g., copy, append hidden behind some function, etc).
Since the call to make happens at runtime, it can make an array of whatever size is needed at this point. A series of calls to append can build up an array. Note that each call to append may have to allocate a new backing array, or may be able to extend the existing array in-place, depending on what's in the capacity. That's why you need x = append(x, elem) or x = append(x, elems...) and not just append(x, elem) or append(x, elems...).
The Go blog entry on slices has a lot more to say on this. I like this page more than the sequence of pages in the Go Tour starting here, but opinions vary.
Could someone provide some proofs, whether Ruby recalculates array size every time when you call array.size, array.length or array.count?
Thanks in advance.
Update
To make things clearer, by recalculate I mean, whether Ruby needs to loop through the whole array again and again to calculate the number of its elements every time when we call array.size.
Pragmatically Speaking, Array#length is Dynamic in Ruby
Your question can't really be answered canonically, because the lookup and storage implementations of arrays is often platform- and VM-specific. However, as a practical matter, from the Ruby interpreter's persepctive the answer is yes because each call sends a message to an Array object, asking it to return its current length.
Some languages store the current length of the array as an element of the array itself. Other approaches exist, too. In Ruby 2.7.1:
static VALUE
rb_ary_length(VALUE ary)
{
long len = RARRAY_LEN(ary);
return LONG2NUM(len);
}
the C implementation appears to retrieve the stored length of the array at the time of the call, but you'd have to dig deeper into the source code if you want to understand all the ins-and-outs of how the VM optimizes this (or not).
is there a way to create a fixed size array in LabView?
I know that I can do some check on the array size, then discard values when an array size become greater than a specific value. But, I think that is a common problem, so there is some built in function in LabView to have a fixed size array?
As far as I know this is impossible, unless they changed something in one of their latest releases but I doubt it: it would probably require a serious rewrite of the core array code.
The closest you can get is writing your own (possibly polymorphic) array class in which you encapsulate an actual array, that you initialize once with a certain size. For the rest your class only exposes methods to get/set by index. No resize etc.
Or, if you are talking about arrays of controls etc on the front panel, you can probably do this at the UI level by hide the indexing control from it and making sure it cannot be resized graphically. Or probably it's also doable to create a custom control and strip lots of array functionality from it.
If the array size is fixed at design time, then you might consider using a cluster instead. There is even a primitive to convert an array to a cluster of fixed size, provided the length is less then 257. (Array To Cluster function.)
There is also a primitive to go the other way if you need to index the array.
One implementation that you could do is a queue with a fixed size. You can use preview queue and flush queue to implement the functionality you want. However a specific custom class is probably a better idea.
In regular desktop LabVIEW, fixed-sized arrays would be something you'd have to code as per the answers you've already gotten here. However, in LabVIEW FPGA with, say, cRIO, all arrays must be fixed-size.
When calling the Call Library Function Node to a WINAPI DLL, there are times where a structure element may be officially be defined as BYTE[130]. So how do you absolutely, positively make sure your cluster has exactly the space for 130 bytes?
You can't do it with arrays no matter what, because LabVIEW arrays are pointers to a structure (the first element being the length), meaning any array you insert will only allocate enough space for a pointer, 4 bytes.
The work-around I came up with is to insert a cluster that includes sixteen U64 and one U16, pass that through an unflatten to string and you'll find it's exactly 130 bytes long.
When the cluster returns from the call, merely type cast the flattened into string results into a U8 array
I'm changing some codes in a database library. The way it works I send a void pointer, to get the size of it I call a query and using the query I calculate the size of the structure. Now the problem is I receive the struct as params but the function fails before/in the middle of the first fetch. After that I need to clear the structure, but I dont even have the size.
I know the best way is send the size of the structure as a param, but I have thousands and thousands programs already compiled, the library is from 1996, so I need to find a way to calculate the structure size even if the type is void.
One idea I had was to calculate the position of the next element that is not in the structure
0x000010 0x000042
[int|char[30]|int|int][int]
So the size is 32, because the 0x00042-0x000010 is 32.
Is there a way to know when I got out of the structure.
the prototype of the function is
int getData(char* fields, void* myStruct)
I need to find out the structure size.
Sorry if I missed some information, the code is HUGE and unfortunately I cannot post it here.
No, in general there's no way, given a void *, to figure out what you're after. The only thing you can do is compare it against NULL, which of course doesn't help here.
Note that there's nothing in the void * that even says it points at a struct, it could just as well be pointing into the middle of an array.
If you have some global means of recording the pointers before they're passed to getData(), you might be able to implement a look-up function that simply compares the pointer value against those previously recorded, but that's just using the pointer value as a key.