I have amount column which has format of number.
I declare 2 dimensional array of type variant, first dimension, I store currency(ex. : GBP, USD) and in other dimension I store amount(eg.: 1234.22 or-1567.69)
myArray(1,0)=GBP
myArray(1,1)= -1234.12
myArray(2,0)=GBP
myArray(2,1)= 1234.12
I am summing myArray(1,1) and myArray(2,1), while summing it is considering format as General/Text instead of Number(which is my column format) and sum is non-zero whereas ideally sum should be 0.
Please suggest, how do I handle this scenario?
To understand that, you will need to understand exactly what a VARIANT is in VBA and exactly what an ARRAY is.
Arrays:
Starting with arrays, a VBA array is actually not really an array of memory locations but a data structure called SAFEARRAY which includes details as shown in the listing below (source):
typedef struct tagSAFEARRAY {
USHORT cDims;
USHORT fFeatures;
ULONG cbElements;
ULONG cLocks;
PVOID pvData;
SAFEARRAYBOUND rgsabound[1];
} SAFEARRAY, *LPSAFEARRAY;
So, you can see that this structure has a pointer to where the data actually is, and what the number of elements are, the number of dimensions and so on and so forth. Because of that VBA is able to ensure that by using its arrays, you will not accidentally mess up some not-to-be-disturbed memory location.
Variants:
With that out of the way, you need to understand what exactly a VARIANT is. VARIANT is also not really a primitive data type but a data structure which makes it able to handle multiple data types easily.
Details of the structure can be found by a simple search but the details are simple:
Total data structure size: 16 bytes
2 bytes: Information about the data type
6 bytes: Reserved bytes (set to 0)
8 bytes: Contain the actual data
Hence when you do a VarType the first two bytes are obtained and that is how the interpreter knows what data type is being used. See here for more details.
So you can understand now what a SAFEARRAY of VARIANT data is.
Finally, the problem in the question::
That has nothing to do with the Variant and everything to do with floating point math. Floating point numbers are not stored exactly as you think they are.
E.g. 2.323 will not be stored as 2.323 but rather as something like 2.322999999999999999999
This rounding error will eventually cause trouble (leading to the entire study of stable and unstable methods, etc.) unless you are very careful about the way you handle this quantization of sorts.
Some algorithms will be such that the errors cancel out and in some they add-up.
So, if you are looking for exact calculations, you need to use a different fixed point data type which might be more suited to your problem domain (e.g. Currency might help in some precision financial calculations)
The Solution:
The Currency data type is a 64-bit data type and interally it's like a very long integer scaled by 10,000. So up to 4 decimal places and 15 digits before the decimal can be accurately represented.
Related
For Mmap static arrays, we need the data to be a "bitstype" (have a fixed storage size in memory, afaik).
If a dataset has missing values, the typical way they are stored is that the column type is Union{Missing,T} where T is the type when it is not missing.
However, isbitstype(Union{Missing,T})=false even if isbitstype(T)=true (it also happens that isbitstype(Missing)=true).
Is there a a bitstype parametric type for allowing missing values in Julia? or generally for defining bitstype unions of bitstypes. I would imagine the size only needs to be 1 more bit (or, for practical reasons, 1 more byte) than the max of the two underlying types.
Otherwise, what is the "right way" to handle this case?
I am trying to make a for loop that will convert any data type to a byte array... this is what I have right now
var arrByte [][]byte
for _,v := range *arr.(*[]interface{}) {
string := fmt.Sprint(v) // What can I put here so that I dont have to convert to string
arrByte = append(arrByte, []byte(string))
}
The problem with this code is that I cant convert it back to its data types. So how can I directly change it so that It keeps the formatting correct so then I can later run this?
var arrInterface []interface{}
for _,v := range arrByte {
data := binary.BigEndian.Uint64(v)
arrInterface = append(arrInterface, data)
}
First, a note on terminology: You appear to be talking about byte slices not byte arrays. Arrays in Go are of fixed length, so calling append on them won't work. See here for more.
Now to your question...
I am trying to make a for loop that will convert any data type to a byte array...
This is not possible, except in the most superficial, meaningless sense. For example, if you have a variable of type net.Conn, converting it to bytes will give you a meaningless value, since it is only meaningful in conjunction with one specific, active network connection.
However, assuming you don't have variables that refer to ephemeral state like this, the simplest way to convert arbitrary variables into a byte slice is with something like gob encoding.
However, this also has limitations, in that when serializing structs, it can only access exported fields, or those exposed through the GobEncoder interface. This Q&A explains why this limitation exists.
Now, in some cases you can work around this limitation with reflection, for example, but doing so is very cumbersome, and is usually unwise, since unexported fields often reference ephemeral state (see the note above).
So in summary: It's impossible to meaningfully do this for all data types. It's difficult to do it for many data types, and it's trivial to do it for "normal" data types.
So I am working on a problem where I am dealing with very large amounts of data and I have come across a limitation I do not fully understand. I need to store sets of 6 integer values and associate each with an index. The approach I chose was to initially create my own type and then create a List(of Type). That failed with an 'Array dimensions exceeded supported range" error. Fine, I presumed that this was due to the Type I defined and perhaps the way the List/Collection was storing the data. I was expecting to make use of the full Integer.MaxValue number of indices in an array, as given in http://msdn.microsoft.com/en-us/library/wak0wfyt.aspx#BKMK_ArraySize but that seems to not apply (why?). I then proceeded to re-write the functions and ended up with an array of type Tuple(int,int,int,int,int,int). But again, I run into the same situation. Same for arrays of a type that has an array as its variable. I tried out several ways to see what the maximum size of the array could be and ended up with a maximum size of around 48E6 indices. The problem is that I need more than 10x that to store the data I have...
The only way I found to make this (sort of) work is to use a List(of List(of Integer())) and then add a new item to the top level list after every 40M indices or so. Nasty solution and not efficient, but it showed that it could be made to work...
Background: VS2010, .NET 4.0, Win7 x64, 32GB Ram.
Any ideas of how I would best store 6 integer values in either a collection or array (I need to be able to access them by index) for more than about 500 million combinations (ideally up to the 2.1B combinations)?
Thanks
The solution is actually quite simple (thanks coffee). Reading through the documentation in the link above, this should not be the problem, but... the maximum size of the array is no longer Int.MaxValue once the type isn't an integer (or so it would seem, though none of the documentation indicates this). The way around this is simply to go from something like this:
Dim _Array(Array_Size) as Tuple(of Integer,Integer,Integer,Integer,Integer,Integer)
to
Dim _Array1(Array_Size) as Integer
Dim _Array2(Array_Size) as Integer
Dim _Array3(Array_Size) as Integer
Dim _Array4(Array_Size) as Integer
Dim _Array5(Array_Size) as Integer
This allows each array the maximum size (or at least the size I need which is close enough to the max size). The only thing is that I then need to expand the rest of the code accordingly.
I am a bit surprised about this, considering that the MSDN states that 'The length of every dimension of an array is limited to the maximum value of the Integer data type' when it looks like it should actually read that 'The Total length [...] is limited to the maximum value'. That would explain that I receive an error (of the original statement) at a size that accounts for the additional 6 integer values plus some for accounting.
I'm in labview working with very constrained ram. I have two arrays that require single precision since I need decimal points. However, single precision takes too much space for what I have, the decimal values I work with are within 0.00-1000.00.
Is there an intuitive way to pack these two arrays together so I can save some space? Or is there a different approach I can take?
If you need to represent 0.00 - 1000.00, you've got 100000 values. That cannot be represented in less than 17 (whole) bits. That means that to fit two numbers in, you'll need 34 bits. 34 bits is obviously more than you can fit in a 32 bit space. I suggest you try to limit your space of values. You could dedicate 11 bits to the integer value (0 - 1023) and 5 bits to the decimal value (0 to 0.96875 in chunks of 1/32 or 0.03125). Then you'll be able to fit two decimal values into one 32 bit space.
Just remember the extra bit manipulation you have to do for this is likely to have a small performance impact on your application.
First of all it would be good general advice to double check you've correctly understood how LabVIEW stores data in memory and whether any of your VI's are using more memory than they need to.
If you still need to squeeze this data into the minimum space, you could do something like:
Instead of a 1D array of n values, use a 2D array of ceiling(n/16) x 17 U16's. Each U16 is going to hold one bit from each of 16 of your data values.
To read value m from the array, get the 17 U16's from row m/16 of the array and get bit (m MOD 16) from each U16, then combine them to create the value you need.
To write to the array, get the relevant 17 U16's, replace the relevant bit of each with the bits representing the new value, and replace the changed U16's in the array.
I guess this won't be fast but maybe you can optimise it for the particular operations you need to do on this data.
Alternatively, could you perhaps use some sort of data compression? I imagine that would work best if you can organise the data into 'pages' containing some set number of values. For example, you could take a 1D array of SGL, flatten it to a string, then apply the compression to the string, and store the compressed string in a string array. I believe OpenG includes zip tools, for example.
Using LabVIEW 2009, I have a VI that outputs an array of U64 integers.
I'd like the user to be able to perform discrete selection from among the elements of this array.
I'm thinking of accomplishing this by programmatically populating a Menu Ring
(as shown at http://digital.ni.com/public.nsf/allkb/FB0409491FAB16FA86256D08004FCE7E).
However, I apparently need to convert my array of U64 ints to an array of strings,
as it is an array of strings that is used to populate the Menu Ring.
My question: how can I convert the array of U64 ints to an array of strings?
Did you try the Number to Decimal String primitive from the String\Conversion palette? It should even accept an array.
I'm not sure if it will work on U64 numbers, because I seem to have a vague memory of it coercing number to I32, but it probably will. In any case, if you want the actual value of the ring control to be the number you want, be sure to change the representation of the ring control to U64.
Format Into String seems to work (although it won't accept arrays, so you'll have to loop over it).