Excel-VBA 2007 appears to have a 64k limit on the size of arrays passed as arguments.
Is anyone aware of a fix or work-around?
Here's the code:
Public Function funA(n)
Dim ar()
ReDim ar(n)
funA = ar
End Function
Public Function funB(x)
funB = UBound(x)
End Function
From Excel:
=funB(funA(2^16-1)) '65536 as expected
=funB(funA(2^16)) 'Gives a #VALUE
Looking inside, funA() works fine but, passed to funB, the argument x is an Error 2015.
I think it's a limitation of the spreadsheet cell itself, rather than VBA. Excel can pass arrays bigger than 2^16 between functions, but apparently it can't contain an array of that size within a cell.
As an experiment, highlight funA(2^16) in the cell formula and hit F9 - it'll give you a '#VALUE!' error.
Because the formula has already calculated the result of funA before it initiates funB, it's then trying to run funB on a function that's already calculated to an error.
It seems a work-around like the one Brad posted (i.e. a third function that calculates funB(funA(n)) within itself) keeps the cell out of the equation until the calculation's completed, so it works fine.
Because single-dimensional arrays are passed back to Excel as a row of columns you have hit the Excel 2007 limit on the number of columns (64K).
If you make your array 2 dimensional and return as rows it should work:
Public Function funA(n)
Dim ar()
ReDim ar(n,1)
funA = ar
End Function
Alternatively you can use Transpose to rotate the array from a row to a column, but this is probably less efficient than creating a 2-dimensional array in the first place.
This seems to be as close to a work around as I can find. Do the inter-function calls from VBA
If you make something like this
Public Function funBA(n As Variant) As Variant
funBA = funB(funA(n))
End Function
it seems to work up to n=2^24=2^8^3 (which doesn't look like any data type break point in VBA which is where the hang up is, but that's a pretty big array)
It's not a VBA issue because you can run this and get no errors
Public Sub test()
x = funB(funA(2 ^ 16 - 1))
y = funB(funA(2 ^ 16))
Debug.Print x; y
End Sub
It seems to be an issue passing it back to Excel - not much documentation but it seems to be a Excel limit.
Here is another link but no solution WorksheetFunction array size limit
and another
http://answers.microsoft.com/en-us/office/forum/office_2007-excel/passing-arrays-to-excel-worksheet-functions-in/56d76732-9a15-4fd2-9cad-41263a4045d4
Related
I have a Function that calculates an average payout. The loop calculates down from x days to y days. As part of this function I have to interpolate a number using a specific range. However this is very slow.
I read that a method to speed up the code would be to read the range as values rather than a Range as VBA is slowed down by going to Excel each time the code is run.
Is this true?
My current code.
Function AveragePayout(Time As Double, period)
Dim i As Integer
Dim sum As Double
Dim interpolate_surface As Range
Set interpolate_surface = Range("A1", "D4")
If Time < period Then
AveragePayout = 0
Else
For i = 1 To period
interpolated_val = Interpolation(interpolate_surface, 5, Time)
sum = sum + CustomPricer(interpolated_value)
Time = Time - 1
Next i
AveragePayout = sum / period
End If
End Function
I was thinking to change line 5 to the below to then run the Interpolation on an VBA matrix/array rather than returning to the Excel document each loop (which apparently slows the function tremendously:
Set interpolate_surface = Range("A1", "D4").Value2
Alternatively are there any other methods to speed up the running of this loop?
Many thanks.
While R.Leruth is very close, there are a few things that need to be elaborated on.
First, the reason why a Range object is slower is because you are working on the Object representation of that value, and there are events bound to that Range. As a result, calculations will run, the sheet will need to be evaluated, and accessing the value has to go through the Object, and not through an in-memory representation of that object.
This performance decrease generally stands true for any Range operations, and the performance decrease is directly tied to the size of the range. Thus, operating on 100 cells is much quicker than operating on 1,000,000 cells.
While performance time of an array is also directly linked, accessing each value is much quicker. This is because the the values are in-memory and easy to access. There are no Objects to depend on with an array. This doesn't mean arrays will always be fast. I have encountered instances of array operations taking many minutes or hours because I took their initial speed for granted. You will notice a performance decrease with arrays, but the rate of performance decrease is much much much lower.
To create an array, we use the Variant type. Keep in mind a Variant can be anything, so you have to be somewhat careful. General convention is to use Dim Foo as Variant() but then any argument that accepts or returns a Variant() must be given a Variant() and not a Variant (minor difference, huge impact on code). Because of this, I tend to use Dim Foo as Variant.
We can then assign the Values from a range back to the array. While Foo = Range("A1:B2") is functionally equivalent to Foo = Range("A1:B2").Value, I strongly recommend full qualification. Thus, I don't rely on implicit properties as much as I can (.Value is the implicit property of Range).
So our code should be:
Dim Foo as Variant
Foo = SomeRange.Value
Where Foo is your variable, and SomeRange is replaced with your range.
As long as your Interpolate function accepts an array, this should cause no issues whatsoever. If the Interpolate function doesn't accept an array, you may need to find another workaround (or write your own).
To output the array, we just need to create a range of the same size as our array. There are different ways of doing this. I tend to prefer this method:
SomeRange.Resize(UBound(SomeArray, 1) - LBound(SomeArray, 1) + 1, Ubound(SomeArray, 2) - LBound(SomeArray, 2) + 1)
All this does is takes some range (should be a single cell) and resizes that range by the number of columns in the array, and the number of rows in the array. I use (Ubound - Lbound) + 1 since, for a 0-based array, this will return Ubound + 1 and for a 1-based array it will return Ubound. It makes things much simpler than creating If blocks for the same purpose.
The last thing to make sure in all of this is that your Range variable is fully-qualified. Notice that Range("A1:B2").Value is functionally equivalent to ActiveSheet.Range("A1:B2").Value but again, relying on implicit calls quickly introduces bugs. Squash those out as much as possible. If you need the ActiveSheet then use it. Otherwise, create a Worksheet variable and point that variable to the correct sheet.
And if you must use the ActiveSheet then Dim Foo as Worksheet : Set Foo = ActiveSheet is much better than just using the ActiveSheet (since the ActiveSheet will generally change when you really need it not to, and then everything will break.
Best of luck in using arrays. They are performance changing but they are never an excuse for bad coding practices. Make sure you use them properly, and that you aren't introducing new inefficiencies just because you now can.
What we usually do in VBA to speed up macros is to decrease the amount of interaction between the code and the sheet.
For example :
Get all necessary values in an array
Dim arr() as Variant
arr = Range("A1:D4")
Treat the value
...
Put them back
Range("A1:D4") = arr
In your case just try to change interpolated_surface from Range to an array type.
I've only been using VBA for a couple of weeks now so bear with me.
I'm trying to change a macro so that it reads from an array instead of a range. One of the sections I need to change uses .formulaR1C1 to run a vlookup, but im having a hard time trying to work out how this can be done with the array.
An example of this is the below line:
.Range("M2:L" & lastrow).FormulaR1C1 = "=VLOOKUP(RC[-1], Sheet2!R1C1:R4C10, 3, 0)"
I'm not too sure on whether I can set the value of an array to a formula, as I've done above, or if I have to maybe store the value as a String and then edit the value of the cell later when printing the column back on to the worksheet.
What I have so far is below:
For i = 2 To lastrow
arr(i, 13).FormulaR1C1 = "=VLOOKUP(RC[-1],Sheet2!R1C1:R4C10,3,0)"
Next
Thanks in advance!
Storing the value as a string would certainly be a valid option. Simply write the formula in your array, presuming it is of Variant / String type, and when you put the data back into a worksheet you can use .FormulaR1C1 on a Cell (Range) object to apply it as a formula.
arr(i, 13) = "=VLOOKUP(RC[-1],WMSPAM!R1C1:R4C10,3,0)"
...
Range("M2").FormulaR1C1 = Arr(1,13)
I believe this approach is likely the most effective and most easily maintained for you. As you are learning and appear to be curious of what is possible, here are a couple more examples of how you could approach this, with some further explanation.
.FormulaR1C1 is a Range object method and so the only way it could be called on an Array item would be if that item was a Range object.
Dim arr(0 To 10) As Range
Set arr(0) = Range("A1")
arr(0).FormulaR1C1 = "=2+2"
Note that as Ranges are an Object (of reference type), this operation will directly effect the Range specified in the array. In the above example, the formula "=2+2" will be placed into cell A1. You can learn more about the difference between Reference and Value types here.
If your array contains only values, the other way to achieve what you need is to use the WorksheetFunction object. With this object you can access the formula functions, that you would use in worksheet, within VBA.
WorksheetFunction.VLookup(Arg1, Arg2, Arg3, Arg4)
As with anything in writing code the WorksheetFunction methods take some trial and error to get them to work how you would expect, but in my experience these specifically can be a tricky to implement, there are though some cases where they can be very useful!
You can read more about the VLookup method here.
You may try to use .Offset():
Sub Test()
lastrow = 11
With Sheets(1)
Set Rng = .Range("M2:L" & lastrow)
End With
For i = 0 To Rng.Rows.Count - 1
Rng.Offset(i, 12).FormulaR1C1 = "=VLOOKUP(RC[-1],Sheet2!R1C1:R4C10,3,0)"
Next
End Sub
I'm not sure what you are asking. Your first code line writes a formula to a two-column range in the most efficient way there is already, your second snippet shows the less efficient way of doing it one cell at the time. If the goal is to use the vlookup to fill cells and then dispose of the formula, one efficient way is this:
.Range("M2:L" & lastrow).FormulaR1C1 = "=VLOOKUP(RC[-1], Sheet2!R1C1:R4C10, 3, 0)"
.Range("M2:L" & lastrow).Value2 = .Range("M2:L" & lastrow).Value2
I'm trying to get the values from an array formula into VBA as an array. Simple example: I have a cell (let's say D1) which has an array formula in it such as
=A1:A10*B1:B10
when I highlight this on the spreadsheet and press F9 I'll get an array of 10 numbers, say, ={5;12;15;24;25;24;42;40;54;70}
I want to be able to store these values in a VBA array: a(0)=5, a(1)=12, a(3)=15 etc; you get the idea.
Tried hunting for an answer on this one, but can't find anything relevant even on MSFT. Lots of answers about how to go the other way from VBA to the worksheet range (I know that one) but not this way. Looked at trying to do it via a ParamArray (the number of elements won't be fixed), by assigning directly to a undimensioned array and via EVALUATE(range) but I can't get any of these to work. I feel I must be missing something obvious.
Not sure what you have tried already. But Evaluate should work.
If I have the following:
The code:
Sub getArrayFormulaResult()
sFormula = Range("D1").FormulaArray
aResult = Evaluate(sFormula)
MsgBox Join(Application.Transpose(aResult), ",")
End Sub
will result in:
try this
Dim var as variant
var=Worksheets("MySheet").Evaluate(Worksheets("MySheet").Range("D1").formula)
Note you should use Worksheet.Evaluate to ensure this works when Mysheet is not the active sheet. The result will always be a 2_D array (with one column for your example array formula)
I have a large range of data in excel that I would like to parse into an array for a user defined function. The range is 2250 x 2250. It takes far too long to parse each cell in via a for loop, and it is too large to be assigned to an array via this method:
dim myArr as Variant
myArr = range("myrange")
Just brainstorming here, would it be more efficient to parse in each column and join the arrays? Any ideas?
Thanks
You're nearly there.
The code you need is:
Dim myArr as Variant
myArr = range("myrange").Value2
Note that I'm using the .Value2 property of the range, not just 'Value', which reads formats and locale settings, and will probably mangle any dates
Note, also, that I haven't bothered to Redim and specify the dimensions of the array: the Value and Value2 properties are a 2-dimensional array, (1 to Rowcount, 1 to Col Count)... Unless it's a single cell, which will be a scalar variant which breaks any downstream code that expected an array. But that's not your problem with a known 2250 x 2250 range.
If you reverse the operation, and write an array back to a range, you will need to set the size of the receiving range exactly to the dimensions of the array. Again, not your problem with the question you asked: but the two operations generally go together.
The general principle is that each 'hit' to the worksheet takes about a twentieth of a second - some machines are much faster, but they all have bad days - and the 'hit' or reading a single cell to a variable is almost exactly the same as reading a seven-million-cell range into a variant array. Both are several million times faster than reading that range in one cell at a time.
Either way, you may as well count any operation in VBA as happening in zero time once you've done the 'read-in' and stopped interacting with the worksheet.
The numbers are all very rough-and-ready, but the general principles will hold, right up until the moment you start allocating arrays that won't fit in the working memory and, again, that's not your problem today.
Remember to Erase the array variant when you've finished, rather than relying on it going out of scope: that'll make a difference, with a range this size.
This works fine.
Sub T()
Dim A() As Variant
A = Range("A2").Resize(2250, 2250).Value2
Dim i As Long, j As Long
For i = 1 To 2250
For j = 1 To 2250
If i = j Then A(i, j) = 1
Next j
Next i
Range("A2").Resize(2250, 2250).Value2 = A
End Sub
I think the best options are:
Try to limit the data to a reasonable number, say 1,000,000 values at a time.
Add some error handling to catch the Out of Memory error and then try again, but cut the size in half, then by a third, a quarter, etc...until it works.
Either way, if we're using data sets in the order of 5,000,000 values and you want to make sure that the program will run, you will need to adjust the code to chop up the data.
This code is a small subset of what I am working on. I have narrowed the problem down to the following piece. I have this UDF SampleFunction, to which I need to pass an Array, {3;4} as the sole argument.
Function SampleFunction(InputVar As Variant) As Integer
SampleFunction = InputVar(LBound(InputVar))
End Function
I call this UDF in two different ways. First, through VBA
Case 1:
Sub testSF()
MsgBox SampleFunction(Array(3, 4))
End Sub
And secondly, through my excel worksheet as
Case 2:
={SampleFunction(ROW(3:4))} -> i.e. as an array function.
The Problem:
The UDF works for Case 1, i.e. the call through VBA, it gives a #VALUE error for Case 2, when I call it through the excel worksheet.
I stepped through the function using F8 for Case 2. Lbound(InputVar) evaluates to 1 (which is different from calling from the sub in Case 1, there it evaluates to 0), yet InputVar(Lbound(InputVar)) shows the "Subscript out of Range" error in Case 2.
All I want to know is how to call SampleFunction function from a worksheet, i.e. Case 2 so that it has the same behaviour as Case 1 shown above. As a bonus, it would be nice if someone could explain why Lbound(InputVar) evaluates differently in the above cases.
Some Other Details:
I am building a UDF to perform some regex manipulations. The argument InputVar above, will be an array {x;y;z;...} specifying the xth, yth, zth ... occurences. The data type of InputVar is kept to Variant because I want to be able to pass either numbers (as a one length array), arrays, or ranges (taken in and converted to array) to the function.
Thanks in advance!!
I believe you have two problems. First, I don't think your code will evaluate if you use SampleFunction in a non-array formula, i.e., if InputVar is a Range. You need to incorporate some way of dealing with different types of input that can be passed into the variant. Second, your code assumes the InputVar is a one-dimensional array. This will lead to an error for any multi-dimensional array. This is the source of the 'Subscript out of range.' error, because array functions pass all array arguments as two-dimensional arrays even if they are can be represented as one-dimensional.
I would suggest declaring a new dynamic array in your function and then setting it equal to InputVar. In the past I've made this work for array and non-array formulas with something like below. Also, note the change to how the first item from the array is retrieved.
Option Explicit
Function SampleFunc(InputVar As Variant) As Integer
Dim tmpArray() As Variant
On Error GoTo ErrHandler
tmpArray = InputVar
'Added extra argument to LBound since dynamic arrays have two dimensions by default.
SampleFunc = tmpArray(LBound(tmpArray, 1), LBound(tmpArray, 2))
Exit Function
ErrHandler:
'Handles the case where InputVar is a Range.
tmpArray = InputVar.Value
Resume Next
End Function
This is quick and dirty, esp. the error handling, but hopefully the basic idea is helpful.