To increase the performance of my VBA code for large spreadsheets, I'm converting the sheets into 2D Arrays of Strings that the program can refer to instead of leaving open and looping through the memory intensive spreadsheets. The sheets dramatically slow down my computer, and at present the macro is slower than doing it by hand (I've removed many unnecessary columns and removed all formatting from these sheets, and no formulas seem to extend past the used range-- they're also stored as .xlsb's. They're already about one-half to two-thirds the size of the originals, so I don't think there's anything else to be done to optimize them).
Note that I'm developing in outlook, but relying heavily on data from excel sheets- the use case is an email auto-responder that searches the sheets for an ID supplied in an email, and then replies to that email with a phone number from the sheets. I have the proper references in place, and the program opens the sheets fine, just (painfully) slowly.
I'd like to use nested For loops to load the spreadsheets into the arrays programmatically, and then store those arrays in another array so they can in turn be looped through. In my research, I've found code to make jagged arrays in VBA (How do I set up a "jagged array" in VBA?), and 2D Arrays in VBA (Multi-dimensional array in VBA for Microsoft Word on Mac) but not arrays of 2D Arrays.
This is the code I wrote to make the 2D arrays- the Dim and ReDim lines throw syntax errors.
For k = LBound(sheetsArr) To UBound(sheetsArr)
Dim myWbksArr(k)() As String
ReDim myWbksArr(k)(sheetsArr(k).UsedRange.Rows.Count, sheetsArr(k).UsedRange.Columns.Count)
Next k
Where sheetsArr is an array of Worksheets into which I copied the sheets I'm referencing to avoid another for loop to iterate through Workbooks as well, using
Dim sheetsArr() As Worksheet, runningIndex As Long
runningIndex = 0
ReDim sheetsArr(1 To totalSheets) 'It would make sense to me to extend this syntax to an array as above, since an array is itself a type/object in other languages
For j = LBound(myWbks) To UBound(myWbks) 'j iterates through the workbooks
Set myWbks(j) = Workbooks.Open(FileName:=arr(j), UpdateLinks:=False) 'false should suppress update links msgbox
For Each sh In myWbks(j).Worksheets 'iterates through the worksheets in each workbook
sheetsArr(runningIndex) = sh 'add the worksheet to an array of worksheets so we don't have to use another for loop to get through the workbook layer too
runningIndex = runningIndex + 1
Next
Next j
What's the correct syntax to create an array of 2D arrays using for loops?
Maybe this will help.
It will read the contents of all the open workbooks into an array of arrays.
You'll probably want to change the method of selected the range to store, and maybe even the method of deciding which workbooks to go through. But this should give you an idea:
Option Explicit
Sub wbtoArr()
Dim wbsArr, wbArr
Dim WB As Workbook, WS As Worksheet
Dim I As Long, J As Long
ReDim wbsArr(1 To Workbooks.Count)
J = 0
For Each WB In Workbooks
J = J + 1
ReDim wbArr(1 To WB.Worksheets.Count)
I = 0
For Each WS In WB.Worksheets
I = I + 1
wbArr(I) = WS.UsedRange
Next WS
wbsArr(J) = wbArr
Next WB
End Sub
Related
I am aware that this;
Arr() =Range("E2:X2500")
...then doing stuff with the Arr, and then dumping back using:
Range("E2:X2500")=Arr()
is many folds more efficient (faster) that looping through and referencing cells directly.
It's light speed!
But, this range-to-array assignment only grabs cells' value.
Is there a way to assign an actual range (continuous or not) into an array (with the same light speed) in such a way that you could then treat array items the way you would refer to cells, like:
arr(23).row 'getting a row number
Or;
If Arr(23).Value ="Pending" then arr(23).font.bold=1 else arr(23).font.bold=0
I know i can dim a range-type array where each item can store an actual single cell range. But this array cannot be handled the same way - with one liner assignment:
Dim Arr () as Range
Set Arr = Range("E2:X2500") 'error
Instead, I would need to iterate each cell and assign it to the next item in the range-type array, which would allow me to treat items the way I'd refer to cells, but take substantially longer to load as I'm dealing with a Loop.
Also how would I dump a range-type array back into the sheet with the same ease and effectiveness of the one liner assignment? I think the only way would be to use a loop yet again, correct?
Side question :
Speedwise, is it any better to refer to cells via a range-type array over referring to cells directly via the sheet, or are both basically the same?
Thanks!
Well, the array use will save a lot of code running time. But, there are some issues which must be understood:
First thing when work in VBA and your project increases, is to properly declare your variables. Try making a reflex in putting Option Explicit on top of all your modules. In the array case, the thing, from this point of view stays like that:
Dim Arr() As variant, arr1 As Variant
Both declarations work in excel. But the second one is recommended (on mai taste), when you need an array from a range. When you want building a, let us say, result array, it will be zero based and you must take care of the range size where the values will be returned.
The array content cannot be retrieved exactly like you tried in your question in case of not fix/known number of elements. Look at the next test code:
Sub testArrays()
Dim sh As Worksheet, rng As Range, arrTest As Variant
Set sh = ActiveSheet
Set rng = sh.Range("A1:F4")
arrTest = rng.value
sh.Range("J1").Resize(UBound(arrTest, 1), UBound(arrTest, 2)).value = arrTest
End Sub
It is recommended to use arrTest = sh.Range("A1:F4").value. Using range Value. Excel is able to understand what you need according to your declaration, but it is good for you to differentiate somehow, from the way of the range definition.
Sometimes, you need to build an array during analyzing of the dynamic range. If you cannot know the new array dimensions and need to Redim (Preserve), only the second dimension of the array can be re-dimensioned and Transpose function must be use, in such a case. And finally the resulted array can be properly loaded in a range, only if you know the array number of rows and columns.
You can deduce the range row, from the array row, in the next way:
If we are referring to the above arrTest we know that its first row is first row of the sheet and it has 5 columns.
So, arrTest(3, 1) will be sh.Range("A3").Value and its row would be 3.
Then, arrTest(3, 4) will be sh.Range("D3").Value and its row would be also 3.
If your array comes from a range starting with the fifth row, you must add four in order to obtain the sheet row extracted from the array row...
So, your example can be transformed in:
If arrTest(3, 4) ="Pending" then sh.Cells(3, 4).Font.Bold=1 Else sh.Cells(3, 4).Font.Bold=0
Now if you need a ranges array, you cannot do it in the way you tried. You must use the ranges address and build the range at the end:
Sub testArraysBis()
Dim sh As Worksheet, rng As Range, rng1 As Range, lastCol As Long
Dim rng2, arrTest As Variant, arrT As Variant, arrF As Variant
Set sh = ActiveSheet
lastCol = sh.Cells(1, Cells.Columns.Count).End(xlToLeft).column
Set rng = sh.Range(sh.Cells(1, 1), sh.Cells(4, lastCol))
Set rng1 = sh.Range("A5:F6")
arrT = Array(rng.Address, rng1.Address)
arrTest = rng.value
Debug.Print UBound(arrTest), LBound(arrTest)
sh.Range("J1").Resize(UBound(arrTest, 1), UBound(arrTest, 2)).value = arrTest
Set rng2 = sh.Range(arrT(0))
Debug.Print rng2.Address
arrF = sh.Range(arrT(0)).value
Debug.Print UBound(arrF, 2)
End Sub
rng2 range will be built using the address string, extracted from arrT array. An array (arrF) can also be extracted from the arrT first element...
Epilog:
The best way, in terms of speed, is to load the range in arrays, make all processing using them (in memory and very fast due to this aspect), but the most important issue is to build another array (or even a range, using Union) and retrieve the data AT ONCE. Sending of each partial processing result to a cell/range consumes a lot of time and other resources, for a big range size...
I like to use an array as if it was an "in RAM memory" spreadsheet, so the code is more clean and it runs faster.
So I would like to ask: generally, is it better start the code with a huge array, say Dim arr(10000), so I can work with it as if it was a blank sheet I fill when necessary, or is that a bad practice, and, instead, I should use Redim Preserve all the time I need to insert data into the array?
Thanks in advance for any help.
The answer to your question depends on how you intend to fill the array (noting that this is in the context of Excel). Here are four simple scenarios:
Filling from a Range. Simply Dim a Variant and let VBA do all the work.
Dim myArray as Variant
Dim myRange as Range
'Set myRange = something
myArray = myRange.Value '<-- Fills a 2-D array
For rowIterator = LBound(myArray,1) to UBound(myArray,1) '<- How identify the size of the first dimension
For colIterator = LBound(myArray,2) to UBound(myArray,2) '<- How identify the size of the second dimension
'Do something to myArray(rowIterator, colIterator)
Next colIterator
Next rowIterator
Filling when you define your bounds. In some code, you can learn early in the code running what your bounds are going to be. In this case, you can ReDim before you get into a loop.
Dim myArray() As String
Dim someCounter As Long
'Some early code
someCounter = CLng(myRange.Value)
ReDim myArray(someCounter) '<-- Do this only once.
For iterator = LBound(myArray) to UBound(myArray)
' Do Something to myArray(iterator)
Next iterator
Filling within a loop. Avoid this when ever you can as ReDimming an array, especially with ReDim Preserve is computationally expensive. However, if you know it is a small loop, you might want to accept the cost.
An alternative to resizing in a loop is to use a Collection.
Oversizing to start with. Rather than continually re-dimensioning the array, you could oversize the array but you then have the additional management overhead of trying to manage the unused array slots. How do you know if the value in the array of "" or 0 is really "" or 0 instead of a default null value?
I have a technical question:
My issue:
I create a
Dim arrTemp as Variant
Dim wbSource as workbook
Dim wbTarget as workbook
because I need to export multiple ranges from multiple worksheets (not fixed range) to another workbook. My code looks like:
' Worksheet 1
arrTemp = wbSource(1).Range("A1:B2").value
wbTarget(1).Range("A1:B2").value = arrTemp
If Not(IsArrayEmpty(arrTemp)) Then Erase arrTemp
' Worksheet 2
arrTemp = wbSource(2).Range("A1:B2").value
wbTarget(2).Range("A1:B2").value = arrTemp
If Not(IsArrayEmpty(arrTemp)) Then Erase arrTemp
' Worksheet 3
arrTemp = wbSource(3).Range("A1:B2").value
wbTarget(3).Range("A1:B2").value = arrTemp
If Not(IsArrayEmpty(arrTemp)) Then Erase arrTemp
(worksheet can be empty in the first place, that's why empty arr handler)
(worksheets can contain int/str/double/... and the size is not that big to define specific arr type)
My question is:
Does it make sense to erase the array every time? or It will be overwritten automatically?
I did a test to check the properties of the array (Lbound & UBound) before and after defining the array with a new range. I can see that It automatically Redim the array. Does it means that I only need to clear it in the end of the procedure?
Or it is a good practice to clear it in between?
Last but not least, do you see any problem in my code? Better way to perform this task?
Many thanks in advance!
Edit:
Bear in mind
The code is not correct for this task, no need to transfer to an array!
In Short (thanks to rory!):
No need to erase the array every time in between.
Only before leaving the procedure.
I have been told there is a way of storing data in dynamic arrays which can then be referred to in a function.
I come from my recent answered question:Showing hidden column in another sheet
I am trying to find out how to store in a dynamic array which row (first array), column (second array) and sheet (third array) my code has to make action on.
I still haven't done arrays in my class, I'm just guessing it is a dynamic one for what I have researched. Do you think this could be done in the same array using different dimensions?
(Edited ->) -- Being more specific: I am looking for a way to store a number of specific rows(or columns) of a specific sheet to then (in a loop I guess) run my function for each element.
I have read the basic documentation (https://msdn.microsoft.com/en-us/library/aa716275(v=vs.60).aspx) and looked for similar posts here but I can't find it. If someone can point out what I should do or read to achieve this I can try and do it myself if you believe it's better.
Edited:
I think I am doing some progress, but I still don't know how to refer to a the sheet within the range.
Would this do the job (if I didn't need sheet referring)?
Public Sub Test()
Dim Rng As Range
Dim Area As Range
Set Rng = Range("A1:A10,B1:B20,G1:G3")
For Each Area In Rng.Areas
Area.Hidden = True
Next Area
End Sub
You can manage that with a single array of Range because the range refer to:
The sheet
The row
The Column
Dim array() as Range
...
' Store with
set array(i) = worksheet.Range(a, b)
...
' Read with
set range = array(i)
The link to msdn in your question explain how to manage Dynamic Arrays
update
The problem in your code is you not refer the worksheet you want.
If no worksheet is indicate, in the best case an error is thrown, in the worst case it takes the "Activesheet" (yes, an error is a better case then work on you don't know what).
Consider you know the name of the sheet (or the position of it), you can pass it in parameters
Public Sub Test(byval sheetname as string)
' First solution: declare a worksheet variable referencing to your worksheet
dim ws as worksheet, rng as range, area as range
set ws = Worksheets(sheetname)
Set rng = ws.Range("A1:A10,B1:B20,G1:G3")
For Each area In rng.Areas
area.Hidden = True
Next Area
' You could replace the dim of ws by a With Worksheets(sheetname)
' and work with .Range() instead
End Sub
Does VBA support using an array of range variables?
dim rangeArray() as range
dim count as integer
dim i as integer
count = 3
redim rangeArray(1 to count)
for i = 1 to count
msgbox rangeArray(i).cells(1,1).value
next
I can't get it to work in this type of application. I want to store a series of ranges in a certain order as a "master copy". I can then add, delete, sort or do whatever to this array and then just print it out to a series of ranges in excel. It doesn't seem like excel supports this - it just forces you to store your data in the spreadsheet and you have to reread it in order to use it.
No, arrays can't hold objects. But oObjects can hold objects. I think what you may want is a Range object that consists of various specific other Range object. In this example, rMaster is my "array" that holds three cells.
Sub StoreRanges()
Dim rMaster As Range
Dim rCell As Range
Set rMaster = Sheet1.Range("A1")
Set rMaster = Union(rMaster, Sheet1.Range("A10"))
Set rMaster = Union(rMaster, Sheet1.Range("A20"))
For Each rCell In rMaster
MsgBox rCell.Address
Next rCell
End Sub
With my new found knowledge that arrays can hold ranges (thnx jtolle), here's an example of how you would store ranges in an array and sort them
Sub UseArray()
Dim aRng(1 To 3) As Range
Dim i As Long
Set aRng(1) = Range("a1")
Set aRng(2) = Range("a10")
Set aRng(3) = Range("a20")
BubbleSortRangeArray aRng
For i = LBound(aRng) To UBound(aRng)
Debug.Print aRng(i).Address, aRng(i).Value
Next i
End Sub
Sub BubbleSortRangeArray(ByRef vArr As Variant)
Dim i As Long, j As Long
Dim vTemp As Variant
For i = LBound(vArr) To UBound(vArr) - 1
For j = i To UBound(vArr)
If vArr(i).Value > vArr(j).Value Then
Set vTemp = vArr(i)
Set vArr(i) = vArr(j)
Set vArr(j) = vTemp
End If
Next j
Next i
End Sub
It's not entirely clear what you want to do, but...
If you want a collection, why not use a VBA Collection Object?
Dim myRanges as New Collection
A Collection.Item can be any object, including a Range.
A Range object doesn't hold data; it holds a reference to worksheet cells. If you want the Range contents in your collection, you'll have to copy them to and from the worksheet.
As with Java, your VBA variables are ephemeral, whether in an Array or Collection. If you want to close the file and have the data there when you open it again, you have to have it in worksheet cells. The worksheets are your persistence mechanism.
I'm going to take a big leap here so if I'm way off, ignore me. What I think you're looking for suggests setting up a separate worksheet as your "database", populated with List/Table objects holding your raw data. In front of that, is your "user sheet" where you do the interesting stuff, referring to the data in the database sheet. Name everything.
It's not completely clear for me what you're talking about.
If you're asking about an ability to create Ranges that map to nothing and exist on their own, then no, there's no way. A Range object is just something that refers to a certain area of a worksheet. It doesn't have any storage of its own or something. Several different instances of Range class can refer to the same worksheet area, too.
And if you just want to store some references in an array, then that's fine, go for it. The only problem with your code is that you don't initialize the array elements before using them: as the Range is a reference type, all elements get initialized with Nothings by default.