I want to populate an array with the values of a range in other sheet different than the active one. I do need to load into arrays data ranges values from many different sheets and then made thousands of operations.
I cannot be thinking about activating sheets, using loops, or even worse accessing the data in the sheets by cells(). And I want to write a neat and clean code avoiding loops for inteligibility.
I started by:
dim claimsarray as variant
claimsArray = Range(Cells(1, 1), Cells(a, b)).Value
a and b are integers
It seems to work. No error. But the values of the array are empty because the array is populated with the values of the active sheet. I want to get the values froma sheet called "claims".
claimsArray = sheets(“claims”).Range(Cells(1, 1), Cells(a, b)).Value
that gives me an error 1004
looking for solutions in stackoverflow I tried the following modifications:
FIRST
worksheets() instead of sheets():
claimsArray = worksheets(“claims”).Range(Cells(1, 1), Cells(a, b)).Value
gives me an error 1004
SECOND
Changing the dimensioning of the array
dim claimsarray as variant
vs
dim claimsarray() as variant
all combinations give me error 1004
It seems to me that you can populate an array in this way only in the active sheet. So I modify:
sheets(”claims”).activate
claimsArray = worksheets(“claims”).Range(Cells(1, 1), Cells(a, b)).Value
it works.
THE QUESTION:
How to populate an array from a different sheet without using loop and without having to activate it.
Why cant I not refer to other sheet to populate the array? Where is the flaw? is it just that VBA is weak code?
NOTE1:
I read many websites about populating arrays with ranges:
http://www.cpearson.com/excel/ArraysAndRanges.aspx
https://excelmacromastery.com/excel-vba-array/
at non avail. They don't really deal with this particular problematic.
NOTE2:
I ended up using the typical loop_:
For i = 1 To a
For j = 1 To b
claimsArray(i, j) = Sheets("claims").Cells(i, j).Value
Next j
Next i
5 lines instead of one. Makes the code so much longer and conbersome...
thanks to #banana I understood where is "the flaw" of passing a range to an array.
claimsArray = sheets(“claims”).Range(Cells(1, 1), Cells(a, b)).Value
does not work properly when "claims" is not the active sheet because Excel gets confused as to which cell is actually cells(1,1)
therefore the elegant and efective solution is simply telling excel this way:
dim ST as worksheet
dim claimsArray as variant
set ST = thisworkbook.sheets("claims")
claimsArray = ST.Range(ST.Cells(1, 1), ST.Cells(a, b)).Value
It is also very important as #banana pointed out in the coments that ST is initialize as to refering to "thisworkbook" to avoid that several workbooks are open having a sheet called "claims".
This whole problematic might be the reason why then populating arrays with loops is at the very end the less problematic way.
In my case it worked when I added the Sheets reference also inside the Range inputs (in each cell reference):
claimsArray = Sheets(“claims”).Range(Sheets(“claims”).Cells(1, 1),Sheets(“claims”).Cells(a, b)).Value
Related
I am aware that this;
Arr() =Range("E2:X2500")
...then doing stuff with the Arr, and then dumping back using:
Range("E2:X2500")=Arr()
is many folds more efficient (faster) that looping through and referencing cells directly.
It's light speed!
But, this range-to-array assignment only grabs cells' value.
Is there a way to assign an actual range (continuous or not) into an array (with the same light speed) in such a way that you could then treat array items the way you would refer to cells, like:
arr(23).row 'getting a row number
Or;
If Arr(23).Value ="Pending" then arr(23).font.bold=1 else arr(23).font.bold=0
I know i can dim a range-type array where each item can store an actual single cell range. But this array cannot be handled the same way - with one liner assignment:
Dim Arr () as Range
Set Arr = Range("E2:X2500") 'error
Instead, I would need to iterate each cell and assign it to the next item in the range-type array, which would allow me to treat items the way I'd refer to cells, but take substantially longer to load as I'm dealing with a Loop.
Also how would I dump a range-type array back into the sheet with the same ease and effectiveness of the one liner assignment? I think the only way would be to use a loop yet again, correct?
Side question :
Speedwise, is it any better to refer to cells via a range-type array over referring to cells directly via the sheet, or are both basically the same?
Thanks!
Well, the array use will save a lot of code running time. But, there are some issues which must be understood:
First thing when work in VBA and your project increases, is to properly declare your variables. Try making a reflex in putting Option Explicit on top of all your modules. In the array case, the thing, from this point of view stays like that:
Dim Arr() As variant, arr1 As Variant
Both declarations work in excel. But the second one is recommended (on mai taste), when you need an array from a range. When you want building a, let us say, result array, it will be zero based and you must take care of the range size where the values will be returned.
The array content cannot be retrieved exactly like you tried in your question in case of not fix/known number of elements. Look at the next test code:
Sub testArrays()
Dim sh As Worksheet, rng As Range, arrTest As Variant
Set sh = ActiveSheet
Set rng = sh.Range("A1:F4")
arrTest = rng.value
sh.Range("J1").Resize(UBound(arrTest, 1), UBound(arrTest, 2)).value = arrTest
End Sub
It is recommended to use arrTest = sh.Range("A1:F4").value. Using range Value. Excel is able to understand what you need according to your declaration, but it is good for you to differentiate somehow, from the way of the range definition.
Sometimes, you need to build an array during analyzing of the dynamic range. If you cannot know the new array dimensions and need to Redim (Preserve), only the second dimension of the array can be re-dimensioned and Transpose function must be use, in such a case. And finally the resulted array can be properly loaded in a range, only if you know the array number of rows and columns.
You can deduce the range row, from the array row, in the next way:
If we are referring to the above arrTest we know that its first row is first row of the sheet and it has 5 columns.
So, arrTest(3, 1) will be sh.Range("A3").Value and its row would be 3.
Then, arrTest(3, 4) will be sh.Range("D3").Value and its row would be also 3.
If your array comes from a range starting with the fifth row, you must add four in order to obtain the sheet row extracted from the array row...
So, your example can be transformed in:
If arrTest(3, 4) ="Pending" then sh.Cells(3, 4).Font.Bold=1 Else sh.Cells(3, 4).Font.Bold=0
Now if you need a ranges array, you cannot do it in the way you tried. You must use the ranges address and build the range at the end:
Sub testArraysBis()
Dim sh As Worksheet, rng As Range, rng1 As Range, lastCol As Long
Dim rng2, arrTest As Variant, arrT As Variant, arrF As Variant
Set sh = ActiveSheet
lastCol = sh.Cells(1, Cells.Columns.Count).End(xlToLeft).column
Set rng = sh.Range(sh.Cells(1, 1), sh.Cells(4, lastCol))
Set rng1 = sh.Range("A5:F6")
arrT = Array(rng.Address, rng1.Address)
arrTest = rng.value
Debug.Print UBound(arrTest), LBound(arrTest)
sh.Range("J1").Resize(UBound(arrTest, 1), UBound(arrTest, 2)).value = arrTest
Set rng2 = sh.Range(arrT(0))
Debug.Print rng2.Address
arrF = sh.Range(arrT(0)).value
Debug.Print UBound(arrF, 2)
End Sub
rng2 range will be built using the address string, extracted from arrT array. An array (arrF) can also be extracted from the arrT first element...
Epilog:
The best way, in terms of speed, is to load the range in arrays, make all processing using them (in memory and very fast due to this aspect), but the most important issue is to build another array (or even a range, using Union) and retrieve the data AT ONCE. Sending of each partial processing result to a cell/range consumes a lot of time and other resources, for a big range size...
To increase the performance of my VBA code for large spreadsheets, I'm converting the sheets into 2D Arrays of Strings that the program can refer to instead of leaving open and looping through the memory intensive spreadsheets. The sheets dramatically slow down my computer, and at present the macro is slower than doing it by hand (I've removed many unnecessary columns and removed all formatting from these sheets, and no formulas seem to extend past the used range-- they're also stored as .xlsb's. They're already about one-half to two-thirds the size of the originals, so I don't think there's anything else to be done to optimize them).
Note that I'm developing in outlook, but relying heavily on data from excel sheets- the use case is an email auto-responder that searches the sheets for an ID supplied in an email, and then replies to that email with a phone number from the sheets. I have the proper references in place, and the program opens the sheets fine, just (painfully) slowly.
I'd like to use nested For loops to load the spreadsheets into the arrays programmatically, and then store those arrays in another array so they can in turn be looped through. In my research, I've found code to make jagged arrays in VBA (How do I set up a "jagged array" in VBA?), and 2D Arrays in VBA (Multi-dimensional array in VBA for Microsoft Word on Mac) but not arrays of 2D Arrays.
This is the code I wrote to make the 2D arrays- the Dim and ReDim lines throw syntax errors.
For k = LBound(sheetsArr) To UBound(sheetsArr)
Dim myWbksArr(k)() As String
ReDim myWbksArr(k)(sheetsArr(k).UsedRange.Rows.Count, sheetsArr(k).UsedRange.Columns.Count)
Next k
Where sheetsArr is an array of Worksheets into which I copied the sheets I'm referencing to avoid another for loop to iterate through Workbooks as well, using
Dim sheetsArr() As Worksheet, runningIndex As Long
runningIndex = 0
ReDim sheetsArr(1 To totalSheets) 'It would make sense to me to extend this syntax to an array as above, since an array is itself a type/object in other languages
For j = LBound(myWbks) To UBound(myWbks) 'j iterates through the workbooks
Set myWbks(j) = Workbooks.Open(FileName:=arr(j), UpdateLinks:=False) 'false should suppress update links msgbox
For Each sh In myWbks(j).Worksheets 'iterates through the worksheets in each workbook
sheetsArr(runningIndex) = sh 'add the worksheet to an array of worksheets so we don't have to use another for loop to get through the workbook layer too
runningIndex = runningIndex + 1
Next
Next j
What's the correct syntax to create an array of 2D arrays using for loops?
Maybe this will help.
It will read the contents of all the open workbooks into an array of arrays.
You'll probably want to change the method of selected the range to store, and maybe even the method of deciding which workbooks to go through. But this should give you an idea:
Option Explicit
Sub wbtoArr()
Dim wbsArr, wbArr
Dim WB As Workbook, WS As Worksheet
Dim I As Long, J As Long
ReDim wbsArr(1 To Workbooks.Count)
J = 0
For Each WB In Workbooks
J = J + 1
ReDim wbArr(1 To WB.Worksheets.Count)
I = 0
For Each WS In WB.Worksheets
I = I + 1
wbArr(I) = WS.UsedRange
Next WS
wbsArr(J) = wbArr
Next WB
End Sub
I've only been using VBA for a couple of weeks now so bear with me.
I'm trying to change a macro so that it reads from an array instead of a range. One of the sections I need to change uses .formulaR1C1 to run a vlookup, but im having a hard time trying to work out how this can be done with the array.
An example of this is the below line:
.Range("M2:L" & lastrow).FormulaR1C1 = "=VLOOKUP(RC[-1], Sheet2!R1C1:R4C10, 3, 0)"
I'm not too sure on whether I can set the value of an array to a formula, as I've done above, or if I have to maybe store the value as a String and then edit the value of the cell later when printing the column back on to the worksheet.
What I have so far is below:
For i = 2 To lastrow
arr(i, 13).FormulaR1C1 = "=VLOOKUP(RC[-1],Sheet2!R1C1:R4C10,3,0)"
Next
Thanks in advance!
Storing the value as a string would certainly be a valid option. Simply write the formula in your array, presuming it is of Variant / String type, and when you put the data back into a worksheet you can use .FormulaR1C1 on a Cell (Range) object to apply it as a formula.
arr(i, 13) = "=VLOOKUP(RC[-1],WMSPAM!R1C1:R4C10,3,0)"
...
Range("M2").FormulaR1C1 = Arr(1,13)
I believe this approach is likely the most effective and most easily maintained for you. As you are learning and appear to be curious of what is possible, here are a couple more examples of how you could approach this, with some further explanation.
.FormulaR1C1 is a Range object method and so the only way it could be called on an Array item would be if that item was a Range object.
Dim arr(0 To 10) As Range
Set arr(0) = Range("A1")
arr(0).FormulaR1C1 = "=2+2"
Note that as Ranges are an Object (of reference type), this operation will directly effect the Range specified in the array. In the above example, the formula "=2+2" will be placed into cell A1. You can learn more about the difference between Reference and Value types here.
If your array contains only values, the other way to achieve what you need is to use the WorksheetFunction object. With this object you can access the formula functions, that you would use in worksheet, within VBA.
WorksheetFunction.VLookup(Arg1, Arg2, Arg3, Arg4)
As with anything in writing code the WorksheetFunction methods take some trial and error to get them to work how you would expect, but in my experience these specifically can be a tricky to implement, there are though some cases where they can be very useful!
You can read more about the VLookup method here.
You may try to use .Offset():
Sub Test()
lastrow = 11
With Sheets(1)
Set Rng = .Range("M2:L" & lastrow)
End With
For i = 0 To Rng.Rows.Count - 1
Rng.Offset(i, 12).FormulaR1C1 = "=VLOOKUP(RC[-1],Sheet2!R1C1:R4C10,3,0)"
Next
End Sub
I'm not sure what you are asking. Your first code line writes a formula to a two-column range in the most efficient way there is already, your second snippet shows the less efficient way of doing it one cell at the time. If the goal is to use the vlookup to fill cells and then dispose of the formula, one efficient way is this:
.Range("M2:L" & lastrow).FormulaR1C1 = "=VLOOKUP(RC[-1], Sheet2!R1C1:R4C10, 3, 0)"
.Range("M2:L" & lastrow).Value2 = .Range("M2:L" & lastrow).Value2
I have a technical question:
My issue:
I create a
Dim arrTemp as Variant
Dim wbSource as workbook
Dim wbTarget as workbook
because I need to export multiple ranges from multiple worksheets (not fixed range) to another workbook. My code looks like:
' Worksheet 1
arrTemp = wbSource(1).Range("A1:B2").value
wbTarget(1).Range("A1:B2").value = arrTemp
If Not(IsArrayEmpty(arrTemp)) Then Erase arrTemp
' Worksheet 2
arrTemp = wbSource(2).Range("A1:B2").value
wbTarget(2).Range("A1:B2").value = arrTemp
If Not(IsArrayEmpty(arrTemp)) Then Erase arrTemp
' Worksheet 3
arrTemp = wbSource(3).Range("A1:B2").value
wbTarget(3).Range("A1:B2").value = arrTemp
If Not(IsArrayEmpty(arrTemp)) Then Erase arrTemp
(worksheet can be empty in the first place, that's why empty arr handler)
(worksheets can contain int/str/double/... and the size is not that big to define specific arr type)
My question is:
Does it make sense to erase the array every time? or It will be overwritten automatically?
I did a test to check the properties of the array (Lbound & UBound) before and after defining the array with a new range. I can see that It automatically Redim the array. Does it means that I only need to clear it in the end of the procedure?
Or it is a good practice to clear it in between?
Last but not least, do you see any problem in my code? Better way to perform this task?
Many thanks in advance!
Edit:
Bear in mind
The code is not correct for this task, no need to transfer to an array!
In Short (thanks to rory!):
No need to erase the array every time in between.
Only before leaving the procedure.
My worksheet requires the following array formula in BG2.
=INDEX('Client'!O$2:O$347473,
MATCH(1, (('Client_Cost'!D$2:D$347473='Client'!BC2)*
('Client_Cost'!E$2:E$347473='Client'!BE2)), 0))
This provides a two-column match (Client_Cost!D:D to Client!BC2 AND Client_Cost!E:E to Client!BE2) and returns the corresponding value from Client!O:O.
The large number of rows makes the array formula very calculation-intensive. I can deal with a few hundred rows (~90 seconds for 500 rows) but I need results all the way down to Client'!BG347473 and I would like them sometime this year.
I've tried using Application Evaluate to return a result from the array formula into an variant array and subsequently returning the array of results to the worksheet en masse but it isn't the improvement I was hoping for. Looking for alternatives.
First off, I would recommend developing alternative methods with a smaller data set. 5K or 10K rows will either demonstrate a noticeable improvement or not; you can always expand to the original data set once you are confident you are not going to get into a long 'Not responding' state that you have to crash out of.
A common method of removing the array from that style of array formula¹ is a 'helper' column that concatenates the two values from column D and E in the Client_Cost worksheet into a single, delimited value. For example, in Client_Cost!Z2 as,
=CONCATENATE(Client_Cost!D2, "|", Client_Cost!E2)
Filled down to Client_Cost!Z347473 should only take a second or two.
Once that is set up, a single INDEX/MATCH function pair can provide a vastly more efficient lookup on a similarly concatenated Client!BC2 and Client'!BE2. In Client!BG2 as,
=INDEX(Client!O$2:O$347473,
MATCH(CONCATENATE(Client!BC2, "|", Client!BE2),
Client_Cost'!Z$2:Z$347473, 0))
That will take 1 hr, 51 minutes for 350K rows. While not yet optimal, it is a big improvement over the estimated ~17.5 hours that the original took.
The next logical step in optimizing that method would be working with a VBA Scripting.Dictionary object. A dictionary holds its own unique index on its keys and the concatenated values could be stuffed into a dictionary object to facilitate virtually instantaneous lookups on a large number of items (i.e. rows).
Sub JR_CSE_in_Array()
Dim olr As Long, rws As Long, JR_Count As Long, JR_Values As Variant
Dim v As Long, vTMP As Variant, vTMPs As Variant, dVALs As Object
Debug.Print Timer
Set dVALs = CreateObject("Scripting.Dictionary")
'get some dimensions to the various data ranges
With Worksheets("Client_Cost")
'only use as many rows as absolutely necessary
olr = Application.Min(.Cells(Rows.Count, "D").End(xlUp).Row, _
.Cells(Rows.Count, "E").End(xlUp).Row)
'store D & E
vTMPs = .Range(.Cells(2, 4), .Cells(olr, 5)).Value2
End With
With Worksheets("Client")
rws = Application.Min(.Cells(Rows.Count, "BC").End(xlUp).Row, _
.Cells(Rows.Count, "BE").End(xlUp).Row, _
UBound(vTMPs, 1))
'override the above statement for sampling
'rws = 5000
'building the Dictionary object takes a fair bit of time but it is worth it
vTMP = .Range(.Cells(2, 15), .Cells(olr, 15)).Value2
For v = LBound(vTMPs, 1) To UBound(vTMPs, 1)
If Not dVALs.Exists(Join(Array(vTMPs(v, 1), vTMPs(v, 2)), ChrW(8203))) Then _
dVALs.Add Key:=Join(Array(vTMPs(v, 1), vTMPs(v, 2)), ChrW(8203)), Item:=vTMP(v, 1)
Next v
'store BC and BE
vTMPs = .Range(.Cells(2, 55), .Cells(olr, 57)).Value2
End With
ReDim JR_Values(1 To rws, 1 To 1) 'force a two-dimension, one-based index on the array
'Debug.Print LBound(JR_Values) & ":" & UBound(JR_Values)
For JR_Count = LBound(JR_Values, 1) To UBound(JR_Values, 1) Step 1
If dVALs.Exists(Join(Array(vTMPs(JR_Count, 1), vTMPs(JR_Count, 3)), ChrW(8203))) Then
JR_Values(JR_Count, 1) = dVALs.Item(Join(Array(vTMPs(JR_Count, 1), vTMPs(JR_Count, 3)), ChrW(8203)))
End If
Next JR_Count
With Worksheets("Client")
.Range("BG2").Resize(UBound(JR_Values), 1) = JR_Values
End With
'Debug.Print dVALs.Count
dVALs.RemoveAll: Set dVALs = Nothing
Debug.Print Timer
End Sub
The elapsed time for that routine to run (without helper column(s)) was 45.72 seconds. Breaking it down, it took a full 13.4 seconds just to build the dictionary and the remainder was largely taken up by the actual lookup with a half-second here and there attributed to the bulk seeding of the variant arrays from the worksheets' values.
So the Scripting.Dictionary is the clear winner here. Unfortunately, it is not automatically calculating updates in the various columns when values change but at this stage of development, the worksheet should be set to manual calculation. Setting one of the formula-based solutions into a recalculation event from a single retyped value seems an inefficient expenditure of time.
All-in-all, this makes perfect sense. The original array formula is analogous to an SQL SELECT statement with an INNER JOIN on two fields and if my SELECT statement was running inefficiently the first thing I would do to improve it would be to look at the tables' indexes.
On a related note, any workbook with this much data should be saved as a Excel Binary Workbook regardless of whether it is macro-enabled or not. The file size of a binary workbook (.XLSB) is typically ¹⁄₃ the size of an equivalent .XLSX or .XLSM. Beyond a faster initial load time, many bulk operations should prove faster.
Anyone wishing to test their own optimizations can find my sample .XLSB workbook here for the time being. Don't blindly run the procedures without seeing what you're getting into first.
¹ Array formulas need to be finalized with Ctrl+Shift+Enter↵. Once entered into the first cell correctly, they can be filled or copied down or right just like any other formula. Try and reduce your full-column references to ranges more closely representing the extents of your actual data. Array formulas chew up calculation cycles logarithmically so it is good practise to narrow the referenced ranges to a minimum. See Guidelines and examples of array formulas for more information.