Combining 2D (2-dimensional) arrays - arrays

I am using VBA in Excel to consume an XML file and dump specific information into individual tabs. I want to be able to combine 2-dimensional arrays. The arrays have a "known" number of columns but an "unknown" number of rows. Consider the following two arrays:
array1:
a b c
d e f
array2:
1 2 3
4 5 6
How do I combine these to arrays if I want the following result:
array3:
a b c
d e f
1 2 3
4 5 6
And just out of curiosity, how would I code if instead I wanted to add to the right instead of the bottom, like this:
array4:
a b c 1 2 3
d e f 4 5 6
I can't seem to find the answer to this anywhere.
Please keep in mind my example above is rather small, but in reality, I'm trying to do this with approx 100,000 rows of data at once. There are only six columns of data, if that matters.
The goal here is to assemble a large array and then write it to an Excel sheet all in one step because when I do it in pieces the performance is really poor.
If possible, I'd prefer a solution that does not require iteration.
The reason I ask about both ways is that in reality I want to add kind of sequentially. For instance, assume I have four arrays, A, B, C, D.
First, add array A:
A
Then, add array B:
A B
Then, add array C:
A B
C
Then, add array D:
A B
C D
and so forth...
Keep in mind that each of the above arrays would be sized such that they "fit" correctly meaning A and B have the same number of rows, but different number of columns. A and C on the other hand have the same number of columns but a different number of rows. And so on...
I wanted to add a demonstration using Macro Man's code from below. Here is what he provided (I added a bit so readers can just copy/paste):
Option Explicit
Sub Testing()
Dim Array1(0 To 1, 0 To 2) As String
Array1(0, 0) = "a"
Array1(0, 1) = "b"
Array1(0, 2) = "c"
Array1(1, 0) = "d"
Array1(1, 1) = "e"
Array1(1, 2) = "f"
Dim Array2(0 To 1, 0 To 2) As String
Array2(0, 0) = "1"
Array2(0, 1) = "2"
Array2(0, 2) = "3"
Array2(1, 0) = "4"
Array2(1, 1) = "5"
Array2(1, 2) = "6"
Dim i As Long
For i = 1 To 25000
With Range("A" & Rows.Count).End(xlUp).Offset(IIf(IsEmpty([A1]), 0, 1), 0)
.Resize(UBound(Array1, 1) - LBound(Array1, 1) + 1, _
UBound(Array1, 2) - LBound(Array1, 2) + 1).Value = Array1
End With
With Range("A" & Rows.Count).End(xlUp).Offset(IIf(IsEmpty([A1]), 0, 1), 0)
.Resize(UBound(Array2, 1) - LBound(Array2, 1) + 1, _
UBound(Array2, 2) - LBound(Array2, 2) + 1).Value = Array2
End With
Next i
End Sub
When you run the above code, which goes back to the spreadsheet each time to write the small amount of data, this takes a long time to run. On my dual Xeon machine, like 25-30 seconds.
However, if you rewrite and populate the array FIRST, then write to the spreadsheet ONCE, it runs in about one second.
Option Explicit
Sub Testing()
Dim Array1(0 To 99999, 0 To 2) As String
Array1(0, 0) = "a"
Array1(0, 1) = "b"
Array1(0, 2) = "c"
Array1(1, 0) = "d"
Array1(1, 1) = "e"
Array1(1, 2) = "f"
Dim i As Long
For i = 0 To 99999
Array1(i, 0) = "a"
Array1(i, 1) = "b"
Array1(i, 2) = "c"
Next i
With Range("A" & Rows.Count).End(xlUp).Offset(IIf(IsEmpty([A1]), 0, 1), 0)
.Resize(UBound(Array1, 1) - LBound(Array1, 1) + 1, _
UBound(Array1, 2) - LBound(Array1, 2) + 1).Value = Array1
End With
End Sub
I would like to see a solution which does the same thing, except being able to add "chunks" of data instead of individual items. Adding arrays to bigger arrays, ideally. Even better would be if the "parent" array somehow dynamically resized itself.
John Coleman's answer below worked great.
I actually combined a bit of Macro Man's with John's test() subroutine and this dynamically re-sizes the range:
Option Explicit
Sub test()
Dim A As Variant, B As Variant
ReDim A(0 To 1, 0 To 1)
ReDim B(0 To 1, 0 To 1)
A(0, 0) = 1
A(0, 1) = 2
A(1, 0) = 3
A(1, 1) = 4
B(0, 0) = 5
B(0, 1) = 6
B(1, 0) = 7
B(1, 1) = 8
Dim Array1 As Variant
Array1 = Combine(A, B)
With Range("A" & Rows.Count).End(xlUp).Offset(IIf(IsEmpty([A1]), 0, 1), 0)
.Resize(UBound(Array1, 1) - LBound(Array1, 1) + 1, _
UBound(Array1, 2) - LBound(Array1, 2) + 1).Value = Array1
End With
End Sub

Here is a VBA function that can combine two 2-dimensional arrays into a single 2-dimensional array. It can be used either from VBA or as an array-formula directly in Excel. Iteration is unavoidable here in VBA since the language doesn't have primitives for things like concatenating arrays:
Function Combine(A As Variant, B As Variant, Optional stacked As Boolean = True) As Variant
'assumes that A and B are 2-dimensional variant arrays
'if stacked is true then A is placed on top of B
'in this case the number of rows must be the same,
'otherwise they are placed side by side A|B
'in which case the number of columns are the same
'LBound can be anything but is assumed to be
'the same for A and B (in both dimensions)
'False is returned if a clash
Dim lb As Long, m_A As Long, n_A As Long
Dim m_B As Long, n_B As Long
Dim m As Long, n As Long
Dim i As Long, j As Long, k As Long
Dim C As Variant
If TypeName(A) = "Range" Then A = A.Value
If TypeName(B) = "Range" Then B = B.Value
lb = LBound(A, 1)
m_A = UBound(A, 1)
n_A = UBound(A, 2)
m_B = UBound(B, 1)
n_B = UBound(B, 2)
If stacked Then
m = m_A + m_B + 1 - lb
n = n_A
If n_B <> n Then
Combine = False
Exit Function
End If
Else
m = m_A
If m_B <> m Then
Combine = False
Exit Function
End If
n = n_A + n_B + 1 - lb
End If
ReDim C(lb To m, lb To n)
For i = lb To m
For j = lb To n
If stacked Then
If i <= m_A Then
C(i, j) = A(i, j)
Else
C(i, j) = B(lb + i - m_A - 1, j)
End If
Else
If j <= n_A Then
C(i, j) = A(i, j)
Else
C(i, j) = B(i, lb + j - n_A - 1)
End If
End If
Next j
Next i
Combine = C
End Function
I tested it in 4 different ways. First I entered your two example arrays in the spreadsheets and used Combine directly in excel as an array formula:
Here A7:C10 contains the array formula
{=combine(A1:C2,A4:C5)}
and A12:F13 contains the array formula
{=combine(A1:C2,A4:C5,FALSE)}
Then, I ran the following sub:
Sub test()
Dim A As Variant, B As Variant
ReDim A(0 To 1, 0 To 1)
ReDim B(0 To 1, 0 To 1)
A(0, 0) = 1
A(0, 1) = 2
A(1, 0) = 3
A(1, 1) = 4
B(0, 0) = 5
B(0, 1) = 6
B(1, 0) = 7
B(1, 1) = 8
Range("A15:B18").Value = Combine(A, B)
Range("C15:F16").Value = Combine(A, B, False)
End Sub
Output:

If possible, I'd prefer a solution that does not require iteration.
Try this:
Function Combine(m, n)
Dim m1&, m2&, n1&, n2&
m1 = UBound(m, 1): m2 = UBound(m, 2)
n1 = UBound(n, 1): n2 = UBound(n, 2)
With Worksheets.Add
.[a1].Resize(m1, m2) = m
.[a1].Resize(n1, n2).Offset(m1) = n
Combine = .[a1].Resize(m1 + n1, m2)
Application.DisplayAlerts = False
Application.ScreenUpdating = False
.Delete
Application.DisplayAlerts = True
Application.ScreenUpdating = True
End With
End Function
Note: this is just a demo to show proof of concept. Currently it does vertical stacking of two 2d arrays. Simple to modify to also do horizontal stacking.
Note: I'm typically opposed to this sort of thing, but if you think about it, an Excel sheet is analogous to a really big 2d array and while this is indeed a sleghammer approach, it is quick and there is no iteration!

You could try re-sizing the destination to match the array's dimensions. Something along the lines of:
(assuming your arrays are called 'Array1' and 'Array2')...
With Range("A" & Rows.Count).End(xlUp).Offset(IIf(IsEmpty([A1]), 0, 1), 0)
.Resize(UBound(Array1, 1) - LBound(Array1, 1) + 1, _
UBound(Array1, 2) - LBound(Array1, 2) + 1).Value = Array1
End With
With Range("A" & Rows.Count).End(xlUp).Offset(IIf(IsEmpty([A1]), 0, 1), 0)
.Resize(UBound(Array2, 1) - LBound(Array2, 1) + 1, _
UBound(Array2, 2) - LBound(Array2, 2) + 1).Value = Array2
End With

Related

The loop over two arrays take LONG

Thanks for your helps,
I have two arrays: A (100k row, 10 col) and B (100k row, 12 col)
The following code (thanks to BSALV) loop through A and B => It takes really long to finish. Is there any way to speedup.
ReDim Preserve B(1 To UBound(B), 1 To UBound(B, 2) + 4)
ReDim arr(1 To UBound(B), 1 To 2)
For i = 1 To UBound(B)
iSell = B(i, 3): mysold = 0
r = Application.Match(B(i, 2), Application.Index(A, 0, 2), 0)
If IsNumeric(r) Then
For i1 = r To UBound(A)
If A(i1, 2) = B(i, 2) And A(i1, 1) <= B(i, 1) Then
x = Application.Max(0, Application.Min(A(i1, 3), iSell))
If x > 0 Then
mysold = mysold + x
iSell = iSell - x
MyValueSold = MyValueSold + x * A(i1, 4)
A(i1, 3) = A(i1, 3) - x
If A(i1, 3) <= 0 Then A(i1, 2) = "~"
End If
If A(i1, 3) > 0 Then Exit For
End If
Next
End If
arr(i, 1) = mysold: arr(i, 2) = MyValueSold
Next
This operation is really slow when using larger arrays:
r = Application.Match(B(i, 2), Application.Index(A, 0, 2), 0)
You can get much better performance just by replacing the Index/Match line with a dictionary lookup.
To illustrate:
Sub Tester()
Const NROWS As Long = 100000
Dim i As Long, r, t
Dim dict As Object
Set dict = CreateObject("scripting.dictionary")
Dim A(1 To NROWS, 1 To 10)
'populate some dummy data
For i = 1 To UBound(A, 1)
A(i, 2) = Application.RandBetween(1, NROWS)
A(i, 3) = i
Next i
'First your existing row lookup...
t = Timer
For i = 1 To 100 'only testing 100 lookups (too slow for more!)
r = Application.Match(i, Application.Index(A, 0, 2), 0)
Next i
Debug.Print "Index/Match lookup", Timer - t, "*100* lookups"
'populate a dictionary for lookups...
t = Timer
For i = 1 To NROWS
dict(A(i, 2)) = i 'mapping second column first occurences to row #
Next i
Debug.Print "Mapping done", Timer - t
'Now the dictionary lookup
t = Timer
For i = 1 To NROWS
If dict.Exists(i) Then
r = dict(i)
End If
Next i
Debug.Print "Dictionary lookup", Timer - t, NROWS & " lookups"
End Sub
Output:
Index/Match lookup 9.62 *100* lookups '<<< slow slow!
Mapping done 0.12
Dictionary lookup 0.26 100000 lookups
EDIT: changes in your existing code
Dim rngMatch As Range '<<< added
'...
'...
Set lo = Sheets("Exc").ListObjects("TBL_Buy")
Set rngMatch = lo.DataBodyRange.Columns(2) '<<< lookup range
With lo.Range
.Sort .Range("B1"), xlAscending, , .Range("A1"), xlAscending, Header:=xlYes
aBuy = lo.DataBodyRange.Value2
.Sort .Range("A1"), xlAscending, , .Range("B1"), xlAscending, Header:=xlYes
End With
'...
For i = 1 To UBound(aResult)
'...
r = Application.Match(aResult(i, 2), rngMatch, 0) '<<<
'...
'...

vba - loop inside a loop freezes excel

i am trying to make a loop to go through an array(47193, 4) and an array 2 named attack(41892,1). The idea here is that the attack array has the values in order from the sheet i want to later on add the values to the next column, this is why i add the values to a third array. So the loop is going to go one by one the value from attack array while looping through arr array to find the common data. i tried copying the values directly to the sheet but excel freezes a lot. Now with this way, excel still freezes at this point. Is there anything wrong with it?
Dim arr3() As Variant
Dim dee As Long
ReDim arr3(UBound(attacks, 1), 1)
For k = 0 To UBound(attacks, 1)
j = 0
For j = 0 To UBound(arr, 1)
If attacks(k, 0) = arr(j, 0) And attacks(k, 1) = arr(j, 2) Then
arr3(dee, 0) = attacks(k, 0)
arr3(dee, 1) = attacks(k, 1)
de = dee + 1
End If
Next j
Next k
Here's some code showing how to use a Dictionary:
Sub Tester()
Const SZ As Long = 10000 'size of test arrays
Dim arr1(1 To SZ, 1 To 2)
Dim arr2(1 To SZ, 1 To 2)
Dim arr3(1 To SZ, 1 To 2) '<<matches go here
Dim n As Long, m As Long, i As Long, t, dict, k
t = Timer
'fill test arrays with random data
For n = 1 To SZ
arr1(n, 1) = CLng(Rnd * 200)
arr1(n, 2) = CLng(Rnd * 200)
arr2(n, 1) = CLng(Rnd * 200)
arr2(n, 2) = CLng(Rnd * 200)
Next n
Debug.Print "Filled test arrays", Timer - t
t = Timer
'test the nested loop approach
For n = 1 To SZ
For m = 1 To SZ
If arr1(n, 1) = arr2(m, 1) And arr1(n, 2) = arr2(m, 2) Then
i = i + 1
arr3(i, 1) = arr1(n, 1)
arr3(i, 2) = arr1(n, 2)
End If
Next m
Next n
Debug.Print "Finished nested loop", Timer - t, i & " matches"
t = Timer
'create a lookup using a dictionary
Set dict = CreateObject("scripting.dictionary")
For n = 1 To SZ
k = arr1(n, 1) & "|" & arr1(n, 2)
dict(k) = dict(k) + 1
Next n
Debug.Print "Filled dictionary", Timer - t
t = Timer
i = 0
Erase arr3
'Perform the match against arr2 using the dictionary
For m = 1 To SZ
k = arr2(m, 1) & "|" & arr2(m, 2)
If dict.exists(k) Then
i = i + 1
arr3(i, 1) = arr2(m, 1)
arr3(i, 2) = arr2(m, 2)
End If
Next m
Debug.Print "Finished dictionary loop", Timer - t, i & " matches"
End Sub
Output:
Filled test arrays 0
Finished nested loop 9.101563 2452 matches
Filled dictionary 0.03125
Finished dictionary loop 0.0078125 2177 matches
Note the # of matches is slightly different - the nested loop catches duplicate matches but the Dictionary only counts unique matches. You might need to make adjustments depending on your use case.

Add column (as first) with 1 to exsiting Variant Array in VBA

I have a array which have 1 or more columns and now I want to add one more column (consists only of 1), but I don't know how do do that. The situation looks like that:
My code:
Dim X() As Variant
X = Range("A1:C3").Value2
It's is important to put column with 1 as first. Probably I need to use ReDim Preserve but nothing works for me.
I think you have some options, but instead of extending the index of the array and transposing, trying to move the values etc which seems too much of a hassle, I would rather add 1 to the Excel range and then create the array:
Range("B1:D3").Value2 = Range("A1:C3").Value2
Range("A1:A3").Value2 = 1
X = Range("A1:D3").Value2
Resize the Array adding a column to the last dimension
Shift all the data to the right.
Assign 1 to the first position in each row
Sub AddColumnShiftData()
Dim X As Variant
Dim i As Long, j As Long
X = Range("A1:C3").Value2
ReDim Preserve X(1 To 3, 1 To 4)
For i = 1 To UBound(X)
For j = UBound(X, 2) To 2 Step -1
X(i, j) = X(i, j - 1)
Next
X(i, 1) = 1
Next
End Sub
Try matrix multiplication by the identify matrix....Well almost identity matrix. Then add 1 to every element in of the resulting matrix. You can use the Excel's Worksheet function for matrix multiplication.
Almost identity matrix
Dim X As Variant
X = Range("A1:C3").Value2
Dim Y As Variant
n = UBound(X, 2)
m = n + 1
Z = UBound(X, 1)
ReDim Y(1 To n, 1 To m)
'Set All values to zero
For i = 1 To n
For j = 1 To m
Y(i, j) = 0
Next j
Next i
' Set offset diagonal to 1
For i = 1 To n
Y(i, i + 1) = 1
Next i
' Matrix MMult
X = Application.WorksheetFunction.MMult(X, Y)
' Add 1 to the first column
For i = 1 To Z
X(i, 1) = 1
Next i
Alternative via Application.Index()
Just for fun (note that the resulting array is a 1-based 2-dim array):
Sub AddFirstIndexColumn()
Const FIXEDVALUE = 1 ' value to replace in new column 1
'[1] get data
Dim v: v = getExampleData()
'[2] define column array inserting first column (0 or 1) and preserving old values (1,2,3)
v = Application.Index(v, _
Application.Evaluate("row(1:" & UBound(v) & ")"), _
Array(1, 1, 2, 3)) ' columns array where 0 reinserts the first column
' [3] add an current number in the first column
Dim i As Long
For i = LBound(v) To UBound(v): v(i, 1) = FIXEDVALUE: Next i
End Sub
Function getExampleData()
' Method: just for fun a rather unusual way to create a 2-dim array
' Caveat: time-consuming for greater data sets; better to assign a range to a datafield array
Dim v
v = Array(Array(2, 3, 5), Array(3, 8, 9), Array(4, 2, 1))
v = Application.Index(v, 0, 0)
getExampleData = v
End Function
Related links
Some pecularities of `Application.Index()
Insert vertical slices into array

A loop for modifying and printing an array in VBA

So, I'm trying to print modified iterations of an array of 100; for the first row I want 1 to 100 of the array, for the second 2 to 100, all the way to the 100th row with just array(100), and all of these rows starting with column A. I can print the first row just fine, but for the subsequent ones I'm not getting any output.
q = 1
For m = 1 To last_age
Sheets("Sheet1").Range(Cells(q, 1), Cells(q, UBound(Data) + 1)) = Data 'Works the first pass, but not for q>1
For p = 0 To UBound(Data) - 1
Data(p) = Data(p + 1)
Next p
If UBound(Data) > 0 Then
ReDim Data(0 To UBound(Data) - 1)
q = q + 1
End If
Next m
All my variables seem to be incrementing correctly, but after the first m loop my Data array isn't being put in the second row. Any thoughts?
Slightly different approach:
Sub Tester()
Dim data(100), i, last_age, sht As Worksheet, q, m
'fill test array
For i = 1 To 100: data(i - 1) = i: Next
Set sht = Sheets("Sheet1")
last_age = 100
q = 1
sht.Cells(q, 1).Resize(1, UBound(data) + 1) = data
For m = 2 To last_age
q = q + 1
sht.Cells(1, 1).Offset(0, m - 1).Resize( _
1, (UBound(data) + 1) - (m - 1)).Copy _
sht.Cells(q, 1)
Next m
End Sub
You are re-dimensioning your array in this line:
ReDim Data(0 To UBound(Data) - 1)
but you are not using the Preserve keyword so your data is getting deleted. Try using:
ReDim Preserve Data(0 To UBound(Data) - 1)

Excel Macro Multi Dimension Array Value Deleted after IF function

I met with some problem with Excel Macro.
I am trying too copy values from various cells of a worksheet into an array for use of comparing with other worksheet's cell value later.
However, I am stuck at the array to store all the value I am trying to assign to it.
Below is the code piece I have done.
Sub singleEntry(suppRow As Integer)
Dim j As Integer
Dim myArray() As Variant
Dim a As Integer
Dim b As Integer
Dim c As Integer
Worksheets("Ind. Supp. Plan Time").Select
Cells(suppRow, "I").Select
For j = 9 To 13
c = j - 8
ReDim myArray(5, 4) As Variant
myArray(c, 1) = c
'ReDim Preserve myArray(5, 4) As Variant
If Cells(suppRow, j).Value = "*" Then
ReDim Preserve myArray(5, 4) As Variant
'myArray(j - 8, 1) = j - 8
myArray(j - 8, 2) = Cells(suppRow, "P").Value
myArray(j - 8, 3) = Cells(suppRow, "Q").Value
myArray(j - 8, 4) = Cells(suppRow, "R").Value
MsgBox "array = {" & myArray(c - 1, 2) & "}"
Else
ReDim Preserve myArray(5, 4) As Variant
myArray(j - 8, 2) = "1"
myArray(j - 8, 3) = "1"
myArray(j - 8, 4) = "1"
MsgBox "array(1,3) = {" & myArray(1, 3) & "}"
End If
Next j
ReDim Preserve myArray(5, 4) As Variant
'For a = 1 To 5
' For b = 1 To 4
' MsgBox "Array = {" & myArray(a, b) & "}"
' Next b
'Next a
End Sub
I put in MsgBox to view the result of executing the code, I am sure the lines are executed as expected.
If I print the value of the array straight away after assign one value to it, the value printed is correct.
However, now I can't solve this problem.
Hopefully anyone know this can give me a help.
Thank you very much!
Not sure why you can't retrieve values. I tested this and it works.
Sub singleEntry(suppRow As Integer)
Dim arrStore(1 To 5, 1 To 4) As Variant, col As Integer, r As Integer, c As Integer
Worksheets("Ind. Supp. Plan Time").Select
For col = 9 To 13
arrStore(col - 8, 1) = col - 8
arrStore(col - 8, 2) = IIf(Cells(suppRow, col) = "*", Cells(suppRow, "P"), 1)
arrStore(col - 8, 3) = IIf(Cells(suppRow, col) = "*", Cells(suppRow, "Q"), 1)
arrStore(col - 8, 4) = IIf(Cells(suppRow, col) = "*", Cells(suppRow, "R"), 1)
Next col
For r = 1 To 5
For c = 1 To 4
Debug.Print arrStore(r, c)
Next c
Next r
End Sub
Points to note:
Given that you always fill the array there is no need to ReDim. It's redundant (and expensive)
I've used the ternary IIF statement to tidy up the code i.e. if "*" then x else 1
I don't think you need the variable c so I've removed it
I've added a simple loop at the end to print out the array (which works for me)
Does this solve it?

Resources