this is my first time posting!
I have a data set similar to
Essentially there is data split into three columns (ID, Record, and Variable). For each "group" (based on the ID they share), I need to be able to reset the order of the records so that 4 and 5 come last in the "group." Then, the function would be able to go to the next one. Finally, I would expect a result like
A few key points:
I cannot use sorting/filtering because none of the IDs/Records/Variables in the true data set are in numeric order
Cannot split it out into different sheets/macros, because there are thousands of unique IDs.
Tried to work through this, but have some issues with my code doing nothing (below). Any ideas?
Sub GrpUpdate()
Dim f As Long
Dim i As Long
Dim last As Long
grpOne = "4"
grpTwo = "5"
i = 2
f = i
last = Range("A:A").Find(what:=Range("A" & f).Value, after:=Range("A" & f), searchdirection:=xlPrevious, LookIn:=xlValues).Row
For f = i To last
If f = last Then
i = last + 1
f = i
last = Range("A:A").Find(what:=Range("A" & f).Value, after:=Range("A" & f), searchdirection:=xlPrevious, LookIn:=xlValues).Row
ElseIf Not IsError(Application.Match(grpOne, "B" & f & ":" & "B" & last, 0)) And Not IsError(Application.Match(grpTwo, "B" & f & ":" & "B" & last, 0)) Then
Rows(Range("B" & f & ":" & "B" & last).Find(what:=grpOne, after:=Range("B" & f), searchdirection:=xlPrevious, LookIn:=xlValues).Row).Cut
Rows(last).Insert
Rows(Range("B" & f & ":" & "B" & last).Find(what:=grpTwo, after:=Range("B" & f), searchdirection:=xlPrevious, LookIn:=xlValues).Row).Cut
Rows(last).Insert
ElseIf Not IsError(Application.Match(grpOne, "B" & f & ":" & "B" & last, 0)) Then
Rows(Range("B" & f & ":" & "B" & last).Find(what:=grpOne, after:=Range("B" & f), searchdirection:=xlPrevious, LookIn:=xlValues).Row).Cut
Rows(last).Insert
ElseIf Not IsError(Application.Match(grpTwo, "B" & f & ":" & "B" & last, 0)) Then
Rows(Range("B" & f & ":" & "B" & last).Find(what:=grpTwo, after:=Range("B" & f), searchdirection:=xlPrevious, LookIn:=xlValues).Row).Cut
Rows(last).Insert
End If
Next f
End Sub
Try this out:
Sub GrpUpdate()
Dim f As Range, first As Long, last As Long
Dim ws As Worksheet, numRows As Long, addrGroups As String, arrLast, g
Set ws = ActiveSheet 'or whatever
first = 2 'start here
arrLast = Array("4", "5") 'items which (if present) should be last for each Id, in order
Do While Len(ws.Cells(first, "A").Value) > 0
With ws.Cells(first, "A") 'find the last value
last = ws.Range("A:A").Find(what:=.Value, after:=.Cells(1), _
searchdirection:=xlPrevious, lookat:=xlWhole).Row
End With
numRows = 1 + (last - first)
If numRows > 1 Then 'ignore single rows
'Using the range address because we're cutting rows which can be
' weird with Range references....
addrGroups = ws.Cells(first, "B").Resize(numRows).Address 'address for the "group" range
For Each g In arrLast 'loop items to be ordered last
Set f = ws.Range(addrGroups).Find(what:=g, lookat:=xlWhole)
If Not f Is Nothing Then
f.EntireRow.Cut 'move the found row to the end of the group
ws.Rows(last + 1).Insert
End If
Next g
End If '>1 row
first = last + 1 'next Id
Loop
End Sub
Related
Every morning I have to compare manually row by row the information in my database with the one sent by the broker. Normally they should have exactly the same information.
The information is about executed trades.
1- I need to compare row by row looking at the values "Stock", "Qty", "Price" and "Date". If any row is not matched from each other (i.e. a value is erroneously different in one of the table or the entire row is missing), I need the unmatched rows to be printed in a third "OUTPUT" table.
My TABLE, BROKER'S TABLE & OUTPUT
2- The thing here is that there are duplicates like for "microsoft" or "nvidia" that are independent trades (different IDs). Duplicates must be kept in the comparison because they are different trades.
How could I manage the duplicates problems? Using Collections in Dictionary could help me? I would compare Table A to Table B and then Table B to Table A.
Or the presence of duplicates (actually independent trades) makes it impossible to perform ?
My file has more than 500 rows.
I'll add an entirely in-memory way to achieve the output.
The one thing worth considering is that we have no way of differentiating one ID from the next if they have matching stock, price, qty, and dates. So, the way I'm handling duplicates is in the order that they appear in the spreadsheet. So if there are two matching entries in your table, and only 1 in the broker table, I assume that the first entry in your table matches the first entry in the broker table, and therefore your second entry will be output.
Try working through the code. I took a few shortcuts for the sake of timing, but I also encapsulated each of the functions so that you can modify as needed. You'll just have to build out the DeserializeKey function to convert a key back to cells in the output sheet (shouldn't be too hard). See the below code, and let me know if it meets expectations.
Note: You will run the "CompareDictionaries" subroutine. The others are helper functions.
Sub CompareDictionaries()
Dim oMine As Object
Dim oBroker As Object
Dim myQueueCount As Long
Dim brokerQueueCount As Long
Dim minQueue As Long
Dim oMinQueue As Object
Set oMine = GetDictionary(Sheet1.Range("A2:E7"))
Set oBroker = GetDictionary(Sheet2.Range("A2:E6"))
For Each oKey In oMine.keys
'The whole row does not exist in the broker table
If Not oBroker.Exists(oKey) Then
Do While oMine(oKey).Count > 0
DeserializeKey oKey, oMine(oKey).dequeue
Loop
Else 'The keys exist in both tables
myQueueCount = oMine(oKey).Count
brokerQueueCount = oBroker(oKey).Count
If myQueueCount = brokerQueueCount Then
'Do nothing. They both have the same number of
'id's, and so we assume they are in sync.
Else
'Determine the minimum queue size, and get rid
'of that many values, since we won't need them
minQueue = IIf(myQueueCount < brokerQueueCount, myQueueCount, brokerQueueCount)
For i = 1 To minQueue
oMine(oKey).dequeue
oBroker(oKey).dequeue
Next i
'Take the remaining IDs out of the dictionary/queue that had more
If brokerQueueCount > myQueueCount Then
Set oMinQueue = oBroker
Else
Set oMinQueue = oMine
End If
Do While oMinQueue(oKey).Count > 0
DeserializeKey oKey, oMinQueue(oKey).dequeue
Loop
End If
End If
Next oKey
'The only remaining thing to test for is keys in the broker dict
'that are not in the myDict
For Each oKey In oBroker.keys
If Not oMine.Exists(oKey) Then
Do While oBroker(oKey).Count > 0
DeserializeKey oKey, oBroker(oKey).dequeue
Loop
End If
Next oKey
End Sub
Function GetDictionary(inputRange As Range) As Object
Dim oDict As Object
Dim sht As Worksheet
Dim cel As Range
Dim theKey As String
Dim oQueue As Object
Set sht = inputRange.Parent
Set oDict = CreateObject("Scripting.Dictionary")
For Each cel In Intersect(inputRange, sht.Columns(1))
theKey = GenerateKey(cel.Resize(, 5))
If oDict.Exists(theKey) Then
oDict(theKey).enqueue cel.Value
Else
Set oQueue = CreateObject("System.Collections.Queue")
oQueue.enqueue cel.Value
oDict.Add theKey, oQueue
End If
Next cel
Set GetDictionary = oDict
End Function
Sub DeserializeKey(ByVal theKey As String, theId As String)
'This is where you'd do some stuff to
'turn the key into individual cells, and store it.
'I'm only writing to the debug widnow to demonstrate
Debug.Print theId & " " & theKey
End Sub
Function GenerateKey(rng As Range) As String
GenerateKey = rng(2) & Format(rng(3), "0") _
& Format(rng(4), "0.00") & Format(rng(5), "mmddyyyy")
End Function
For those interested in this method, I'm editing this answer to add the "deserializekey" function:
Dim r As Long
Worksheets("Output").Activate
r = 1
'What we are doing here with "loopcell" is to check if the destination cells in the "output" sheet are empty or free.
'If not, we go down 1 row.
loopcell:
If IsEmpty(Range("A" & r).Value) = True Then
Range("A" & r).Value = "_" & theId & "_" & theKey
Else
r = r + 1
GoTo loopcell
End If
'The key is wrote to the the cell but we need to split every element of the key in multiple cells.
splitOutput = Range("A" & r).Value
splitArray = Split(splitOutput, "_")
For i = 1 To UBound(splitArray)
Cells(r, i).Value = splitArray(i)
Next i
Debug.Print theId & " " & theKey
End Sub ```
** New GetDictionary and Deserialize methods used to store more info **
Sub DeserializeKey(ByVal theKey As String, theId As Variant)
'This is where you'd do some stuff to
'turn the key into individual cells, and store it.
'I'm only writing to the debug widnow to demonstrate
Dim output As String
'Keep in mind that we have a 2d array, and we are reading
'one row at a time. So get the number of columns in the
'array, and then do whatever you need with them.
For i = LBound(theId, 2) To UBound(theId, 2)
output = output & " " & theId(1, i)
Next i
Debug.Print theKey & " -->" & output
End Sub
Function GetDictionary(inputRange As Range) As Object
Dim oDict As Object
Dim sht As Worksheet
Dim cel As Range
Dim theKey As String
Dim oQueue As Object
Dim columnCount As Long
Dim rngAsArray As Variant
Set sht = inputRange.Parent
'Get the column count of the input range. Since we don't
'hardcode it in, this function is more flexible to
'future changes
columnCount = inputRange.Columns.Count
Set oDict = CreateObject("Scripting.Dictionary")
For Each cel In Intersect(inputRange, sht.Columns(1))
theKey = GenerateKey(cel.Resize(, 5))
'Put the full row into an array, which we will then
'store as the content of the queue
rngAsArray = cel.Resize(, columnCount).Value
If oDict.Exists(theKey) Then
oDict(theKey).enqueue rngAsArray
Else
Set oQueue = CreateObject("System.Collections.Queue")
oQueue.enqueue rngAsArray
oDict.Add theKey, oQueue
End If
Next cel
Set GetDictionary = oDict
End Function
First of all, please read DS_London's comment.
If you would like to have a result sheet, then you can use below macro:
Option Explicit
Sub CompareData()
Dim wbk As Workbook
Dim wshMyData As Worksheet, wshBrokersData As Worksheet, wshResult As Worksheet
Dim i As Integer, j As Integer, k As Integer
Dim sTmp As String
On Error Resume Next
Set wbk = ThisWorkbook
Set wshResult = wbk.Worksheets("Result")
On Error GoTo Err_CompareData
'if there_s no result sheet
If Not wshResult Is Nothing Then
Application.DisplayAlerts = False
wbk.Worksheets("Result").Delete
Application.DisplayAlerts = True
End If
Set wshMyData = wbk.Worksheets("Sheet1")
Set wshBrokersData = wbk.Worksheets("Sheet2")
Set wshResult = wbk.Worksheets.Add(After:=wshBrokersData)
wshResult.Name = "Result"
wshResult.Range("A1") = "ID"
wshResult.Range("B1") = "Stock"
wshResult.Range("C1") = "Qty"
wshResult.Range("D1") = "Price"
wshResult.Range("E1") = "Date"
wshResult.Range("F1") = "My"
wshResult.Range("G1") = "Broker"
wshResult.Range("A1:G1").Interior.Color = vbGreen
'find last entry in your data
i = wshMyData.Range("A" & wshMyData.Rows.Count).End(xlUp).Row
'find last entry in brokers data
j = wshBrokersData.Range("A" & wshBrokersData.Rows.Count).End(xlUp).Row
'copy data into result sheet
k = 2
wshMyData.Range("A2:E" & i).Copy wshResult.Range("A" & k)
k = k + i - 1
wshBrokersData.Range("A2:E" & j).Copy wshResult.Range("A" & k)
k = k + j - 2
'remove duplicates
wshResult.Range("$A$1:$E$" & k).RemoveDuplicates Columns:=Array(2, 3, 4, 5), Header:=xlYes
k = wshResult.Range("A" & wshResult.Rows.Count).End(xlUp).Row
'start comparison ;)
'my data
sTmp = "(" & wshMyData.Name & "!" & wshMyData.Range("B1:B" & i).AddressLocal & "=B2)"
sTmp = sTmp & "*(" & wshMyData.Name & "!" & wshMyData.Range("C1:C" & i).AddressLocal & "=C2)"
sTmp = sTmp & "*(" & wshMyData.Name & "!" & wshMyData.Range("D1:D" & i).AddressLocal & "=D2)"
sTmp = sTmp & "*(" & wshMyData.Name & "!" & wshMyData.Range("E1:E" & i).AddressLocal & "=E2)"
sTmp = "=SUM(IF(" & sTmp & ", 1, 0))"
wshResult.Range("F2").BorderAround LineStyle:=xlContinuous
wshResult.Range("F2").FormulaArray = sTmp
wshResult.Range("F2:F" & k).FillDown
'brokres data
sTmp = "(" & wshBrokersData.Name & "!" & wshBrokersData.Range("B1:B" & i).AddressLocal & "=B2)"
sTmp = sTmp & "*(" & wshBrokersData.Name & "!" & wshBrokersData.Range("C1:C" & i).AddressLocal & "=C2)"
sTmp = sTmp & "*(" & wshBrokersData.Name & "!" & wshBrokersData.Range("D1:D" & i).AddressLocal & "=D2)"
sTmp = sTmp & "*(" & wshBrokersData.Name & "!" & wshBrokersData.Range("E1:E" & i).AddressLocal & "=E2)"
sTmp = "=SUM(IF(" & sTmp & ", 1, 0))"
wshResult.Range("G2").BorderAround LineStyle:=xlContinuous
wshResult.Range("G2").FormulaArray = sTmp
wshResult.Range("G2:G" & k).FillDown
'autofit
wshResult.Range("A:G").Columns.AutoFit
Exit_CompareData:
On Error Resume Next
Set wshMyData = Nothing
Set wshBrokersData = Nothing
Set wshResult = Nothing
Set wbk = Nothing
Exit Sub
Err_CompareData:
MsgBox Err.Description, vbExclamation, Err.Number
Resume Exit_CompareData
End Sub
Result:
As you can see, 0 means that there's no corresponding data in selected sheet.
What above macro does?
Adds new sheet: Result, then adds column headers (ID, Stock, Qty, Price, Date, My data, Broker in row 1 respectively in columns A-G)
Copies all data from your sheet (Sheet1) to Result sheet
Copies all data from broker's sheet (Sheet2) to Result sheet
Removes duplicates in Result sheet (based on all columns excluding ID)
Inserts formula-array in cell F2 and G2 and fill it down.
Important note: There's at least few other ways to achieve that...
Final note: Feel free to change the code to your needs.
I managed to sync selected data from three sheets into a fourth sheet. But the data doesn't align properly after empty cells beginning with the 14th row.
Now I'm trying to use arrays to align my data better. I have 3 sheets with columns Area, Zone, Employee and 6 numeric columns for each employee.
The data in Area, Zone & Employee is repeating itself in multiple rows so I need to add the numbers for every employee to have the Employee Name displayed only once with added data in other 6 columns.
I don't really have problem with filtering the names and adding data, but I'm not sure how to do it using arrays.
Or if anyone could help me find a mistake in my code that's causing the data to not align properly, I would also appreciate it. Below is my code so far, hopefully it would help.
Private Sub cmd_button1_Click()
Dim WS1 As Worksheet, ws2 As Worksheet, ws3 As Worksheet, ws4 As Worksheet
Set WS1 = Sheets("Sheet2")
Set ws2 = Sheets("Distribution")
Set ws3 = Sheets("Sheet3")
Set ws4 = Sheets("Sheet4")
Dim LastRow As Long
Dim R As Long, LR As Long, n As Long
Application.ScreenUpdating = False
'Getting the row number of last cell
LastRow = ws2.Range("A" & Rows.Count).End(xlUp).Row
'Deleting any previous data from destination sheet
ws2.Range("A2:AX10000").ClearContents
For i = 1 To 10
'If value in V column of the row is "" then copy the row to destination sheet
If WS1.Cells(i, "V").Value = "" Then
WS1.Range("E:E").Copy Destination:=ws2.Range("A1")
WS1.Range("F:F").Copy Destination:=ws2.Range("B1")
WS1.Range("G:G").Copy Destination:=ws2.Range("C1")
WS1.Range("A:A").Copy Destination:=ws2.Range("E1")
WS1.Range("O:O").Copy Destination:=ws2.Range("F1")
WS1.Range("P:P").Copy Destination:=ws2.Range("G1")
WS1.Range("R:R").Copy Destination:=ws2.Range("H1")
WS1.Range("S:S").Copy Destination:=ws2.Range("I1")
WS1.Range("Q:Q").Copy Destination:=ws2.Range("J1")
WS1.Range("T:T").Copy Destination:=ws2.Range("K1")
ws3.Range("E:E").Copy Destination:=ws2.Range("L1")
ws3.Range("F:F").Copy Destination:=ws2.Range("M1")
ws3.Range("G:G").Copy Destination:=ws2.Range("N1")
ws3.Range("A:A").Copy Destination:=ws2.Range("O1")
ws3.Range("S:S").Copy Destination:=ws2.Range("P1")
ws3.Range("T:T").Copy Destination:=ws2.Range("Q1")
ws3.Range("V:V").Copy Destination:=ws2.Range("R1")
ws3.Range("W:W").Copy Destination:=ws2.Range("S1")
ws3.Range("X:X").Copy Destination:=ws2.Range("T1")
ws4.Range("F:F").Copy Destination:=ws2.Range("U1")
ws4.Range("G:G").Copy Destination:=ws2.Range("V1")
ws4.Range("H:H").Copy Destination:=ws2.Range("W1")
ws4.Range("A:A").Copy Destination:=ws2.Range("X1")
ws4.Range("L:L").Copy Destination:=ws2.Range("Y1")
ws4.Range("M:M").Copy Destination:=ws2.Range("Z1")
ws4.Range("N:N").Copy Destination:=ws2.Range("AA1")
ws4.Range("O:O").Copy Destination:=ws2.Range("AB1")
ws4.Range("P:P").Copy Destination:=ws2.Range("AC1")
ws4.Range("Q:Q").Copy Destination:=ws2.Range("AD1")
End If
Next i
LR = Cells(Rows.Count, "C").End(xlUp).Row
Range("A2:AX" & LR).Sort Key1:=Range("A2"), Order1:=xlAscending
For R = 2 To LR
'Count the number of duplicates for third row
n = Application.CountIf(Columns(3), Cells(R, 3).Value)
'Sum up the values for every duplicate
Range("F" & R).Value = Evaluate("=Sum(F" & R & ":F" & R + n - 1 & ")")
Range("G" & R).Value = Evaluate("=Sum(G" & R & ":G" & R + n - 1 & ")")
Range("H" & R).Value = Evaluate("=Sum(H" & R & ":H" & R + n - 1 & ")")
Range("I" & R).Value = Evaluate("=Sum(I" & R & ":I" & R + n - 1 & ")")
Range("J" & R).Value = Evaluate("=Sum(J" & R & ":J" & R + n - 1 & ")")
Range("K" & R).Value = Evaluate("=Sum(K" & R & ":K" & R + n - 1 & ")")
Range("E" & R).Value = Evaluate("=Count(E" & R & ":E" & R + n - 1 & ")")
'Go to next value in third column
R = R + n - 1
Next R
On Error Resume Next
'Remove all duplicates
ws2.Range("$A$1:$K$7979").RemoveDuplicates Columns:=3, Header:=xlYes
On Error GoTo 0
'Fill out the table with values
Columns("A:K").AutoFit
Application.ScreenUpdating = True
Range("A1").Select
End Sub
The code above is for synchronizing the sheets with Distribution and filter the data from Sheet2, and I have 2 more buttons made to filter the other 2 sheets.
The code below is my attempt to align the data but it's not working correctly.
Sub LineEmUp()
Dim i As Long, j As Long, LR As Long
Application.ScreenUpdating = False
LR = Range("C" & Rows.Count).End(xlUp).Row
Columns("A:K").Sort Key1:=Range("A2"), _
Header:=xlYes, OrderCustom:=1, MatchCase:=False, _
Orientation:=xlTopToBottom, DataOption1:=xlSortNormal
Columns("L:T").Sort Key1:=Range("L2"), _
Header:=xlYes, OrderCustom:=1, MatchCase:=False, _
Orientation:=xlTopToBottom, DataOption1:=xlSortNormal
Columns("U:AD").Sort Key1:=Range("U2"), _
Header:=xlYes, OrderCustom:=1, MatchCase:=False, _
Orientation:=xlTopToBottom, DataOption1:=xlSortNormal
i = 2
Do
If Cells(i, "C") > Cells(i, "N") And Cells(i, "C") > "" Then
Cells(i, "A").Resize(1, 10).Insert xlShiftDown
ElseIf Cells(i, "N") > Cells(i, "W") And Cells(i, "N") > "" Then
Cells(i, "L").Resize(1, 10).Insert xlShiftDown
ElseIf Cells(i, "W") > Cells(i, "C") And Cells(i, "W") > "" Then
Cells(i, "U").Resize(1, 10).Insert xlShiftDown
ElseIf Cells(i, "C") < Cells(i, "N") And Cells(i, "C") > "" Then
Cells(i, "L").Resize(1, 10).Insert xlShiftDown
ElseIf Cells(i, "N") < Cells(i, "W") And Cells(i, "N") > "" Then
Cells(i, "U").Resize(1, 10).Insert xlShiftDown
ElseIf Cells(i, "W") < Cells(i, "C") And Cells(i, "W") > "" Then
Cells(i, "A").Resize(1, 10).Insert xlShiftDown
End If
i = i + 1
Loop Until Cells(i, "C") = "" And Cells(i, "W") = ""
Application.ScreenUpdating = True
End Sub
Hope I explained it properly. Thanks
Organization (without unnecessary repetition) is always important in coding, and especially key when troubleshooting. For example, your 29 copy-paste statements can be tidied up considerably - which shows some inconsistencies.
...I sorted them by source worksheet and then by source column, and grouped them together, also pasting into columns instead of single cells.
Edit:
There's a number of "weird things" going on here that require some explanation so I know whether they're designed this way intentionally.
**See my "'<<<<<<" notes below (There are some specific questions, starting with *what happens if you don't disable screen updating, and don't ignore the errors with On Error Resume Next...?
Option Explicit
Private Sub cmd_button1_Click()
Dim WS1 As Worksheet, ws2 As Worksheet, ws3 As Worksheet, ws4 As Worksheet
Set WS1 = Sheets("Sheet2")
Set ws2 = Sheets("Distribution")
Set ws3 = Sheets("Sheet3")
Set ws4 = Sheets("Sheet4")
'Dim LastRow As Long
Dim R As Long, LR As Long, n As Long, i As Integer
' <<<<< always ALLOW screen updating during troubleshooting, until your code
' <<<<< is functioning perfectly: It may give a clue to the problem.
'Application.ScreenUpdating = False
'Getting the row number of last cell '<<<<< variable [LastRow] is not being used.
'LastRow = ws2.Range("A" & Rows.Count).End(xlUp).Row
'Deleting any previous data from destination sheet
'ws2.Range("A2:AX10000").ClearContents
ws2.UsedRange.ClearContents ' <<<<<< instead of specifying a range, just clear what's used
For i = 1 To 10
'If value in V column of the row is "" then copy the row to destination sheet
If WS1.Cells(i, "V").Value = "" Then
WS1.Range("A:A").Copy Destination:=ws2.Range("E:E") '<<< there's no pattern to what's being copied,
WS1.Range("E:G").Copy Destination:=ws2.Range("A:C") '<<< (and in a strange criss-cross),
WS1.Range("O:S").Copy Destination:=ws2.Range("F:I") '<<< are you sure nothing's being missed?
WS1.Range("T:T").Copy Destination:=ws2.Range("K:K")
ws3.Range("A:A").Copy Destination:=ws2.Range("O:O")
ws3.Range("E:G").Copy Destination:=ws2.Range("L:N")
ws3.Range("S:T").Copy Destination:=ws2.Range("P:Q")
ws3.Range("V:X").Copy Destination:=ws2.Range("R:T")
ws4.Range("A:A").Copy Destination:=ws2.Range("X1")
ws4.Range("F:H").Copy Destination:=ws2.Range("U:W")
ws4.Range("L:Q").Copy Destination:=ws2.Range("Y:AD")
End If
Next i
LR = Cells(Rows.Count, "C").End(xlUp).Row
Range("A2:AX" & LR).Sort Key1:=Range("A2"), Order1:=xlAscending '<<<<< this could be a problem??
For R = 2 To LR
'Count the number of duplicates for third row
n = Application.CountIf(Columns(3), Cells(R, 3).Value)
'Sum up the values for every duplicate
Range("F" & R).Value = Evaluate("=Sum(F" & R & ":F" & R + n - 1 & ")") '<<<<<< this is a strange way to do this...,
Range("G" & R).Value = Evaluate("=Sum(G" & R & ":G" & R + n - 1 & ")") '<<<<<< can you explain the purpose of these lines?
Range("H" & R).Value = Evaluate("=Sum(H" & R & ":H" & R + n - 1 & ")") '<<<<<< why not just add the cells normally instead like this?
Range("I" & R).Value = Evaluate("=Sum(I" & R & ":I" & R + n - 1 & ")")
Range("J" & R).Value = Evaluate("=Sum(J" & R & ":J" & R + n - 1 & ")")
Range("K" & R).Value = Evaluate("=Sum(K" & R & ":K" & R + n - 1 & ")")
Range("E" & R).Value = Evaluate("=Count(E" & R & ":E" & R + n - 1 & ")")
'Go to next value in third column
R = R + n - 1 '<<<<< WOAH! don't change the value of R when it's being used inside a loop!!!
Next R
'On Error Resume Next '<<<<< Errors mean something - Don't ignore them! (especially during troubleshooting)
'Remove all duplicates
ws2.Range("$A$1:$K$7979").RemoveDuplicates Columns:=3, Header:=xlYes '<<< this shifts cells around, might be a problem
On Error GoTo 0
'Fill out the table with values
Columns("A:K").AutoFit
Application.ScreenUpdating = True
Range("A1").Select
End Sub
This works to put the values in the column:
Sub JR_ArrayToDebugPint2()
' written by Jack in the UK for [url]www.OzGrid.com[/url]
' our web site [url]www.excel-it.com[/url]
' Excel Xp+ 14th Aug 2004
' [url]http://www.ozgrid.com/forum/showthread.php?t=38111[/url]
Dim JR_Values(500)
Dim JR_Count As Integer
Dim R As Long
R = 2
For JR_Count = 1 To 500 Step 1
JR_Values(JR_Count) = Evaluate("=INDEX('Client'!$O$2:$O$347473,MATCH(1,(('Client_Cost'!$D$2:$D$347473=BC" & CStr(R) & ")*('Client_Cost'!$E$2:$E$347473=BE" & CStr(R) & ")),0))")
Sheet1.Range("BG" & CStr(R) & "").Value = JR_Values(JR_Count)
R = R + 1
'Debug.Print JR_Values(JR_Count)
Next JR_Count
End Sub
I've modified the original code I found on mrexcel.com
I get the correct list of values whether I Debug.Print or print to the worksheet. So in my mind, I ought to be able to put the values in an array as they are calculated, then use Range("BG2:BG500").Value = Application.Transpose(myarray).
I am assuming if I do this the values will be placed in the cells in the column all at once, rather than one at a time, which is what this code, and all others I've tried, is doing. I am also assuming that, if the values are placed in the cells in the column all at once, it is MUCH faster than placing the values in the cells one at a time.
What I'm not able to do is get the code to put the value in an array once the formula is evaluated. I've tried variations of the following with no success - statements to set the array and have the array take the value of the calculation are in caps and marked by ==>. The most common error I get is type mismatch.
Sub JR_ArrayToDebugPint2()
Dim JR_Values(500)
Dim JR_Count As Integer
Dim R As Long
==> DIM arrPRICE(0 TO 500) AS VARIANT
R = 2
For JR_Count = 1 To 500 Step 1
JR_Values(JR_Count) = Evaluate("=INDEX('Client'!$O$2:$O$347473,MATCH(1,(('Client_Cost'!$D$2:$D$347473=BC" & CStr(R) & ")*('Client_Cost'!$E$2:$E$347473=BE" & CStr(R) & ")),0))")
==> arrPRICE(R) = JR_VALUES(JR_COUNT)
R = R + 1
'Debug.Print JR_Values(JR_Count)
Next JR_Count
End Sub
When you dimension the variant array like Dim JR_Values(500) you are creating a one-dimensioned array based upon a zero-based index. This first element within the array is JR_Values(0) and the last is JR_Values(500) for a total of 501 array elements. While you could work from 0 to 499 with a little math, you can also force a one-based index on the variant array by declaring it that way.
The assumed worksheet parentage of the BC and BE columns where the individual row data criteria comes from is not definitive when using Application Evaluate like it is when the same formula is used on a worksheet. A worksheet knows who it is; VBA may or may not know what worksheet you are implying.
Sub JR_ArrayToDebugPint2()
Dim olr As Long, rws As Long, JR_Count As Long, JR_Values As Variant
'get some dimensions to the various data ranges
With Worksheets("Client_Cost")
'only use as many rows as absolutely necessary
olr = Application.Min(.Cells(Rows.Count, "C").End(xlUp).Row, _
.Cells(Rows.Count, "E").End(xlUp).Row)
End With
With Worksheets("Client")
rws = Application.Min(.Cells(Rows.Count, "BC").End(xlUp).Row, _
.Cells(Rows.Count, "BE").End(xlUp).Row)
'override the above statement unless you want to run this overnight
rws = 500
End With
ReDim JR_Values(1 To rws) 'force a one-based index on the array
'Debug.Print LBound(JR_Values) & ":" & UBound(JR_Values)
For JR_Count = LBound(JR_Values) To UBound(JR_Values) Step 1
'Debug.Print Evaluate("INDEX('Client'!O2:O" & olr & _
", MATCH(1, (('Client_Cost'!D2:D" & olr & "='Client'!BC" & JR_Count+1 & ")" & _
"*('Client_Cost'!E2:E" & olr & "='Client'!BE" & JR_Count+1 & ")), 0))")
'R would be equal to JR_Count + 1 if R was still used (starts as R = 2)
JR_Values(JR_Count) = _
Evaluate("INDEX('Client'!O2:O" & olr & _
", MATCH(1, (('Client_Cost'!D2:D" & olr & "='Client'!BC" & JR_Count + 1 & ")" & _
"*('Client_Cost'!E2:E" & olr & "='Client'!BE" & JR_Count + 1 & ")), 0))")
'Debug.Print JR_Values(JR_Count)
Next JR_Count
With Worksheets("Client")
.Range("BG2").Resize(UBound(JR_Values), 1) = Application.Transpose(JR_Values)
End With
End Sub
I've left a lot of comments for you to review and subsequently clean up. I recently wrote a narrative of declaring one-dimension and two-dimension variant arrays in How to assign the variable length of an array to integer variable.
I have the following data
Name ID Value
Alice 12C 500
Bob 14 60
Dan 15C 64
Dan 1C 25
Alice 4 556
Bob 11 455
In my data, Alice has both numerical (4) and string+numerical ID (12C) and I want to delete all Alice rows, while I want to hold on to data of names where their ID is strictly numeric (Bob 11, 14) or strictly string+numeric (Dan 15C , 1C).
First I make an array of unique Name entries:
FinalRow = 7
Name_column = 1
n = 1
Dim FID_Array() As Variant
ReDim Preserve FID_Array(1 To 1)
FID_Array(1) = Cells(2, Name_column)
For j = 3 To FinalRow
If Cells(j, Name_column).Value <> FID_Array(n) Then
ReDim Preserve FID_Array(1 To n + 1)
FID_Array(n + 1) = Cells(j, Name_column).Value
n = n + 1
End If
Next j
Then I make an Array of the row numbers that contain a particular Name
ReDim Preserve Count_FID_Array(1 To 1) As Variant
n = 1
range_FID = A2:A7
' In my actual code this is Range_FID
' range_FID = Cells(2, FolderId_column).Address & ":" & Cells(FinalRow, FolderId_column).Address
For Each itm5 In FID_Array()
Count_FID_Array(n) = Application.CountIf(" & range_FID & ", " & itm5 & ")
ReDim Preserve Count_FID_Array(1 To n + 1)
n = n + 1
Next itm5
I don't think my CountIf is working. I have tried to store the value of Count_FID_Array in another cell in a different sheet but I am getting #value!
If I got the countIf to work then I was going to sort the data by name, then double loop to check the ID variable the next "n" times to see if the last digit was "C" for all of them or to check if the ID was numeric for all of them.
Can you please point out why my countif is not working and is there a smarter way to do this?
I am using arrays of names here because in the end I want to feed the array into an autofilter and delete the rows that I don't want.
Update 1 3:45 PM Nov 21 2013: I have solved this as following:
I basically created three columns. First column was 0 or 1 depending on if the the ID was all numbers. The second column was 0 or 1 depending on if the last digit was "C" (in my real work the last two digits are "IB" ) and finally I compared the frequency of these occurences to the frequency of the Name itself. If any of those match then I give it the number 1 else 0. I use this index later to autofilter.
Now I'll try to use zx8754's shorter formula in the VBA code and I will try to address the issues regarding Countif that Joe has raised.
Sub conditionsforsubfolders()
FinalColumn = Cells(1, Columns.Count).End(xlToLeft).Column
FinalRow = Cells(Rows.Count, 1).End(xlUp).Row
ActiveWorkbook.ActiveSheet.Columns(FinalColumn + 1).Insert
ActiveWorkbook.ActiveSheet.Columns(FinalColumn + 2).Insert
ActiveWorkbook.ActiveSheet.Columns(FinalColumn + 3).Insert
Isnumber_Column = FinalColumn + 1
Is_IB_Column = FinalColumn + 2
Exceptions_Column = FinalColumn + 3
Cells(1, Isnumber_Column) = "Number"
Cells(1, Is_IB_Column) = "Letters"
Cells(1, Exceptions_Column) = "Exceptions"
For j = 1 To FinalColumn
If Cells(1, j).Value = "TradeId" Then
TradeId_column = j
ElseIf Cells(1, j).Value = "Total Notional per folder" Then
Total_Notional_Per_Folder_Column = j
ElseIf Cells(1, j).Value = "ExternalId" Then
ExternalId_Column = j
ElseIf Cells(1, j).Value = "FolderId" Then
FolderId_column = j
End If
Next j
range_FolderId_fixed = Cells(2, FolderId_column).Address & ":" & Cells(FinalRow, FolderId_column).Address
range_TradeId_fixed = Cells(2, TradeId_column).Address & ":" & Cells(FinalRow, TradeId_column).Address
range_Isnumber = Cells(2, Isnumber_Column).Address & ":" & Cells(FinalRow, Isnumber_Column).Address(RowAbsolute:=False, ColumnAbsolute:=False)
range_Isnumber_fixed = Cells(2, Isnumber_Column).Address & ":" & Cells(FinalRow, Isnumber_Column).Address
range_Is_IB = Cells(2, Is_IB_Column).Address & ":" & Cells(FinalRow, Is_IB_Column).Address(RowAbsolute:=False, ColumnAbsolute:=False)
range_Is_IB_fixed = Cells(2, Is_IB_Column).Address & ":" & Cells(FinalRow, Is_IB_Column).Address(RowAbsolute:=False, ColumnAbsolute:=False)
range_FolderId_cell = Cells(2, FolderId_column).Address(RowAbsolute:=False, ColumnAbsolute:=False)
range_TradeId_cell = Cells(2, TradeId_column).Address(RowAbsolute:=False, ColumnAbsolute:=False)
range_Exceptions = Cells(2, Exceptions_Column).Address & ":" & Cells(FinalRow, Exceptions_Column).Address(RowAbsolute:=False, ColumnAbsolute:=False)
Range(range_Isnumber).Formula = "=Isnumber(" & range_TradeId_cell & ")*1"
Range(range_Is_IB).Formula = "=(RIGHT(" & range_TradeId_cell & ",2)= ""IB"")*1"
Range(range_Exceptions).Formula = "=(SUMIF(" & range_FolderId_fixed & "," & range_FolderId_cell & "," & range_Isnumber_fixed & ")= COUNTIF(" & range_FolderId_fixed & "," & range_FolderId_cell & "))*1 +(SUMIF(" & range_FolderId_fixed & "," & range_FolderId_cell & "," & range_Is_IB_fixed & ")= COUNTIF(" & range_FolderId_fixed & "," & range_FolderId_cell & "))*1 "
Worksheets("Sheet1").UsedRange.AutoFilter Field:=7, Criteria1:="=1"
End Sub
Formula solution, no VBA:
=IF(SUMPRODUCT(--($A$2:$A$7=A2),--(ISNUMBER($B$2:$B$7)))=1,"delete","keep")
The problem with your CountIF call is that you're passing a poorly-formed string. You're literally passing "range_FID & ", " & itm5".
First, you set to properly define range_fid:
Dim range_fid As Range
Set range_fid = [A2:A7]
The call CountIF with:
count_fid_array(n) = Application.WorksheetFunction.CountIf(range_fid, itm5)
With that said, I would go about it differently:
Dim c As Range
Dim people As Collection: Set people = New Collection
Dim person As Collection
Dim code As String
For Each c In Range(Range("a2"), Range("a2").End(xlDown)) ' loop through all rows
If IsNumeric(c.Offset(0, 1)) Then ' check if the ID is numeric or not
code = "num"
Else
code = "alphanum"
End If
On Error Resume Next ' Needed in order to avoid error when person already exists in collection
Set person = New Collection
person.Add c.Value, "name"
person.Add code, "code"
people.Add person, c.Value ' will only be added if name doesn't already exist in collection
On Error GoTo 0
If people(c.Value)("code") <> code Then ' if the format (alpha/num) of the ID on the current row is different than the format of a previous row for this name....
people(c.Value).Remove ("code") ' then set the code to "diff"
people(c.Value).Add "diff", "Code"
End If
Next
For Each person In people ' just display the content; you can take appropriate action here
Debug.Print person("name") & ": " & person("code")
Next
The result is a Collection containing names and a code for each. The code will be one of:
num: all values for a name are numeric (Bob)
alphanum: all values for a name are alphanumeric (Dan)
diff: name has at least one each of numeric and alphanumeric (Alice)
Note that this could be done a little clearer with a Dictionary instead of a Collection, or with a Class, but I chose to take the most straightforward approach.
The following VBA code takes very long time to execute. I ran it 25 minutes ago for 48,000 rows and it's still running. How can I shorten the execution time?
Sub delrows()
Dim r, RowCount As Long
r = 2
ActiveSheet.Columns(1).Select
RowCount = UsedRange.Rows.Count
userresponse = MsgBox("You have " & RowCount & " rows", vbOKOnly, "Info")
Rows(RowCount).Delete Shift:=xlUp
' Trim spaces
Columns("A:A").Select
Selection.Replace What:=" ", Replacement:="", LookAt:=xlPart, _
SearchOrder:=xlByRows, MatchCase:=False, searchFormat:=False, _
ReplaceFormat:=False
' Delete surplus columns
Range("L:T,V:AA,AE:AG,AR:AR,AU:AU,AZ:AZ").Select
Selection.Delete Shift:=xlToLeft
' Delete surplus rows
Do
If Left(Cells(r, 1), 1) = "D" _
Or Left(Cells(r, 1), 1) = "H" _
Or Left(Cells(r, 1), 1) = "I" _
Or Left(Cells(r, 1), 2) = "MD" _
Or Left(Cells(r, 1), 2) = "ND" _
Or Left(Cells(r, 1), 3) = "MSF" _
Or Left(Cells(r, 1), 5) = "MSGZZ" _
Or Len(Cells(r, 1)) = 5 _
Or Cells(r, 3) = 0 Then
Rows(r).Delete Shift:=xlUp
ElseIf Int(Right(Cells(r, 1), 4)) > 4000 Then
Rows(r).Delete Shift:=xlUp
Else: r = r + 1
End If
Loop Until (r = RowCount)
End Sub
The biggest issue is probably the amount of data you are looping through. I've updated your code to create a formula to check if the row needs to be deleted, then you can filter on that formula result and delete all rows at once.
I've made a bunch of comments to both help you clean your code and understand what I did. I prefaced my comments with '=>.
One last note, loading the values into an array may help as well, but if you have many, many columns of data, this may be more difficult. I don't have a ton of experience with it, but I know it makes things worlds faster!
Good luck and have fun!
Option Explicit
Sub delrows()
With Application
.ScreenUpdating = False
.Calculation = xlCalculationManual
End With
Dim r As Long, RowCount As Long
r = 2
Dim wks As Worksheet
Set wks = Sheets(1) '=> change to whatever sheet index (or name) you want
'=> rarely a need to select anything in VBA [ActiveSheet.Columns(1).Select]
With wks
RowCount = .Range("A" & .Rows.Count).End(xlUp).Row '=> as opposed to [RowCount = UsedRange.Rows.Count], as UsedRange can be misleading
'NOTE: this also assumes Col A will have your last data row, can move to another column
userresponse = MsgBox("You have " & RowCount & " rows", vbOKOnly, "Info")
.Rows(RowCount).Delete Shift:=xlUp
' Trim spaces
'=> rarely a need to select anything in VBA [Columns("A:A").Select]
.Range("A1:A" & RowCount).Replace What:=" ", Replacement:="", lookat:=xlPart, _
SearchOrder:=xlByRows, MatchCase:=False, searchFormat:=False, _
ReplaceFormat:=False
' Delete surplus columns
'=> rarely a need to select anything in VBA [Range("L:T,V:AA,AE:AG,AR:AR,AU:AU,AZ:AZ").Select]
.Range("L:T,V:AA,AE:AG,AR:AR,AU:AU,AZ:AZ").Delete Shift:=xlToLeft ' as opposed to Selection.Delete Shift:=xlToLeft
' Delete surplus rows
'=> Now, here is where we help you loop:
'=> First insert column to the right to capture your data
.Columns(1).Insert Shift:=xlToRight
.Range("A1:A" & RowCount).FormulaR1C1 = "=If(OR(Left(RC[1],1) = ""D"",Left(RC[1],1) = ""H"", Left(RC[1],1) = ""I"", Left(RC[1],2) = ""MD"",Left(RC[1],2) = ""ND"",Left(RC[1],3) = ""MSF"",Left(RC[1],5) = ""MSGZZ"",Len(RC[1])=5),""DELETE"",If(Int(Right(RC[1],4)) > 4000,""DELETE"",""""),""""))"
'=> Now, assuming you something to delete ...
If Not .Columns(1).Find("DELETE", LookIn:=xlValues, lookat:=xlWhole) Is Nothing Then
'=> filter and delete
.Range("A1:A" & RowCount).AutoFilter 1, "DELETE"
Intersect(.UsedRange, .UsedRange.Offset(1), .Range("A1:A" & RowCount)).SpecialCells(xlCellTypeVisible).EntireRow.Delete
End If
'=> Get rid of formula column
.Columns(1).EntireColumn.Delete
End With
With Application
.ScreenUpdating = True
.Calculation = xlCalculationAutomatic
End With
End Sub
the reason its so slow is you are iterating over each cell. Below copies to an array, finds the rows that need deleting and then deletes. Update Sheet4 to your sheet and Range("A2").CurrentRegion to the area you require:
Dim data() As Variant
Dim count As Double, i As Double, z As Double, arrayCount As Double
Dim deleteRowsFinal As Range
Dim deleteRows() As Double
Application.ScreenUpdating = False
data = Sheet4.Range("A2").CurrentRegion.Value2
For i = 1 To UBound(data, 1)
count = count + 1
If (data(i, 1) = "D" Or Left(data(i, 1), 1) = "H" Or Left(data(i, 1), 1) = "I" Or Left(data(i, 1), 2) = "MD" _
Or Left(data(i, 1), 2) = "ND" Or Left(data(i, 1), 3) = "MSF" Or Left(data(i, 1), 5) = "MSGZZ" _
Or Len(data(i, 1)) = 5 Or data(i, 3) = 0 Or Int(Right(IIf(Cells(i, 1) = vbNullString, 0, Cells(i, 1)), 4)) > 4000) Then
ReDim Preserve deleteRows(arrayCount)
deleteRows(UBound(deleteRows)) = count
arrayCount = arrayCount + 1
End If
Next i
Set deleteRowsFinal = Sheet4.Rows(deleteRows(0))
For z = 1 To UBound(deleteRows)
Set deleteRowsFinal = Union(deleteRowsFinal, Sheet4.Rows(deleteRows(z)))
Next z
deleteRowsFinal.Delete Shift:=xlUp
Application.ScreenUpdating = True
Turn off the screen updates to start with. Add your observations post the following.
You can disable calculations as well, if you think it isn't affecting anything as such.
Application.ScreenUpdating = False
your code...
Application.ScreenUpdating = True
EDIT: I have uploaded a file here - https://dl.dropbox.com/u/24702181/TestDeleteRowsInChunk.xls
The workbook is macro enabled.
After opening, click on "Recover Data" followed by "Start Deleting".
Take a look at the code for details. I suppose it can be optimized further.
A couple of hints
Do a reverse loop.
Get cell contents in an array, use array to check for values.
Build a string for rows to be deleted.
Delete it in chunks.