Creating a Macro to Index Data from Government Website - database

I provide the link here: http://quickstats.nass.usda.gov/results/320F1D82-1064-30F1-809E-F77E509EC508. I would like to learn how to perform this task using a macro/script in Excel VBA. The problems which I have been unable to overcome myself are:
1) Not all the weeks in the year are represented in the data. I am not sure how to sort the data by particular week, not sure what command I would need.
2) Is is possible to the Excel function VLOOKUP in my vba script in order to select all the distinct values, say Week14 or Week15, before performing the calculation for each week? If so, how is it incorporated? My idea was to just perform a VLOOKUP in the macro to search for the five different crop conditions.
My first question on this topic was about how to download the information off the website, see Building a Macro to Download Data off a Website into Excel. I suggest that this is the first button, "download", and then there should be a second button to sort the data called "sort".
CommandButton_1 is for download data, see linked question.
Private Sub CommandButton1_Click()
Dim thisWb, downloadWb As Workbook
Set thisWb = ActiveWorkbook
Set downloadWb = Workbooks.Open("http://quickstats.nass.usda.gov/data/spreadsheet/4C43034A-0EAA-3171-B4FC-84CC95FC6E0C.csv")
downloadWb.Worksheets(1).Range("A1:U2613").Copy Destination:=thisWb.Worksheets(3).Range("A8")
downloadWb.Close
End Sub
This works very well. Now the second button (Sort Data) it was suggested below that
Private Sub CommandButton2_Click()
Public Function WeekTotal(Condition As String, Table As Range) As Integer
Dim rRow As Range
Dim xVal As Integer
xVal = 0
For Each rRow In Table.Rows
If InStr(LCase(rRow.Cells(1, 10).Value), "very") > 0 And LCase(Condition) = "poor" Then
'skip if row includes 'very', but the condition is 'poor' and not 'very poor'
ElseIf InStr(LCase(rRow.Cells(1, 10).Value), LCase(Condition)) > 0 Then
xVal = xVal + rRow.Cells(1, 13).Value
End If
Next rRow
WeekTotal = xVal
End Function
End Sub
This does not seem to work. Here is a picture, slightly altered width of columns, of what I am trying to deal with.

In my opinion, a macro is over-complicating a simple problem.
Simply add a new column with a VLOOKUP on the value that corresponds to your rating. To accomplish this, you will need a reference range that can provide such a value. (See screenshot 1).
In a new worksheet, list your weeks, and use SUMIF to calculate the total of the values in your vlookup column for each corresponding week. (See screenshot 2).
If you need to do a more sophisticated calculation, then provide additional detail and I'll modify this answer to help.

I45 = "Excellent"
J2:J36 is the Data Item column
M2:M36 is the Value column
=SUM(IF(ISERROR(SEARCH(I45,$J$2:$J$36)),0,$M$2:$M$36))
Returns: 14
This is an array formula, so you have to press Ctrl+Shift+Enter, not just Enter. (You will know you did it correctly because it will look like this in the formula bar: {=SUM(IF(ISERROR(SEARCH(I45,$J$2:$J$36)),0,$M$2:$M$36))}
It automatically adds { and } when you press Ctrl+Shift+Enter - you do not type them in.
What this formula does:
Search("Excellent", $J$2) would look for the string "Excellent" in J2 and return the position it found the string or an error if it did not find the string. We don't care about the actual value - we just want to know if it throws an error or not. We turn this into an array by using $J$2:$J$36. You can't see it, but what this actually does is return an array of values that might look like this :
{44;#VALUE!;#VALUE!;#VALUE!;#VALUE!;44;#VALUE!;#VALUE!;#VALUE!;#VALUE!;44;#VALUE!;#VALUE!;#VALUE!;#VALUE!;44;#VALUE!;#VALUE!;#VALUE!;#VALUE!;44;#VALUE!;#VALUE!;#VALUE!;#VALUE!;44;#VALUE!;#VALUE!;#VALUE!;#VALUE!;44;#VALUE!;#VALUE!;#VALUE!;#VALUE!}
By using ISERROR(SEARCH("Excellent", $J$2:$J$36)) we change that internal array into this:
{FALSE;TRUE;TRUE;TRUE;TRUE;FALSE;TRUE;TRUE;TRUE;TRUE;FALSE;TRUE;TRUE;TRUE;TRUE;FALSE;TRUE;TRUE;TRUE;TRUE;FALSE;TRUE;TRUE;TRUE;TRUE;FALSE;TRUE;TRUE;TRUE;TRUE;FALSE;TRUE;TRUE;TRUE;TRUE}
If it fails to find "Excellent", ISERROR will be TRUE, so we want to find the value for all the FALSE values in this array. We do this using IF. If the SEARCH is an error, 0, else give us the value another array - $M$2:$M$36 (the value column).
=IF(ISERROR(SEARCH("Excellent", $J$2:$J$36)), 0,$M$2:$M$36)
This gives us the following:
{2;0;0;0;0;2;0;0;0;0;2;0;0;0;0;2;0;0;0;0;2;0;0;0;0;2;0;0;0;0;2;0;0;0;0}
Since your sample data had 2 for excellent for each week.
Finally we add SUM() around it all to sum up the values.
=SUM(IF(ISERROR(SEARCH("Excellent", $J$2:$J$36)), 0,$M$2:$M$36))
Again, since this is an array formula, if you just hit enter with this formula, it will return 0, but if you hit Ctrl+Shift+Enter it will give you the correct result of 14.

If you want to use VBA, simply as practice, here is how you could use a custom formula.
To use this function you would type the following into a cell:
=WeekTotal("Excellent",$A$2:$M$36)
VBA code:
Public Function WeekTotal(Condition As String, Table As Range) As Integer
Dim rRow As Range
Dim xVal As Integer
xVal = 0
For Each rRow In Table.Rows
If InStr(LCase(rRow.Cells(1, 10).Value), "very") > 0 And LCase(Condition) = "poor" Then
'skip if row includes 'very', but the condition is 'poor' and not 'very poor'
ElseIf InStr(LCase(rRow.Cells(1, 10).Value), LCase(Condition)) > 0 Then
xVal = xVal + rRow.Cells(1, 13).Value
End If
Next rRow
WeekTotal = xVal
End Function

Related

Excel VBA: How to concatenate variant array elements (row numbers) into a range object?

I did research this question but could not find the specific answer I was looking for and am actually even more confused at present.
I created a macro that would run through rows on a sheet and run boolean checks on a number of cells in each row that looked for the presence or absence of specific values, or calculated the outcome of a specific inequality. On the basis of those checks, the macro may or may not pass the row number into a specific array. That part is working fine.
My issue is, now that I have the row numbers (stored in variant arrays) - I cannot figure out how to properly concatenate that data into a range and then take a bulk excel action on those items. What I'd like to do is create a range of those values and then delete all of those rows at once rather than looping through.
My macro is on my work computer, but here's something I wrote that should explain what I'm trying to do:
Sub Test()
Dim Str As String
Dim r As Range
Dim i, a As Integer
Dim Count As Integer
Dim RngArray()
Count = ThisWorkbook.Sheets("Sheet1").Cells(Rows.Count, "A:A").End(xlUp).Row
ReDim RngArray(Count)
a = 0
For i = 1 To Count
If Not i = Count Then
RngArray(a) = i
Str = Str & RngArray(a) & ":" & RngArray(a) & ", "
a = a + 1
ElseIf i = Count Then
RngArray(a) = i
Str = Str & RngArray(a) & ":" & RngArray(a)
a = a + 1
Else: End If
Next i
Set r = Range(Str)'Error Can Appear here depending on my concatenation technique
Range(Str).EntireRow.Delete 'error will always appear here
End Sub
I've combined a few steps here and left out any Boolean checks; in my actual macro the values in the arrays are already stored and I loop from LBound to UBound and concatenate those values into a string of the form ("1:1, 2:2, 3:3, ...., n:n")
The reason why I did this is that the rows are all over the sheet and I wanted to get to a point where I could pass the argument
Range("1:1, 2:2, 3:3, ..., n:n").EntireRow.Delete
I think it's clear that I'm just not understanding how to pass the correct information to the range object. When I try to run this I get a "Method Range of Object Global" error message.
My short term fix is to just loop through and clear the rows and then remove all of the blank rows (the macro keeps track of absolute positions of the rows, not the rows after an iterative delete) - but I'd like to figure out HOW to do this my way and why it's not working.
I'm open to other solutions as well, but I'd like to understand what I'm doing wrong here. I should also mention that I used the Join() to try to find a workaround and still received the same type of error.
Thank you.
After some experimentation with my dataset for the macro above, I discovered that it worked on small sets of data in A:A but not larger sets.
I ran Debug.Print Len(Str) while tweaking the set size and macro and found that it appears Range() can only accept a maximum of 240 characters. I still don't understand why this is or the reason for the specific error message I received, but the macro will work if Len(Str) < 240.
I'll have to loop backwards through my array to delete these rows if I want to use my present method...or I may just try something else.
Thanks to Andrew for his attention to this!

Error 13 Type mismatch in filtered Excel worksheet

I have a filtered worksheet in which I need the values which are in column N.
This worksheet also has labels in row 1. Everything works perfect, except when I have only one row left after the filter. On that moment I get Error 13.
I see that the value the program is working with is the value from cel A1, this is a textvalue from the label and not the value in column N. Can anybody tell me what I do wrong ?
Dim Rng As Range
Set Rng = Range("N2", Range("N2").End(xlDown)).Cells.SpecialCells(xlCellTypeVisible)
For Each cell In Rng.SpecialCells(xlCellTypeVisible)
score(y) = cell
totaal_k = totaal_k + score(y)
y = y + 1
Next cell
Does this happen on every single-after-filter row or just row 641?
I think xlDown is the problem here. Is there anyway for you to predefine the last row range, or does it have to be dynamic? If there is, then I think it might be the solution. If not I'd try defining the data range differently, for example by implementing a quick function to determine last row of data. Sample can be found under this link: https://www.extendoffice.com/documents/excel/1721-excel-find-last-row-column-with-data.html
Also, as a part of troubleshooting, I advise putting a breakpoint on line 4 of your excerpt above and see what values are being applied to cell and Rng.
Now that I mentioned it - it just occurred to me that you use xlCellTypeVisible twice - both on your Rng and later referring to that Rng. If you applied it when setting the variable, why use it later? Try removing that part from one of the places and see how that works.
Try checking if Rng has actually been set before interacting with its methods/properties.
Dim Rng As Range
Set Rng = Range("N2", Range("N2").End(xlDown))
If Not (Rng is nothing) then
For Each cell In Rng.SpecialCells(xlCellTypeVisible)
score(y) = cell
totaal_k = totaal_k + score(y)
y = y + 1
Next cell
End if

Variable Range as Input

I am a new user of VBA for Excel (2013), and I've been struggling with a problem for quite a while.
I want to create a function that, among the other input variables that the user has to provide, has several ranges of cells, that should be variable and "easily selectable" by the user themselves.
It should be something like (for example) the Excel "SUM" formula, where you can input several ranges such as "A1:C1,F1:H1,...", selecting them with your mouse.
Moreover, I have to transfer the values selected by the user in a 2D array. Every range selected by the user has to become a row of my 2D array.
Further to what I've said so far, the number of ranges that the user inputs should be anyone, and only the first one should be mandatory.
Given the "structure" of my spreadsheet, all the data that the user has to select are arranged in rows. (Does it make any difference if they are arranged in rows or in columns?)
Please note that I shouldn't use the "Input Box", because it slows down a lot the procedure when I "drag down" a row to apply the formula to many rows below the active one.
Below, I report the only parts of the code that I was able to write (not very much...)
Function Trial_Version(MyRange1 As Range, MyRange2 As Range, MyRange3 As Range) As Double
''' The number of ranges that the user is be able to input
'''should be anyone. Only the first Range should be mandatory
Dim TotalArray()() As Double
'''The user should be able to input as many "sub-arrays" as they want,
'''so I do not know how many parentheses to write
TotalArray = (MyRange1)(MyRange2)
'''here, the same problem as the previous line
RANGE_TO_ARRAY = TotalArray
'''This is the output of my function
End Function
Thank you very much in advance,
Orlando
Not the full answer, but hopefully points you in the right direction.
A standard Range object can already hold multiple 'areas' (e.g. C5:D9,G9:H16,B14:D18). You can loop through these using the .Areas property.
You can get easily get a 2D array of values from a Range object by using a Variant variable (e.g. Dim a as Variant, a = SomeRange). If multiple areas are selected however, this method will only get the first area, so you'll need to loop through each area to get all values.
The code below requires validation added, plus possibly handling any column / row offsets (i.e. See comment in loop related to indexes).
Public Function ProcessRange(ByRef rng As Range)
'# PURPOSE: Process multi-area range
Dim rngArea As Range ' Single area in multi-area selection
Dim lngCol As Long ' Column in selected range(area), 1 based
Dim lngRow As Long ' Row in selected range(area), 1 based
Dim avarValues As Variant ' 2D variant array for range(area) values, 1 based
' --> Add validation here <--
' Go through each area in selected range
' --------------------------------------
For Each rngArea In rng.Areas
' Store the values into 2d array (Row, Column), 1 based
avarValues = rngArea
' Show each value held in the array
For lngRow = 1 To UBound(avarValues, 1)
For lngCol = 1 To UBound(avarValues, 2)
' Note row / column index is NOT necessarily the same as the row/column on the worksheet
' Index 1 will be the first cell in the selected area
Debug.Print "Area " + rngArea.Address + ", value in AREA row " & lngRow & ", column " & lngCol & ": " & avarValues(lngRow, lngCol)
Next lngCol
Next lngRow
Next rng
End Function

Need help, Excel Formula, Count number of cells in a range that match the content of a different range

I am pretty good on Excel but this has got me.
I have a range of entries - $D7:$AH7
I want to know how many times these entries appear in this range - $K$7:$V$10
I have tried countif array but I think it is trying to match row with row rather than each entry in my range against the entire lookup range. I also tried transposing the lookup range onto the same line but then I think it tries to match column by column in the same fashion.
If sumproduct works I can't use that is too calculation extensive with this document.
I am happy to use a VBA solution if you have that suggestion.
Non VBA solution:
=SUMPRODUCT(COUNTIF($K$7:$V$10,$D7:$AH7))
VBA solution:
Founds how many times values from rng1 appears in rng2:
Can it ignore blanks?
UPD:
Function countEntries(rng1 As Range, rng2 As Range) As Long
Dim arr1, arr2, a1, a2
arr1 = rng1.Value
arr2 = rng2.Value
For Each a1 In arr1
If Trim(a1) <> "" Then
For Each a2 In arr2
If a2 = a1 Then
countEntries = countEntries + 1
Exit For
End If
Next
End If
Next
End Function
call it like this: =countEntries($D7:$AH7,$K$7:$V$10).
P.S. I use loop for determining whether array contains value or not rather than Application.Match/Application.CountIf because it's up to 10 times faster. See this link for details: Matching values in string array

compare two worksheets (with different number of rows on each sheet) and to get the list of not matching rows

Below is my requirement
1.Two sheets are available , say Sh1, Sh2
2.Row count differs between both the sheets(row count may/may not be the same between them)
3.Used Column of the sheets varies for data considered for comparison
But at any given point of time, both sheets will have same number of Used columns
( ideally speaking range is Dynamic for both sheets)
4.Now I need to perform comparison between them and find out the excess records in Sh1, Sh2 and save them in diff worksheets Sh3, Sh4 or so.
5.The comparison performed needs to be a Row level Comparison between Sh1 & Sh2
6.I prefer using Arrays (Loading the worksheets to Arrays) and perform comparison on it and return value - PERFORMANCE PLAYS A MAJOR ROLE SINCE THE DATA MAY BE OFF MILLIONS OF RECORDS
Is there a way to perform comparison of two Arrays whose size are non-identical ?
Matched rows can be Ignored
Is there a way to apply some Join() functions and read a complete row from Array1 and compare them with Array2 ?
Sorry If im violating the Forum rules by Any means !
I hope there is no Discussion on Comparing Two sheets with different Dynamic ranges to find the mismatches either here or anywhere
I mostly see ppl limiting the comparison to only one Column , or just for fixed ranges
My ULTIMATE AIM IS TO DO SOME VBA CODE WHICH OPERATES KIND OF "BEYOND COMPARE"(Well not exact behavior, read it a like)
I have been working on a similar kind of vba macro.
hope the below code helps !
All you need is to picture and include is
1)Row level comparison if possible
2)Looping through arrays and finding the Difference
** I am
Code shown below :
'Comparison Code
Sub CompareMacro()
Application.ScreenUpdating = True
Dim sheet1, sheet2, sheet4, sheet3 As Worksheet
Dim rComp, rcomp1 As Range, addy As String, addz As String
Dim iRow As Long, jCol As Long
Dim kRow As Long, lCol As Long
Dim strFileRange As String
Dim strDBRange As String
Set sheet1 = Sheets("File")
Set sheet2 = Sheets("DB")
Set sheet3 = Sheets("File(-)DB")
Set sheet4 = Sheets("DB(-)File")
sheet1.Select
Set rComp = sheet1.UsedRange
sheet2.Select
Set rcomp1 = sheet2.UsedRange
addy = rComp.Address
addz = rcomp1.Address
ary1 = rComp
ary2 = rcomp1
ary3 = sheet3.Range(addy)
ary4 = sheet4.Range(addz)
'*********File MINUS DB code goes here*********
'Step 1
'VALIDATE IF THE ROW1 OF File matches with Any of the Row in DB
'This step should include Iteration of Rows in DB range
'Step 2
'Proceed with Next row (ROW1+1 OF File) if Match Occurs
'This step contains incremental of File row & Starting comparison from First Row of DB
'Step 3
'If no Match occurs , Add the Specific Row to ary3
'This step captures the complete Mismatch record of File Row
'*********DB MINUS File code goes here*********
'Similar to the Flow of File MINUS DB
'adding the Array3 & 4 resultant Sheets
sheet3.Range(addy) = ary3
sheet4.Range(addz) = ary4
End Sub

Resources