Loop until website has no additional JSON arrays to parse - arrays

I am looking for a loop function/syntax that will allow my loop to cease once the website I am pulling JSON arrays from has no additional arrays left to parse (variable / unknowable number of arrays).
Thank you for the insight.
sheetCount = 1
i = 1
urlArray = Array("URL array list")
Dim MyRequest As Object
Set MyRequest = CreateObject("WinHttp.WinHttpRequest.5.1")
Dim MyUrls
MyUrls = urlArray
Dim k As Long
Dim Json As Object
For k = LBound(MyUrls) To UBound(MyUrls)
With MyRequest
.Open "GET", MyUrls(k)
.Send
Set Json = JsonConverter.ParseJson(.ResponseText)
Do Until ''[NEED HELP HERE]
Sheets("Sheet" & sheetCount).Cells(i, 1) = Json("cars")(i)("carType")
Sheets("Sheet" & sheetCount).Cells(i, 2) = Json("cars")(i)("fare")("carprice")
i = i + 1
Loop
End With
sheetCount = sheetCount + 1
Next

You are missing the UBound function.
Other notes
No code without Option Explicit, period. No exceptions.
Make small functions that do one thing only.
Add references to the libraries you use instead of using CreateObject. It will make your life a lot easier because this way you get compile-time type checking and Intellisense.
It's safer to use the Exists() method to check if a dictionary key exists before you try to access it. Trying to access a non-existing key will throw a run-time error.
I'm silently assuming that you are using https://github.com/VBA-tools/VBA-JSON.
This should be close enough:
Option Explicit
Function GetJson(ByVal url As String) As Dictionary
With New WinHttpRequest ' see http://stackoverflow.com/a/3119794/18771
.Open "GET", url
.Send
Set GetJson = JsonConverter.ParseJson(.ResponseText)
End With
End Function
Sub FillCarInfo(data As Dictionary, sheet As Worksheet)
Dim i As Integer, car As Dictionary
For i = 0 To UBound(data("cars")) - 1
Set car = data("cars")(i)
' you probably should use If car.Exists("carType") Then
sheet.Cells(i, 1) = car("carType")
sheet.Cells(i, 1) = car("fare")("carprice")
Next i
End Sub
Sub FillMultipleCarInfo(urls As Variant, book As Workbook)
Dim i As Integer, data As Dictionary, sheet As Worksheet
For i = 0 To UBound(urls) - 1
Set data = GetJson(urls(i))
Set sheet = book.Sheets(i + 1)
FillCarInfo data, sheet
Next i
End Sub
Usage
Dim myUrls As Variant
myUrls = Array("URL array list")
FillMultipleCarInfo myUrls, ActiveWorkbook

Related

merge several bits of data from different sheets

I am trying to merge several bits of data from different sheets. After a few questions and attempts (at arrays, thanks to Stackoverflow before for help with this), i think a dictionary may be best. The final outcome is a populated table that holds all data for each individual entry. (depending on the entry the data in a column in raw data could be in different locations)
The data can include multiple entries for one person. But the data for each entry is different depending on the stage of entry. For example, if the data in column 3 would be in column 5 of the final table if a condition was stage 1, however if condition was stage 2, the same data that was in column 3 could be column 10 of the final table.
https://www.youtube.com/watch?v=o8fSY_4p93s
Following this video tutorial ondictionaires, i think i could o through the dtaa and find each individual entry and then add the corresponding variables for the case. E.g. find data for Steve Smith, if steve smith exists then if stage 1, add data to variable stagedate1, if stage2 add data to stage2date and so on. If not found add entry and find the stage.
Similar to the video, where he finds the corresponding data for each customer, and adds sales and volumes, i could do the same if an if function is round before to identify which datastage and to then put the value in correct variable.
I know there will be a million way to do it, but this seems simple and effective.
Sub Dictionary()
Dim dict As Dictionary
Set dict = ReadData()
Call WriteDict(dict)
End Sub
Function ReadData()
Dim dict As New Dictionary
Dim DataWs As Worksheet: Set DataWs = ThisWorkbook.Sheets("DATA")
Dim PoolOfWeekWs As Worksheet: Set PoolOfWeekWs = ThisWorkbook.Sheets("Pool of the week")
Dim LastrowData As Long: LastrowData = DataWs.range("A" & Rows.Count).End(xlUp).Row
Dim LastColData As Long: LastColData = DataWs.Cells(1 & DataWs.Columns.Count).End(xlToLeft).Column
Dim LastColDataString As String: LastColDataString = Split(Cells(1, LastColData).Address, "$")(1)
Dim DataRange As range: Set DataRange = DataWs.range("A1:" & LastColDataString & LastrowData)
Dim DataArr As Variant: DataArr = DataWs.range("A1:AO" & LastrowData)
Dim range As range: Set range = DataWs.range("A1").CurrentRegion
Dim i As Long
Dim CandidateProcessID As String, CandidateName As String, FirstName As String, ProcessStatus As String, PQLDate As Date, oCandidate As ClsCandidate
For i = 2 To range.Rows.Count
CandidateProcessID = range.Cells(i, 10).Value
CandidateName = range.Cells(i, 16).Value
FirstName = range.Cells(i, 17).Value
ProcessStatus = range.Cells(i, 9).Value
If dict.Exists(CandidateProcessID) = True Then
Set oCandidate = dict(CandidateProcessID) 'CODE ERRORS HERE AFTER A FEW ROWS (Comes across a
Else an entry that is already in the dictionary)
Set oCandidate = New ClsCandidate
dict.Add CandidateProcessID, oCustomer
End If
oCandidate.CandidateName = oCandidate.CandidateName
oCandidate.FirstName = oCandidate.FirstName
oCandidate.ProcessStatus = oCandidate.ProcessStatus
oCandidate.PQLDate = oCandidate.PQLDate
Next i
Set ReadData = dict
End Function
Sub WriteDict(dict As Dictionary)
Dim key As Variant, oCandidate As ClsCandidate
For Each key In dict
Set oCandidate = dict(key)
Debug.Print key, oCandidate.CandidateName, oCandidate.FirstName, oCandidate.ProcessStatus, oCandidate.PQLDate
Next key
End Sub
I believe the error is object error. It stops and debugs.
That would be "Object Required", and it's absolutely consistent with a very common trap: add Option Explicit at the top of every module, and now VBA won't let you run the code until it knows what oCustomer is.
Option Explicit being missing has allowed the code to run with oCustomer undeclared: simply, a new Variant/Empty pointer is created on-the-spot for that local identifier ("oCustomer"), such that every iteration that "works" is just preparing the ground to make the ones that don't, blow up:
If dict.Exists(CandidateProcessID) = True Then
Set oCandidate = dict(CandidateProcessID) '<~ Variant/Empty is retrieved here: "object required"
Else
Set oCandidate = New ClsCandidate
dict.Add CandidateProcessID, oCustomer '<~ Variant/Empty is added here
End If
The Variant/Empty is successfully retrieved from the dictionary: the problem is that the Set oCandidate instruction on the left side of that = operator says the right-hand side of the operation must be an object reference. Variant/Empty fails to satisfy this requirement, and an "object required" run-time error is raised.
The bug isn't with the retrieval: it's with the storage!
You can easily find bugs like this with static code analysis tooling, such as Rubberduck (disclosure: that's my website).

Sometimes can't assign to array and sometimes can

I have the next code:
Function findRanges(keyword) As Variant()
Dim foundRanges(), rngSearch As Range
Dim i, foundCount As Integer
i = 0
foundCount = 0
ReDim foundRanges(0)
Set rngSearch = ActiveDocument.Range
Do While rngSearch.Find.Execute(FindText:=keyword, MatchWholeWord:=True, Forward:=True) = True
Set foundRanges(i) = rngSearch.Duplicate
i = i + 1
ReDim Preserve foundRanges(UBound(foundRanges) + 1)
rngSearch.Collapse Direction:=wdCollapseEnd
Loop
ReDim Preserve foundRanges(UBound(foundRanges) - 1)
findRanges = foundRanges
End Function
And:
Sub test()
Dim rngIAM_Code() As Range
...
Dim rngIAM_Title() As Range
rngIAM_Code = findRanges("IAM_Code")
...
rngIAM_Title = findRanges("IAM_Title")
End Sub
What is very confuding is that sometimes the compiler says "Can't assign to array" and sometimes it works fine. For example, when I only try to search one value and populate one array, the code works. When I try to populate both array, there is an error "Can't assign to an array". I can then switch lines of code like this:
rngIAM_Title = findRanges("IAM_Title")
...
rngIAM_Code = findRanges("IAM_Code")
And then the error happens with another array. The error can happen anywhere: on the first line, in the middle, or in the end, but it is consistent as long as I don't move lines. And again, if I leave only one-two lines of code with arrays in sub "test"everything works fine.
The following works for me.
In this code, every object variable is explicitly assigned a type. In VBA, every variable must be typed, else it's assigned the type Variant by default. In the following declaration line, for example, foundRanges() is of type Variant because it's not followed by As with a data type. The same with i in the next line of code in the question.
Dim foundRanges(), rngSearch As Range
And since the arrays in the calling procedure are of type Range the function should return the same type.
I also took the liberty of passing the Document object to the function as, conceivably, some day the document in question might not be ActiveDocument but a Document object assigned using Documents.Open or Documents.Add. If this is not desired it can be changed back, but not relying on ActiveDocument is more reliable...
Additionally, I added the Wrap parameter to Find.Execute - it's always a good idea to specify that when executing Find in a loop to prevent the search from starting again at the beginning of the document (wdFindContinue).
Sub testRangesInArrays()
Dim rngIAM_Code() As Range
Dim rngIAM_Title() As Range
rngIAM_Code = findRanges("You", ActiveDocument)
rngIAM_Title = findRanges("change", ActiveDocument)
End Sub
Function findRanges(keyword As String, doc As Word.Document) As Range()
Dim foundRanges() As Range, rngSearch As Range
Dim i As Integer, foundCount As Integer
i = 0
foundCount = 0
ReDim foundRanges(0)
Set rngSearch = doc.content
Do While rngSearch.Find.Execute(findText:=keyword, MatchWholeWord:=True, _
Forward:=True, wrap:=wdFindStop) = True
Set foundRanges(i) = rngSearch.Duplicate
ReDim Preserve foundRanges(UBound(foundRanges) + 1)
i = i + 1
rngSearch.Collapse Direction:=wdCollapseEnd
Loop
findRanges = foundRanges
End Function
Here is an alternative based on a Collection instead of an Array:
I used also included Cindys Input regarding passing the document and adding wrap.
I don't exactly know what the you use the return value for, but in general a collection is a bit more flexible than an Array.
I also removed the underscores since they indicate a function of an implemented Interface and may cause problems later down the line. are used when implementing an Interface (improves readability).
As explained here you can use wrap or collapse to prevent a continuous Loop.
Option Explicit
Sub test()
Dim rngIAMCode As Collection
Dim rngIAMTitle As Collection
Set rngIAMCode = findRanges("IAM_Code", ActiveDocument)
Set rngIAMTitle = findRanges("IAM_Title", ActiveDocument)
Debug.Print "Code found : " & rngIAMCode.Count & " Times."
Debug.Print "Title found : " & rngIAMTitle.Count & " Times."
End Sub
Function findRanges(ByVal keyword As String, doc As Document) As Collection
Set findRanges = New Collection
Dim rngSearch As Range
Set rngSearch = doc.Content
With rngSearch.Find
.Text = keyword
.MatchWholeWord = True
.Forward = True
.Wrap = wdFindStop
While .Execute
findRanges.Add rngSearch.Duplicate
rngSearch.Collapse Direction:=wdCollapseEnd
Wend
End With
End Function

Assigning Ranges to Array

I have been struggling with this for quite some time, but the error dialogue box that pops up isn't exactly the most helpful. I'm trying to extract a list of names from the worksheet and assigning them to an array using the range function. I tried and tried, but I couldn't seem to get it to work, so I tried reading in the cells 1 by 1 instead, using the Do Until Loop. I didn't expect to be posting this here, so the code of what I was doing before, is already gone, but here's an example:
Dim RangeList As Variant
RangeList = ThisWorkbook.Worksheets("Plan").Range("H1:H132").Value2
I switched it to the next method in hopes that it would lead to a more straightforward approach:
ReDim ResourceList(ResourceLength - 1)
I = 1
Do Until ThisWorkbook.Worksheets("Plan").Cells(I, 8).Value = ""
ResourceList(I) = ThisWorkbook.Worksheets("Plan").Cells(I, 8).Value
Workbooks("NEW PROJECT PLAN").Worksheets("Console").Cells(I, 2).Value = Resource
I = I + 1
Loop
The first one returns an empty range that 'Can't find any cells' and the second one gave me an array of empty strings 169 items long. I feel like I'm pounding my head against a brick wall on this one, any help would be appreciated.
Here is the entirety of the code that I'm trying to troubleshoot:
'Collects the List of Resources
Dim ResourceLength As Long, I As Integer
Dim ResourceList() As String
ResourceLength = ThisWorkbook.FinalRow(8, "Plan")
MsgBox ("Final Row is: " & ResourceLength) 'The Last row used in column 8
ReDim ResourceList(ResourceLength - 1)
I = 1
Do Until ThisWorkbook.Worksheets("Plan").Cells(I, 8).Value = ""
ResourceList(I - 1) = ThisWorkbook.Worksheets("Plan").Cells(I, 8).Value
Workbooks("NEW PROJECT PLAN").Worksheets("Console").Cells(I, 2).Value = Resource
I = I + 1
Loop
ResourceList = ThisWorkbook.FilterArray(ResourceList)
Dim myCount As Integer
Dim Source As Variant
For Each Source In ResourceList
Worksheets("Console").Cells(myCount, 1).Value = Source
myCount = myCount + 1
Next Source
Here is the FilterArray Function:
Public Function FilterArray(UnsortedArray As Variant) As Variant
Dim Intermediate() As Variant
Dim UItem As Variant
' Runs through each item and compares it to the list of items found, if it finds repeats, it throws them out.
For Each UItem In UnsortedArray
If Not ArrayItemExist(Intermediate, UItem) Then
' The Item does not Exist
ReDim Intermediate(UBound(Intermediate) + 1)
Intermediate(UBound(Intermediate)) = UItem
End If
Next UItem
' Returns the Sorted Array.
FilterArray = Intermediate
End Function
Private Function ArrayItemExist(TargetArray() As Variant, TargetItem As Variant) As Boolean
'Searches an Array for TargetItem and returns a boolean stating whether it exists within the Array or not.
Dim ItemFound As Boolean
Dim SItem As Variant
ItemFound = False
For Each SItem In TargetArray
If TargetItem = SItem Then
ItemFound = True
Exit For
End If
Next SItem
ArrayItemExist = ItemFound
End Function
Public Function FinalRow(Column As Integer, Sheet As String) As Long
' Finds the last Row used in the spreadsheet.
FinalRow = Worksheets(Sheet).Cells(Rows.Count, Column).End(xlUp).Row
End Function
When I try to run the software, I receive an error that the For Loop is not initialized, which I traced back to the 'ResourceList' Array/Range being empty.
[Edit]
This function is used to prep an array of names that are extracted from a list of dropdown box resources. This list may contain multiple instances of the same name, so it's sent to the FilterArray function to sort the array into an array with just one instance of each name. Example:
Before and after sorting
After this, it's sent to a module that will inject each name into a dictionary with a corresponding amount of hours that the person is scheduled to work.

Access Values In Nested Array in JSON response

I have a multidimensional JSON array returned via a web request (http://www.coincap.io/history/365day/BTC). I want to cycle through the 2nd entry and retrieve its nested values.
If this was a normal array, I'd use:
For Each item In response
logic, logic, logic
currentRow = currentRow + 1
Next
This web request returns a JSON-response with 3 entries: market_cap, price, and volume. I just want to cycle through response(1) and get the price values. Each entry in price contains two keys, 0 and 1.
I would imagine that I could accomplish this by doing
For Each item in response(1)
Cells(currentRow, 1).Value = item(0)
Cells(currentRow, 2).Value = item(1)
currentRow = currentRow + 1
Next
I've also considered For Each item in response("price"). Neither works.
Sub Tester()
Dim json As String
Dim sc As Object
Dim o, n, i, p
Set sc = CreateObject("scriptcontrol")
sc.Language = "JScript"
json = HttpGet("http://www.coincap.io/history/365day/BTC")
sc.Eval "var obj=(" & json & ")" 'evaluate the json response
'add a couple of accessor functions
sc.AddCode "function numPrices(){return obj.price.length;}"
sc.AddCode "function getPrice(i){return obj.price[i];}"
n = sc.Run("numPrices")
For i = 0 To n - 1
p = Split(sc.Run("getPrice", i), ",")
Debug.Print i, p(0), p(1)
Next i
End Sub
Function HttpGet(url As String) As String
Dim oHTML As Object
Set oHTML = CreateObject("Microsoft.XMLHTTP")
With oHTML
.Open "GET", url, False
.send
HttpGet = .responsetext
End With
End Function

Building and comparing arrays

I have the below code I'm trying to get to work. This is my first time dealing with arrays in VBA. Here's the plain english version of what I'm going for:
Load SSBarray with column A from worksheet SSB.
Load EDMarray with Column I from worksheet EDM.
Compare the above arrays and sort into two new arrays IDarray and noIDarray based on a possible match.
Output the new arrays into their respective worksheets.
Step 4 is temporary to just make sure the code is even working. The entire project is compiling all the data from 3 sheets into these two lists. Worksheet 1 has Data point A only, Worksheet 2 may or may not have Data point A, B, and/or C, and Worksheet 3 may or may not have Data point A, B, and/or C. The code I have is my start to check for all of the data point A's in worksheet 1 are in worksheet 2. Run time is also a factor. I'll take any and all the help I can get at this point. Thanks.
'Build Arrays
Dim i As Long, j As Long
Dim SSBarray
Dim EDMarray
Dim IDarray
Dim noIDarray
Dim YCounter As Long
Dim NCounter As Long
Dim inArray As Boolean
endSSB = SSB.Range("A" & Rows.Count).End(xlUp).Row
endEDM = EDM.Range("A" & Rows.Count).End(xlUp).Row
BBlast = BB.Range("A" & BB.Range("A" & Rows.Count).End(xlUp).Row)
ReDim SSBarray(1 To endSSB)
ReDim EDMarray(1 To endEDM)
For i = 2 To endSSB
SSBarray(i) = SSB.Cells(i, 1).Value2
Next i
For i = 2 To endEDM
EDMarray = EDM.Cells(i, 9).Value2
Next i
For i = 2 To endSSB
inArray = False
For j = 2 To endEDM
If SSBarray(i) = EDMarray(j) Then
inArray = True
YCounter = YCounter + 1
ReDim Preserve IDarray(1 To YCounter)
IDarray(YCounter) = SSBarray(i)
Exit For
End If
Next j
If inArray = False Then
NCounter = NCounter + 1
ReDim Preserve noIDarray(1 To NCounter)
noIDarray(NCounter) = SSBarray(i)
End If
Next i
For i = 1 To UBound(IDarray)
Identifiers.Cells(i, 4) = IDarray(i)
Next i
For i = 1 To UBound(noIDarray)
NoIdentifiers.Cells(i, 4) = noIDarray(i)
Next i
End Sub
Revised Code:
'Sort and Compile Data
Dim i As Long
endSSB = SSB.Range("A" & Rows.Count).End(xlUp).Row
endEDM = EDM.Range("A" & Rows.Count).End(xlUp).Row
BBlast = BB.Range("A" & BB.Range("A" & Rows.Count).End(xlUp).Row)
Public Type otherIDs
SEDOL As Variant
ISIN As Variant
End Type
Dim SSBIds As New Scripting.Dictionary
Dim IDs As otherIDs
For i = 2 To endSSB
'Add an ID\row number pair
SSBIds.Add SSB.Cells(i, 1).Value2
Next i
Dim EDMIds As New Scripting.Dictionary
For i = 2 To endEDM
IDs.SEDOL = EDM.Cells(i, 8).Value2
IDs.ISIN = EDM.Cells(i, 7).Value2
EDMIds.Add EDM.Cells(i, 9).Value2, IDs.SEDOL, IDs.ISIN
Next i
Dim IdMatches As New Scripting.Dictionary
Dim IdMisMatches As New Scripting.Dictionary
Dim key As Variant
For Each key In SSBIds
'If it's in the other dictionary...
If EDMIds.Exists(key) Then
'...add the row to the matches...
IdMatches.Add key, EDMIds(key)
Else
'...otherwise add the row to the mismatches.
IdMisMatches.Add key, EDMIds(key)
End If
Next
i = 1
For Each key In IdMatches.Keys
Identifiers.Cells(i, 4) = key
Identifiers.Cells(i, 5) = IdMatches.IDs.SEDOL
Identifier.Cells(i, 6) = IdMatches.IDs.ISIN
i = i + 1
Next
i = 1
For Each key In IdMisMatches.Keys
NoIdentifiers.Cells(i, 4) = key
i = i + 1
Next
Arrays aren't the best containers to be using here. Dictionaries have an .Exists method that uses a much faster hash lookup than a simple iteration that compares every value.
Not only that, repeatedly calling Redim Preserve is incredibly inefficient compared to adding items to a Dictionary. Every time you increase the array dimension, the entire data set gets copied to a newly allocated area of memory and the data pointer for the array gets updated to point to it.
Example using Dictionaries (you'll need to add a reference to Microsoft Scripting Runtime):
Dim SSBIds As New Scripting.Dictionary
For i = 2 To endSSB
'Add an ID\row number pair
SSBIds.Add SSB.Cells(i, 1).Value2, i
Next i
Dim EDMIds As New Scripting.Dictionary
For i = 2 To endEDM
EDMIds.Add EDM.Cells(i, 9).Value2, i
Next i
Dim IdMatches As New Scripting.Dictionary
Dim IdMisMatches As New Scripting.Dictionary
Dim key As Variant
For Each key In SSBIds
'If it's in the other dictionary...
If EDMIds.Exists(key) Then
'...add the row to the matches...
IdMatches.Add key, EDMIds(key)
Else
'...otherwise add the row to the mismatches.
IdMisMatches.Add key, EDMIds(key)
End If
Next
i = 1
For Each key In IdMatches.Keys
Identifiers.Cells(i, 4) = key
i = i + 1
Next
i = 1
For Each key In IdMisMatches.Keys
NoIdentifiers.Cells(i, 4) = key
i = i + 1
Next
Note that this assumes that your key columns have unique values. If they don't, you can either test for the presence of the key before adding a value (this matches your code's behavior of only taking the first match), or you can create a Collection of values to store in the Dictionary for each key, or something else entirely depending on your requirement.

Resources