Deserialize complex JSON with Newtonsoft.Json - Visual Basic - arrays

Need help to deserialize complex json file using Visual Basic and Newtonsoft.Json library.
Here is the sample of Json file:
{
"value": [
{
"Id": 12345,
"Name": "Test",
"CardCount": 0,
"DailySpendLimit": 0.00,
"WeeklySpendLimit": 50.00,
"MonthlySpendLimit": 100.00,
"YearlySpendLimit": 1000.00,
"LifetimeSpendLimit": 0.00,
"MerchantCategories": [
"#{Id=11111; Name=Associations \u0026 Organizations; Description=Post Office, local and federal government services, religious organizations.; TransactionLimit=0.00; DailySpendLimit=0.00; WeeklySpendLimit=0.00; MonthlySpendLimit=0.00; YearlySpendLimit=0.00; LifetimeSpendLimit=0.00}",
"#{Id=22222; Name=Automotive Dealers; Description=Vehicle dealerships (car, RV, motorcycle, boat, recreational).; TransactionLimit=0.00; DailySpendLimit=0.00; WeeklySpendLimit=0.00; MonthlySpendLimit=0.00; YearlySpendLimit=0.00; LifetimeSpendLimit=0.00}"
]
}
],
"Count": 1
}
I had setup classes:
Public Class TEST_AdvRules
<JsonProperty("value")>
Public Property ValueDetails As ValueList()
Public Class ValueList
<JsonProperty("Id")>
Public Property RuleId As Integer
<JsonProperty("Name")>
Public Property RuleName As String = ""
Public Property CardCount As Integer
Public Property DailySpendLimit As Double
Public Property WeeklySpendLimit As Double
Public Property MonthlySpendLimit As Double
Public Property YearlySpendLimit As Double
Public Property LifetimeSpendLimit As Double
<JsonProperty("MerchantCategories")>
Public Property MerchantCategories() As List(Of MerchantCategoriesList)
End Class
Public Class MerchantCategoriesList
Public Property Id As Integer
Public Property Name As String = ""
Public Property Description As String = ""
Public Property TransactionLimit As Double
Public Property DailySpendLimit As Double
Public Property WeeklySpendLimit As Double
Public Property MonthlySpendLimit As Double
Public Property YearlySpendLimit As Double
Public Property LifetimeSpendLimit As Double
End Class
End Class
The code for deserilization:
Dim jsonStringP = IO.File.ReadAllText(PathJson & SecFileName)
Dim serSettings = New JsonSerializerSettings()
serSettings.ContractResolver = New CamelCasePropertyNamesContractResolver()
Dim RulesPEX = JsonConvert.DeserializeObject(Of TEST_AdvRules)(jsonStringP, serSettings)
But it errors out on the deserialization line - can't convert MerchantCategories.
I have never dealt with Json with [ "# array sequence. Am I processing it wrong?
Also - there is a character sequence '\u0026 ' within Merchant categories. Does it require special handling? Thank you!

The problem is that MerchantCategories is a string collection. You will have to convert each string to object using a custom string analyzer. Last time I worked with Vb.net was more then 10 years ago, so I can give you a solution in C# , your part would be to convert it to VB.Net. I don't feel like installing VB.NET compilliar at my mashine.
using Newtonsoft.Json;
var jsonObject = JObject.Parse(jsonStringP);
var jsonArr=new JArray();
foreach (var item in (JArray) jsonObject["value"][0]["MerchantCategories"])
{
jsonArr.Add(ConvertToJObject(item.ToString()));
}
jsonObject["value"][0]["MerchantCategories"]=jsonArr;
TEST_AdvRules RulesPEX = jsonObject.ToObject<TEST_AdvRules>();
this function converts a string to a JObject
public JObject ConvertToJObject(string str)
{
var strStart=str.IndexOf("{")+1;
var strEnd=str.IndexOf("}")-2;
var arr = str.Substring(strStart, strEnd).Split(";").Select(c => c.Split("="));
var jObj = new JObject();
foreach (var item in arr)
jObj.Add(new JProperty(item[0].Trim(), item[1].Trim()));
return jObj;
}

let Visual Studio create the class for you:
copy the whole json file content to the clipboard
in VS, add a new empty class -> Edit -> Paste Special -> paste json as class
I tend to add a shared Read-Method for easier construction.
Of course you can rename your classes as you wish :-)
I'm on german VS but you get the idea :-)

Translating Serge's answer into VB (note untested code so it's possible there are some minor errors)...
Imports Newtonsoft.Json
'...
Dim jsonObject = JObject.Parse(jsonStringP)
Dim jsonArr = new JArray()
For Each item In CType(jsonObject("value")(0)("MerchantCategories"), JArray)
jsonArr.Add(ConvertToJObject(item.ToString()))
Next
jsonObject("value")(0)("MerchantCategories") = jsonArr
Dim RulesPEX As TEST_AdvRules = jsonObject.ToObject(Of TEST_AdvRules)()
and
Public Function ConvertToJOBject(ByVal str As String) As JObject
Dim strStart = str.IndexOf("{") + 1
Dim strEnd = str.IndexOf("}") - 2
Dim arr = str.Substring(strStart, strEnd).Split(";").Select(Function(c) c.Split("="))
Dim jObj = New JObject()
For Each item In arr
jObj.Add(New JProperty(item(0).Trim(), item(1).Trim()))
Next
Return jObj
End Function

Related

How to initialise a jagged array of bytes as a class member

I'm trying to find an alternative way to solve the problem I'm stuck on here. I'm using MSTest to select one of a set of arrays of bytes to pass to a function under test.
I'm trying this approach as I haven't been able to get MSTest working directly passing an array of bytes to the test function.
I want to set up a Private ReadOnly jagged array of arrays of Bytes (TestMsgs) as part of my test class to allow the test subroutine to access elements one by one. Currently I'm getting error BC30201 "Expression Expected" as below. Something is missing in my initialisation, but I can't find any example on how to initialise this jagged array.
Public Class DecoderTests
Private ReadOnly TestMsgs As Byte()() = New Byte(2)() {
New Byte() {&HA1, &HB2, &HC3}, 'Test array should Pass
New Byte() {&HA2, &HB3}, 'Test array should Fail
} <========= Error BC30201 Here
Private DecoderInstance
Here is the full code of my test (Simplified to debug the original issue)
Test Class
Imports System.Text
Imports Microsoft.VisualStudio.TestTools.UnitTesting
Namespace TestDecoder.Tests
<TestClass>
Public Class DecoderTests
Private ReadOnly TestMsgs As Byte()() = New Byte(2)() {
New Byte() {&HA1, &HB2, &HC3}, 'Test array should Pass
New Byte() {&HA2, &HB3}, 'Test array should Fail
}
Private DecoderInstance
<DataTestMethod>
<DataRow(0)>
<DataRow(1)>
Public Sub ParseTestData(message_number)
Dim result As Boolean
DecoderInstance = New Decoder()
result = DecoderInstance.parse(TestMsgs(message_number)(0))
Assert.IsTrue(result, "Failed the dummy test")
End Sub
End Class
End Namespace
Simplified Class under test:
Imports Microsoft.VisualBasic
Public Class Decoder
Function parse(rxchar As Byte) As Boolean
Return rxchar = &H41
End Function
End Class
I just tried this code and there were no errors:
Private ReadOnly TestMsgs As Byte()() = {
New Byte() {&HA1, &HB2, &HC3},
New Byte() {&HA2, &HB3}
}
I think the issue with your original code was that you were specifying the size of the array explicitly and also initialising it, therefore specifying the size of the array implicitly. I just tested this code and it worked too:
Private ReadOnly TestMsgs As Byte()() = New Byte()() {
New Byte() {&HA1, &HB2, &HC3},
New Byte() {&HA2, &HB3}
}

how to deserialize this json on vb.net?

i have this data in json format:
{
"status":{
"timestamp":"2021-01-07T17:13:48.471Z",
"error_code":0,
"error_message":null,
"elapsed":12,
"credit_count":1,
"notice":null,
"total_count":4115
},
"data":[
{
"id":1,
"name":"Bitcoin",
"symbol":"BTC",
"slug":"bitcoin",
"num_market_pairs":9732,
"date_added":"2013-04-28T00:00:00.000Z",
"tags":[
"mineable",
"pow",
"sha-256",
"store-of-value",
"state-channels"
],
"max_supply":21000000,
"circulating_supply":18593700,
"total_supply":18593700,
"platform":null,
"cmc_rank":1,
"last_updated":"2021-01-07T17:12:02.000Z",
"quote":{
"USD":{
"price":39362.167971369854,
"volume_24h":78135138852.86674,
"percent_change_1h":2.61231359,
"percent_change_24h":12.47756102,
"percent_change_7d":36.98956944,
"market_cap":731888342609.2596,
"last_updated":"2021-01-07T17:12:02.000Z"
}
}
},
{
"id":1027,
"name":"Ethereum",
"symbol":"ETH",
"slug":"ethereum",
"num_market_pairs":5934,
"date_added":"2015-08-07T00:00:00.000Z",
"tags":[
"mineable",
"pow",
"smart-contracts"
],
"max_supply":null,
"circulating_supply":114155463.749,
"total_supply":114155463.749,
"platform":null,
"cmc_rank":2,
"last_updated":"2021-01-07T17:12:02.000Z",
"quote":{
"USD":{
"price":1261.606649005652,
"volume_24h":39345516218.36576,
"percent_change_1h":3.1558102,
"percent_change_24h":7.80752209,
"percent_change_7d":71.81090319,
"market_cap":144019292086.06207,
"last_updated":"2021-01-07T17:12:02.000Z"
}
}
}
]
}
and, in vb.net, with special paste i got this definition:
Public Class Rootobject
Public Property status As Status
Public Property data() As Datum
End Class
Public Class Status
Public Property timestamp As Date
Public Property error_code As Integer
Public Property error_message As Object
Public Property elapsed As Integer
Public Property credit_count As Integer
Public Property notice As Object
Public Property total_count As Integer
End Class
Public Class Datum
Public Property id As Integer
Public Property name As String
Public Property symbol As String
Public Property slug As String
Public Property num_market_pairs As Integer
Public Property date_added As Date
Public Property tags() As String
Public Property max_supply As Integer
Public Property circulating_supply As Integer
Public Property total_supply As Integer
Public Property platform As Object
Public Property cmc_rank As Integer
Public Property last_updated As Date
Public Property quote As Quote
End Class
Public Class Quote
Public Property USD As USD
End Class
Public Class USD
Public Property price As Single
Public Property volume_24h As Single
Public Property percent_change_1h As Single
Public Property percent_change_24h As Single
Public Property percent_change_7d As Single
Public Property market_cap As Single
Public Property last_updated As Date
End Class
but when i try with this command to deserialize
Dim m As IEnumerable(Of Rootobject) =
JsonConvert.DeserializeObject(Of IEnumerable(Of Rootobject))(res)
i get this error:
Newtonsoft.Json.JsonSerializationException: 'Cannot deserialize the current JSON object
(e.g. {"name":"value"}) into type
'System.Collections.Generic.IEnumerable`1[CoinMarketCap.Rootobject]'
because the type requires a JSON array (e.g. [1,2,3])
to deserialize correctly. To fix this error either change the JSON to a
JSON array (e.g. [1,2,3]) or change the deserialized type so that it is a
normal .NET type (e.g. not a primitive type like integer, not a
collection type like an array or List) that can be deserialized from a
JSON object. JsonObjectAttribute can also be added to the type to force
it to deserialize from a JSON object. Path 'status', line 1, position 10.'
im stucked... any idea?
Thanks in advance.
To fix this error either change the JSON to a JSON array (e.g. [1,2,3]) or change the deserialized type so that it is a normal .NET type (e.g. not a primitive type like integer, not a collection type like an array or List) that can be deserialized from a JSON object.
In other words, change
Dim m As IEnumerable(Of Rootobject) = JsonConvert.DeserializeObject(Of IEnumerable(Of Rootobject))(res)
to
Dim m As Rootobject = JsonConvert.DeserializeObject(Of Rootobject)(res)

What is the most efficient way to extract information from a text file formatted like this

Consider a text file stored in an online location that looks like this:
;aiu;
[MyEditor45]
Name = MyEditor 4.5
URL = http://www.myeditor.com/download/myeditor.msi
Size = 3023788
Description = This is the latest version of MyEditor
Feature = Support for other file types
Feature1 = Support for different encodings
BugFix = Fix bug with file open
BugFix1 = Fix crash when opening large files
BugFix2 = Fix bug with search in file feature
FilePath = %ProgramFiles%\MyEditor\MyEditor.exe
Version = 4.5
Which details information about a possible update to an application which a user could download. I want to load this into a stream reader, parse it and then build up a list of Features, BugFixes etc to display to the end user in a wpf list box.
I have the following piece of code that essentially gets my text file (first extracting its location from a local ini file and loads it into a streamReader. This at least works although I know that there is no error checking at present, I just want to establish the most efficient way to parse this first. One of these files is unlikely to ever exceed more than about 250 - 400 lines of text.
Dim UpdateUrl As String = GetUrl()
Dim client As New WebClient()
Using myStreamReader As New StreamReader(client.OpenRead($"{UpdateUrl}"))
While Not myStreamReader.EndOfStream
Dim line As String = myStreamReader.ReadLine
If line.Contains("=") Then
Dim p As String() = line.Split(New Char() {"="c})
If p(0).Contains("BugFix") Then
MessageBox.Show($" {p(1)}")
End If
End If
End While
End Using
Specifically I'm looking To collate the information about Features, BugFixes and Enhancements. Whilst I could construct what would in effect be a rather messy if statement I feel sure that there must be a more efficient way to do this , possibly involving linq. I'd welcome any suggestions.
I have added the wpf tag on the off chance that someone reading this with more experience of displaying information in wpf listboxes than I have might just spot a way to effectively define the info I'm after in such a way that it could then be easily displayed in a wpf list box in three sections (Features, Enhancements and BugFixes).
Dom, Here is an answer in C#. I will try to convert it to VB.Net momentarily. First, since the file is small, read all of it into a list of strings. Then select the strings that contain an "=" and parse them into data items that can be used. This code will return a set of data items that you can then display as you like. If you have LinqPad, you can test thecode below, or I have the code here: dotnetfiddle
Here is the VB.Net version: VB.Net dotnetfiddle
Imports System
Imports System.Collections.Generic
Imports System.Linq
Public Class Program
Public Sub Main()
Dim fileContent As List(Of String) = GetFileContent()
Dim dataItems = fileContent.Where(Function(c) c.Contains("=")).[Select](Function(c) GetDataItem(c))
dataItems.Dump()
End Sub
Public Function GetFileContent() As List(Of String)
Dim contentList As New List(Of String)()
contentList.Add("sb.app; aiu;")
contentList.Add("")
contentList.Add("[MyEditor45]")
contentList.Add("Name = MyEditor 4.5")
contentList.Add("URL = http://www.myeditor.com/download/myeditor.msi")
contentList.Add("Size = 3023788")
contentList.Add("Description = This is the latest version of MyEditor")
contentList.Add("Feature = Support for other file types")
contentList.Add("Feature1 = Support for different encodings")
contentList.Add("BugFix = Fix bug with file open")
contentList.Add("BugFix1 = Fix crash when opening large files")
contentList.Add("BugFix2 = Fix bug with search in file feature")
contentList.Add("FilePath = % ProgramFiles %\MyEditor\MyEditor.exe")
contentList.Add("Version = 4.5")
Return contentList
End Function
Public Function GetDataItem(value As String) As DataItem
Dim parts = value.Split("=", 2, StringSplitOptions.None)
Dim dataItem = New DataItem()
dataItem.DataType = parts(0).Trim()
dataItem.Data = parts(1).Trim()
Return dataItem
End Function
End Class
Public Class DataItem
Public DataType As String
Public Data As String
End Class
Or, in C#:
void Main()
{
List<string> fileContent = GetFileContent();
var dataItems = fileContent.Where(c => c.Contains("="))
.Select(c => GetDataItem(c));
dataItems.Dump();
}
public List<string> GetFileContent()
{
List<string> contentList = new List<string>();
contentList.Add("sb.app; aiu;");
contentList.Add("");
contentList.Add("[MyEditor45]");
contentList.Add("Name = MyEditor 4.5");
contentList.Add("URL = http://www.myeditor.com/download/myeditor.msi");
contentList.Add("Size = 3023788");
contentList.Add("Description = This is the latest version of MyEditor");
contentList.Add("Feature = Support for other file types");
contentList.Add("Feature1 = Support for different encodings");
contentList.Add("BugFix = Fix bug with file open");
contentList.Add("BugFix1 = Fix crash when opening large files");
contentList.Add("BugFix2 = Fix bug with search in file feature");
contentList.Add("FilePath = % ProgramFiles %\\MyEditor\\MyEditor.exe");
contentList.Add("Version = 4.5");
return contentList;
}
public DataItem GetDataItem(string value)
{
var parts = value.Split('=');
var dataItem = new DataItem()
{
DataType = parts[0],
Data = parts[1]
};
return dataItem;
}
public class DataItem
{
public string DataType;
public string Data;
}
The given answer only focuses on the first part, converting the data to a structure that can be shaped for display. But I think you main question is how to do the actual shaping.
I used a somewhat different way to collect the file data, using Microsoft.VisualBasic.FileIO.TextFieldParser because I think that makes coding just al little bit easier:
Iterator Function GetTwoItemLines(fileName As String, delimiter As String) _
As IEnumerable(Of Tuple(Of String, String))
Using tfp = New TextFieldParser(fileName)
tfp.TextFieldType = FieldType.Delimited
tfp.Delimiters = {delimiter}
tfp.HasFieldsEnclosedInQuotes = False
tfp.TrimWhiteSpace = False
While Not tfp.EndOfData
Dim arr = tfp.ReadFields()
If arr.Length >= 2 Then
Yield Tuple.Create(arr(0).Trim(), String.Join(delimiter, arr.Skip(1)).Trim())
End If
End While
End Using
End Function
Effectively the same thing happens as in your code, but taking into account Andrew's keen caution about data loss: a line is split by = characters, but the second field of a line consists of all parts after the first part with the delimiter re-inserted: String.Join(delimiter, arr.Skip(1)).Trim().
You can use this function as follows:
Dim fileContent = GetTwoItemLines(file, "=")
For display, I think the best approach (most efficient in terms of lines of code) is to group the lines by their first items, removing the numeric part at the end:
Dim grouping = fileContent.GroupBy(Function(c) c.Item1.TrimEnd("0123456789".ToCharArray())) _
.Where(Function(k) k.Key = "Feature" OrElse k.Key = "BugFix" OrElse k.Key = "Enhancement")
Here's a Linqpad dump (in which I took the liberty to change one item a bit to demonstrate the correct dealing with multiple = characters:
You could do it with Regular Expressions:
Imports System.Text.RegularExpressions
Private Function InfoReader(ByVal sourceText As String) As List(Of Dictionary(Of String, String()))
'1) make array of fragments for each product info
Dim products = Regex.Split(sourceText, "(?=\[\s*\w+\s*])")
'2) declare variables needed ahead
Dim productProperties As Dictionary(Of String, String)
Dim propertyNames As String()
Dim productGroupedProperties As Dictionary(Of String, String())
Dim result As New List(Of Dictionary(Of String, String()))
'2) iterate along fragments
For Each product In products
'3) work only in significant fragments ([Product]...)
If Regex.IsMatch(product, "\A\[\s*\w+\s*]") Then
'4) make array of property lines and extract dictionary of property/description
productProperties = Regex.Split(product, "(?=^\w+\s*=)", RegexOptions.Multiline).Where(
Function(s) s.Contains("="c)
).ToDictionary(
Function(s) Regex.Match(s, "^\w+(?=\s*=)").Value,
Function(s) Regex.Match(s, "(?<==\s+).*(?=\s+)").Value)
'5) extract distinct property names, ignoring numbered repetitions
propertyNames = productProperties.Keys.Select(Function(s) s.TrimEnd("0123456789".ToCharArray)).Distinct.ToArray
'6) make dictionary of distinctProperty/Array(Of String){description, description1, ...}
productGroupedProperties = propertyNames.ToDictionary(
Function(s) s,
Function(s) productProperties.Where(
Function(kvp) kvp.Key.StartsWith(s)
).Select(
Function(kvp) kvp.Value).ToArray)
'7) enlist dictionary to result
result.Add(productGroupedProperties)
End If
Next
Return result
End Function

et Return the Whole Subdocument Arrary from MongoDB

I need to get all of the subdocuments array from the Courses Class where the User.UserId = whatever and Courses.Status=active
Public Class User
Public Property UserId As String 'this is unique so i would like to index this, unless you think otherwise
Public Property CourseData() As List(Of Courses) '
Public Property Groups As List(Of String)
Public Property BU As List(Of String)
End Class
Public Class Courses
Public Property id As String 'this can be dynamic
Public Property description As String
Public Property CompletionDate As String
Public Property Hours As String
Public Property Status As String
End Class
Using vb.net , I tried a few ways, I only want the courses returned that have a Status="Active" to be dumped into Ienumberable
I tried (_users is a collection of User) (_uid is a variable passed into it)
Return _users.FindAs(Of User)(Query.And(query.EQ("LearningHours.Status", "Active"), (Query.EQ("UserId", _uid))))
Return _users.FindAs(Of User)(Query.And(query.EQ("LearningHours.Status", "Active"), (Query.EQ("UserId", _uid)))).SetFields("Courses", "1")
Return _users.FindAs(Of Courses)(Query.And(query.EQ("LearningHours.Status", "Active"), (Query.EQ("UserId", _uid))))
Return _users.FindAs(Of Courses)(Query.And(query.EQ("LearningHours.Status", "Active"), (Query.EQ("UserId", _uid)))).SetFields("Courses", "1")
None seem to work, they usually come back with the fields from Class User or both Class User and Class Course, but the Course fields are blank
I even am trying linq.. this works - but only returns 1 row result
Dim uc = From _u In _users.AsQueryable(Of User)()
Where _u.userid = _userid _
Select _
CourseID = _u.Courses.Where(Function(c) c.State = "Submitted").Select(Function(c) c.CourseId)(0), _
CourseDescription = _u.Courses.Where(Function(c) c.State = "Submitted").Select(Function(c) c.CourseDescription)(0)
Seems easy enough to do, just cant get it
Got It, I think I was over thinking it
Once I declare an instance of the class, I can iterate through the subdocument
Dim _u as new User
For Each c In _user.Courses.Where(Function(cs) cs.Status= "Active").Select(Function(cs) cs)
console.writeline(c.id & c.description & "so on...")
Next

parsing by Linq to JSON

I was trying to parse this using System.Json but I'm a bit confused how to use LINQ on this. The JSON string returned was something like:
{"SearchResults":[{"PageCount":"1"},
{"SEARCHVAL":"Result","CATEGORY":"Category1","X":"1","Y":"2"},
{"SEARCHVAL":"AnotherResult","CATEGORY":"Category1","X":"2","Y":"2"}]}
My code starts out as:
WebClient client = new WebClient();
client.OpenReadCompleted +=
new OpenReadCompletedEventHandler(client_OpenReadCompleted);
client.OpenReadAsync(uri);
then proceeds to the event handler where the result is stored in a Stream. Any example how to get just the SEARCHVALs, CATEGORYs, Xs and Ys from the stream using LINQ?
The DataContractJSonSerializer is one way to do it but the presence of the "PageCount" object makes its use awkard. It looks to me as though someone has goofed on the server end, it would make much more sense to have the page count a property of the outer object and therefore leave the search results array to contain an homogeneous set of objects.
The System.Json namespace is handy for this sort of JSON since its very flexible.
JsonObject searchResults = (JsonObject)JsonValue.Load(e.Result);
Now you could use some Linq to get a set of objects out of this but first so we need to create a DTO type for it:-
public class SearchResult
{
public string SearchValue {get; set;}
public string Category {get; set;}
public string X {get; set;}
public string Y {get; set;}
}
So now the results query is:-
var resultList = ((JsonArray)searchResults["SearchResults"])
.OfType<JsonObject>()
.Where(o => o.ContainsKey("SEARCHVAL"))
.Select(o => new SearchResult() {
SearchValue = o["SEARCHVALUE"],
Category = o["CATEGORY"].
X = o["X"],
Y = o["Y"]
}).ToList();
The resultList is List<SearchResult> which you can now bind into some Xaml for display.

Resources