Parsing JSON to do math? - arrays

I'm new to programming (especially JSON format), so please forgive me not for using proper terminology :)
Using Python 3.7 Requests module, I receive a JSON response. To keep things simple, I made an example:
{
"Bob":
{
"Age": "15",
"LastExamGrade": "45",
},
"Jack":
{
"Age": "16",
"LastExamGrade": "58",
}
}
What I would like to do is parse the JSON responses to extract two items from each response/structure and save it to a list like this (I think this is called a tuple of tuples?):
[("Bob","45"),("Jack","58")]
Then, after receiving doing this, I will receive another similar response, such as the following (where the only thing that changed is the exam grade):
{
"Bob":
{
"Age": "15",
"LastExamGrade": "54",
},
"Jack":
{
"Age": "16",
"LastExamGrade": "70",
}
}
I want to also save the name and grade into a tuple of tuples (list).
Lastly, I would like to subtract the first exam score of each person from their last exam score, and save this into a final list, which includes the name, final exam grade, and grade improvement, like this:
[("Bob","54","9"),("Jack","67","12")]
What is the simplest way to do this using Python 3? As for my own research, I've searched all throughout StackOverflow, but couldn't find out how to parse a JSON like mine (For example, in mine, the name is outside of the curly braces), and had difficulty doing math operations for JSON items.

I'd recommend using a dedicated package for calculations like pandas:
first_exam_grades = pd.DataFrame.from_dict(first_exam_results, orient='index').astype(int)
second_exam_grades = pd.DataFrame.from_dict(second_exam_results, orient='index').astype(int)
improvements = second_exam_grades.LastExamGrade.to_frame()
improvements['Improvement'] = second_exam_grades.LastExamGrade - first_exam_grades.LastExamGrade
This will give you something that looks like this:
Now you can output it anyway you'd like
list(zip(*([improvements.index.tolist()] + [improvements[c].values.tolist() for c in improvements])))
This will give you [('Bob', 54, 9), ('Jack', 70, 12)] as you want.

One possible solution, using coroutines. Coroutine receive_message holds up to last two values LastExamGrade from the message for each student and produces list of student name, last grade and improvement over last grade:
json_messages = [
# 1st message:
{
"Bob":
{
"Age": "15",
"LastExamGrade": "45",
},
"Jack":
{
"Age": "16",
"LastExamGrade": "58",
}
},
# 2nd message
{
"Bob":
{
"Age": "15",
"LastExamGrade": "54",
},
"Jack":
{
"Age": "16",
"LastExamGrade": "70",
}
},
# 3nd message (optional)
{
"Bob":
{
"Age": "15",
"LastExamGrade": "14",
},
"Jack":
{
"Age": "16",
"LastExamGrade": "20",
}
}
]
def receive_message():
d, message = {}, (yield)
while True:
for k, v in message.items():
d.setdefault(k, []).append(v['LastExamGrade'])
d[k] = d[k][-2:] # store max last two messages
message = yield [(k, *tuple(v if len(v)==1 else [v[1], str(int(v[1])-int(v[0]))])) for k, v in d.items()]
receiver = receive_message()
next(receiver) # prime coroutine
for msg in json_messages:
print(receiver.send(msg))
Prints:
[('Bob', '45'), ('Jack', '58')]
[('Bob', '54', '9'), ('Jack', '70', '12')]
[('Bob', '14', '-40'), ('Jack', '20', '-50')]

Related

How to get a specific map out of a list of maps that has a bigger than or smaller than value than the rest?

I'm working on a app where I retrieve json data via http, and I have made the listview.builder and all of that. Now I want to make a filter button to show the lists that only have values below a certain integer.
for example:
my list of maps goes something like this
[
{
"name": "jess",
"age": "28",
"job": "doctor"
},
{
"name": "jack",
"age": "30",
"job": "jobless"
},
{
"name": "john",
"age": "24",
"job": "doctor"
},
{
"name": "sara",
"age": "23",
"job": "teacher"
}...etc
]
Now I want to press that filter button and in my listview show only those that are below or above the age of 25.
You can map over it and add your "filter" inside the condition. Since you save age as String you have to parse it to an int:
var filteredList = myMapList.map((e){
int? parsedAge = int.tryParse(e["age"]!);
if(parsedAge != null && parsedAge >= 25){
return e;
}
}).toList();
I got the answer and it was there all the time I just didn't thought about that much.
In case anyone faces the same issue, here's how it worked for me.
list filteredList = peopleDetailsList.where((element) => int.parse(element['age'] < 25).toList();

Can we change the sequence of json items?

Can we change the sequence of json items?
For example:
[
{
"category": "Science: Mathematics",
"type": "multiple",
"difficulty": "medium",
"question": "In a complete graph G, which has 12 vertices, how many edges are there?",
"correct_answer": "66",
"incorrect_answers1": "67",
"incorrect_answers2 : "34",
"incorrect_answers3 : "11"
},
{
"category": "Science: Mathematics",
"type": "multiple",
"difficulty": "medium",
"question": "In base 2, what is 1 + 1?",
"incorrect_answers1": "2",
"incorrect_answers2 : "01",
"correct_answer": "10",
"incorrect_answers3 : "11"
},
{
"category": "Science: Mathematics",
"type": "multiple",
"difficulty": "medium",
"question": "In the hexadecimal system, what number comes after 9?",
"incorrect_answers1": "10",
"incorrect_answers2 : "The Number 0",
"correct_answer": "The Letter A",
"incorrect_answers3 : "16"
}
]
As mentioned in the comments that re-ordering JSON data is impossible and also not required.
For the current situation, you have to loop across the parent array containing the JSON objects, create a local array variable to get the list of re-ordered answers, push all the answers to this array and shuffle it using a library like lodash (https://lodash.com/docs/#shuffle) or a custom random functions, whatever works for your needs.
You can place this array within each object of this JSON while looping and give it a meaningful key like shuffled_answers.
So now your item structure will look like this:
{
"category": "Science: Mathematics",
"type": "multiple",
"difficulty": "medium",
"question": "In base 2, what is 1 + 1?",
"incorrect_answers1": "2",
"incorrect_answers2" : "01",
"correct_answer": "10",
"incorrect_answers3" : "11",
"shuffled_answers": ["2", "11", "10", "11"]
}
You can then use this array to display answer choices in your UI to ask user the question and use the correct_answer key to compare the correct answer from the user's choice.
I'm linking here a video tutorial for a very similar App developed in Vue using the above logic. You can refer for more insight about apps like these and use it for your requirements.
Hope that helps.

Import data - with firebase keys

I am trying to import some data into firebase
{
"people":
[
{
"name": "John Smith",
"age": 23,
},
{
"name": "Tony Jones",
"age": 61,
},
]
}
This is fine but it adds a "traditional" array index in firebase (0,1) - which I believe is bad?
When I insert a new value via my web form I get a mix
"0" : {
"name": "John Smith",
"age": 23,
},
"1" : {
"name": "Tony Jones",
"age": 61,
},
"-LgWkhX2DdD_ChbWJkXo" : { // inserted via form it has a firebase index
"name": "Simon Green",
"age": 37,
}
How can I get the initial inserted data to use firebase indexes it is just a normal .json file.
{
"people":
[
{
"name": "John Smith",
"age": 23,
},
{
"name": "Tony Jones",
"age": 61,
},
]
}
When you write array type JSON data into Realtime Database, you are going to get array type numeric indexes in the database. If you don't want to write like this, you will have to convert the array yourself - there is no API that's going to do that for you. You'll have to read the JSON, iterate each element of the array, and write each item into the database the way you want it to be written. It looks like perhaps you want to add each item using an automatic push ID, since you are trying to create something that looks like "-LgWkhX2DdD_ChbWJkXo".

Obtaining keys and values from JSON nested array in nest

First time posting! I am converting JSON data (dictionary) from a server into a csv file. The keys and values taken are fine apart from the nest "Astronauts", which is an array. Basically every individual JSON string is a datum that may contains from 0 to an unlimited number of astronauts which features I would like to extract as independent values. For instance something like this:
Astronaut1_Spaceships_First: Katabom
Astronaut1_Spaceships_Second: The Kraken
Astronaut1_name: Jebeddia
(...)
Astronaut2_gender: Hopefully female
and so on. The problem here is that the nest is set as an array and not a dictionary so I do not know what to do. I have tried the dpath library as well as flattering the nest but nothing did change. Any ideas?
import json
import os
import csv
import datetime
import dpath.util #Dpath library needs to be installed first
datum = {"Mission": "Make Earth Greater Again", "Objective": "Prove Earth is flat", "Astronauts": [{"Spaceships": {"First": "Katabom", "Second": "The Kraken"}, "Name": "Jebeddiah", "Gender": "Hopefully male", "Age": 35, "Prefered colleages": [], "Following missions": [{"Payment_status": "TO BE CONFIRMED"}]}, {"Spaceships": {"First": "The Kraken", "Second": "Minnus I"}, "Name": "Bob", "Gender": "Hopefully female", "Age": 23, "Prefered colleages": [], "Following missions": [{"Payment_status": "TO BE CONFIRMED"}]}]}
#Parsing process
parsed = json.loads(datum) #datum is the JSON string retrieved from the server
def flattenjson(parsed, delim):
val = {}
for i in parsed.keys():
if isinstance(parsed[i], dict):
get = flattenjson(parsed[i], delim)
for j in get.keys():
val[i + delim + j] = get[j]
else:
val[i] = parsed[i]
return val
flattened = flattenjson(parsed,"__")
#process of creating csv file
keys=['Astronaut1_Spaceship_First','Astronaut2_Spaceship_Second', 'Astronaut1_Name] #reduced to 3 keys for this example
writer = csv.DictWriter(OD, keys ,restval='Null', delimiter=",", quotechar="\"", quoting=csv.QUOTE_ALL, dialect= "excel")
writer.writerow(flattened)
.
#JSON DATA FROM SERVER
{
"Mission": "Make Earth Greater Again",
"Objective": "Prove Earth is flat",
"Astronauts": [ {
"Spaceships": {
"First": "Katabom",
"Second": "The Kraken"
},
"Name": "Jebeddiah",
"Gender": "Hopefully male",
"Age": 35,
"Prefered colleages": [],
"Following missions": [
{
"Payment_status": "TO BE CONFIRMED"
}
]
},
{
"Spaceships": {
"First": "The Kraken",
"Second": "Minnus I"
},
"Name": "Bob",
"Gender": "Hopefully female",
"Age": 23,
"Prefered colleages": [],
"Following missions": [
{
"Payment_status": "TO BE CONFIRMED"
}
]
},
]
}
]
Firstly, the datum you have defined here is not the datum that would be extracted from the server. The datum from the server would be a string. The datum you have in this program is already processed. Now, assuming datum to be:
datum = '{"Mission": "Make Earth Greater Again", "Objective": "Prove Earth is flat", "Astronauts": [{"Spaceships": {"First": "Katabom", "Second": "The Kraken"}, "Name": "Jebeddiah", "Gender": "Hopefully male", "Age": 35, "Prefered colleages": [], "Following missions": [{"Payment_status": "TO BE CONFIRMED"}]}, {"Spaceships": {"First": "The Kraken", "Second": "Minnus I"}, "Name": "Bob", "Gender": "Hopefully female", "Age": 23, "Prefered colleages": [], "Following missions": [{"Payment_status": "TO BE CONFIRMED"}]}]}'
You don't need the the dpath library. The problem here is that your json flattener doesn't handle embedded lists. Try using the one I've put below.
Assuming that you want a one line csv file,
import json
def flattenjson(data, delim, topname=''):
"""JSON flattener that can handle embedded lists and dictionaries"""
flattened = {}
def internalflat(int_data, name=topname):
if type(int_data) is dict:
for key in int_data:
internalflat(int_data[key], name + key + delim)
elif type(int_data) is list:
i = 1
for elem in int_data:
internalflat(elem, name + str(i) + delim)
i += 1
else:
flattened[name[:-len(delim)]] = int_data
internalflat(data)
return flattened
#If you don't want mission or objective in csv file
flattened_astronauts = flattenjson(json.loads(datum)["Astronauts"], "__", "Astronaut")
keys = flattened_astronauts.keys().sort()
writer = csv.DictWriter(OD, keys ,restval='Null', delimiter=",", quotechar="\"", quoting=csv.QUOTE_ALL, dialect= "excel")
writer.writerow(flattened_astronauts)

How to convert JSON Http response to Array in AngularJS 2

I'm doing a Http get in Angular 2 and the response is a JSON. However, i'm trying to use this in a ngFor but i can't because it isn't an Array.
How can I convert JSON to Array in Angular 2? I searched in many websites but didn't discover a effective way to do that.
Edit 1:
The response is like that:
{
"adult": false,
"backdrop_path": "/fCayJrkfRaCRCTh8GqN30f8oyQF.jpg",
"belongs_to_collection": null,
"budget": 63000000,
"genres": [
{
"id": 18,
"name": "Drama"
}
],
"homepage": "",
"id": 550,
"imdb_id": "tt0137523",
"original_language": "en",
"original_title": "Fight Club",
"overview": "A ticking-time-bomb insomniac and a slippery soap salesman channel primal male aggression into a shocking new form of therapy. Their concept catches on, with underground \"fight clubs\" forming in every town, until an eccentric gets in the way and ignites an out-of-control spiral toward oblivion.",
"popularity": 0.5,
"poster_path": null,
"production_companies": [
{
"name": "20th Century Fox",
"id": 25
}
],
"production_countries": [
{
"iso_3166_1": "US",
"name": "United States of America"
}
],
"release_date": "1999-10-12",
"revenue": 100853753,
"runtime": 139,
"spoken_languages": [
{
"iso_639_1": "en",
"name": "English"
}
],
"status": "Released",
"tagline": "How much can you know about yourself if you've never been in a fight?",
"title": "Fight Club",
"video": false,
"vote_average": 7.8,
"vote_count": 3439
}
I think if you want to pass from json to array you could do the following command:
var arr = []
for(i in json_object){
arr.push(i)
arr.push(json_object[i])
}
Then you have every keys in the even index and every contents in the odd index
Well, I really don't see the point here. Arrays are for operating with lists of similar objects or types, not for complex structures. If you had a bunch of objects similar to the one you show, then it would make sense. Anyways, if you really want an array then you could do it with recursion and create a flat array of the properties.
var flatPropertyArray = [];
function flatten(obj) {
for (var property in obj) {
if (obj.hasOwnProperty(property)) {
if (typeof obj[property] == "object")
flatten(obj[property]);
else
flatPropertyArray.push(property);
}
}
}
pass your JSON into the flatten func.

Resources