Python how to save dictionary with cyrillic symbols into json file - arrays

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import json
d = {'a':'текст',
'b':{
'a':'текст2',
'b':'текст3'
}}
print(d)
w = open('log', 'w')
json.dump(d,w, ensure_ascii=False)
w.close()
It gives me:
UnicodeEncodeError: 'ascii' codec can't encode characters in position 1-5: ordinal not in range(128)

Post the full traceback, the error could be coming from the print statement when it fails to decode the dictionary object. For some reason print statement cannot decode all contents if you have Cyrillic text in it.
Here is how I save to json my dictionary that contains Cyrillics:
mydictionary = {'a':'текст'}
filename = "myoutfile"
with open(filename, 'w') as jsonfile:
json.dump(mydictionary, jsonfile, ensure_ascii=False)
The trick will be reading in json back into dictionary and doing things with it.
To read in json back into dictionary:
with open(filename, 'r') as jsonfile:
newdictonary = json.load(jsonfile)
Now when you look at the dictionary, the word 'текст' looks (encoded) like '\u0442\u0435\u043a\u0441\u0442'. You simply need to decode it using encode('utf-8'):
for key, value in newdictionary.iteritems():
print value.encode('utf-8')
Same goes for lists if your Cyrillic text is stored there:
for f in value:
print f.encode('utf-8')
# or if you plan to use the val somewhere else:
f = f.encode('utf-8')

Related

ijson.common.IncompleteJSONError: lexical error: invalid char in json text

hi all ijson newbie I have a very large .json file 168 (GB) I want to get all possible keys, but in the file some values are written as NaN. ijson creates a generator and outputs dictionaries, in My code value. When a specific item is returned, it throws an error. How can you get a string instead of a dictionary instead of value? Tried **parser = ijson.items (input_file, '', multiple_values = True, map_type = str) **, didn't help.
def parse_json(json_filename):
with open('max_data_error.txt', 'w') as outfile:
with open(json_filename, 'r') as input_file:'''
# outfile.write('[ '
parser = ijson.items(input_file, '', multiple_values=True)
cont = 0
max_keys_list = list()
for value in parser:
for i in json.loads(json.dumps(value, ensure_ascii=False, default=str)) :
if i not in max_keys_list:
max_keys_list.append(i)
print(value)
print(max_keys_list)
for keys_item in max_keys_list:
outfile.write(keys_item + '\n')
if __name__ == '__main__':
parse_json('./email/emailrecords.bson.json')
Traceback (most recent call last):
File "panda read.py", line 29, in <module>
parse_json('./email/emailrecords.bson.json')
File "panda read.py", line 17, in parse_json
for value in parser:
ijson.common.IncompleteJSONError: lexical error: invalid char in json text.
litecashwire.com","lastname":NaN,"firstname":"Mia","zip":"87
(right here) ------^
Your file I not valid JSON (NaN is not a valid JSON value); therefore any JSON parsing library will complain about this, one way or another, unless they have an extension to handle this non-standard content.
The ijson FAQ found in the project description has a question about invalid UTF-8 characters and how to deal with them. Those same answers apply here, so I would suggest you go and try one of those.

Saving json array output to txt file and getting errors when attempting to parse

I am unable to parse a JSON array from a text file due to errors and my limited knowledge of JSON.
The file looks something like this [{"random":"fdjsf","random56":128,"name":"dsfjsd", "rid":1243,"rand":674,"name":"dsfjsd","random43":722, "rid":126},{"random":"fdfgfgjsf","random506":120,"name":"dsfjcvcsd", "rid":12403,"rando":670,"name":"dsfooojsd","random4003":720, "rid":120}] It has more than one object({}) in the entire array however I did not want to include all 600. The layout shown above is basically how all of them look.
r = s.get(getAPI, headers=header, verify=False)
f = open('text.txt', 'w+')
f.write(r.text)
f.close
output_file = open ('text.txt', 'r')
json_array = json.load(output_file)
json_list = []
for item in json_array:
name = "name"
rid = "rid"
json_items = {name:None, rid:None}
json_items = [name] = item[name]
json_items = [rid] = item[rid]
json_list.append(json_items)
print(json_list)
I would like to loop through an array and find any time it says "name":... eventually followed by "rid":... and store those in a dictionary as key value pairs.
Errors:
ValueError: too many values to unpack (expected 1)
There is a syntax error when you assign values to json_items, change it to:
json_items[name] = item[name]
json_items[rid] = item[rid]

How to convert a line having string into dictionary using python?

I have a line as:
Name:sample Location:(xyz)
I want to to convert it to dictionary as follows:
{'Name':'sample','Location':'(xyz)'}
I want to do this using python script. So, please suggest how do i make this possible. The platform i am working on is linux.
# First split at whitespaces ==> ['Name:sample', 'Location:(xyz)']
# Next split each item at ':' and convert them into list of tuples
# ==>[('Name', 'sample'), ('Location', '(xyz)')]
# Convert the list of tuples to dictionary
sample_string = "Name:sample Location:(xyz)"
split_sample_string = sample_string.split()
tuple_string = [tuple(item.split(":")) for item in split_sample_string]
final_dictionary = dict(tuple_string)
print final_dictionary
# final_dictionary = {'Name': 'sample', 'Location': '(xyz)'}

storing data into a file rather than returning to the terminal

i have this function: write_reversed_file(input_filename, output_filename) that writes to the given output file the contents of the given input file with the lines in reversed order. i just need the output to be written to the file (output_filename) rather than to the terminal (python shell).
the only part im missing is to store the output into the file.
i successfully managed to complete the reversing lines part.
def write_reversed_file(input_filename, output_filename):
for line in reversed(list(open(filename))):
print(line.rstrip())
def write_reversed_file(input_filename, output_filename):
s = ""
f = open(input_filename,"r")
lines = f.read().split("\n")
f.close()
for line in reversed(lines):
s+=line.rstrip()+"\n"
f = open(outPutFile.txt,"w")
f.write(s)
f.close()
It is good practice to use 'with open as' format when working with files since it is automatically closing the file for us. (as recommended in docs.python.org)
def write_reversed_file(input_filename, output_filename):
with open(output_filename, 'w') as f:
with open(input_filename, 'r') as r:
for line in reversed(list(r.read())):
f.write(line)
write_reversed_file("inputfile.txt", "outputfile.txt")

How to append or change data within a file

Hiya i have made a program that stores the player name and strength..Here is the code:
data = {
"PLAYER":name2,
"STRENGTH":str(round(strength, 2)),
}
with open("data2.txt", "w", encoding="utf-8") as file:
file.write(repr(data))
file.close()
So this stores the data so what to i do if i wanna append/change the value after a certain action usch as a 'BATTLE'
Is it possible the get the variable of 'STRENGTH' and then change the number?
At the moment to read data from the external file 'DATA1.txt'i am using this code:
with open("data1.txt", "r", encoding="utf-8") as file:
data_string = file.readline()
data = eval(data_string)
# (data["STRENGTH"])
S1 = (float(data["STRENGTH"]))
file.close()
Now i can do something with the variable --> 'S1'
Here is the external text file 'data1.txt'
{'PLAYER': 'Oreo', 'STRENGTH': '11.75'}
... But i wanna change the strength value after a "battle" many thanks
Maybe you're not understanding Python dict semantics?
Seems to me you're doing a lot of unnecessary things like S1 = (float(data['STRENGTH'])) to try to manipulate and change values when you could be doing really simple stuff.
>>> data = {'PLAYER': 'Oreo', 'STRENGTH': '11.75'}
>>> data['STRENGTH'] = float(data['STRENGTH'])
>>> data
{'PLAYER': 'Oreo', 'STRENGTH': 11.75}
>>> data['STRENGTH'] += 1
>>> data
{'PLAYER': 'Oreo', 'STRENGTH': 12.75}
Maybe you should give Native Data Types -- Dive Into Python 3 a read to see if it clears things up.

Resources