I am using elephantbird project to load a json file to pig.
But i am not sure how i can define the schema at load. Did not find a description about the same.
data:
{"id":22522,"name":"Product1","colors":["Red","Blue"],"sizes":["S","M"]}
{"id":22523,"name":"Product2","colors":["White","Blue"],"sizes":["M"]}
code:
feed = LOAD '$INPUT' USING com.twitter.elephantbird.pig.load.JsonLoader() AS products_json;
extracted_products = FOREACH feed GENERATE
products_json#'id' AS id,
products_json#'name' AS name,
products_json#'colors' AS colors,
products_json#'sizes' AS sizes;
describe extracted_products;
result:
extracted_products: {id: chararray,name: bytearray,colors: bytearray,sizes: bytearray}
how i can give the correct schema to them (int,string,array,array) and how can i flatten array elements into rows?
thanks in advance
to convert json array to tuple:
feed = LOAD '$INPUT' USING com.twitter.elephantbird.pig.load.JsonLoader() AS products_json;
extracted_products = FOREACH feed GENERATE
products_json#'id' AS id:chararray,
products_json#'name' AS name:chararray,
products_json#'colors' AS colors:{t:(i:chararray)},
products_json#'sizes' AS sizes:{t:(i:chararray)};
to flatten a tuple
flattened = foreach extracted_products generate id,flatten(colors);
Related
Let's say I want to use the range function (inside a ForEach loop) in Azure Data Factory to create an array which consists of integers. These integers represent API pages related to some ID which was given to us as a parameter in the ForEach loop.
I would use it like #range(1, int(varMaxApiPages)).
This gives me what I expect; an array of integers:
[1, 2, 3]
But would it be possible to append the related ID to these integers? So the result would be something like: [{"someID", 1},{"someID", 2},{"someID", 3}]?
Such as:
def appendToArray(varMaxApiPages):
arr1 = list(range(varMaxApiPages))
json_array = [];
for item in arr1:
jsonObejct = {"someID",item}
json_array.append(jsonObejct)
for item in json_array:
print(item)
appendToArray(3)
The correct json array should look like this [{"someID": 1},{"someID": 2},{"someID": 3}], we can achieve that. If you don’t want the colon, you can think of a way to replace it.
My debug result is as follows:
I declared 3 array type variables. Variable res is used to review the debug result.
In Set variable1 activity, assign the value to it via #range(1,3).
Then Foreach the arr1.
Inside Foreach activity, we can use Append variable activity, add expression #json(concat('{"someID":',item(),'}')). It will convert json string to json Object and append to the array jsonArray.
Outside Foreach activity, assign the value of array jsonArray to array res to review the result, you can omit this step and use array jsonArray directly.
That's all.
I am using a pod called iOSDropDown to display a dropdown selection menu for a textfield. I am getting a list of data from php to populate that selection menu that Im storing in a variable.
The PHP data passed stored in a swift variable looks like this when printed -> "option01","option02","option03"... and so on. This is dynamic data that will change that is why I am retrieving from PHP/MYSQL Database instead of just manually typing in the options in the Swift array.
Below is my code. What I am trying to do is use the "dropdownData" variable that holds the options for the array. Each option should be in its own row and separately selectable. What I am getting is one option, one string of coding with all my options as shown in the picture below.How would I use the dropdownData variable to display options instead of one string, one option?
dropdownData = "option01","option02","option03"... etc. ALL OPTIONS STORED IN THIS ONE ARRAY
let dropdownData : String = (dumpsArray[indexPath.row] as AnyObject).value(forKey: "dropdownData") as! String
.
cell.nameField.optionArray = [dropdownData]
Image
In the image above there should be no comma after styrofoam cooler... the next product should be the next option displaying under the styrofoam cooler and separately selectable.
Seems like dumpsArray[indexPath.row] as AnyObject).value(forKey: "dropdownData") returns a String where names are comma separated,
so
if let nameString = dumpsArray[indexPath.row] as AnyObject).value(forKey: "dropdownData") as? String {
let namesArray = nameString.components(separatedBy: ",")
cell.nameField.optionArray = namesArray
}
That should do
I have two JSON. In the both I have name. How can I get names from first JSON and add to array? Later I want do this same with second JSON later i want both array compare? How can I do this?
jsonArray1 = [{'name': "doug", 'id':5}, {'name': "dofug", 'id':23}];
jsonArray2 = [{'name': "goud", 'id':1}, {'name': "doaaug", 'id':52}];
for example I want:
a = [doug, dofug] b = [goud, doaaug]
and later check if these are the same arrays
i don't know how can i do this in jmeter, help
To convert one array to another:
Add JSR223 PostProcessor as a child of the request which returns first JSON Array
Put the following code into "Script" area:
def builder = new groovy.json.JsonBuilder()
builder(com.jayway.jsonpath.JsonPath.read(prev.getResponseDataAsString(),'$..name').collect())
vars.put('array1', builder.toPrettyString())
That's it, you should be able to access the newly generated JSON Array as ${array1} where required
Repeat the same steps for the second JSON array.
There are several options on how to compare 2 JSON entities, depending on what you're trying to achieve you can go for:
Response Assertion
a JSR223 Assertion and a 3rd-party library like:
Jackson
Gson
JSONassert
I'm using this json content (I'm open to suggestions on better formatting here):
{"forwardingZones": [{"name": "corp.mycompany.com","dnsServers": ["192.168.0.1","192.168.0.2"]}]}
Note: we may add more items to this list as we scale out, both more IPs and more names, hence the "join(',')" in the end of the code below.
And I'm trying to loop through it to get this result:
corp.mycompany.com=192.168.0.1;192.168.0.2
Using this code:
forward_zones = node['DNS']['forward_zones'].each do |forwarded_zone|
forwarded_zone_name = forwarded_zone['name']
forwarded_zone_dns_servers = forwarded_zone['dns_servers'].join(';')
"#{forwarded_zone_name}=#{forwarded_zone_dns_servers}"
end.join(',')
This is the result that I get:
{"dnsServers"=>["192.168.0.1", "192.168.0.2"], "name"=>"corp.mycompany.com"}
What am i doing wrong...?
x.each returns x. You want x.map.
I have two lists that look like this:
List("key1", "key2")
List("val1", "val2")
I'm trying to get it into this JSON-like format:
[{"key1":"val1"},{"key2","val2"}]
Right now, this is what I'm using:
val output = List(attrKeys.zip(attrValues).toMap)
But it gives me this:
[{"key1":"val1","key2":"val2"}]
How do I get this into a list of separate map objects?
attrKeys.zip(attrValues).map(Map(_))