How can I pass an object like a list or dictionary in python behave .feature file - python-behave

How can I pass an object like a list or dictionary as an argument in behave .feature file so I can use that argument in my python function step? See an example of what I'm trying to achieve below:
Feature:
Scenario: Given the inputs below
Given a "<Dictionary>" and "<List>"
When we insert "<Dictionary>" and "<List>"
Then we confirm the result in the database
Examples: Input Variables
|Input1 |Input2 |
|Dictionary(json) |List |

You can provide the data as json, and parse it using json.loads in the steps.
Note, to use Examples: we need a Scenario Outline instead of a
Scenario.
# features/testing_objects.feature
Feature: Testing objects
Scenario Outline: Given the inputs below
Given a <Dictionary> and <List>
When we insert them
Then we confirm the result in the database
Examples: Input Variables
|Dictionary |List |
|{"name": "Fred", "age":2} |[1,2,"three"]|
Parse it using json.loads in the steps:
# features/steps/steps.py
import json
from behave import given, when, then
#given('a {dictionary} and {a_list}')
def given_dict_and_list(context, dictionary, a_list):
context.dictionary = json.loads(dictionary)
context.a_list = json.loads(a_list)
#when('we insert them')
def insert_data(context):
print('inserting dictionary', context.dictionary)
print('inserting list', context.a_list)
#then('we confirm the result in the database')
def confirm(context):
print('checking dictionary', context.dictionary)
print('checking list', context.a_list)
Instead of using Examples: you could also use a multi-line string literal
and then access each object in a separate step, via context.text.
Feature: String literal JSON
Scenario:
Given a dictionary
"""
{
"name": "Fred",
"age": 2
}
"""
And a list
"""
[1, 2, "three"]
"""
Then we can check the dictionary
And check the list
#given('a dictionary')
def given_a_dictionary(context):
context.dictionary = json.loads(context.text)
#given('a list')
def given_a_list(context):
context.a_list = json.loads(context.text)
#then('we can check the dictionary')
def check_the_dictionary(context):
assert context.dictionary == {
'name': 'Fred',
'age': 2
}
#then('check the list')
def check_the_list(context):
assert context.a_list == [1, 2, 'three']

Related

GenericRowWithSchema ClassCastException in Spark 3 Scala UDF for Array data

I am writing a Spark 3 UDF to mask an attribute in an Array field.
My data (in parquet, but shown in a JSON format):
{"conditions":{"list":[{"element":{"code":"1234","category":"ABC"}},{"element":{"code":"4550","category":"EDC"}}]}}
case class:
case class MyClass(conditions: Seq[MyItem])
case class MyItem(code: String, category: String)
Spark code:
val data = Seq(MyClass(conditions = Seq(MyItem("1234", "ABC"), MyItem("4550", "EDC"))))
import spark.implicits._
val rdd = spark.sparkContext.parallelize(data)
val ds = rdd.toDF().as[MyClass]
val maskedConditions: Column = updateArray.apply(col("conditions"))
ds.withColumn("conditions", maskedConditions)
.select("conditions")
.show(2)
Tried the following UDF function.
UDF code:
def updateArray = udf((arr: Seq[MyItem]) => {
for (i <- 0 to arr.size - 1) {
// Line 3
val a = arr(i).asInstanceOf[org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema]
val a = arr(i)
println(a.getAs[MyItem](0))
// TODO: How to make code = "XXXX" here
// a.code = "XXXX"
}
arr
})
Goal:
I need to set 'code' field value in each array item to "XXXX" in a UDF.
Issue:
I am unable to modify the array fields.
Also I get the following error if remove the line 3 in the UDF (cast to GenericRowWithSchema).
Error:
Caused by: java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to MyItem
Question: How to capture Array of Structs in a function and how to return a modified array of items?
Welcome to Stackoverflow!
There is a small json linting error in your data: I assumed that you wanted to close the [] square brackets of the list array. So, for this example I used the following data (which is the same as yours):
{"conditions":{"list":[{"element":{"code":"1234","category":"ABC"}},{"element":{"code":"4550","category":"EDC"}}]}}
You don't need UDFs for this: a simple map operation will be sufficient! The following code does what you want:
import spark.implicits._
import org.apache.spark.sql.Encoders
case class MyItem(code: String, category: String)
case class MyElement(element: MyItem)
case class MyList(list: Seq[MyElement])
case class MyClass(conditions: MyList)
val df = spark.read.json("./someData.json").as[MyClass]
val transformedDF = df.map{
case (MyClass(MyList(list))) => MyClass(MyList(list.map{
case (MyElement(item)) => MyElement(MyItem(code = "XXXX", item.category))
}))
}
transformedDF.show(false)
+--------------------------------+
|conditions |
+--------------------------------+
|[[[[XXXX, ABC]], [[XXXX, EDC]]]]|
+--------------------------------+
As you see, we're doing some simple pattern matching on the case classes we've defined and successfully renaming all of the code fields' values to "XXXX". If you want to get a json back, you can call the to_json function like so:
transformedDF.select(to_json($"conditions")).show(false)
+----------------------------------------------------------------------------------------------------+
|structstojson(conditions) |
+----------------------------------------------------------------------------------------------------+
|{"list":[{"element":{"code":"XXXX","category":"ABC"}},{"element":{"code":"XXXX","category":"EDC"}}]}|
+----------------------------------------------------------------------------------------------------+
Finally a very small remark about the data. If you have any control over how the data gets made, I would add the following suggestions:
The conditions JSON object seems to have no function in here, since it just contains a single array called list. Consider making the conditions object the array, which would allow you to discard the list name. That would simpify your structure
The element object does nothing, except containing a single item. Consider removing 1 level of abstraction there too.
With these suggestions, your data would contain the same information but look something like:
{"conditions":[{"code":"1234","category":"ABC"},{"code":"4550","category":"EDC"}]}
With these suggestions, you would also remove the need of the MyElement and the MyList case classes! But very often we're not in control over what data we receive so this is just a small disclaimer :)
Hope this helps!
EDIT: After your addition of simplified data according to the above suggestions, the task gets even easier. Again, you only need a map operation here:
import spark.implicits._
import org.apache.spark.sql.Encoders
case class MyItem(code: String, category: String)
case class MyClass(conditions: Seq[MyItem])
val data = Seq(MyClass(conditions = Seq(MyItem("1234", "ABC"), MyItem("4550", "EDC"))))
val df = data.toDF.as[MyClass]
val transformedDF = df.map{
case MyClass(conditions) => MyClass(conditions.map{
item => MyItem("XXXX", item.category)
})
}
transformedDF.show(false)
+--------------------------+
|conditions |
+--------------------------+
|[[XXXX, ABC], [XXXX, EDC]]|
+--------------------------+
I am able to find a simple solution with Spark 3.1+ as new features are added in this new Spark version.
Updated code:
val data = Seq(
MyClass(conditions = Seq(MyItem("1234", "ABC"), MyItem("234", "KBC"))),
MyClass(conditions = Seq(MyItem("4550", "DTC"), MyItem("900", "RDT")))
)
import spark.implicits._
val ds = data.toDF()
val updatedDS = ds.withColumn(
"conditions",
transform(
col("conditions"),
x => x.withField("code", updateArray(x.getField("code")))))
updatedDS.show()
UDF:
def updateArray = udf((oldVal: String) => {
if(oldVal.contains("1234"))
"XXX"
else
oldVal
})

Snowpark - split a dataframe column

Is there a way to split a snowpark dataframe column based on a string?
Here is what I have tried so far
from snowflake.snowpark import Session
from snowflake.snowpark import functions as SF
connection_parameters = {
"account": "myaccount",
"user": "myuser",
}
session = Session.builder.configs(connection_parameters).create()
dh_session = session.table('tableName')
dh = dh_session.select(SF.to_timestamp(SF.col("timestamp")).as_("timestamp"),SF.col("name"))
# Split the name column by string delimiter '-AA' and get the first part
dh.select(SF.split("name",SF.lit("-AA").get_value(0)).as_("test")).show()
However I get an error message
AttributeError: 'Column' object has no attribute 'getItem'
Thanks
Instead of using get_value, try using get like so,
from snowflake.snowpark import Session
from snowflake.snowpark import functions as SF
connection_parameters = {
"account": "myaccount",
"user": "myuser",
}
session = Session.builder.configs(connection_parameters).create()
dh_session = session.table('tableName')
dh = dh_session.select(SF.to_timestamp(SF.col("timestamp")).as_("timestamp"),SF.col("name"))
# Split the name column by string delimiter '-AA' and get the first part
dh = dh.select(SF.split("name",SF.lit("-AA")).as_("split_name"))
dh.select(SF.get(dh.split_name, SF.lit(0)).as_("name[0]")).show()
Example of first using a filter then select with sample dataframe. Note this uses a 'like' match which may or may not be an accurate regex for your data.
from snowflake.snowpark import Session
from snowflake.snowpark import functions as SF
connection_parameters = {
"account": "myaccount",
"user": "myuser",
}
session = Session.builder.configs(connection_parameters).create()
##
## Create a dummy dataframe
##
dh = session.create_dataframe([["morning-CC", 2], ["hello-AA", 4],
["bye-BB", 7], ["another-AA", 3]], schema=["name", "col2"])
dh.filter("name like '%-AA%'").select(col("name"),
SF.split(col("name"),SF.lit("-"))[0].alias("test")).show()
"NAME"
"TEST"
hello-AA
"hello"
another-AA
"another"

How to extract data from an API and create an array to send to the another API in Jmeter?

Example:
API A:
{
"customer":[
{
"name":"Jane",
"phone":"9999999",
"email":"jane#test.com"
},
{
"name":"John",
"phone":"8888888",
"email":"john#test.com"
},
{
"name":"Joe",
"phone":"7777777",
"email":"Joe#test.com"
}
]
}
Using the JSON extractor, I want to get the names of all the customers
so: Jane, John, Joe
How do I get these values and turn them into an array
[{"name":"Jane", "name":"John", "name":"Joe"}]
And pass it onto the next API?
Note: That it has to be dynamic so API A could show different 2 names or 1 name or more and needs to be adjusted into the array
First of all your [{"name":"Jane", "name":"John", "name":"Joe"}] is not a valid JSON, you can check it yourself:
so I strongly doubt that this is the string you need to generate.
So if you really need to construct this value you can do something like:
Add JSR223 PostProcessor as a child of the request which returns this "customers" data
Put the following code into "Script" area:
def response = new groovy.json.JsonSlurper().parse(prev.getResponseData())
def payload = new StringBuilder()
payload.append('[{')
0.upto(response.customer.size - 1, { index ->
payload.append('"name": "').append(response.customer[index].name).append('"')
if (index != response.customer.size - 1) {
payload.append(',')
}
})
payload.append('}]')
vars.put('payload', payload as String)
Refer the generated value as ${payload} where required
Demo:
More information:
JsonSlurper
Apache Groovy - Parsing and producing JSON
Apache Groovy - Why and How You Should Use It

Parse Array of JSON Objects in Crystal lang

Suppose I've got a simple JSON mapped object in Crystal lang, e.g.:
class Item
JSON.mapping(
id: UInt32,
name: String,
)
end
I can parse individual objects from JSON strings easily like so:
foo = Item.from_json(%({"id":1,"name":"Foo"}))
puts "OK: foo=#{foo}"
# => OK: foo=Item(#id=1, #name="Foo")
But how would I parse an array of Items from a JSON string? I've tried a few approaches but am not sure how to proceed, e.g.:
items_str = %([{"id":1,"name":"Foo"},{"id":2,"name":"Bar"}])
items : Array(Item) = JSON.parse(items_str)
# => Error in foo.cr:15: type must be Array(Item), not JSON::Any
Of course, I'd also like to be able to do this with a JSON pull parser, so presumably there's some mapping trick or type hint I'm missing. Ideas?
Found it in this spec. So, you can use Array(Item).from_json:
items = Array(Item).from_json %([{"id":1,"name":"Foo"},{"id":2,"name":"Bar"}])
items.first.id #=> 1
items.first.name #=> "Foo"
items.last.id #=> 2
items.last.name #=> "Bar"

Json creation in ruby for a list

I am new to Ruby.
I want to create a JSON file for a group of elements.
For this, I am using eachfunction to retrieve the datas. I want to create json as follows for the 4 length array,
'{
"desc":{
"1":"1st Value",
"2":"2nd value"
"3":"3rd Value",
"4":"4th value"
},
}'
This is my array iteration,
REXML::XPath.each( doc, "//time" ) { |element1|
puts element1.get_text
}
I know here is the simple code to generate a JSON,
require 'json/add/core'
class Item < Struct.new(:id, :name); end
chair = Item.new(1, 'chair')
puts JSON.pretty_generate(chair)
This syntax will generate a json as follows,
{
"json_class": "Item",
"v": [
1,
"chair"
]
}
But I'm not sure how to do that to make JSON for my elements as stated above. Google search didn't give me a proper way to do this.
Can anyone help me here?
it means this?:
require 'json'
my_arr= ["1st Value","2nd Value","3rd Value","4th Value"]
tmp_str= {}
tmp_str["desc"] = {}
my_arr.each do |x|
tmp_str["desc"]["#{x[0]}"] = x
end
puts JSON.generate(tmp_str)
you can iterate the string array ,then take the strings to hash object.JSON can easy to parse Hash objcect .

Resources