Parameter validation failed:\nInvalid type for parameter Body in endpoint - amazon-sagemaker

error
{
"errorMessage": "Parameter validation failed:\nInvalid type for parameter Body, value: [[[[-0.4588235, -0.27058822, -0.44313723], [-0.4823529, -0.47450978, -0.6], [-0.7490196, -0.70980394, -0.75686276],
....
[0.18431377, 0.19215691, 0.15294123], [0.0196079, 0.02745104, -0.03529412], [-0.0745098, -0.05882353, -0.16862744], [-0.4823529, -0.5058824, -0.62352943], [-0.38039213, -0.372549, -0.4352941], [-0.4588235, -0.41960782, -0.47450978], [-0.56078434, -0.58431375, -0.62352943], [-0.4352941, -0.41960782, -0.4588235], [-0.40392154, -0.41176468, -0.45098037], [-0.30196077, -0.34117645, -0.372549], [-0.30196077, -0.29411763, -0.34117645], [-0.26274508, -0.2235294, -0.27843136], [0.00392163, -0.01176471, 0.09019613], [0.09019613, 0.09803927, -0.01176471], [0.06666672, 0.12941182, -0.05098039], [0.03529418, 0.09019613, -0.03529412], [0.09019613, 0.10588241, 0.00392163], [-0.01960784, -0.01176471, -0.05882353], [0.03529418, 0.04313731, -0.01960784], [0.13725495, 0.15294123, 0.06666672], [0.06666672, 0.07450986, 0.02745104], [0.16078436, 0.1686275, 0.20000005], [0.43529415, 0.52156866, 0.5686275], [0.5764706, 0.64705884, 0.7019608], [0.67058825, 0.7490196, 0.75686276], [0.5764706, 0.654902, 0.6627451], [0.5921569, 0.67058825, 0.6784314]]]], type: <class 'list'>, valid types: <class 'bytes'>, <class 'bytearray'>, file-like object",
"errorType": "ParamValidationError",
"stackTrace": [
[
"/var/task/lambda_function.py",
16528,
"lambda_handler",
"[ 0.5921569 , 0.67058825, 0.6784314 ]]]])"
],
[
"/var/runtime/botocore/client.py",
357,
"_api_call",
"return self._make_api_call(operation_name, kwargs)"
],
[
"/var/runtime/botocore/client.py",
649,
"_make_api_call",
"api_params, operation_model, context=request_context)"
],
[
"/var/runtime/botocore/client.py",
697,
"_convert_to_request_dict",
"api_params, operation_model)"
],
[
"/var/runtime/botocore/validate.py",
297,
"serialize_to_request",
"raise ParamValidationError(report=report.generate_report())"
]
]
}
lambda code
import os
import io
import boto3
import JSON
ENDPOINT_NAME = "tensorflow-training-2021-01-24-03-35-44-884"
runtime= boto3.client('runtime.sagemaker')
def lambda_handler(event, context):
print("Received event: " + json.dumps(event, indent=2))
data = json.loads(json.dumps(event))
payload = data['data']
print(payload)
response = runtime.invoke_endpoint(EndpointName=ENDPOINT_NAME,
Body=payload)
print(response)
result = json.loads(response['Body'].read().decode())
print(result)
return result[0]
I trained the model in sagemaker using TensorFlow
Estimator part
pets_estimator = TensorFlow(
entry_point='train.py',
role=role,
train_instance_type='ml.m5.large',
train_instance_count=1,
framework_version='2.1.0',
py_version='py3',
output_path='s3://imageclassificationtest202/data',
sagemaker_session=sess
)
I don't know why but I can't send data using JSON for
{
"data" : [[[[90494]...]]
}
My model simply accept NumPy array of dimension (1,128,128,3), and I m sending that data in JSON data field, but its saying invalid format, need byte or byte array


If you are following the reference from the AWS Blog, one way to solve the error would be:
data= json.dumps(event).encode('utf-8')
Then in the invokeEndPoint method, we would set Body=data .
Therefore, in reference to your code, I would suggest you set:
payload=json.dumps(data).encode('utf-8')


you have to set a serializer on your predictor, not sure how to do in your example but when I deploy a model I set this on the returned predictor.
from sagemaker.serializers import CSVSerializer
xgb_predictor.serializer = CSVSerializer() # set the serializer type

Related

Pass array of connection strings in parameters file to bicep file to be used for loop for application service

I am trying to create an application service to deploy via a CI/CD pipeline, however I am struggling create a For loop for the connection strings.
Im my parameters file I have the following:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
// Required Parameters
...
// Optional Parameters
"connectionStrings": [
{
"name": "Tracking",
"connectionString": "...",
"type": "SQLServer"
}
{
"name": "Logs",
"connectionString": "...",
"type": "SQLServer"
}
],
...
}
}
I am passing this into the main.bicep file and the appService.bicep module file as a param connectionStrings array.
In the appService.bicep file I have the following for loop.
resource appServiceWeb 'Microsoft.Web/sites#2022-03-01' = {
name: nameWeb
location: location
tags: tags
kind: kind
identity: {
type: 'SystemAssigned'
}
properties: {
serverFarmId: appServicePlan_id
siteConfig: {
appSettings: [
...
]
connectionStrings: [for connectionString in connectionStrings: {
name: connectionString.name
connectionString: connectionString.connectionString
type: connectionString.type
}]
}
}
}
I am getting the following error:
{
"code": "InvalidRequestContent",
"message": "The request content was invalid and could not be deserialized: 'Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'Azure.Deployments.Core.Definitions.DeploymentParameterDefinition' because the type requires a JSON object (e.g. {\"name\":\"value\"}) to deserialize correctly.\r\nTo fix this error either change the JSON to a JSON object (e.g. {\"name\":\"value\"}) or change the deserialized type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List<T> that can be deserialized from a JSON array. JsonArrayAttribute can also be added to the type to force it to deserialize from a JSON array.\r\nPath 'properties.parameters.connectionStrings', line 1, position 280.'."
}

I have managed to fix my issue. While my Json structure was legal, it wasn't the correct structure for bicep files.
I needed to add the value part below:
"connectionStrings": {
"value": [
{
"name": "TrackingEntities",
"connectionString": "...",
"type": "SQLServer"
}
]
},

Instead of passing connectionStrings in parameters.json file, Include it with an array type in main.bicep file. If you pass in parameters.json it is expecting an object type and that is why for loop is throwing deserialized object error.
Usually to avoid this error, Pass for loop as
[for (connectionStrings, i) in connectionStrings: {
connectionStrings : <ResourceName>.properties.connectionStrings[i]
}
Alternatively, I would suggest you add in main.bicep file with param keyword as detailed above.
By referring to the sample bicep template, I made a few changes to your code and was able to deploy it successfully.
main.bicep:
param webAppName string = <AppNAme>
param sku string = '<SKU>'
param linuxFxVersion string = ''
param location string = resourceGroup().location
var appServicePlanName = <appServicePlanName>
var webSiteName = <webSiteName>
param connectionStrings array = [
{
name: 'Tracking'
connectionString: ''
type: 'SQLServer'
}
{
name: 'Logs'
connectionString: ''
type: 'SQLServer'
}
]
resource appServicePlan 'Microsoft.Web/serverfarms#2020-06-01' = {
name: appServicePlanName
location: location
properties: {
reserved: true
}
sku: {
name: sku
}
kind: 'linux'
}
resource appService 'Microsoft.Web/sites#2020-06-01' = {
name: webSiteName
location: location
properties: {
serverFarmId: appServicePlan.id
siteConfig: {
linuxFxVersion: linuxFxVersion
connectionStrings: [for connectionString in connectionStrings: {
name: connectionString.name
connectionString: connectionString.connectionString
}]
}
}
}
Build & deployment got succeeded:
Deployed a WebApp successfully:
Added connection Strings in WebApp -> Configuration:

How to parse json array which contains multiple data types?

I am getting two types of data in json array from API Response: String and json object
JSON::
{
"elements": [
"task.",
"weeks.",
{
"image_url": "https://www.graph-2x.jpg"
}
]
}
data class::
#SerializedName("elements" ) var bodyElements : ArrayList<String> = arrayListOf()
It throws error as I have mentioned data type is String. Can any one please suggest that how can I parse the multiple type json array in Kotlin? Thanks in advance.

For this specific JSON format you can use the following:
Use Any as data type:
data class ElementsResponse(#SerializedName("elements") val elements: ArrayList<Any>)
And to consume the response, check if Any is String or LinkedTreeMap
when(element) {
is String -> {
// task. or weeks.
println("string: $element")
}
is LinkedTreeMap<*, *> -> {
// key: image_url / value: https://www.graph-2x.jpg
println("image_url: ${element["image_url"]}")
}
}

Accept list input and list response output using Django Rest Framework

I am getting this part 1 input and part 1 output successfully & documentation is also available for this type of problems Single array
Part 1 input
{
"someid:"1"
"lecture_id": 1,
"subject_name": "English",
"teacher_id": 1,
"teacher_name": "Cirillo Kierans",
"room": "Room A",
"schedule": "1534567899"
}
Part 1
output posting-->
{
"id:1,
"someid" :1
"lecture_id": 1,
"subject_name": "English",
"teacher_id": 1,
"teacher_name": "Cirillo Kierans",
"room": "Room A",
"schedule": "1534567899",
}
but my data is not coming like above format and i need to get post response not like above Part 1 , i am expecting this multiple record (list) Part 2 input and Part 2 output
Part 2 input
{
"someid:"1"
"lectures": [{
"lecture_id": 1,
"subject_name": "English",
"teacher_id": 1,
"teacher_name": "Cirillo Kierans",
"room": "Room A",
"schedule": "1534567899"
}]
}
Part 2
output
{
"lectures": [{
"id:1,
"lecture_id": 1,
"subject_name": "English",
"teacher_id": 1,
"teacher_name": "Cirillo Kierans",
"room": "Room A",
"schedule": "1534567899",
"someid" :1
}]
}
I have succeeded in getting the listing of data by overriding def list this function
serializers-->
class LecturesSerializer(serializers.ModelSerializer):
schedule_timestamp=serializers.IntegerField()
class Meta:
model=Lectures
fields = ('id','lecture_id','schedule_timestamp','subject_name','teacher_id','teacher_name','room')
def validate(self, attrs):
travel_date = Lectures.to_date_time(attrs.pop('schedule_timestamp'))
attrs.update({'schedule_date': travel_date})
attrs = super().validate(attrs)
return attrs
def to_representation(self, instance):
data = super(LecturesSerializer, self).to_representation(instance)
result_data={}
result_data["lectures"]=data
return result_data
views-->
class LecturesViewSet(viewsets.ModelViewSet):
queryset = Lectures.objects.all()
def get_serializer_class(self):
return LecturesSerializer
def dispatch(self, request, *args, **kwargs):
response = super(LecturesViewSet, self).dispatch(request, *args, **kwargs)
data = {}
data['lectures'] = response.data
response.data = data
return response
it is giving listing as expected but posting and updating is not working for me, how to define a posting method and getting customized array object response for POST and PUT depicted above
Thanks in advance for helping

It is very likely a job for a nested serializer:
class SomeModelSerializer(serializers.ModelSerialiser):
lectures = LecturesSerializer(many=true, )
class Meta:
model = SomeModel
fields = ('some_id', 'lectures', )
and remove to_representation from LecturesSerializer. Editing can be done trough this concept as well.

I am trying to validate schema having integer array in karate

I am trying to validate integer array schema but getting Error assertion failed:actual :{applications=[2,5,7], assigned_doctors=[100014,90], additional_roles=[10,19]
expected: {applications=##[], assigned_doctors=#[], additional_roles=#[]}
I am not able to find correct syntax to validate array of integer.
Response:
"userdetail": {
"applications": [
2,
5,
7
],
"assigned_doctors": [
100014,
90
],
"additional_roles": [
10,
19
],
Structure:
"""
{
userdetail:{
applications: ##[],
assigned_doctors: #[],
additional_roles: #[]
},
When def getUserByIdUrl = 'http:xyz//UserRegistration/getUserById/105'
And header Content-Type = 'application/json; charset=utf-8
And header auth_token = token
And url getUserByIdUrl
And method GET
Then status 200
And def getUserByIdData = response
And print getUserByIdData
"""
{
userdetail:{
applications: ##[],
assigned_doctors: #[],
additional_roles: #[]
},
actual: {applications=[2,5,7], assigned_doctors=[100014,90], additional_roles=[10,19]
expected:
expected: {applications=##[], assigned_doctors=#[], additional_roles=#[]}

Please, next time paste your response JSON properly so that it is well-formed. You can paste the below into a new Scenario and see it work. And read the docs: https://github.com/intuit/karate#schema-validation
* def response =
"""
{
"userdetail": {
"applications": [
2,
5,
7
],
"assigned_doctors": [
100014,
90
],
"additional_roles": [
10,
19
]
}
}
"""
* match response ==
"""
{
userdetail: {
applications: '#[] #number',
assigned_doctors: '#[] #number',
additional_roles: '#[] #number'
}
}
"""

Groovy & json: How to get the length of an array from json?

I am using curl to get some data, then parsing it with JsonSlurper.
The data structure is
"results" : [
{
"uri" : "http://localhost:8081/artifactory/api/storage/...",
"created" : "2015-11-27"
},
{
"uri" : "http://localhost:8081/artifactory/api/storage/...",
"created" : "2015-11-27"
},
{
"uri" : "http://localhost:8081/artifactory/api/storage/...",
"created" : "2015-11-30"
}
]
So if I'm not mistaken, the entire thing is considered an object. But there is an array in the object (results) which contains objects in that array.
I need to get the length of the results array. When I try to do json.result.length, I receive a null.
How can I get the length of the results array?
def list = ['curl', '-u', 'user:pass', "http://localhost:8081/..."].execute()
def json = new JsonSlurper().parseText(list.text)
println json.result.size()

What I see is, you are using incorrect property to get the size. Need to use results instead of result. Thats is the reason you are seeing that error as result is null(because there is no such property)
Here is working script and gets output as 3:
import net.sf.json.groovy.JsonSlurper
def jsonText='''{
"results" : [
{
"uri" : "http://localhost:8081/artifactory/api/storage/...",
"created" : "2015-11-27"
},
{
"uri" : "http://localhost:8081/artifactory/api/storage/...",
"created" : "2015-11-27"
},
{
"uri" : "http://localhost:8081/artifactory/api/storage/...",
"created" : "2015-11-30"
}
]
}'''
def json = new JsonSlurper().parseText(jsonText)
println json.results.size()
assert 3 == json.results.size(), "Array size is not matching"

It will be:
def response = ... // your response as text, stream..
def parsed = new JsonSluper().parse(response) // parseText if string
println parsed.results.size()
BTW: size() is for Collection, length for String, note method vs field. In groovy you can use size() for String as well.

{
"projects":
[
{
"platform": "java",
"name":"abc"
},
{
"platform": ".net",
"name":"abcd"
}]
}
for this json file list.json how to get json size or number of applications count.
below code worked for me in groovy as i was using in it my jenkinsfile.
import groovy.json.JsonBuilder
import groovy.json.JsonOutput
import groovy.io.FileType
def applicationCount()
{
jsonFile1 = 'list.json'
def json1 = readJSON file: jsonFile1
totalApplication = json1.projects.size()
println "count" + totalApplication
println "applicationname" + json1['projects'][0]['name']
}

Resources