I'm trying to write a json validator to check files before runtime but there's a really odd issue happening with using "type".
I know "type" is a reserved word but jsonSchema doesn't have an issue with it if it doesn't have a value pairing as I found in this other question: Key values of 'key' and 'type' in json schema.
The solution to their problem was encasing "type": { "type": "string"} inside "properties" and it does work. However, my implementation requires it to be inside an array. Here's the snippet of my code:
{
"type": "object",
"additionalProperties": false,
"properties":{
"method":{
"type": "array",
"items":{
"type": {
"type": "string"
},
"name":{
"type": "string"
},
"provider": {
"type": "array"
}
}
}
}
}
Oddly enough, VScode doesn't have a problem with it in a file when it's isolated, but when it's included in the main code, it doeesn't like it and yields no solution. Regardless, validating it with python yields:
...
raise exceptions.SchemaError.create_from(error)
jsonschema.exceptions.SchemaError: {'type': 'string'} is not valid under any of the given schemas
Failed validating 'anyOf' in metaschema['allOf'][1]['properties']['properties']['additionalProperties']['$dynamicRef']['allOf'][1]['properties']['items']['$dynamicRef']['allOf'][3]['properties']['type']:
{'anyOf': [{'$ref': '#/$defs/simpleTypes'},
{'items': {'$ref': '#/$defs/simpleTypes'},
'minItems': 1,
'type': 'array',
'uniqueItems': True}]}
On schema['properties']['method']['items']['type']:
{'type': 'string'}
What further confuses me is that https://www.jsonschemavalidator.net/ tells me
Expected array or string for 'type', got StartObject. Path 'properties.method.items.type', line 8, position 17. yet JSON Schema Faker is able to generate a fake file without any problems. The generated fake json also returns the same error when validated with python and JSONSchemaValidator.
I'm a beginner and any help or insight will be greatly appreciated, thanks for your time.
Edit: here's the snippet of the input data as requested.
{
...
"method": [
{
"type": "action",
"name": "name of the chaos experiment to use here",
"provider": [
]
}
}
]
}
Arrays don't have properties; arrays have items. The schema as you have included it is not valid; are you sure you don't mean to have this?
{
"type": "object",
"additionalProperties": false,
"properties":{
"method":{
"type": "array",
"items":{
"type": "object",
"properties": {
"type": {
"type": "string"
},
"name":{
"type": "string"
},
"provider": {
"type": "array"
}
}
}
}
}
}
Related
I'm trying to parse some data in Nifi (1.7.1) using UpdateRecord Processor.
Original data are json files, that I would like to convert to Avro, based on a schema.
The Avro conversion is ok, but in that convertion I also need to parse one array element from the json data to a different structure in Avro.
This is a sample data of the input json:
{ "geometry" : {
"coordinates" : [ [ 4.963087975800593, 45.76365595859971 ], [ 4.962874487781098, 45.76320922779652 ], [ 4.962815443439148, 45.763116079159374 ], [ 4.962744732112515, 45.763010484202866 ], [ 4.962096825239138, 45.762112721939246 ] ]} ...}
Being its schema (specified in RecordReader):
{ "type": "record",
"name": "features",
"fields": [
{
"name": "geometry",
"type": {
"type": "record",
"name": "geometry",
"fields": [
{
"name": "coordinatesJson",
"type": {
"type": "array",
"items": {
"type": "array",
"items": "double"
}
}
},
]
}
},
....
]
}
As you can see, coordinates is an array of arrays.
And I need to parse those data to Avro, based on this schema (specified in RecordWriter):
{
"name": "outputdata",
"type": "record",
"fields": [
{"name": "coordinatesAvro",
"type": {
"type": "array",
"items" : {
"type" : "record",
"name" : "coordinatesAvro",
"fields" : [ {
"name" : "X",
"type" : "double"
}, {
"name" : "Y",
"type" : "double"
} ]
}
}
},
.....
]
}
The problem here is that I'm not being able to parse from coordinatesJson to coordinatesAvro, using RecordPath functions
I tried several mappings, like:
Property: Value:
/coordinatesJson[0..-1]/X /geometry/coordinatesAvro[*][0]
/coordinatesJson[0..-1]/Y /geometry/coordinatesAvro[*][1]
It should be a pretty straighforward parsing step, but as I said, I've been going in circles to achive this for a while.
Any help would be really appreciated.
When I collide with something like that I do next:
1) Transofrm Json into Json with strcuture that I need (for example in your case: coordinatesAvro) by ExecuteScript Processor. I have used ECMAScript cause you can simple parse JSON and work with objects (transform them).
2) ConvertJsonToAvro with one common schema (coordinatesAvro in your case) for Reader and Writer.
It works very good and I have used it on BigData cases. This is one of possible resolutions for your problem.
Background
I am making a form using angular-schema-form
Setup
I am trying to make an array of items that a user can make using a form. So, the user can add as many items into the array as they want.
For now the items in the array contain a command type.
Command Type should be a dropdown containing SSH, REST, and whatever the user enters in as the personalized command type.
Code so far
SCHEMA
{
"type": "object",
"properties": {
"personalizedCommandType": {
"title": "Personalized Command Type",
"type": "string"
},
"commands": {
"type": "array",
"title": "Actions",
"items": {
"type": "object",
"properties": {
"commandType": {
"title": "Command Type",
"type": "string",
"enum": [
"REST",
"SSH"
]
}
}
}
}
}
}
FORM
[
{
"type": "help",
"helpvalue": "<h5>Command</h5>"
},
{
"key":"personalizedCommandType"
},
{
"title":"Command",
"key": "commands",
"items": [
"commands[].commandType"
]
}
]
One can test this code here: http://schemaform.io/examples/bootstrap-example.html . Just copy and paste in my code.
Question
As one can see, the code I have now has a field with Personalized Command Type and an array of dropdowns with the 2 options SSH and REST. But I want to drop to also contain the value of the Personalized Command Type once the user has entered it.
NOTE
copyValueTo does not seem to have the functionality that I want given that it can only change values in the model, but I want it to change the enum array in the schema.
Use the onChange option:
[
{
"type": "help",
"helpvalue": "<h5>Command</h5>"
},
{
"key":"personalizedCommandType"
onChange: "updateSchema(modelValue,form)"
},
{
"title":"Command",
"key": "commands",
"items": [
"commands[].commandType"
]
}
]
Update the Schema:
var defaultEnum = ["REST","SSH"];
$scope.updateSchema = function(modelValue,form) {
var currentEnum = $scope.schema.commands.items.properties.commandType.enum;
angular.copy(defaultEnum, currentEnum);
if (modelValue) {
currentEnum.push(modelValue);
};
$scope.$broadcast('schemaFormRedraw');
};
Currently I am trying to create a swagger file for my software.
Now I would like to create a definition for a timeRange.
My problem is that this array looks like this:
timeRange: {
"2016-01-15T09:00:00.000Z", // this is the start date
"2017-01-15T09:00:00.000Z" // this is the end date
}
How can I create an example value that works out of the box?
It is an "array of strings" with a minimum of two.
"timeRange": {
"type": "array",
"items": {
"type": "string",
"example": "2017-01-15T09:00:00.000Z,2017-01-15T09:00:00.000Z"
}
}
This generates an example like this:
"timeRange": [
"2017-01-15T09:00:00.000Z,2017-01-15T09:00:00.000Z"
]
This example does not work, because it is an array and not an object.
All together:
How can I realize an example value that exists out of two different strings (without a name).
Hope you can help me!
Cheers!
timeRange: {
"2016-01-15T09:00:00.000Z", // this is the start date
"2017-01-15T09:00:00.000Z" // this is the end date
}
is not valid JSON – "timeRange" needs to be enclosed in quotes, and the object/array syntax should be different.
If using the object syntax {}, the values need to be named properties:
"timeRange": {
"start_date": "2016-01-15T09:00:00.000Z",
"end_date": "2017-01-15T09:00:00.000Z"
}
Otherwise timeRange needs to be an [] array:
"timeRange": [
"2016-01-15T09:00:00.000Z",
"2017-01-15T09:00:00.000Z"
]
In the first example ({} object), your Swagger would look as follows, with a separate example for each named property:
"timeRange": {
"type": "object",
"properties": {
"start_date": {
"type": "string",
"format": "date-time",
"example": "2016-01-15T09:00:00.000Z"
},
"end_date": {
"type": "string",
"format": "date-time",
"example": "2017-01-15T09:00:00.000Z"
}
},
"required": ["start_date", "end_date"]
}
In case of an [] array, you can specify an array-level example that is a multi-item array:
"timeRange": {
"type": "array",
"items": {
"type": "string",
"format": "date-time"
},
"example": [
"2016-01-15T09:00:00.000Z",
"2017-01-15T09:00:00.000Z"
]
}
In my data, I have two fields that I want to use as an index together. They are sensorid (any string) and timestamp (yyyy-mm-dd hh:mm:ss).
So I made an index for these two using the Cloudant index generator. This was created successfully and it appears as a design document.
{
"index": {
"fields": [
{
"name": "sensorid",
"type": "string"
},
{
"name": "timestamp",
"type": "string"
}
]
},
"type": "text"
}
However, when I try to make the following query to find all documents with a timestamp newer than some value, I am told there is no index available for the selector:
{
"selector": {
"timestamp": {
"$gt": "2015-10-13 16:00:00"
}
},
"fields": [
"_id",
"_rev"
],
"sort": [
{
"_id": "asc"
}
]
}
What have I done wrong?
It seems to me like cloudant query only allows sorting on fields that are part of the selector.
Therefore your selector should include the _id field and look like:
"selector":{
"_id":{
"$gt":0
},
"timestamp":{
"$gt":"2015-10-13 16:00:00"
}
}
I hope this works for you!
I'm writing a simple JSON schema and using minItems to validate the number of items in a given array. My schema is as follows:
{
"title": "My Schema",
"type": "object",
"properties": {
"root": {
"type": "array",
"properties": {
"id": {
"type": "string"
},
"myarray": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 4,
"uniqueItems": true
},
"boolean": {
"type": "boolean"
}
},
"required": ["id","myarray","boolean"]
}
},
"required": [
"root"
],
"additionalProperties": false
}
Now I would expect the following JSON to fail validation given the element myarray has nothing in it. But when using this online validator, it passes. Have I done something wrong or is the schema validator I'm using faulty?
{
"root":[
{
"id":"1234567890",
"myarray":[],
"boolean":true
}
]
}
I am not sure why or what it is called, but the correct schema definition for your requirement should be as shown further down.
From what I understand from the JSON Schema definitions, you should declare the properties of an array inside the items declaration. In your schema you where defining properties outside of the array item declaration.
In your schema you have the two different types of array declaration:
Once with just a single object (a string for the "myarray" object)
Once with a complex object (the object name "myComplexType" in the code below)
Have a look at the definitions of both, how they are structured and how they would be interpreted.
The corrected schema:
{
"title": "My Schema",
"type": "object",
"properties": {
"root": {
"type": "array",
"items": { <-- Difference here - "items" instead of "properties"
"type": "object", <-- here - define the array items as a complex object
"title": "myComplexType", <-- here - named for easier referencing
"properties": { <-- and here - now we can define the actual properties of the object
"id": {
"type": "string"
},
"myarray": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 4,
"uniqueItems": true
},
"boolean": {
"type": "boolean"
}
}
},
"required": [
"id",
"myarray",
"boolean"
]
}
},
"required": [
"root"
],
"additionalProperties": false
}
Remove the comments I added with <-- when copying over to your code, added for pointing where there changes are.
As a note, I do however don't understand why the validator didn't give an error for the 'malformed' schema, but might just be that it saw the definition as you had it as additional properties, not entirely sure.
The only thing wrong with your schema is that the root property should have type object instead of array. Because the properties keyword is not defined for arrays, it is ignored. Therefore, the part of the schema you were trying to test was completely ignored even though it was correct.
Here is the relevant passage from the specification
Some validation keywords only apply to one or more primitive types. When the primitive type of the instance cannot be validated by a given keyword, validation for this keyword and instance SHOULD succeed.
http://json-schema.org/latest/json-schema-validation.html#rfc.section.4.1