AJV validator and custom or user-friendly error message - ajv

I have the following schema and json to validate using ajv. I am developing a REST API that takes a JSON and gets validated against the schema and it returns the error (400- with the ajv error) or (200 - when successfully validated)
const schema = {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": [ "countries" ],
"definitions": {
"europeDef": {
"type": "object",
"required": ["type"],
"properties": { "type": {"const": "europe"} }
},
"asiaDef": {
"type": "object",
"required": ["type"],
"properties": { "type": {"const": "asia"} }
}
},
"properties": {
"countries": {
"type": "array",
"items": {
"oneOf":[
{ "$ref": "#/definitions/europeDef" },
{ "$ref": "#/definitions/asiaDef"}
]
}
}
}
}
const data = {
"countries":[
{"type": "asia1"},
{"type": "europe1"}
]
}
const isValid = ajv.validate(schema, data); //schema, data
if(! isValid){
console.log(ajv.errors);
}
and the error is:
[ { keyword: 'const',
dataPath: '/countries/0/type',
schemaPath: '#/definitions/europeDef/properties/type/const',
params: { allowedValue: 'europe' },
message: 'should be equal to constant' },
{ keyword: 'const',
dataPath: '/countries/0/type',
schemaPath: '#/definitions/asiaDef/properties/type/const',
params: { allowedValue: 'asia' },
message: 'should be equal to constant' },
{ keyword: 'oneOf',
dataPath: '/countries/0',
schemaPath: '#/properties/countries/items/oneOf',
params: { passingSchemas: null },
message: 'should match exactly one schema in oneOf' },
{ keyword: 'const',
dataPath: '/countries/1/type',
schemaPath: '#/definitions/europeDef/properties/type/const',
params: { allowedValue: 'europe' },
message: 'should be equal to constant' },
{ keyword: 'const',
dataPath: '/countries/1/type',
schemaPath: '#/definitions/asiaDef/properties/type/const',
params: { allowedValue: 'asia' },
message: 'should be equal to constant' },
{ keyword: 'oneOf',
dataPath: '/countries/1',
schemaPath: '#/properties/countries/items/oneOf',
params: { passingSchemas: null },
message: 'should match exactly one schema in oneOf' } ]
I know why the error is appearing (reason: as I have used 'asia1' & 'europe1' and it is not conforming the schema standard)
My question is, as I have derived this schema so I can pretty much understand the error. But for a third person it would definitely take some time to figure it out (and it may take more time, if the schema/errors are more complex).
If I returned that whole error message as a response as it is, it will be more complex error message to understand and to present to the enduser.
So, Is there any way by which I can provide more meaningful & user friendly error message to understand ?
ex: Invalid countries values found in JSON
I have checked: ajv-errors, better-ajv-errors but they are not providing the exact way I want?
Can someone suggest how to do that in a more user friendly way or any alternative mechanism?

I am using below code for generating the human readable error message
let msg: string = "Wrong body" // fallback error message;
if (errors && errors.length > 0) {
const error = errors[0];
msg = `${error.instancePath} ${error.message}`;
}
res.status(4xx).json({
errorMsg: msg,
});
I am using below dependencies to generate the validate functions in runtime with the below code
"dependencies": {
"ajv": "^8.11.0",
... // other dependencies
}
"devDependencies": {
"ajv-cli": "^5.0.0"
}
Below code gets run before project build and hence creating runtime generated validation files
const ajv = new Ajv({
schemas: schemas, // list of parsed json *.json schemas
code: { source: true, esm: true },
});
let moduleCode = standaloneCode(ajv);
Below is the few examples of error messaged diaplayed
Case of missing property:
"/items/0/price must have required property 'currency_code'"
Case of additional property: "/address must NOT have additional properties"
Case when quantity is fraction(allowed is +ve number): "/items/0/quantity must be integer"
Case when quantity is -ve(allowed is +ve number): "/items/0/quantity must be >= 1"
Case when passed value is not allowed(case of the enum): /items/0/price/currency_code must be equal to one of the allowed values

Related

How to avoid specifying every leaf attribute in graphql query with gatsby

I uploaded some json to my graphql using gatsby-transformer-json and gatsby-source-filesystem.
I would like to request an object of that data, which contains subfields.
When I try to make this request, I receive the error
Field \"sections\" of type \"DataJsonSections\" must have a selection of subfields. Did you mean \"sections { ... }\"?
Below is my json file. I am trying to request the section called "Sections". I would really like to find a way, so that I don't have to specify spiel and abbr for every section.
My json file
{
"slogan": "some slogan",
"drawIn": "Something to draw people in",
"fullName": "My full title",
"shortName": "My title",
"metadata": {
"desc": "...........",
"title": "............"
},
"sections": {
"about": {
"spiel": "..........."
},
"web-design": {
"abbr": "Websites",
"spiel": ".............."
},
"app-design": {
"abbr": "Apps",
"spiel": "..........."
},
"seo": {
"spiel": ".........."
},
"contact": {
"title": "Contact",
"subtitle": "Contact subtitle",
}
}
}
This is the request I would like to make, but I receive the error above when I do
query MyQuery {
allDataJson {
edges {
node {
sections
}
}
}
}
Extra
Within my gatsby-config.js(This is how I'm using gatsby-transformer-json and gatsby-source-filesystem)
`gatsby-transformer-json`,
{
resolve: `gatsby-source-filesystem`,
options: {
name: `data`,
path: `${__dirname}/src/data`
}
},
Field "sections" of type "DataJsonSections" must have a selection of subfields

Update nested subdocuments in MongoDB with arrayFilters

I need to modify a document inside an array that is inside another array.
I know MongoDB doesn't support multiple '$' to iterate on multiple arrays at the same time, but they introduced arrayFilters for that.
See: https://jira.mongodb.org/browse/SERVER-831
MongoDB's sample code:
db.coll.update({}, {$set: {“a.$[i].c.$[j].d”: 2}}, {arrayFilters: [{“i.b”: 0}, {“j.d”: 0}]})
Input: {a: [{b: 0, c: [{d: 0}, {d: 1}]}, {b: 1, c: [{d: 0}, {d: 1}]}]}
Output: {a: [{b: 0, c: [{d: 2}, {d: 1}]}, {b: 1, c: [{d: 0}, {d: 1}]}]}
Here's how the documents are set:
{
"_id" : ObjectId("5a05a8b7e0ce3444f8ec5bd7"),
"name" : "support",
"contactTypes" : {
"nonWorkingHours" : [],
"workingHours" : []
},
"workingDays" : [],
"people" : [
{
"enabled" : true,
"level" : "1",
"name" : "Someone",
"_id" : ObjectId("5a05a8c3e0ce3444f8ec5bd8"),
"contacts" : [
{
"_id" : ObjectId("5a05a8dee0ce3444f8ec5bda"),
"retries" : "1",
"priority" : "1",
"type" : "email",
"data" : "some.email#email.com"
}
]
}
],
"__v" : 0
}
Here's the schema:
const ContactSchema = new Schema({
data: String,
type: String,
priority: String,
retries: String
});
const PersonSchema = new Schema({
name: String,
level: String,
priority: String,
enabled: Boolean,
contacts: [ContactSchema]
});
const GroupSchema = new Schema({
name: String,
people: [PersonSchema],
workingHours: { start: String, end: String },
workingDays: [Number],
contactTypes: { workingHours: [String], nonWorkingHours: [String] }
});
I need to update a contact. This is what I tried using arrayFilters:
Group.update(
{},
{'$set': {'people.$[i].contacts.$[j].data': 'new data'}},
{arrayFilters: [
{'i._id': mongoose.Types.ObjectId(req.params.personId)},
{'j._id': mongoose.Types.ObjectId(req.params.contactId)}]},
function(err, doc) {
if (err) {
res.status(500).send(err);
}
res.send(doc);
}
);
The document is never updated and I get this response:
{
"ok": 0,
"n": 0,
"nModified": 0
}
What am I doing wrong?
So the arrayFilters option with positional filtered $[<identifier>] does actually work properly with the development release series since MongoDB 3.5.12 and also in the current release candidates for the MongoDB 3.6 series, where this will actually be officially released. The only problem is of course is that the "drivers" in use have not actually caught up to this yet.
Re-iterating the same content I have already placed on Updating a Nested Array with MongoDB:
NOTE Somewhat ironically, since this is specified in the "options" argument for .update() and like methods, the syntax is generally compatible with all recent release driver versions.
However this is not true of the mongo shell, since the way the method is implemented there ( "ironically for backward compatibility" ) the arrayFilters argument is not recognized and removed by an internal method that parses the options in order to deliver "backward compatibility" with prior MongoDB server versions and a "legacy" .update() API call syntax.
So if you want to use the command in the mongo shell or other "shell based" products ( notably Robo 3T ) you need a latest version from either the development branch or production release as of 3.6 or greater.
All this means is that the current "driver" implementation of .update() actually "removes" the necessary arguments with the definition of arrayFilters. For NodeJS this will be addressed in the 3.x release series of the driver, and of course "mongoose" will then likely take some time after that release to implement it's own dependencies on the updated driver, which would then no longer "strip" such actions.
You can however still run this on a supported server instance, by dropping back to the basic "update command" syntax usage, since this bypassed the implemented driver method:
const mongoose = require('mongoose'),
Schema = mongoose.Schema,
ObjectId = mongoose.Types.ObjectId;
mongoose.Promise = global.Promise;
mongoose.set('debug',true);
const uri = 'mongodb://localhost/test',
options = { useMongoClient: true };
const contactSchema = new Schema({
data: String,
type: String,
priority: String,
retries: String
});
const personSchema = new Schema({
name: String,
level: String,
priority: String,
enabled: Boolean,
contacts: [contactSchema]
});
const groupSchema = new Schema({
name: String,
people: [personSchema],
workingHours: { start: String, end: String },
workingDays: { type: [Number], default: undefined },
contactTypes: {
workingHours: { type: [String], default: undefined },
contactTypes: { type: [String], default: undefined }
}
});
const Group = mongoose.model('Group', groupSchema);
function log(data) {
console.log(JSON.stringify(data, undefined, 2))
}
(async function() {
try {
const conn = await mongoose.connect(uri,options);
// Clean data
await Promise.all(
Object.entries(conn.models).map(([k,m]) => m.remove() )
);
// Create sample
await Group.create({
name: "support",
people: [
{
"_id": ObjectId("5a05a8c3e0ce3444f8ec5bd8"),
"enabled": true,
"level": "1",
"name": "Someone",
"contacts": [
{
"type": "email",
"data": "adifferent.email#example.com"
},
{
"_id": ObjectId("5a05a8dee0ce3444f8ec5bda"),
"retries": "1",
"priority": "1",
"type": "email",
"data": "some.email#example.com"
}
]
}
]
});
let result = await conn.db.command({
"update": Group.collection.name,
"updates": [
{
"q": {},
"u": { "$set": { "people.$[i].contacts.$[j].data": "new data" } },
"multi": true,
"arrayFilters": [
{ "i._id": ObjectId("5a05a8c3e0ce3444f8ec5bd8") },
{ "j._id": ObjectId("5a05a8dee0ce3444f8ec5bda") }
]
}
]
});
log(result);
let group = await Group.findOne();
log(group);
} catch(e) {
console.error(e);
} finally {
mongoose.disconnect();
}
})()
Since that sends the "command" directly through to the server, we see the expected update does in fact take place:
Mongoose: groups.remove({}, {})
Mongoose: groups.insert({ name: 'support', _id: ObjectId("5a06557fb568aa0ad793c5e4"), people: [ { _id: ObjectId("5a05a8c3e0ce3444f8ec5bd8"), enabled: true, level: '1', name: 'Someone', contacts: [ { type: 'email', data: 'adifferent.email#example.com', _id: ObjectId("5a06557fb568aa0ad793c5e5") }, { _id: ObjectId("5a05a8dee0ce3444f8ec5bda"), retries: '1', priority: '1', type: 'email', data: 'some.email#example.com' } ] } ], __v: 0 })
{ n: 1,
nModified: 1,
opTime:
{ ts: Timestamp { _bsontype: 'Timestamp', low_: 3, high_: 1510364543 },
t: 24 },
electionId: 7fffffff0000000000000018,
ok: 1,
operationTime: Timestamp { _bsontype: 'Timestamp', low_: 3, high_: 1510364543 },
'$clusterTime':
{ clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 3, high_: 1510364543 },
signature: { hash: [Object], keyId: 0 } } }
Mongoose: groups.findOne({}, { fields: {} })
{
"_id": "5a06557fb568aa0ad793c5e4",
"name": "support",
"__v": 0,
"people": [
{
"_id": "5a05a8c3e0ce3444f8ec5bd8",
"enabled": true,
"level": "1",
"name": "Someone",
"contacts": [
{
"type": "email",
"data": "adifferent.email#example.com",
"_id": "5a06557fb568aa0ad793c5e5"
},
{
"_id": "5a05a8dee0ce3444f8ec5bda",
"retries": "1",
"priority": "1",
"type": "email",
"data": "new data" // <-- updated here
}
]
}
]
}
So right "now"[1] the drivers available "off the shelf" don't actually implement .update() or it's other implementing counterparts in a way that is compatible with actually passing through the necessary arrayFilters argument. So if you are "playing with" a development series or release candiate server, then you really should be prepared to be working with the "bleeding edge" and unreleased drivers as well.
But you can actually do this as demonstrated in any driver, in the correct form where the command being issued is not going to be altered.
[1] As of writing on November 11th 2017 there is no "official" release of MongoDB or the supported drivers that actually implement this. Production usage should be based on official releases of the server and supported drivers only.
I had a similar use case. But my second level nested array doesn't have a key. While most examples out there showcase an example with arrays having a key like this:
{
"id": 1,
"items": [
{
"name": "Product 1",
"colors": ["yellow", "blue", "black"]
}
]
}
My use case is like this, without the key:
{
"colors": [
["yellow"],
["blue"],
["black"]
]
}
I managed to use the arrayfilters by ommiting the label of the first level of the array nest. Example document:
db.createCollection('ProductFlow')
db.ProductFlow.insertOne(
{
"steps": [
[
{
"actionType": "dispatch",
"payload": {
"vehicle": {
"name": "Livestock Truck",
"type": "road",
"thirdParty": true
}
}
},
{
"actionType": "dispatch",
"payload": {
"vehicle": {
"name": "Airplane",
"type": "air",
"thirdParty": true
}
}
}
],
[
{
"actionType": "store",
"payload": {
"company": "Company A",
"is_supplier": false
}
}
],
[
{
"actionType": "sell",
"payload": {
"reseller": "Company B",
"is_supplier": false
}
}
]
]
}
)
In my case, I want to:
Find all documents that have any steps with payload.vehicle.thirdParty=true and actionType=dispatch
Update the actions set payload.vehicle.thirdParty=true only for the actions that have actionType=dispatch.
My first approach was withour arrayfilters. But it would create the property payload.vehicle.thirdParty=true inside the steps with actionType store and sell.
The final query that updated the properties only inside the steps with actionType=dispatch:
Mongo Shell:
db.ProductFlow.updateMany(
{"steps": {"$elemMatch": {"$elemMatch": {"payload.vehicle.thirdParty": true, "actionType": "dispatch"}}}},
{"$set": {"steps.$[].$[i].payload.vehicle.thirdParty": false}},
{"arrayFilters": [ { "i.actionType": "dispatch" } ], multi: true}
)
PyMongo:
query = {
"steps": {"$elemMatch": {"$elemMatch": {"payload.vehicle.thirdParty": True, "actionType": "dispatch"}}}
}
update_statement = {
"$set": {
"steps.$[].$[i].payload.vehicle.thirdParty": False
}
}
array_filters = [
{ "i.actionType": "dispatch" }
]
NOTE that I'm omitting the label on the first array at the update statement steps.$[].$[i].payload.vehicle.thirdParty. Most examples out there will use both labels because their objects have a key for the array. I took me some time to figure that out.

angular-schema-form default, hidden values

I want to build a (section of a) form which produces the following output:
{
...
offers: {
context: "http://schema.org",
minPrice: 3
}
...
}
The catch is, context should always be present - the only field the user gets to manipulate is minPrice. Immediately, a hidden field with a value comes to mind. So here's the schema definition:
$scope.schema = {
...
offers: {
type: 'object',
properties: {
minPrice: {
type: 'number'
}
}
}
...
};
And here the form definition:
$scope.form = [
...
{
key: 'offers',
type: 'fieldset',
items: [
{
key: 'offers.minPrice',
type: 'number'
},
{
key: 'offers.context',
type: 'hidden',
default: 'http://schema.org'
}
]
}
...
];
However, observing the generated model it's obvious the entry context is not present. I have successfully used a combination of type: 'hidden' and default with a tabarray, but I just can't get it right with an object. I am using version 0.8.13 of angular-schema-forms - the latest at the time of this writing.
I'd appreciate any insights, thanks.
You must include context and it's default value within the schema in that version.
I expect that your issue relates to a bug which should have been fixed in the alphas for v1.0.0
It should work with:
Schema
{
"type": "object",
"properties": {
"offers": {
"type": "object",
"properties": {
"minPrice": {
"type": "number"
},
"context": {
"type": "string",
"default": "http://schema.org"
}
}
}
}
}
Form
[
{
"type": "fieldset",
"items": [
{
"key": "offers.minPrice",
"type":"number"
},
{
"key": "offers.context",
"type": "hidden",
"notitle": true
}
]
}
]

search as you type with elasticsearch,angularjs

I am working on search as you type functionality with angularjs and elastic search.I am passing the $viewValue to factory written in angular and it fetches data from angular.Please check code below.
services.factory('instantSearch',['$q', 'esFactory', '$location', function($q, elasticsearch, $location){
return{
instantResult : function(term){
var client = elasticsearch({
// host: $location.host() + ':9200'
host: 'localhost:9200'
});
var deferred = $q.defer();
client.search({
"index": 'stocks',
"type": 'stock',
"body": {
"from" : 0, "size" : 20,
"query": {
"bool":{
"should":[
{
"match_phrase":{
"name": term
}
},
{
"match_phrase":{
"symbol": term
}
},
{
"match":{
"industry": term
}
}
]
}
}
}
}).then(function(result) {
var hits = result.hits.hits;
deferred.resolve(hits);
},
function (err) {
console.trace(err.message);
}, deferred.reject);
return deferred.promise;
}
};
}]);
This code is working fine but the problem is that I get result when input matches complete term in elasticsearch index's field.So I want to implement token analyzer which will match token(ngram - 1,2,3) and provide result on typing of each character.
So to add analyzer code we have to add settings in te elasticserach index as below:
"settings": {
"analysis": {
"filter": {
"autocomplete_filter": {
"type": "edge_ngram",
"min_gram": 1,
"max_gram": 20
}
},
"analyzer": {
"autocomplete": {
"type": "custom",
"tokenizer": "standard",
"filter": [
"lowercase",
"autocomplete_filter"
]
}
}
}
}
But I am not getting the way to pass the argument here.Every example I checked shows output with curl command.How can we mix analyzer with the working code above.
Thanks for help.
Have you added the analyzer to the fields name, symbol and industry in your elastic search mapping?
curl -XPUT 'http://localhost:9200/index/type/_mapping?ignore_conflicts=true' -d'
{
"type": {
"properties": {
"name": {
"type": "string",
"analyzer": "autocomplete"
}
}
}
}'
Use ignore_conflicts=true without fail.
If you still face issues, then you might have to create a new index, add analyzer and filter to setting, create the desired mapping and then upload the data again.

Loopback, AngularJS and validation

I followed this tutorial to create a project with Loopback and AngularJs. https://github.com/strongloop/loopback-example-angular
Now, I have an application with:
HTML files (with Bootstrap)
AngularJS controllers
AngularJS service (generated with syntax lb-ng server/server.js client/js/services/lb-services.js)
Model (located in ./common folder)
MongoDB backend
The model "Device" is defined in ./common/models/device.js
module.exports = function(Device) {
};
And in ./common/models/device.json
{
"name": "Device",
"base": "PersistedModel",
"idInjection": true,
"properties": {
"name": {
"type": "string",
"required": true
},
"description": {
"type": "string",
"required": true
},
"category": {
"type": "string",
"required": true
},
"initialDate": {
"type": "date"
},
"initialPrice": {
"type": "number",
"required": true
},
"memory": {
"type": "number"
}
},
"validations": [],
"relations": {},
"acls": [],
"methods": []
}
In the "AddDeviceController", I have an initialization part with:
$scope.device = new DeviceToBuy({
name: '',
description: '',
category: '',
initialPrice: 0,
memory: 8
initialDate: Date.now()
});
And I am able to save the $scope.device when executing the following method:
$scope.save = function() {
Device.create($scope.device)
.$promise
.then(function() {
console.log("saved");
$scope.back(); // goto previous page
}, function (error) {
console.log(JSON.stringify(error));
});
}
When everything is valid, the model is saved in the backend. If something is not valid in the $scope.device, I receive an error from my backend. So everything is working fine.
Now, I would like to use the model to perform client-side validation before sending my model to the backend and put some "error-class" on the bootstrap controls.
I tried something in the $scope.save function before sending to the backend:
if ($scope.device.isValid()) {
console.log("IsValid");
} else {
console.log("Not Valid");
}
But I get an exception "undefined is not a function" --> isValid() doesn't exist.
And I cannot find any example on how to execute this client-side validation.
LoopBack models are unopinionated and therefore do not provide client side validation out-of-box. You should use Angular validation mechanisms before calling $save.

Resources