Creating ruleset for API Governance - mulesoft

I'm trying creaiting ruleset for RAML to check if there are the example for responses and description for uriParams.
/example:
/{uriParams}:
get:
uriParameters:
uriParams:
description: Example description uriParams
body:
application/json:
example: !include examples.example.json
And for this I create two ruleset but it's not working:
response-example:
message: Provide example.
targetClass: apiContract.Example
and:
- propertyConstraints:
apiContract.returns:
atLeast:
count: 1
validation:
propertyConstraints:
apiContract.structuredValue:
pattern: "^!include"
uri-descriptions:
message: Provide descriptions.
targetClass: apiContract.Parameter
if:
propertyConstraints:
apiContract.Parameter:
pattern: uri
then:
propertyConstraints:
core.description:
minCount: 1

Checking examples is non-trivial. AMF (RAML parser used for Governance) resolves them to the schema of the parameter, payload, header, etc. Just clarifying with an example: imagine that if you have a response of type Person and an example there, the example will be transferred from the response to the Person type.
Fortunately AMF keeps an annotation called tracked-element that points back to the original parameter, payload, header, etc. where the example was defined. Unfortunately checking that these are the same element must be done with custom Rego code.
Here's the resulting ruleset:
profile: My Ruleset
description: Example ruleset
violation:
- provide-examples-on-payloads
- provide-description-on-parameters
validations:
provide-examples-on-payloads:
message: Always include examples in request and response bodies
targetClass: apiContract.Payload
rego: |
schema = find with data.link as $node["http://a.ml/vocabularies/shapes#schema"]
nested_nodes[examples] with data.nodes as object.get(schema, "http://a.ml/vocabularies/apiContract#examples", [])
examples_from_this_payload = { element |
example = examples[_]
sourcemap = find with data.link as object.get(example, "http://a.ml/vocabularies/document-source-maps#sources", [])
tracked_element = find with data.link as object.get(sourcemap, "http://a.ml/vocabularies/document-source-maps#tracked-element", [])
tracked_element["http://a.ml/vocabularies/document-source-maps#value"] = $node["#id"]
element := example
}
$result := (count(examples_from_this_payload) > 0)
provide-description-on-parameters:
message: Always include examples in URI parameters
targetClass: apiContract.Parameter
if:
propertyConstraints:
apiContract.binding:
in: ['path']
then:
propertyConstraints:
core.description:
minCount: 1

Related

How to display complex objects in multipart/form-data requests in Swagger UI?

I am using OpenAPI 3.0.1 and I am trying to send a request as multipart/form-data. The request body is defined as follows:
requestBody:
content:
multipart/form-data: # Media type
schema: # Request payload
type: object
properties: # Request parts
media: # Part 1 (string value)
type: string
address: # Part2 (object)
type: object
properties:
street:
type: string
city:
type: string
profileImage: # Part 3 (an image)
type: string
format: binary
Here, the address field is an object. Swagger UI shows the address field itself, but does not show its properties street, city. Why is that?
This issue was fixed in Swagger UI v. 3.51.0. Make sure you're using the latest version.

google cloud endpoints body array

We have a rest API that is written in Java (hosted in Wildfly). Our service is running in kubernetes (GKE). We want to leverage Cloud Endpoints to track usage and responsiveness of our API. The API is not new, we have been shipping software that interacts with it for years. It is also quite large (thousands of public methods). We have Swagger documentation for our API, and have no validation errors. When I try to deploy our Swagger using:
gcloud beta service-management deploy swagger.yaml
It is not successful. I get the following error repeated 237 times:
ERROR: unknown location: http: body field path 'body' must be a non-repeated message.
I have tracked it down to 237 methods that include a json array in a body parameter. In our API these are methods that either accept or return a list of objects.
Is there any way I can get this accepted by service-management deploy? Changing our API isn't an option, but we would really like to be able to use endpoints.
For example, this method signature:
#PUT
#Path ("/foobars/undelete")
#Consumes (MediaType.APPLICATION_JSON)
#Produces (MediaType.APPLICATION_JSON)
#ApiOperation (value = "Undelete foobars")
#ApiResponses (value =
{
#ApiResponse (
code = 200,
message = "foobars undeleted",
response = FooBar.class,
responseContainer = "List"
) , #ApiResponse (
code = 206,
message = "Not all foobars undeleted",
response = FooBar.class,
responseContainer = "List"
) , #ApiResponse (
code = 410,
message = "Not found"
) , #ApiResponse (
code = 500,
message = "Server Error"
)
})
public Response undeleteFooBars (#ApiParam (value = "FooBar ID List") List<UUID> entityIds)
generates this swagger snippet:
"/foobars/undelete":
put:
tags:
- foo
summary: Undelete FooBars
description: ''
operationId: undeleteFooBars
consumes:
- application/json
produces:
- application/json
parameters:
- in: body
name: body
description: FooBar ID List
required: false
schema:
type: array
items:
type: string
format: uuid
responses:
'200':
description: Foo Bars undeleted
schema:
type: array
items:
"$ref": "#/definitions/FooBar"
'206':
description: Not all FooBars undeleted
schema:
type: array
items:
"$ref": "#/definitions/FooBar"
'410':
description: Not found
'500':
description: Server Error
I have had the exact same problem with Endpoints, where it does not seem to think that passing an array of objects is valid as a body parameter. I worked around this by just using a generic object and a decent description. The description will not programatically fix anything, but using a generic object allows Endpoints to work and the description gives information to the consumer of the API for what is expected.
parameters:
- in: body
name: body
description: Array of FooBar objects
required: false
schema:
type: object
This seems like an oversight on the part of the Endpoints team IMHO as using an array of objects in the body fits fine within the OpenApi spec and works with tools like http://editor.swagger.io/
Edit: I should also add that it is generally bad practice to use just a raw array as a request body or response body as it can cause a contract breaking change if additional properties are desired in the future, like say a count or pagination information.
If this is an existing API and you are just documenting the existing contract, then this solution will work to get the job done, but if you are designing a new API, then a better definition would be:
parameters:
- in: body
name: body
description: All the FooBar objects
required: false
schema:
type: object
properties:
items:
type: array
items:
$ref: '#/definitions/FooBarResource'
Since this could later be extended to add additional properties like
parameters:
- in: body
name: body
description: All the FooBar objects
required: false
schema:
type: object
properties:
count:
type: integer
description: The total count of resources
callbackUrl:
type: string
description: The URL to trigger once creation is complete
items:
type: array
items:
$ref: '#/definitions/FooBarResource'
description: The resources to create
You can do better than a plain object. Yyou can specify an array as the the value of an object with a single key. This way you preserve your type information:
parameters:
- description: "Your items to add"
in: body
name: listings
required: true
schema:
type: object
properties:
payload:
type: array
maxItems: 1000
minItems: 1
$ref: "#/definitions/listing"
It's ugly but at least it documents whatever the model you are passing in should look like.

angularjs trying to understand and save a resource state with params

here's the relevant snippet of code:
$scope.submit = function() {
console.log(this);
weekly_habits = $resource('/api/users/:user_id/weekly_habits/:id/', {user_id: '#user'});
entry = weekly_habits.save({name: $scope.newAccomp, count: 0});
$scope.accomplishments.unshift(entry);
$scope.newAccomp = '';
}
my error is that no route matches /api/users/weekly_habits... My project is built in rails and I don't understand what this line means {user_id: '#user'}. Where is this #user object supposed to be? It would be much appreciated if someone explained what's going on here.
Thanks!
From the documentation:
If the parameter value is prefixed with # then the value of that parameter is extracted from the data object (useful for non-GET operations).
And later on they have an example
var User = $resource('/user/:userId', {userId:'#id'});
So the second argument to the resource service are the default parameters. In this case, by default, the userId will be extracted from the resource object you call a method on.
Previously I had an example of a GET operation, which doesn't really make sense in this case. But imagine you were doing a POST ($save) request with a data object. Angular would automatically extract the userId from the data object and inject it into the URL for you. You don't have to manually specify the value for the URL.
More info can be found on the doc page.
http://docs.angularjs.org/api/ngResource.$resource
Quoting the documentation: "If the parameter value is prefixed with # then the value of that parameter is extracted from the data object (useful for non-GET operations)."
The '#user' tells Angular to extract the value of user_id from the 'user' property of the object. So if you call .save() on an object that has an 'user' property of 'abc123', then the :user_id in your URL will be replaced with 'abc123'.

GAE: Error when downloading data, if ndb.KeyProperty(repeated=True)

I am creating the bulkloader.yaml automatically from my existing schema and have trouble downloading my data due the repeated=True of my KeyProperty.
class User(ndb.Model):
firstname = ndb.StringProperty()
friends = ndb.KeyProperty(kind='User', repeated=True)
The automatic created bulkloader looks like this:
- kind: User
connector: csv
connector_options:
# TODO: Add connector options here--these are specific to each connector.
property_map:
- property: __key__
external_name: key
export_transform: transform.key_id_or_name_as_string
- property: firstname
external_name: firstname
# Type: String Stats: 2 properties of this type in this kind.
- property: friends
external_name: friends
# Type: Key Stats: 2 properties of this type in this kind.
import_transform: transform.create_foreign_key('User')
export_transform: transform.key_id_or_name_as_string
This is the error message I am getting:
google.appengine.ext.bulkload.bulkloader_errors.ErrorOnTransform: Error on transform. Property: friends External Name: friends. Code: transform.key_id_or_name_as_string Details: 'list' object has no attribute 'to_path'
What can I do please?
Possible Solution:
After Tony's tip I came up with this:
- property: friends
external_name: friends
# Type: Key Stats: 2 properties of this type in this kind.
import_transform: myfriends.stringToValue(';')
export_transform: myfriends.valueToString(';')
myfriends.py
def valueToString(delimiter):
def key_list_to_string(value):
keyStringList = []
if value == '' or value is None or value == []:
return None
for val in value:
keyStringList.append(transform.key_id_or_name_as_string(val))
return delimiter.join(keyStringList)
return key_list_to_string
And this works! The encoding is in Unicode though: UTF-8. Make sure to open the file in LibreOffice as such or you would see garbled content.
The biggest challenge is import. This is what I came up with without any luck:
def stringToValue(delimiter):
def string_to_key_list(value):
keyvalueList = []
if value == '' or value is None or value == []:
return None
for val in value.split(';'):
keyvalueList.append(transform.create_foreign_key('User'))
return keyvalueList
return string_to_key_list
I get the error message:
BadValueError: Unsupported type for property friends: <type 'function'>
According to Datastore viewer, I need to create something like this:
[datastore_types.Key.from_path(u'User', u'kave#gmail.com', _app=u's~myapp1')]
Update 2:
Tony you are to be a real expert in Bulkloader. Thanks for your help. Your solution worked!
I have moved my other question to a new thread.
But one crucial problem that appears is that, when I create new users I can see my friends field shown as <missing> and it works fine.
Now when I use your solution to upload the data, I see for those users without any friend entries a <null> entry. Unfortunately this seems to break the model since friends can't be null.
Changing the model to reflect this, seems to be ignored.
friends = ndb.KeyProperty(kind='User', repeated=True, required=False)
How can I fix this please?
update:
digging further into it:
when the status <missing> is shown in the data viewer, in code it shows friends = []
However when I upload the data via csv I get a <null>, which translates to friends = [None]. I know this, because I exported the data into my local data storage and could follow it in code. Strangely enough if I empty the list del user.friends[:], it works as expected. There must be a beter way to set it while uploading via csv though...
Final Solution
This turns out to be a bug that hasn't been resolved since over one year.
In a nutshell, even though there is no value in csv, because a list is expected, gae makes a list with a None inside. This is game breaking, since retrieval of such a model ends up in an instant crash.
Adding a post_import_function, which deletes the lists with a None inside.
In my case:
def post_import(input_dict, instance, bulkload_state_copy):
if instance["friends"] is None:
del instance["friends"]
return instance
Finally everything works as expected.
When you are using repeated properties and exporting to a CSV, you should be doing some formatting to concatenate the list into a CSV understood format. Please check the example here on import/export of list of dates and hope it can help you.
EDIT : Adding suggestion for import transform from an earlier comment to this answer
For import, please try something like:
`from google.appengine.api import datastore
def stringToValue(delimiter):
def string_to_key_list(value):
keyvalueList = []
if value == '' or value is None or value == []: return None
for val in value.split(';'):
keyvalueList.append(datastore.Key.from_path('User', val))
return keyvalueList
return string_to_key_list`
if you have id instead of name , add like val = int(val)

Griffon checkbox binding won't work

i am trying the following griffon code
on model:
#Bindable boolean hello1=false
on view:
checkBox(id:1,text: 'hello1', constraints:'wrap',selected:bind(target: model, targetProperty:'hello1'))
but it does say
ERROR org.codehaus.griffon.runtime.builder.UberBuilder - An error occurred while building test.TestView#1132e76
groovy.lang.MissingMethodException: No signature of method: java.lang.Object.setVariable() is applicable for argument types: (java.util.Collections$EmptyMap, java.util.Arrays$ArrayList) values: [[:], [1, javax.swing.JCheckBox[,0,0,0x0,invalid,alignmentX=0.0,alignmentY=0.5,border=javax.swing.plaf.synth.SynthBorder#b101cf,flags=288,maximumSize=,minimumSize=,preferredSize=,defaultIcon=,disabledIcon=,disabledSelectedIcon=,margin=javax.swing.plaf.InsetsUIResource[top=0,left=0,bottom=0,right=0],paintBorder=false,paintFocus=true,pressedIcon=,rolloverEnabled=true,rolloverIcon=,rolloverSelectedIcon=,selectedIcon=,text=]]]8-mar-2012 12.03.41 groovy.util.FactoryBuilderSupport createNode
AVVERTENZA: Could not find match for name 'setVariable'
i dont get what's the deal, i copied that from working examples on internet....
Use a String instead of a number for the value of the id: property, like this
checkBox(id: 'c1', ...)

Resources