First, it would be awesome if Apigee had a built in policy to validate requests against an OAS, like for XSDs, but currently it’s not available. It would be even more awesome, if only one validation policy was required, since the OAS already has all the definitions.
Even though it’s not built in, we can get SO CLOSE with just one simple Javascript policy using tv4, and the OAS. It relies on the fact that the OAS "operationId" === proxy "current.flow.name" on conditional flows.
The OAS fragment
... "paths": { "/nodes": { "post": { "operationId": "createNode", "parameters": [{ "name": "node", "in": "body", "required": true, "schema": { "$ref": "#/definitions/NodeCreateRequest" } }], ...
Create a Javascript policy to validate the requests using the OAS as an included resource.
The JS-ValidateRequest Policy
<Javascript async="false" continueOnError="false" enabled="true" timeLimit="200" name="JS-ValidateRequest"> <DisplayName>JS-ValidateRequest</DisplayName> <Properties/> <IncludeURL>jsc://oas.js</IncludeURL> <IncludeURL>jsc://tv4-min.js</IncludeURL> <ResourceURL>jsc://ValidateRequest.js</ResourceURL> </Javascript>
The ValidateRequest.js script is pretty straight forward
try { var verb = context.getVariable('request.verb'); var pathsuffix = context.getVariable('proxy.pathsuffix'); var flowname = context.getVariable('current.flow.name'); var schema = getMessageSchema( flowname ); if( schema === undefined || schema === null ) { throw "Missing schema definition for: " + verb + " " + pathsuffix; } else { var bodyContent = context.getVariable('request.content'); var body = JSON.parse(bodyContent); tv4.addSchema('/schema', oas); var result = tv4.validateMultiple(body, schema); // A missing schema validates to true, but we want that to be an error // Override missing entry with full schema value if( result.missing[0] ) { result.errors[0] = {"schema":schema}; throw "Schema definition not found" + JSON.stringify(result.errors); } else if( result.valid === false ) { throw "Validation failed for: " + verb + " " + pathsuffix + ": " + JSON.stringify(result.errors); } } } catch( err ) { // This raises fault named "ScriptExecutionFailed", // and sets errorMessage to the validation result context.setVariable('javascript.errorMessage', err); throw err; } function getMessageSchema( flowname ) { // Find the operationId that matches the flowname // Return the schema from the parameter that is "in" "body" // More efficient than using jsonPath var paths = oas.paths; for ( var path in paths ) { var verbs = paths[path]; for( var verb in verbs ) { if( verbs[verb].operationId === flowname ) { var params = verbs[verb].parameters; for ( var param in params ) { if( params[param].hasOwnProperty( 'in' ) && params[param].in === 'body' ) { return params[param].schema; } } } } } return undefined; }
SO CLOSE: As you can see, we almost have a “standard” policy. All we need is access to the OAS! We know that when a proxy is created using an OAS, the spec is referenced via the association.js resource, but that’s just a link to the spec, not the entire spec. AFAIK, there is no way to access the full spec at runtime. If there was, we wouldn’t have to copy and paste the OAS into a resource file.
Even so, this single Javascript policy can now be placed on any conditional flow where validation needs to be performed.
See the attached proxy and Postman collection.
Thanks for the details on validating JSON schema draft v4 however, I will have to validate against JSON Schema draft v6 so is there something like tv6 similar to tv4 ?
Hey @Kurt Kanaskie - great work.
I would like to propose a solution to the problem you mentioned at the end of the article "... we wouldn't have to copy and paste the OAS into a resource file".
The solution that I have is to expose the OA description (in YAML or JSON) from proxies, e.g. /products/v1/description.yaml for Product API v1. This url structure should be standardised across all APIs so they are well known.
There are three benefits of this approach:
This proposal promotes /description.yaml as the single source for OA descriptions which all other processes rely on. So when description changes, that API team will just need to modify the response of this endpoint and all other processes will get updated automatically. By exposing the description from the API itself, we are also enforcing the fact that the API team is responsible for exposure and maintenance of their OA description; just like any other resource of their API.
Hi, I'm coming back to the API world, and I'm trying to understand why the validation in the proxy/middleware itself.
Although I understand the value of comparing/validating the JSON input against a schema,
it sounds it might affect performance? also, more options to fail?
I would like to know your thoughts, I might find this check super useful first time after integrating with apps, but then not so much
any thoughts ?
Hey @Ozan Seymen,
That's a neat idea, I've often considered how to make the OAS available via an API as a standard design approach, I imagined GET /products/v1/specification.json.
In any case, challenge is to incorporate into CI / CD. Since I typically associate the OAS with the proxy code, would need to come up with a scheme to create the AssignMessage from the actual spec and not maintain a separate copy. Similar problem with creating the JS resource.
Its typical to validate requests during integration testing and then turn off in production.
However, there are times when you must ensure a valid message (e.g. minimum required fields) before sending to a backend system.
thank you, I guess then it becomes a question of "where does it make sense to validate this? " FE . | API | BackEnd not sure what is the best pattern. from my experience, the design is to make the other team validate 😉 . but I wonder if this should be one of these cases of "validate it in API" since it might get used by multiple parties
Very useful article. Agree that it should be an enhancement on Apigee to add its own policy to do this.
Very useful article!Is there a way to implement this as a shared flow where the oas.js file gets picked up at the proxy level?
Do you know if there is a way that you can allow null as value that is defined as string in the OpenAPI spec?
I'm using the approach you suggest using tv4 with OpenApi.
exampleField:
type: string
request sample:
{
"exampleField": null
}
tv4 is returning "Invalid type: null (expected string)", but for me this request is valid and because I haven't configured as required I want to accept it, same way as if it was not coming.
Have you tried to define the element as an array of types?
"type": [ "null", "string" ],
Something like this?
{ "type": "object", "properties": { "exampleField": { "type": [ "null", "string" ], "maxLength": 16 } }, "required": [ "exampleField" ] }
Hi @Kurt Googler Kanaskie, the current implementation works fine only for required parameters defined in schema ex:
definitions:
sms:
type: object
required:
- smsType
properties:
smsType:
type: string
title: smsType
My client wants me to implement schema validation, for the whole JSON schema. is it possible?.
Very good article,
I am also exporing the tv4 for validating the JSON. I need some understanding on couple of the items -
What is the use of -
"additionalProperties": true
Its part of the JSON Schema spec, for more details see: https://json-schema.org/understanding-json-schema/reference/object.html
I understand this validates json request against OAS specified in json . Is there a way to handle OAS specified in yaml ?
I am planning to use this approach where I want to validate mainly POST request against swagger because every click to backend cost money.
I liked the idea of creating seperate proxy just for specs and calling that during runtime to fetch spec and validate against it.
I provided an updated solution that supports OAS 2 and 3, plus validates required headers and query params here: https://community.apigee.com/articles/88441/validate-json-requests-using-openapi-spec-20-or-30.html