Using the OpenAPI Spec to validate JSON requests

First, it would be awesome if Apigee had a built in policy to validate requests against an OAS, like for XSDs, but currently it’s not available. It would be even more awesome, if only one validation policy was required, since the OAS already has all the definitions.

Even though it’s not built in, we can get SO CLOSE with just one simple Javascript policy using tv4, and the OAS. It relies on the fact that the OAS "operationId" === proxy "current.flow.name" on conditional flows.

The OAS fragment

...
"paths": {
        "/nodes": {
            "post": {
                "operationId": "createNode",
                "parameters": [{
                    "name": "node",
                    "in": "body",
                    "required": true,
                    "schema": {
                        "$ref": "#/definitions/NodeCreateRequest"
                    }
                }],  
...    

Create a Javascript policy to validate the requests using the OAS as an included resource.

  1. Create the oas.js resource and JSON object by simply pasting the entire JSON spec into oas.js and assigning it to a variable (e.g. var oas = { ...spec... }; ).
  2. Add the tv4-min.js resource, just copy and paste.

The JS-ValidateRequest Policy

<Javascript async="false" continueOnError="false" enabled="true" timeLimit="200" name="JS-ValidateRequest">
    <DisplayName>JS-ValidateRequest</DisplayName>
    <Properties/>
    <IncludeURL>jsc://oas.js</IncludeURL>
    <IncludeURL>jsc://tv4-min.js</IncludeURL>
    <ResourceURL>jsc://ValidateRequest.js</ResourceURL>
</Javascript>

The ValidateRequest.js script is pretty straight forward

  1. Get the schema for the operationId from the OAS .
  2. Load the OAS schema into tv4
  3. Validate the request
  4. If validation fails set “javascript.errorMessage” variable and throw an error triggering a “ScriptExecutionFailed” fault.
  5. In fault rules, you can differentiate from other script errors by checking the policy name that failed. (e.g. javascript.JS-ValidateRequest.failed === true).
try {
    var verb = context.getVariable('request.verb');
    var pathsuffix = context.getVariable('proxy.pathsuffix');
    var flowname = context.getVariable('current.flow.name');
    var schema = getMessageSchema( flowname );

    if( schema === undefined || schema === null ) {
        throw "Missing schema definition for: " + verb + " " + pathsuffix;
    }
    else {
        var bodyContent = context.getVariable('request.content');
        var body = JSON.parse(bodyContent);
        
        tv4.addSchema('/schema', oas);
        
        var result = tv4.validateMultiple(body, schema);
    
        // A missing schema validates to true, but we want that to be an error
        // Override missing entry with full schema value
        if( result.missing[0] ) {
            result.errors[0] = {"schema":schema};
            throw "Schema definition not found" + JSON.stringify(result.errors);
        }
        else if( result.valid === false ) {
            throw "Validation failed for: " + verb + " " + pathsuffix + ": " + JSON.stringify(result.errors);
        }
    }
}
catch( err ) {
    // This raises fault named "ScriptExecutionFailed", 
    // and sets errorMessage to the validation result
    context.setVariable('javascript.errorMessage', err);
    throw err;
}

function getMessageSchema( flowname ) {
    // Find the operationId that matches the flowname
    // Return the schema from the parameter that is "in" "body"
    // More efficient than using jsonPath
    var paths = oas.paths;
    for ( var path in paths ) {
        var verbs = paths[path];
        for( var verb in verbs ) {
            if( verbs[verb].operationId === flowname ) {
                var params = verbs[verb].parameters;
                for ( var param in params ) {
                    if( params[param].hasOwnProperty( 'in' ) && params[param].in === 'body' ) {
                        return params[param].schema;
                    }
                }
            }
        }
    }
    return undefined;
}

SO CLOSE: As you can see, we almost have a “standard” policy. All we need is access to the OAS! We know that when a proxy is created using an OAS, the spec is referenced via the association.js resource, but that’s just a link to the spec, not the entire spec. AFAIK, there is no way to access the full spec at runtime. If there was, we wouldn’t have to copy and paste the OAS into a resource file.

Even so, this single Javascript policy can now be placed on any conditional flow where validation needs to be performed.

See the attached proxy and Postman collection.

Comments
Not applicable

Thanks for the details on validating JSON schema draft v4 however, I will have to validate against JSON Schema draft v6 so is there something like tv6 similar to tv4 ?

ozanseymen
Staff

Hey @Kurt Kanaskie - great work.

I would like to propose a solution to the problem you mentioned at the end of the article "... we wouldn't have to copy and paste the OAS into a resource file".

The solution that I have is to expose the OA description (in YAML or JSON) from proxies, e.g. /products/v1/description.yaml for Product API v1. This url structure should be standardised across all APIs so they are well known.

There are three benefits of this approach:

  1. Developers can fetch the description from those URLs and use them however they wish. They also could import it to any other editor (swagger UI, etc) or integrate into other processes during their integration testing and CI.
  2. Automated deployment of this API (product api in this case), can fetch the OA description from this URL after proxy deployment and push it to developer portal to render smartdocs.
  3. Your ValidateRequest code can do an http callout to fetch the description (and cache it?) rather than relying on a copy of it in JS resources to decrease maintenance and errors.

This proposal promotes /description.yaml as the single source for OA descriptions which all other processes rely on. So when description changes, that API team will just need to modify the response of this endpoint and all other processes will get updated automatically. By exposing the description from the API itself, we are also enforcing the fact that the API team is responsible for exposure and maintenance of their OA description; just like any other resource of their API.

elartigue
New Member

Hi, I'm coming back to the API world, and I'm trying to understand why the validation in the proxy/middleware itself.

Although I understand the value of comparing/validating the JSON input against a schema,

it sounds it might affect performance? also, more options to fail?

I would like to know your thoughts, I might find this check super useful first time after integrating with apps, but then not so much

any thoughts ?

kurtkanaskie
Staff

Hey @Ozan Seymen,

That's a neat idea, I've often considered how to make the OAS available via an API as a standard design approach, I imagined GET /products/v1/specification.json.

In any case, challenge is to incorporate into CI / CD. Since I typically associate the OAS with the proxy code, would need to come up with a scheme to create the AssignMessage from the actual spec and not maintain a separate copy. Similar problem with creating the JS resource.

kurtkanaskie
Staff

Its typical to validate requests during integration testing and then turn off in production.

However, there are times when you must ensure a valid message (e.g. minimum required fields) before sending to a backend system.

elartigue
New Member

thank you, I guess then it becomes a question of "where does it make sense to validate this? " FE . | API | BackEnd not sure what is the best pattern. from my experience, the design is to make the other team validate 😉 . but I wonder if this should be one of these cases of "validate it in API" since it might get used by multiple parties

Not applicable

Very useful article. Agree that it should be an enhancement on Apigee to add its own policy to do this.

Not applicable

Very useful article!Is there a way to implement this as a shared flow where the oas.js file gets picked up at the proxy level?

egutierrez
New Member

Do you know if there is a way that you can allow null as value that is defined as string in the OpenAPI spec?

I'm using the approach you suggest using tv4 with OpenApi.

exampleField:

type: string

request sample:

{

"exampleField": null

}

tv4 is returning "Invalid type: null (expected string)", but for me this request is valid and because I haven't configured as required I want to accept it, same way as if it was not coming.

Not applicable

Have you tried to define the element as an array of types?

"type": [
        "null",
        "string"
      ],

Something like this?

{
  "type": "object",
  "properties": {
    "exampleField": {
      "type": [
        "null",
        "string"
      ],
      "maxLength": 16
    }
  },
  "required": [
    "exampleField"
  ]
} 
vjosyula
New Member

Hi @Kurt Googler Kanaskie, the current implementation works fine only for required parameters defined in schema ex:

definitions:

sms:

type: object

required:

- smsType

properties:

smsType:

type: string

title: smsType

My client wants me to implement schema validation, for the whole JSON schema. is it possible?.

itravindrasingh
New Member

Very good article,

I am also exporing the tv4 for validating the JSON. I need some understanding on couple of the items -

What is the use of -

"additionalProperties": true
kurtkanaskie
Staff

Its part of the JSON Schema spec, for more details see: https://json-schema.org/understanding-json-schema/reference/object.html

mrds91
New Member

I understand this validates json request against OAS specified in json . Is there a way to handle OAS specified in yaml ?

bbhatia
New Member

I am planning to use this approach where I want to validate mainly POST request against swagger because every click to backend cost money.

I liked the idea of creating seperate proxy just for specs and calling that during runtime to fetch spec and validate against it.

kurtkanaskie
Staff

I provided an updated solution that supports OAS 2 and 3, plus validates required headers and query params here: https://community.apigee.com/articles/88441/validate-json-requests-using-openapi-spec-20-or-30.html

Version history
Last update:
‎06-12-2017 03:26 PM
Updated by: