formparam is not validating

[supriyokumar274]

i have one proxy where i am trying to validate request.formparam.documentGuid should not be null 

but somehow its not taking through javascript , if i am adding any dummy extract variable policy before that then its working not sure why 

kindly help i am sending it in body and content type-- application/x-www-form-urlencoded

@anilsagar 

[supriyokumar274]

we are using OPDk  4.52.00.00 @anilsagar @dino 

[dchiesa1]

hi

---

@supriyokumar274 wrote:

somehow its not taking through javascript

---

I'm glad to try to help but I don't understand what you mean by "it's not taking". Can you be more precise in your explanation please? 

How exactly are you validating the formparam? Can you show the code?   What request are you sending in?  (and how?). What result do you observe?  (and how do you observe it?) What result do you expect to see?  

[supriyokumar274]

below is the js we have written to validate :

var validationErrors = [];
var errorCode = 'gateway.documents.badRequest';
var documentGuids = context.getVariable('request.formparam.documentGuids') || "";

if ( _.isEmpty(documentGuids)) {
validationErrors.push({
"field": "documentGuids",
"location": "formparam",
"reason": "A valid value of [documentGuids] should be passed."
});
}

print ("documentGuids", documentGuids);

if (validationErrors.length > 0) {
context.setVariable('flow.myapi.error.httpStatusCode', '400');
context.setVariable('flow.myapi.error.httpStatusReason', 'Bad Request');
context.setVariable('flow.myapi.error.code', errorCode);
context.setVariable('flow.myapi.error.message', 'Request Validation Failed');
context.setVariable('flow.myapi.error.errors', validationErrors);
throw Error();
}

even if we are passing documentGuids into header still everytime its taking null value @dchiesa1 

[dchiesa1]

It looks like you're using an underscore method - _isEmpty().  Probably underscore is not available in your JS callout, unless you've done something extraordinary. But the good news is you can check an empty string in JS without the need for the underscore library.

This code works for me

var validationErrors = [];
var errorCode = "gateway.documents.badRequest";
var documentGuids =
  context.getVariable("request.formparam.documentGuids") || "";

const isEmpty = (obj) => obj == null || !obj;

if (isEmpty(documentGuids)) {
  validationErrors.push({
    field: "documentGuids",
    location: "formparam",
    reason: "A valid value of [documentGuids] should be passed."
  });
}

print("documentGuids", documentGuids);

if (validationErrors.length > 0) {
  context.setVariable("flow.myapi.error.httpStatusCode", "400");
  context.setVariable("flow.myapi.error.httpStatusReason", "Bad Request");
  context.setVariable("flow.myapi.error.code", errorCode);
  context.setVariable("flow.myapi.error.message", "Request Validation Failed");
  context.setVariable(
    "flow.myapi.error.errors",
    JSON.stringify(validationErrors)
  );
  throw Error();
}

 Sample requests and responses: 

$ curl -i $apigee/supriyo-validate-in-js/t1 -d "documentGuids=hello" 
HTTP/2 200 
content-length: 23
content-type: application/json
apiproxy: supriyo-validate-in-js r3
x-request-id: eb069ce2-d465-4d87-919e-9fd3ea9ba47d
date: Tue, 28 Nov 2023 20:44:47 GMT
via: 1.1 google, 1.1 google
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

{
    "status" : "ok"
}

$ curl -i $apigee/supriyo-validate-in-js/t1 -d "docs=hello"         
HTTP/2 400 
content-type: application/json
apiproxy: supriyo-validate-in-js r3
x-request-id: 62e37c16-3404-4f1a-a28d-af751e311a8b
content-length: 238
date: Tue, 28 Nov 2023 20:44:54 GMT
via: 1.1 google
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

{
 "error" : {
    "code" : "gateway.documents.badRequest",
    "message" : "Request Validation Failed",
    "notices" : [{"field":"documentGuids","location":"formparam","reason":"A valid value of [documentGuids] should be passed."}]
 }
}

My test proxy is attached here.

[supriyokumar274]

thanks for the solution but somehow in my environment request.formparam.documentGuids always take null .. even after sending value also its taking null always and raising error @dchiesa1 

[supriyokumar274]

sending value 0385f705-d115-4c56-a5ad-4805cd156925
but its taking null as below :

[dchiesa1]

How are you sending it? (I asked you this before, and you didn't answer) Are you sure you are using the correct name on the sending side?  Maybe there is no trailing s.  Maybe there is all lowercase?  Maybe you are not sending it as content-type:application/x-www-form-urlencoded

If you are using Apigee trace , you can examine the inbound payload (request.content) on the VERY FIRST stage of the trace.  That would let you see the encoded form data. 

If you see the correct form parameter at the first stage, then there is a step between the first one and the one where you validate, that is resetting the form parameter.

[supriyokumar274]

below is the request i am sending :

and yes content-type:application/x-www-form-urlencoded is being send , also parameter name is same exactly 

curl --location --request DELETE 'https://{host}/v1/documents/bulk' \
--header 'AppKey: {appkey}' \
--header 'Authorization: {token}' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'documentGuids=0385f705-d115-4c56-a5ad-4805cd156925'

[dchiesa1]

Ahh yes, thanks for that. I'm sure this is frustrating. BTW, that request works for me on my test proxy.

Did you verify if the formparam is present on the first stage of the trace session, as I suggested above? This is important. (Everything I say and ask is important 😃 )

If you see no formparam there, it means somewhere between the client and Apigee, there is something removing the payload from the DELETE request. Regarding that, I have one idea. The HTTP Specification says that the payload on a DELETE request has no semantics, no meaning. To conform to the specification, you should identify the entire resource in the URL, with a DELETE request. So the request should be DELETE /v1/documents/038ff705-28728-33 (or whatever the documentID is) and carry no payload. But I guess you want to enable a bulk delete, in which there are multiple different document IDs, and you cannot pass them all on the URL path. So that is why you want to use a payload(body) with a DELETE request. I understand your aim there.

But that might be problematic. Per this answer on Stackoverflow, RFC 9110 says that

content received in a DELETE request has no generally defined semantics, cannot alter the meaning or target of the request, and might lead some implementations to reject the request and close the connection because of its potential as a request smuggling attack (Section 11.2 of [HTTP/1.1]). A client SHOULD NOT generate content in a DELETE request unless it is made directly to an origin server that has previously indicated, in or out of band, that such a request has a purpose and will be adequately supported.

That bold section was my emphasis. DIRECT to an origin server (which in this case is an Apigee API Proxy). I suspect that in virtually every enterprise, the path from client to Apigee is not direct. There is at least one HTTP-aware intermediary (router, gateway, firewall, or other system) between the client and your Apigee proxy.  Given the semantics of DELETE, it is possible that an intermediary is removing the payload you send with the DELETE request. (as an example, This article states that an Akamai WAF will strip the content from a DELETE request, by default).  If there is no form parameter visible in the first stage of the Apigee trace, that is a good indication that the body/content/payload of the DELETE request has been removed by something outside of Apigee. And per RFC 9110, there are good reasons to do so (to avoid smuggling attacks).

How to avoid the problem? Convert your API to use a POST , not a DELETE. and probably also convert the URL to be /v1/documents/bulkDelete , to indicate the behavior. Keep the form content the same.

if there IS a form parameter present in the first stage of the Apigee trace, then you have a policy step between that point, and the point at which you are validating the form parameter , that is removing the parameter, or changing the content-type, or causing some other manipulation of the payload such that your validation code no longer sees the original body. 

[supriyokumar274]

Thanks @dchiesa1  for all the info , we are checking what is the issue , meanwhile i have added one EV policy to extract as below :

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ExtractVariables async="false" continueOnError="false" enabled="true" name="Extract-Variables-1">
<DisplayName>Extract Variables-1</DisplayName>
<Properties/>
<FormParam name="documentGuids">
<Pattern>{documentGuids}</Pattern>
</FormParam>
<Source clearPayload="false">request</Source>
<VariablePrefix>requestData</VariablePrefix>
</ExtractVariables>

and using requestData.documentGuids variable inside the js policy for validation and it is working for me , but we are checking what is the issue with request.formparam.documentGuids, and why its taking null 
thanks for your help ..

[dchiesa1]

oh! Wow! And that works?? OK that suggests it's something inside the API proxy, something between the initial receipt of the message, and the actual validation logic.

Did you verify for me that a browserified version of underscore is in your JS IncludeURL tag? You are using _.isEmpty() so you would need something to define that method. Or maybe you are using my version of the isEmpty() method? The reason I suggest this: if you don't have underscore, or if you otherwise do not have _.isEmpty() defined, then you could be getting a JS exception in your validation code.

[supriyokumar274]

Yes @dchiesa1  we are using lodash_js library to achieve this