Parsing text/html response

Good day,

I did a search and couldn't find a response. So here's a newbie's question ...

I created a reverse proxy with this end point:

http://dummy.restapiexample.com/api/v1/employee/66150

The response content type is text/html; charset=UTF-8. But the body is json:

{"id":"66150","employee_name":"jfkjdbfjkabfbadjf","employee_salary":"29166","employee_age":"23","profile_image":""}

I can't parse employee_name out. I read that the content type has to be json/application for the extract variable policy to work. I also read somewhere that someone simply changed the content type header to json and it worked for them. But here it doesn't seem to work. I was able to use the extract variable policy with another API.

What might be going wrong? How should we parse the output?

Also a more generic question: how does one parse free text API response (say from an html page)?

P.S.: the end point above changes daily. You may need to get this:

http://dummy.restapiexample.com/api/v1/employees

to find a working employee ID.

Solved Solved
1 2 2,796
1 ACCEPTED SOLUTION

I did a search and couldn't find a response.

Thank you for trying!

So here's a newbie's question ...

ok great!

You are correct that ExtractVariables will work only if the content-type header matches what you are trying to extract: if you are using XPath to find the thing to extract, then the message needs to have content-type set to application/xml or text/xml or similar. Whereas, if you are using JSONPath to extract, then the message must have content-type header set to application/json or similar. (having charset=UTF-8 as a suffix in the content-type header is fine in either case)

Your question suggested that the required value is "json/application" . Not sure if that's a typo or if you have inadvertently switched that in your actual code. But json/application is not a valid content-type, and, trying to extract information using ExtractVariables with JSONPath from a message with content-type header set to json/application will not succeed. You need application/json.

If your backend (upstream) system delivers an incorrect content-type, and if you're certain that the content-type assertion is wrong, then, no problem. You can forcibly set the content-type in your Apigee Edge API Proxy.

Setting a header means modifying the message. The general-purpose policy for modifying a message in Apigee Edge is "AssignMessage". The policy configuration will look like this:

<AssignMessage name="AM-ForceContentType">
  <Set>
    <Headers>
      <Header name='content-type'>application/json</Header>
    </Headers>
  </Set>
</AssignMessage>

You will need to attach this AssignMessage policy in the flow BEFORE ExtractVariables runs, and AFTER the response message has been received.

Then the ExtractVariables should work, if you have the JSONPath set properly.

See attached for an example.

restexample-rev1-2019-03-21.zip

View solution in original post

2 REPLIES 2

I did a search and couldn't find a response.

Thank you for trying!

So here's a newbie's question ...

ok great!

You are correct that ExtractVariables will work only if the content-type header matches what you are trying to extract: if you are using XPath to find the thing to extract, then the message needs to have content-type set to application/xml or text/xml or similar. Whereas, if you are using JSONPath to extract, then the message must have content-type header set to application/json or similar. (having charset=UTF-8 as a suffix in the content-type header is fine in either case)

Your question suggested that the required value is "json/application" . Not sure if that's a typo or if you have inadvertently switched that in your actual code. But json/application is not a valid content-type, and, trying to extract information using ExtractVariables with JSONPath from a message with content-type header set to json/application will not succeed. You need application/json.

If your backend (upstream) system delivers an incorrect content-type, and if you're certain that the content-type assertion is wrong, then, no problem. You can forcibly set the content-type in your Apigee Edge API Proxy.

Setting a header means modifying the message. The general-purpose policy for modifying a message in Apigee Edge is "AssignMessage". The policy configuration will look like this:

<AssignMessage name="AM-ForceContentType">
  <Set>
    <Headers>
      <Header name='content-type'>application/json</Header>
    </Headers>
  </Set>
</AssignMessage>

You will need to attach this AssignMessage policy in the flow BEFORE ExtractVariables runs, and AFTER the response message has been received.

Then the ExtractVariables should work, if you have the JSONPath set properly.

See attached for an example.

restexample-rev1-2019-03-21.zip

Fantastic! This works. Thanks for your quick and helpful response. Including an example was super helpful. Thanks.