{ Community }
  • Academy
  • Docs
  • Developers
  • Resources
    • Community Articles
    • Apigee on GitHub
    • Code Samples
    • Videos & eBooks
    • Accelerator Methodology
  • Support
  • Ask a Question
  • Spaces
    • Product Announcements
    • General
    • Edge/API Management
    • Developer Portal (Drupal-based)
    • Developer Portal (Integrated)
    • API Design
    • APIM on Istio
    • Extensions
    • Business of APIs
    • Academy/Certification
    • Adapter for Envoy
    • Analytics
    • Events
    • Hybrid
    • Integration (AWS, PCF, Etc.)
    • Microgateway
    • Monetization
    • Private Cloud Deployment
    • 日本語コミュニティ
    • Insights
    • IoT Apigee Link
    • BaaS/Usergrid
    • BaaS Transition/Migration
    • Apigee-127
    • New Customers
    • Topics
    • Questions
    • Articles
    • Ideas
    • Leaderboard
    • Badges
  • Log in
  • Sign up

Get answers, ideas, and support from the Apigee Community

  • Home /
  • Edge/API Management /
avatar image
2
Question by Philip Sinclair · Oct 29, 2016 at 01:03 PM · 3.7k Views service calloutresponse cache

How can I store the results of the ServiceCallout policy in cache? and later, retrieve it from cache?

Hi,

I am trying to mashup the data from a number of ServiceCallout policies, but only the proxy itself is getting returned from cache. What is the best way to return the result from a service callout from cache rather than making a new request each time to the backend?

When I try to do this, it doesn't seem to hit the cache for the service callouts which really slows down the overall response time.

Thanks

Comment
Add comment
10 |5000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by Apigeeks only
  • Viewable by the original poster
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Close

1 Answer

  • Sort: 
avatar image
5
Best Answer

Answer by Dino   · Oct 29, 2016 at 08:49 PM

I suspect you are using the "response cache". That works only for the target response. I think you want to use the cache primitives - LookupCache and PopulateCache - to store the result of the service callout.

For example, your flow might be like this:

      <Request>
        <!-- check the cache for the data previously retrieved -->
        <Step><Name>CacheLookup-MyData</Name></Step>
        <Step>
          <Name>SC-RetrieveMyData</Name>
          <Condition>cached.my.data = null</Condition>
        </Step>
        <Step>
          <Name>Extract-MyData</Name>
          <Condition>cached.my.data = null</Condition>
        </Step>
        <Step>
          <Name>CacheInsert-MyData</Name>
          <Condition>cached.my.data = null</Condition>
        </Step>
        <Step>
          <Name>AV-MyData</Name>
          <Condition>cached.my.data != null</Condition>
        </Step>
         ....


What's going on there? The first step is a LookupCache . It looks in the cache for a particular key, and stores the resulting data into the context variable "cached.my.data". If the cache entry does not exist, then that context variable will be null after executing the first policy. If the cache entry exists, that context variable will be non-null.

The next 4 policies execute conditionally. The SC-RetrieveMyData is a ServiceCallout that calls some external thing. The Extract-MyData is an ExtractVariables policy that pulls something from the ServiceCallout response into a variable. (This assumes you do not want the entire response.content to be cached, but rather, some subset or portion of it) You can call this variable anything, let's call it "my.data". The CacheInsert-MyData is a PopulateCache policy that inserts the extracted value ("my.data") into the cache, so that it is available for the next request.

Finally, the AV-MyData runs only if the original LookupCache succeeded. Basically this just copies the retrieved value from the "cached.my.data" variable into the "my.data" variable .

When the cache is warm, the first and final policies in the above snip will execute. When the cache is cold, the first 4 policies will execute. Either way, "my.data" holds the correct value at the end of this sequence. Subsequent policies can reference "my.data" which has been filled either by the AV-MyData, in the case of a cache hit, or by the Extract-MyData policy, in the case of a cache miss.

Does this make sense? This is a handy re-usable pattern for caching data retrieved from ServiceCallout policies.

For some further detail, here's an example of what the CacheLookup-MyData might look like:

<LookupCache name='CacheLookup-MyData'>
  <CacheResource>cache1</CacheResource>
  <AssignTo>cached.my.data</AssignTo> <!-- name of flow variable -->
  <Scope>Application</Scope>
  <CacheKey>
    <Prefix>something</Prefix>
    <KeyFragment ref="client_id"/> <!-- as an example -->
  </CacheKey>
</LookupCache>

The CacheInsert is something like this:

<PopulateCache name='CacheInsert-MyData'>
  <CacheResource>cache1</CacheResource> <!-- choose a proper name -->
  <Source>my.data</Source>
  <Scope>Application</Scope>
  <CacheKey> <!-- use same cache key as the LookupCache -->
    <Prefix>something</Prefix>
    <KeyFragment ref="client_id"/>
  </CacheKey>
  <ExpirySettings>
    <TimeoutInSec>7200</TimeoutInSec>
  </ExpirySettings>
</PopulateCache>

And the AV-MyData is like this:

<AssignMessage name='AV-MyData'>
  <IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
  <AssignVariable>
    <Name>my.data</Name>
    <Ref>cached.my.data</Ref>
  </AssignVariable>
</AssignMessage>

I will leave the ServiceCallout and ExtractVariables policy for you to implement. The key is that the Extract should operate on the response message that has been set by ServiceCallout, and should set "my.data".

As I said, you can omit the ExtractVariables if you wish to cache the entire response.content from the servicecallout.

The cache key can be anything. The example code here shows client_id, but you could use anything that identifies the data to be cached. a query param, or a header, or whatever is appropriate.

Comment
Add comment Show 4 · Link
10 |5000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by Apigeeks only
  • Viewable by the original poster
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image Philip Sinclair · Oct 30, 2016 at 09:06 AM 0
Link

Dino, thanks for the comprehensive answer, after some more head scratching I was finally able to get my head around the entire concept. I think the documentation should be updated with a good example like yours! Philip

avatar image Hari Philip Sinclair · Oct 30, 2016 at 10:39 AM 0
Link

I had implemented the same thing and was coming here to post the answer, but you explained it better than I ever could, Dino.

avatar image Dino ♦♦ Philip Sinclair   · Oct 31, 2016 at 01:02 PM 0
Link

Glad to be able to help!

avatar image jonesfloyd ♦♦ · Oct 31, 2016 at 06:17 PM 0
Link

Thanks, @Dino. And great question, @Philip Sinclair. I've added a link to this from the Service Callout policy doc, in the About section.

Follow this Question

Answers Answers and Comments

44 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

Can Service Callout Policy / be configured using Variables? 1 Answer

Access Service Call out XML Response 1 Answer

Cache Keys - Any disallowed characters? My Cache Policy is resulting in a miss which should be a hit 1 Answer

Service callout from node.js app 1 Answer

lookup cache is not working as expected 1 Answer

  • Products
    • Edge - APIs
    • Insights - Big Data
    • Plans
  • Developers
    • Overview
    • Documentation
  • Resources
    • Overview
    • Blog
    • Apigee Institute
    • Academy
    • Documentation
  • Company
    • Overview
    • Press
    • Customers
    • Partners
    • Team
    • Events
    • Careers
    • Contact Us
  • Support
    • Support Overview
    • Documentation
    • Status
    • Edge Support Portal
    • Privacy Policy
    • Terms & Conditions
© 2021 Apigee Corp. All rights reserved. - Apigee Community Terms of Use - Powered by AnswerHub
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Create an article
  • Post an idea
  • Spaces
  • Product Announcements
  • General
  • Edge/API Management
  • Developer Portal (Drupal-based)
  • Developer Portal (Integrated)
  • API Design
  • APIM on Istio
  • Extensions
  • Business of APIs
  • Academy/Certification
  • Adapter for Envoy
  • Analytics
  • Events
  • Hybrid
  • Integration (AWS, PCF, Etc.)
  • Microgateway
  • Monetization
  • Private Cloud Deployment
  • 日本語コミュニティ
  • Insights
  • IoT Apigee Link
  • BaaS/Usergrid
  • BaaS Transition/Migration
  • Apigee-127
  • New Customers
  • Explore
  • Topics
  • Questions
  • Articles
  • Ideas
  • Badges