Circuit breaker pattern which uses cached responses in error scenarios

[#1]

I'd like to implement a circuit breaker pattern such that slow responses from the target service trip the circuit and consequently my Apigee proxy changes behavior. The desired behavior when the circuit is tripped is for Apigee to temporarily stop passing requests to my target service for 5-10 minutes, thereby giving my target service a chance to recover, and rather than Apigee directly returning an error to clients when the circuit is tripped, I'd like to fallback to a cached response if one is available and if no cached response is available, then return an error to client.

I've been advised that I can use the connection timeout property to the target server and implement some fault handling and have had success implementing the fault handling, however in my scenario there are multiple services deployed on the host target server, so slow connection times to the host target server do not indicate slow response times from my service. To resolve this issue, I was thinking of using a service callout to a health check endpoint (or perhaps a generic search endpoint) in the target service wherein a fault is raised when response times exceed some pre-defined threshold much like described here: https://community.apigee.com/questions/38742/a-circuit-breaker-in-apigee-edge.html

To achieve the fallback to cached responses, I've added a postflow step which populates a cache entry for each requested URI. In this way, I intended to use a fault flow when the circuit is tripped which first looks for a cached response for the specified request URI and if available, then that is returned to clients, however is none is available, then an error is returned to clients.

I've experienced limited success handling faults from service callout policies, but that does not seem like the most difficult part of my scenario. The piece which I've been having trouble with is the circuit breaker piece.

Has anyone implementing something similar to what I've described?

[DChiesa]

I don't think you need ServiceCallout to do the healthchecks "manually".

I think that the HealthMonitor described in the docs does the HTTP check, using the HTTP Monitor.

Example

    <HealthMonitor>
      <IsEnabled>true</IsEnabled>
      <IntervalInSec>5</IntervalInSec>
      <HTTPMonitor>
        <Request>
          <ConnectTimeoutInSec>10</ConnectTimeoutInSec>
          <SocketReadTimeoutInSec>30</SocketReadTimeoutInSec>
          <Port>80</Port>
          <Verb>GET</Verb>
          <Path>/maps/api/geocode/json?address=08008</Path>
        </Request>
        <SuccessResponse>
          <ResponseCode>200</ResponseCode>
        </SuccessResponse>
      </HTTPMonitor>
    </HealthMonitor>

You would configure this to ping YOUR service. You can specify the timeouts and so on.

Then in the postflow you could handle faults as you are currently doing, satisfying requests through cache if the appropriate response is available.

Does this work? I'm thinking maybe I misunderstood your goals, because it seems kinda simple.