How to add a notification to a HealthMonitor

Not applicable

We are utilizing a HealthMonitor to perform a fail over from one backend system to another, should the original backend system not return a 200 OK. Once the original system is back online, it will revert back to the original. We would like to add email notifications each time a failover and revert back occurs. Is there a way to do this?

0 22 502
22 REPLIES 22

Dear @matthew.rinkenberger@infogroup.com ,

Yes, API Health supports this out of the box. You can add email notifications on failover in second step of Adding Probe.

For Example,

1. Add Probe

846-screen-shot-2015-07-28-at-80924-am.png

2. Configure Notifications,

  • Check - If Failure exceeds
  • Give count as 1 in the box if you want immediate alert on 1 instance of failover
  • Check Email Notifications, Provide Email Addresses separated by comma to get notified.

847-apihealth.png

Please let us know if you have any questions.

Cheers,

Anil Sagar

I tried setting up an API probe. I have 2 questions:

1) Does the "select locations" step automatically saves the probe upon completion?

2) How do I verify that the probe is set up right and running? After what seemed like an error-free creation, the probe seemingly does nothing. In the Edit steps page, I saw a "Validate" button - which on being hit (all probe setup completed) just throws up a status bar that says "Loading" but the var is just 2% completed, and doesn't progress much.

Thanks for the help in advance.

Best,

Balaji.

Not applicable

The configuration we have is on the TargetEndPoint and we set up a HealthMonitor based on HttpMonitor to determine the health of the primary in a LoadBalancer configuration. Matt and I work on this and I'd like to add to the above to make it clear. I follow the above conceptually. However I have 2 issues:

1) I don't see an API probe that I can from the GUI. The way we configured the LoadBalancer was based on XML config code available in the help pages. Is there an xml example you can share?

2) Are there any requirements to set up mailer configs - or will even the xml config work out of the box?

I can also trace as to what is the current state of the load balancer - whether it is in fallback mode or not. This is the trigger I'd like to use (isFallback=true/false) to send out notifications across that toggle.

Thank you.

Not applicable

@Balaji.Rajamani , AFAIK there are no notifications for health monitor as of now .

But is there notifications on the load balancer itself (the switch between fallback and primary and vice versa)?

Is health.api.com a service add-on? I don't get a response when I try to hit this link. As I pointed out in my previous pose, enterprise.apigee.com does not have a step available for the probe.

As @Maruti Chand points out, there seem to be no health monitor notification support. Does anyone have a workaround for this? Basically, I need to trigger notification on the load balancer fallback switchover and vice versa.

Thanks again.

You should use https://health.apigee.com.

Secure link worked - thanks.

Not applicable

@Balaji.Rajamani

Looks like we are mixing up things here .

#1 Health monitor that you have in your target definition is just to help you with your routing and it doesn't have any notifications .

#2 API health is to monitor any API health , it can be Northbound or Southbound API .

#3 In your case , you can either use API health to trigger notifications when your backend returns 503 or 504 .(or any http response code based on your requirement) if health domain can reach your backend.

else

Have a healthcheck passthrough to your backend APIs and configure the northbound API in health to do the same and by looking at the response codes,f you can understand if fallback happened or not .

(Basically you need to do health check twice one for routing and one for notifications)

Lets see if any one in the community has more ideas .

@Maruti Chand Agreed - though I didn't think of it as a mixing of things to start with. After all, when a fallback occurs the system would know about it, and I thought we can add a notification hook right there.

Having said that, I'm ok with treating them as separate and I even attempted to create a separate Health probe. This I did in addition to the loadbalancer with fallback I already have working.

However, the probe doesn't seem to be doing much. I have since opened a support ticket - the essence of which I'm attaching hereunder. The bottom-line, the health probe doesn't seem to do much. It has stayed in the no-activity state for close to 24 hrs now. When I hit on the "Validate" button, it just stalls in the "loading" state forever. I'm not sure what I'm missing here. All help most welcome.

Original ticket description:

I created the "IC3 failover probe" to monitor the health of the primary server and report any downtime. I could create the probe step seemingly straight-forward. However, I don't see any activity in the probe. I even tried to validate the probe using the "Validate" button - it seems to just start some validation and get stuck in the initial "loading" stage. There is no further feedback. I understand that the API health portal is in beta - however, this functionality (or some way of identifying the primary server is down) is critical for our roll out into production for billing purposes.

Thanks,

Balaji.

vinoth
Participant III

@Balaji.Rajamani I can see the Failures in the probes. The failures are mostly because of Timeouts. The system disconnects the probe call after 30 sec. I can see those failure cases in the message view

881-infogroup.png

Let me know if you are recieving the notifications.

@Balaji.Rajamani in order to get to the message view, in the Dashboard click on the icon which is next to the delete icon.

Yes - I see the message view. The requests should be faster. Do you think this is a firewall issue? I assumed that since the enterprise apigee proxy is able to hit this end point, the health probe will also be able to. Is that assumption incorrect? Typically the call is a sub-second call (not 30+ secs as shown above).

Thanks.

Balaji.

The url defined in the probe, im not able to hit it from my macbook. The DNS is not resolving. It tried hitting the box from 2-3 different locations, DNS is not resolving.

This is the exact URI we have set up for the API proxy. The proxy is able to see it and resolve it - that is why I asked the question about firewall or related issue. Do we know how/why the proxy is able to resolve, but the health is not?

Im not able to ping this url even from pingdom. Im not sure how apigee proxy is able to resolve this. Can you please check with support ? Api health checks the availability of an API from outside in view, to check the real availability from outside. I was not able to resolve this DNS from anywhere outside

@vinothapigee Can you let me know from where I should expect the request to hit our server - and I'll open up the firewall to allow the request. I see apigee proxy is enabled in our firewall, but apparently the health probe request comes from a different address.

Thank you.

@Balaji.Rajamani the IPs are dynamic. We are running agents which are dynamic in nature. And its spread across various locations in the globe. This is the first of its kind request for us. We will discuss about this and get back to you on the feasibility of this. Unfortunately, it would take sometime for that.

More importantly - no, I'm not getting notifications for this failure. As you can see, the Step response is defined as "200". I assumed that that is the success response expected by the probe step - anything else will trigger a notification (on the first failure as the subsequent page config sets).

Am I correct in the above expectation?

@Balaji.Rajamani Im able to get alerts for your probe. I configured myself to get the alerts. I was able see that in my inbox. Can you look at your spam folder ?

882-infogroup-email.jpg

@vinothapigee Yes - we have firewall rules to allow requests from select addresses. Apparently, the api proxy and the health api probe don't share the same subnet/domain. At this time, I need to involve IT to get over this issue I'd think.

As for the question on the email notification, no, I don't see it in the junk/spam folder either. I need to see if our mailer filters it out automatically - now that I know the emails do happen.

Thanks for the support. Will keep the thread posted on any further updates.

Quick update:

1) If a publicly visible endpoint for the same service is put - the probe returns success - this corroborates the firewall issue we discussed before.

2) If I give another email address from a public vendor (not my enterprise email) - the email alerts work too - I therefore suspect either a reject by our enterprise mailing system, or a filtering. I'll probe further on this.

I definitely need to set up against our internal server (not the public end point). I'll work with our IT to see how to go about that and take it from there.

What is the cost of using the API probe? Since it seems separate from the api proxy, I'm not sure of the cost involved - since this is beta I can think that this is not well supported and hence the no cost to use.