java callout run time issue

kalyaninturi
Participant II

I have an issue with the java-callout policy. The issue is intermittent and message processor restart

is solving my problem.

Issue: Some of the times java callout is not getting executed, I don't see either execution failure

or java exception it is simply not getting executed. I am setting a lot of variables in java callout and

everything is becoming null and this makes me believe that java callout is not executing. This is

happening in one of our production environments [on-prem] so I don't have the help of trace tool.

PS: The above problem is getting solved by restarting the message processor and I still don't have a

reason how it's happening.

0 7 206
7 REPLIES 7

kalyaninturi
Participant II

If I were troubleshooting this I would make sure my Java callout code included a try...catch within the execute() method, and then I'd populate a context variable with the stacktrace.

Example here

https://github.com/DinoChiesa/ApigeeEdge-Nanotime/blob/master/callout/src/main/java/com/google/apige...

public ExecutionResult execute(final MessageContext msgCtxt, final ExecutionContext execContext) {
    try {
      // ..
      // do the thing here
      // ..
      msgCtxt.setVariable(varName("success"), "true");
      return ExecutionResult.SUCCESS;
    } catch (java.lang.Exception exc1) {
      msgCtxt.setVariable(varName("error"), exc1.getMessage());
      msgCtxt.setVariable(varName("stacktrace"), exceptionStackTrace(exc1));
      return ExecutionResult.ABORT;
    }
  }
}

@Dino-at-Google

Please go through my question again, I am saying java callout simply not executing. I have try/catch, context variable and probably all of the things you mentioned above. Context variables in JC are becoming null when I tried to read them in policies followed by Java Callout.

I know that it is weird but its true java callout is not getting executed and restarting message processor made it work. This situation is happening intermittently.

oh I'm sorry. I misunderstood.

In that case I would suggest looking in the MP system logs. And checking the VM vital signs (memory, CPU). It may simply be a process or memory issue with the MP.

@Dino-at-Google

How can I identify memory issue as soon as it occurs? Is there any automated process to identify it and resolve the issue or should we keep checking MP system.log every day??

@Dino-at-Google can you suggest some advice, I tried to contact apigee support but

they aren't helpful.

Detecting a memory issue as soon as it occurs is an issue that is generic to running any service. It's not specific to Apigee.

I would suggest using a system monitoring strategy and tool. Maybe something like Elastic (formerly "the ELK stack") which includes ElasticSearch, Logstash, and Kibana. There's also a "TICK stack", which includes Telegraf, InfluxDB, Chronograf, and Kapacitor.

Or some people use Splunk with appropriate alerts set up.

Or Datadog.

There are many other alternatives.

but I would guess you already have something like that.