HTTP 504 and HTTP 502 error in APIGEE

Hi All,

We are seeking assistance on our current issue we are facing with the below details.Hopefully somebody had encountered the same situation and have at least a work around.

Current Setup:

  • APIGEE Private Cloud
  • APIGEE Edge Version: Version 4.16.01.04

Issues:

  • Our APIGEE APIs returns http 504 intermittently. We noticed that after a bounce in the message processor server it will be ok but, will be back again in a short while.
  • Our APIGEE Message Processor has default 57 seconds timeout as per apigee documentation.
  • 5773-504.jpg
  • When also noticed that most of our API calls just keeps on loading. If they even stop loading http 504 was already thrown in our calls. Also the first policy (regardless of what kind) will always have a large processing time for some reason.
    • 5776-capture.jpg
  • We also experience such situation where APIGEE blocks every request and returns http 502 bad gateway. Checking on the trace nothing shows up when we hit this scenario.
    • 5775-502.jpg
1 7 2,544
7 REPLIES 7

@Harry Reynoso have you checked your mp logs or mgmt server logs. It will show where the exact error is occurring and why.

@Christian Brown,

Thanks! let us see what we can find in mgmt server logs. We do not see much details in MP logs when we are getting 504 or 502.

sjm2000
Participant V

You can increase timeout but that won't solve issue as per your first point .

can you use jconsole and monitor the usage of router and message processor using jmx ?

Thanks & best regards

Sujith Mathew

@Harry Reynoso Can you please check the nginx access logs and the corresponding system logs on the MPs?

The Edge Private Cloud Troubleshooting Guide should be able to help you with more insights.

http://docs.apigee.com/private-cloud/content/edge-private-cloud-troubleshooting-guide

@Akash Prabhashankar Thank you, we will look into the nginx access logs. We will check the guide also for troubleshooting.

Hi All, Thank you for those who had share their thoughts on this. It seems like we found the issue. For some reason the Message Logging Policy on one of our proxy chained API Proxy is causing the issue as it continuously throws a "Broken Pipe Exception" when checking the MP Logs. What we did is to disable the message logging and everything has been ok since then.

Hi, could you please explain what kind of MessageLogging policy are you using?

We are currently having the same "Broken Pipe" issues.

We have a MessageLogging conditional policy in the "PostClientFlow" (following the instructions from the Docu) as this:

<MessageLogging async="false" continueOnError="false" enabled="true" name="MessageLoggingJavaCallout">
    <DisplayName>MessageLoggingJavaCallout</DisplayName>
    <BufferMessage>false</BufferMessage>
    <File async="true">
        <FileName>trace.log</FileName>
        <FileRotationOptions rotateFileOnStartup="true">
            <MaxFileSizeInMB>10</MaxFileSizeInMB>
            <MaxFilesToRetain>10</MaxFilesToRetain>
            <RotationFrequency unit="day">1</RotationFrequency>
            <FileRotationType>SIZE</FileRotationType>
        </FileRotationOptions>
        <FlushAfterEachWrite>false</FlushAfterEachWrite>
        <Message>[{iso_system_timestamp}] [{system.timestamp}] [{response_statusCode}:{response_reasonPhrase}] {response_content}
</Message>
    </File>
    <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
    <logLevel>INFO</logLevel>
    <NotificationIntervalInSec>0</NotificationIntervalInSec>
</MessageLogging>

If really you did only deactivate the policy, then I guess there is an general Issue with the MessageLogging that should be addressed by Apigee

Thanks in advance,

Oscar