Getting 500 Internal Server Error with error.class as com.apigee.errors.http.server.BadGateway from backend server

[adevegowda]

We are getting 500 Internal Server as a response to our API. When we enabled the UI trace, we found that the 500 Internal Server error from the Backend server and noticed the following variables:

error             Received non success response code
error.class       com.apigee.errors.http.server.BadGateway

Attaching a part of the trace file showing this error:

screen-shot-2018-03-04-at-94906-am.png

I have a secure Backend server configured in the Target Endpoint as

<URL>https://<target server host>/path<URL>

[adevegowda · Accepted solution]

I investigated the issue as follows:

1. I tried to run the direct call to the backend server from one of the Message Processors associated with the org, using curl with -k option. But I got the error :

   curl -v -k "https://<target server host>/path"
   * About to connect() to <target server host> port 9443 (#0)
   * Trying <target IP>...
   * Connected to <target server host> (<IP address>) port 9443 (#0)
   * Initializing NSS with certpath: sql:/etc/pki/nssdb
   * skipping SSL peer certificate verification
   * NSS: client certificate not found (nickname not specified)
   * NSS error -12227 (SSL_ERROR_HANDSHAKE_FAILURE_ALERT)
   * SSL peer was unable to negotiate an acceptable set of security parameters.
   * Closing connection 0
   curl: (35) NSS: client certificate not found (nickname not specified)
   

2. The above error indicated that the Server was requesting the client (Message Processor in this case) to send the Certificate.
3. However, the target endpoint in the API Proxy was not configured to be a 2 way SSL to send the certificate to the backend server. Since the Certificate is not passed it gave the error "client certificate not found".
4. Collected a ssldump and confirmed that the Server is requesting the Client to send the Certificate. When the Client doesn't send the certificate it sends an alert and terminates the connection.
5. Here's a snippet of the ssldump showing the issue:

   1 5    0.1988   (0.0000)    S>C    Handshake
           CertificateRequest
             certificate_types                       rsa_sign
             certificate_types                       dss_sign
             certificate_types                   ServerHelloDone
   1 6    0.2028   (0.0040)    C>S    Handshake
           Certificate
           ClientKeyExchange
   Not enough data. Found 64 bytes (expecting 16384)
   1 7    0.2028   (0.0000)    C>S    ChangeCipherSpec
   1 8    0.2028   (0.0000)    C>S      Handshake
   1 9    0.2330   (0.0301)    S>C    Alert
         level               fatal
         value               handshake_failure
   

Solution:

1. Created Keystore and Truststore and added the relevant certificates.
2. Configured the target endpoint to be 2 way SSL as documented here.

With this the issue was resolved.