MTLS to targetServer still works after certificate expires

[mark_hammelman]

We have a target server defined so that it enables MTLS by following Apigee's documentation here: https://docs.apigee.com/api-platform/system-administration/configuring-ssl-edge-backend-service#defi....

Within that section, it says "If the server certificate is signed by a third party, then you will need to upload the complete cert chain, including the root CA cert, to the truststore."

In this case, the cert is issued by "DigiCert", which I'd consider meets the definition of a third-party, so the complete chain was uploaded. I need to obfuscate this picture for security reasons, but this should give the picture that its all loaded within the same 'alias' of the trustStore.

Pretend that's the certificate for 'host.example.com'. It's stored in the trustStore that is referenced by the targetServer's "trustStore" property.  Here is how the targetServer object is defined:

Show More

{
"host": "host.example.com",
"isEnabled": true,
"name": "host-example-com-targetServer",
"port": 443,
"sSLInfo": {
"ciphers": [],
"clientAuthEnabled": "true",
"commonName": {
"value": "host.example.com",
"wildcardMatch": false
},
"enabled": "true",
"ignoreValidationErrors": false,
"keyAlias": "keystore-outbound",
"keyStore": "ref://keystore-outbound-ref",
"protocols": [],
"trustStore": "ref://trustStore-outbound-ref"
} }

What would we need to change to make it require a valid certificate?

[apickelsimer]

Can you clarify what you mean by "make it require a valid certificate"? Are you referring to the backend? Are you referring to an Apigee keystore? Without knowing that part, I can only speculate. 

From the screenshot it appears the final certificate in the chain (a.k.a. client certificate) is expired (Jan 14 2023). Thats the certificate Apigee sends to the server. The server then validates it is genuine (or not). Of course before the client certificate is sent, Apigee does validate the server certificate is valid as well. It appears, from the information you provided, the server is not validating the expiration of the client certificate sent by Apigee, and perhaps other important stuff too.

[mark_hammelman]

Here is a sequence diagram to help clarify:

When Apigee initiates the connection to the backend (step 20) the server "host.example.com" will present it's certificate (step 30).  Apigee should be checking that this certificate is trusted by looking for a match within the "trustStore" defined in the targetServer definition.  

In this case, the cert in Apigee's trustStore for "host.example.com" has expired, but the server is sending a non-expired cert.  The point is, the cert from the server does not match the certificate in our trustStore, so the TLS handshake should have stopped there.  At this stage, this is just '1-way TLS'.

The server has to ask for a client-certificate, at which point Apigee would send the certificate defined in the "keyStore" and "keyAlias" properties of the targetServer object.  So in this case, we're sending a different cert with a host name like "api.myCompany.com" so then the targetServer must have that cert in their trustStore too.. but that's not the problem here.

I want to know why Apigee is not stopping the TLS connection when the certificate in Apigee's trustStore does not match the certificate that the server sent. (Step 30/40)

Hope this helps clarify the problem!

[balaji17]

@mark_hammelman Seems like you have CA (DigiCert Root and Intermediate) and Leaf (Server) certificate added in your trust-store, since CA certs already in the trust-store, Apigee might use the CA certs to trust the TLS connection from client perspective as it usually has longer validity.

May be you could try only adding a Leaf (server) certificate which is expired in the trust-store and see Apigee is trusting the TLS connection. I hope not.

[mark_hammelman]

Right.. just to be sure we're thinking the same thing:

The "root" and "intermediate" certs are not loaded as different 'aliases' in the trustStore. When I created the alias for this "host.example.com", I uploaded one file that contains the entire chain.  

I can upload only the "leaf" certificate to test it, but this seems to contradict the documentation I mentioned in the first post "If the server certificate is signed by a third party, then you will need to upload the complete cert chain, including the root CA cert, to the truststore."

[mark_hammelman]

Alright - I have some updates!

1. I deleted the cert chain and confirmed the connection starts to fail with TLS error. 
2. I uploaded ONLY the expired 'leaf' certificate, and confirmed this also fails with a TLS error.
3. I deleted the expired 'leaf' and replaced it with the new, non-expired 'leaf' cert, and the connection started to work again. 

So, it appears Apigee's documentation is flat out wrong when it says "If the server certificate is signed by a third party, then you will need to upload the complete cert chain, including the root CA cert, to the truststore. There are no implicitly-trusted CAs."

If your certificate is signed by a 'well-known' CA, then you only need to upload the 'leaf' certificate.  If you load the entire cert-chain, then Apigee will allow ANY leaf certificates from that CA.

[apickelsimer]

It's not guaranteed that every 'well-known' CA will always be available in the truststore. Since Apigee uses JDK for its certificate store, JDK does come with a default cacerts store.

I was able to find the root CA you're using in the store, which confirms the behavior you are describing.

Owner: CN=DigiCert Global Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Issuer: CN=DigiCert Global Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Serial number: 83be056904246b1a1756ac95991c74a
Valid from: Fri Nov 10 00:00:00 UTC 2006 until: Mon Nov 10 00:00:00 UTC 2031
Certificate fingerprints:
        SHA1: A8:98:5D:3A:65:E5:E5:C4:B2:D7:D6:6D:40:C6:DD:2F:B1:9C:54:36
        SHA256: 43:48:A0:E9:44:4C:78:CB:26:5E:05:8D:5E:89:44:B4:D8:4F:96:62:BD:26:DB:25:7F:89:34:A4:43:C7:01:61
Signature algorithm name: SHA1withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3