GenerateJWT failing again when using password protected private key

[amitsharma]

I am facing a strange issue on GenerateJWT policy in which I am truing the using a password protected private key. I had a similar issue in July 2019 which was resolved when Apigee deployed support for different type key encryption algorithms. I followed exactly the same sets for creating password protected private key but i keep on getting following error when executing the api proxy.

{
    "fault": {
        "faultstring": "Failed to parse key: policy(Generate-JWT-Token) ",
        "detail": {
            "errorcode": "steps.jwt.KeyParsingFailed"
        }
    }
}

Here is my setup

1. I have generated PEM (password protected private key) using following command

openssl pkcs12 -in tokensigning.pfx -nocerts -out token-private-key.pem

Here is the encryption method details from PFX

openssl pkcs12 -info -in tokensigning.pfx -nooutPKCS7 Data
Output:
Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2000
PKCS7 Encrypted data: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2000

Create a encrypted Key Value Map using Portal with following keys

privatekey-password: *********


privatekey:

-----BEGIN ENCRYPTED PRIVATE KEY-----
key data.......
-----END ENCRYPTED PRIVATE KEY-----

Private Key

Bag Attributes
    Microsoft Local Key set: <No Values>
    localKeyID: 01 00 00 00 
    Microsoft CSP Name: Microsoft RSA SChannel Cryptographic Provider
    friendlyName: some-friendly-name-16a8f7db122f
Key Attributes
    X509v3 Key Usage: 10 
-----BEGIN ENCRYPTED PRIVATE KEY-----
key data.......
-----END ENCRYPTED PRIVATE KEY-----

KeyValueMap Policy

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<KeyValueMapOperations name="KVM-GetTokenSigningPrivateKey" mapIdentifier="oauth-token-signing">
    <Scope>environment</Scope>
    <ExpiryTimeInSecs>15</ExpiryTimeInSecs>
    <Get assignTo="private.private-key">
        <Key>
            <Parameter>privatekey</Parameter>
        </Key>
    </Get>
    <Get assignTo="private.privatekey-password">
        <Key>
            <Parameter>privatekey-password</Parameter>
        </Key>
    </Get>
</KeyValueMapOperations>

Generate JWT policy

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<GenerateJWT name="Generate-JWT-Token">
    <Algorithm>RS256</Algorithm>
    <PrivateKey>
        <Value ref="private.private-key"/>
        <Password ref="private.privatekey-password"/>
        <Id>unique-identifier-for-privatekey-here</Id>
    </PrivateKey>
    <Subject>ABCD</Subject>
    <Issuer>urn://1a4b40567d5a</Issuer>
    <Audience>urn://api.dev</Audience>
    <ExpiresIn>60m</ExpiresIn>
    <AdditionalClaims>
        <Claim name="apigee-proxy" ref="apiproxy.name"/>
        <Claim name="messageid" ref="messageid"/>
        <Claim name="request-path" ref="request.path"/>
        <Claim name="apigee-org" ref="organization.name"/>
        <Claim name="apigee-env" ref="environment.name"/>
    </AdditionalClaims>
    <OutputVariable>output-jwt</OutputVariable>
</GenerateJWT>

Trace session are not providing any useful information. I have validate whether the KeyValueMapOperation is loading the values in variables by printing the variables using AssignMessage Policy. Similar setup is working fine in other tenants. I am not sure what is going wrong here. Please help.

[amitsharma]

Hey @Dino-at-Google.. You helped me with a similar issue in past. I have followed exactly the same steps but GenerateJWT is failing. Please help

[dchiesa1]

wow, it sounds like you know what you're doing.

Similar setup is working fine in other tenants.

Same encryption algorithm is used on the private key in those other tenants? But a different key and password, I assume?

Is it possible the password you are trying to use is incorrect? Have you verified it independently? (with openssl)

The only other possibility I can think of is that the private key PEM is somehow corrupted in the KVM. Because it is encrypted, you don't NEED to store it securely. That's the whole point of encryption - you can treat the encrypted private key as non-secret, and only store the password.

So maybe try hard-coding the encrypted private key value in the configuration (I can't remember if this actually works), or using AssignMessage to set a variable _explicitly_ with the PEM (including newlines) of the private key. For testing you can also try setting the private key password in AssignMessage.

Maybe like this:

<AssignMessage name='AM-SetKey'>
  <AssignVariable>
    <Name>private.private-key</Name>
    <Value>
-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIJnzBJBgkqhkiG9w0BBQ0wPDAbBgkqhkiG9w0BBQwwDgQIC5fD1UCTj1kCAggA
MB0GCWCGSAFlAwQBFgQQ2JVJHo5VBtXvKF8MgMuxJASCCVAXz0r2tKf/FQySdJeC
wA1kuX5+kLrV9Dtzl1tCcaGwLw1wuPOEsVW53uIDAC1eKydBUhiLQqmPMX67b6xp
s25CR7vJ6EpNlR28ozgX4C9DmPxpEjGCQpabHhBQeAweOpO8UWjNkT6Cy2jdU+sx
j/1gPeyQ+kKW36MtK7Blr0ECxOn1c53qA/PmsYaFNwCE+XGTHSeNMMyoOoAzcJzE
...
g1+1zOuH48Tj2GbfYHrkZjmAn5I0e9I51243h+3ve6TgPPQHElKEskrMGgZgNEpS
jXGB18BTfLDM2ylIuKn1ANRIew==
-----END ENCRYPTED PRIVATE KEY-----
    </Value>
  </AssignVariable>
</AssignMessage>

That's what I would try.

[amitsharma]

Thank you @Dino-at-Google . I have tried hard-coding the values in AssignMessage and set the private variables. Unfortunately this didn't work and I got the same error message.

Then I went down the route of removing the password from the private key using openssl rsa -in key.pem -out key_with_no_pw.key command and used key which was not password protected. This test worked.

The password for the private key is correct. I have validated it by executing following command

openssl rsa -in private-key.pem -check

Just a hunch, most probably a silly one. There is something strange I have noticed with the size of pem file generated by openssl. I was looking at the private key file which is working file in other tenant. The size of this file is 1.99kb. However if I create a new pem from the same pfx and using same password, the file size is 2.05 kb. I do understand that the algorithm generates different files.

[dchiesa1]

Hmm, strange. Some inconsistency there. Are you certain that the encryption algorithm is the same? There are some crypto algorithms supported by openssl for encrypting, that are not supported by Apigee Edge when decrypting a private key.

Apigee uses BouncyCastle for decrypting, and the BC library does not support every algo that openssl supports.

[amitsharma]

Hey @Dino-at-Google, thank you for pointing me in the right direction. I tried loading the key using BouncyCastle and it didn't work. So I checked the pem file for hidden characters and found that the pem file which is working has LF at end of each line. However the pem file which is not working has CRLF.

I had a suspicion about the version of openssl I was using to generate the private key. It turnout that I was using openssl which was installed with Git for windows (OpenSSL 1.1.1d 10 Sep 2019, git version 2.24.1.windows.2). This version is adding CRLF. However when I tried generating the file with openssl which is installed with Ruby 2.5.3 (windows version) OpenSSL 1.0.2p 14 Aug 2018 and the private key file generate had LF at end of each line. This file worked properly with BouncyCastle and Apigee.

I don't understand the intricacy of the encryption so not sure what's going on here. But for now my API proxy is working as expected.

[dchiesa1]

WOW, thank you Amit.

I'm glad you were able to solve the problem.

Thanks for the reply and for all the followup. What you found is very interesting. I would like Apigee to be more resilient in the face of such differences, so we'll look into making that happen (complying with Postel's law).