Providing KVM Content APIs for Apigee X and hybrid

Update June 2022: 

KVM APIs are now provided for Apigee X. The runtime proxy is no longer required to edit KVM entries.
Please see the following Apigee API docs pages for the specification of the KVM API operations:

 

In this short post we will talk about the implementational difference between Apigee Edge (SaaS offering in the public cloud) and Apigee X or Apigee hybrid with regard to managing the contents of Apigee Key Value Maps (KVMs).

Whilst most of the paradigms and interfaces for how to implement API proxies remain unchanged between Apigee Edge and Apigee X and hybrid, there are (as of the writing of this post) a few architectural differences that require a slightly different way of thinking. Most notably to highlight here:

  • KVM contents are no longer managed via the Apigee API. Read below for how to manage KVM contents from within the API runtime plane.
  • Hosted targets are no longer supported but can be easily migrated to Cloud Run as described in a previous blog post.
  • Edge extensions are not supported. Some common scenarios are covered by reusable shared flows that are available as open source community assets in Apigee DevRel

Why are KVMs handled differently in Apigee Edge vs. Apigee hybrid and Apigee X?

The difference in how KVM content is handled comes down to a few factors. In Apigee Edge the KVMs are managed in a true multi-tenant fashion. The authorization and the optional encryption of the content is handled transparently by Google. In Apigee X and hybrid the KVMs are kept at the runtime level. This means that the customer is in control of the encryption and the content. Because of that, all operations on the KVM’s content need to be performed from within the runtime itself.

As a consequence of this, the previous Management API functionality to manage KVM entries no longer apply to Apigee X or hybrid. The KVMs in Apigee X or hybrid still need to be created via the management API or the Apigee UI but their content can only be created, read or deleted from within the runtime from a KeyValueMapOperations policy.

Operating KVM entries via a custom API Proxy

Assuming you already created a KVM via either the Apigee API or the UI, you can edit its content using the KeyValueMapOperations policy. If you have used this policy before you might remember the mapIdentifier attribute of the root element of this policy that is used to statically define the name of the KeyValueMap that the policy operates on. You can still use this attribute to create dedicated API proxies for each KVM you wish to manage entries in. If you have several KVMs that need this functionality then adding proxies with static KVM names for each one of them will quickly become impractical.Luckily, Apigee X as well as Apigee hybrid from version 1.4 onwards added the option to use dynamic KVM names in the KeyValueMapOperations policy in the form of the MapName element. This allows you to obtain the name of the KVM from the incoming request and use it in the KVM operations policy. For a restful URI the kvm name could be read into a flow variable called kvm.name using an ExtractVariables policy as follows:

<ExtractVariables name="EV-PathParameters">
    <URIPath>
        <Pattern ignoreCase="true">/kvms/{kvm.name}/entries/{kvm-entry.key}</Pattern>
        <Pattern ignoreCase="true">/kvms/{kvm.name}/entries</Pattern>
    </URIPath>
</ExtractVariables>

A full example of an implementation of the GET, POST and DELETE methods for KVMs with a dynamic name can be found in the Apigee DevRel Github repository.

Operational considerations

As briefly mentioned before, KVMs have to be created before their name can be used in the MapName element. Depending on the automation requirement, the KVMs could also be automatically created by calling the Apigee API from within the Apigee proxy. For most use cases creating the KVMs would still be done as part of the configuration automation.

A second important aspect then it comes to operating a KVM admin proxy is authentication and authorization. As KVMs often contain important configuration, it is important that only authorized users can modify its content. Users are therefore strongly advised to enforce additional security policies e.g. via OAuth2 on the KVM admin API proxy before making it available.

Contributors
Comments
priyashikhar
Bronze 3
Bronze 3

Thanks strebel.

Using KVM-admin-api repo for create KVM name, push entries in KVM name.

There is any way to retrieve all entries in particular KVM using management Api or KVM-admin-api  in Apigee X?

 

strebel
Staff

Sorry for the late reply @priyashikhar I somehow missed this. 

As of today listing the keys is not a supported feature of either the KVM admin proxy or the Apigee API. 
The two workarounds that I am aware of are:

a) Store the keys in another service like firebase
b) Use a dedicated known key like "keys" and store all the keys used in its value.

b) of course requires you to respect the maximum size of allowed values as per the Apigee limits as well as well as ensuring that the write to the keys list is ACID.

debjitd18
Bronze 3
Bronze 3

@strebel 

Could you please advise on the below:

I have used the Management API to create a KVM at Organization level, however when I tried to view/list, it does not show up in the Apigee X UI.

Create:

https://cloud.google.com/apigee/docs/reference/apis/apigee/rest/v1/organizations.keyvaluemaps/create

I tried the to use the list method but the Try It gives me an error message. Is there something I am missing to do here

List:

https://cloud.google.com/apigee/docs/reference/apis/apigee/rest/v1/organizations.keyvaluemaps/list

 

strebel
Staff

hi @debjitd18 which version of Apigee are you using? 

Please note that if you'r using Apigee hybrid < 1.8. the KVM Admin API isn't supported.
https://cloud.google.com/apigee/docs/reference/apis/apigee/rest/v1/organizations.keyvaluemaps.entrie...

debjitd18
Bronze 3
Bronze 3

@strebel 

Sure thank you, but the error I am seeing while trying to access the List method is there something I am missing to do:

https://cloud.google.com/apigee/docs/reference/apis/apigee/rest/v1/organizations.keyvaluemaps/list

debjitd18_0-1666962019615.png

 

debjitd18
Bronze 3
Bronze 3

@strebel 

sorry for a late follow up and response. yes we are using a version of Apigee X > 1.8x.

Could you let me know why the list method returns the error I had pointed out in my previous post.
Also as per the Key Value Map Operations policy, it gives us a feature to:

a) Update an existing entry since it uses a PUT but while Management APIs allows only a POST functionality which would not allow a user to update an existing entry.
b) Also it allows to create more than one Value for a Key and retrieve it using the GET based on the index, but I did not find a similar functionality here with the Management APIs.

Is there something am missing to look up.

Best Regards,

Debjit

strebel
Staff

There seems to be a UI issue that I am following up on:

Meanwhile the curl should work as documented

curl https://apigee.googleapis.com/v1/organizations/YOUR_ORG/keyvaluemaps -H "Authorization: Bearer $(gcloud auth print-access-token)"

# Result
[
  "foomap1",
  "kvmtestmap",
  "org-kvm"
]

I understand your confusion about he multiple values. Probably some improvement in the documentation is needed on that point:

For Apigee Edge: The Management API allows you to create multiple values per key
For Apigee X/hybrid: The Management API performs an Upsert operation (create or override the existing value)

debjitd18
Bronze 3
Bronze 3

@strebel 

Thanks for your response. I tried the POST option to create a new Key Value pair for a KVM and then I did try to update with a new Value for the similar/existing Key it did not perform an Upsert but returned a 409 :

debjitd18_0-1670238322615.png

 

Version history
Last update:
‎06-14-2022 12:10 AM
Updated by: