What's the maximum recommended size for KVM?

Not applicable

Hi experts,

Several posts mention that KVMs should not be used for storing large data sets. The provided reasons make sense. At the same time, it's not clear what is a large data set. Any idea what's the maximum recommended size? Is it 10, 100, 1000, 10K entries per KVM? Thanks a lot in advance!

https://community.apigee.com/content/kbentry/2825/storing-credentialssensitive-config-kvm-vs-vault.h...

https://community.apigee.com/questions/7289/do-we-have-a-constraint-on-the-length-of-key-in-kv.html

2 7 1,714
7 REPLIES 7

"A key value map has a 15 MB limit. Writing to the map may fail close to or beyond this limit. If the amount of data might grow beyond 15 MB, be sure to use the ExpiryTimeInSecs element to expire entries before exceeding the limit."
(Copied from Docs).

And i think there is no limit on Number of entries, if you are not close to 15MB size.

ExpiryTimeInSecs refers to the time in which Edge refreshes its cached value from the KeyValueMap. It is not about expiring or removing items automatically from the datastore. Therefore this setting has no benefit towards controlling the data size.

Hi @oseymen@apigee.com

Thank you for clarification!

Hi @Abdullah Ahmed

Thanks for your answer! It's good to know about the limit of the KVM. I'm trying to find out if there is a maximum recommended size. My guess is that performance is going to suffer significantly when KVM size gets close to 15 MB. I was curious about real-life examples of dependency between KVM size and KVM access performance. For example, will it take twice more time to read entries from a 2 MB KVM vs 1 MB KVM? Will performance get exponentially worse as KVM size increases? Is there a "sweet spot" on the KVM size chart when performance overhead is negligible?

Hi @Andrei Filippov,

Performance depends on the server configuration, heap size allocated to processes and network latency. Also, depends on type of other policies getting executed as part of the proxies.

Based on my experience, I would suggest to keep KVM's size less than 1 MB. Keep an eye on response time once KVM size is around MBs. Also, I would suggest to start looking at alternative design options like optimizing size of data getting stored, BaaS, Multiple KVMs, etc.

Cheers,

Hi @rdoda

Thanks a lot for sharing your experience! It's very helpful.

Hi @Andrei Filippov

Thanks for sharing the info. I was wondering if there is a way to estimate if my KVM would exceed 15 MB.