{ Community }
  • Academy
  • Docs
  • Developers
  • Resources
    • Community Articles
    • Apigee on GitHub
    • Code Samples
    • Videos & eBooks
    • Accelerator Methodology
  • Support
  • Ask a Question
  • Spaces
    • Product Announcements
    • General
    • Edge/API Management
    • Developer Portal (Drupal-based)
    • Developer Portal (Integrated)
    • API Design
    • APIM on Istio
    • Extensions
    • Business of APIs
    • Academy/Certification
    • Analytics
    • Events
    • Hybrid
    • Integration (AWS, PCF, Etc.)
    • Microgateway
    • Monetization
    • Private Cloud Deployment
    • Insights
    • IoT Apigee Link
    • BaaS/Usergrid
    • BaaS Transition/Migration
    • Apigee-127
    • New Customers
    • Topics
    • Questions
    • Articles
    • Ideas
    • Leaderboard
    • Badges
  • Log in
  • Sign up

Get answers, ideas, and support from the Apigee Community

  • Home /
  • Microgateway /
avatar image
1
Question by Kristopher Kleva · Dec 31, 2015 at 08:40 PM · 1.3k Views node.jsmicrogatewayperformance

Apigee Edge Microgateway Vertical and Horizontal scaling for 1000 TPS and beyond.

Hello,

I'm looking for some quick insights regrading best practices for vertically and horizontally scaling Apigee Micro-gateway instances.

Vertical - Can / should I run multiple micro-gateway node.js processes on the same physical server?

Horizontal - What should I consider when running multiple micro-gateways across physical servers?

Apigee Edge Analytics - When sending 1000+ TPS into scaled micro-gateways, how should we plan capacity on Apigee Edge Cloud with Analytics Ingest?

Your thoughts are appreciated!

Comment
Add comment
10 |5000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by Apigeeks only
  • Viewable by the original poster
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Close

3 Answers

  • Sort: 
avatar image
1
Best Answer

Answer by prabhat · Jan 01, 2016 at 08:07 PM

Hi Kristopher,

Our recommendation is to run multiple instances of MGWY on different servers front by load balancers like nginx.

It's possible to run multiple instances of MGWY on same server using node's cluster module. In upcoming releases we are working on rearranging few things so that multiple instances can "easily" run on same server but our recommendation to stick with multiple servers.

Comment
Add comment Show 4 · Link
10 |5000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by Apigeeks only
  • Viewable by the original poster
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image Kristopher Kleva · Jan 08, 2016 at 10:45 PM 1
Link

There are some aspects of running Node.js at the scale that I think we are leaving out here. I've found pretty quickly 'open file ulimit' is at least one BLOCKER I've found attempting to get Edge Micro to run at scale. See: https://community.apigee.com/questions/15162/edge-microgateway-error-during-load-testing.html

I've still not been able to reach my 1000TPS goal so I will continue to make attempts and tuning the operating system network and test.

In terms of load balancers. What sort of indicators would prompt you to use auto scaling? Would it be advisable if we are good a predicting traffic?

avatar image Kristopher Kleva Kristopher Kleva · Jan 20, 2016 at 10:37 PM 1
Link

Update. We've succeeded running our 1000 TPS load test with just a bit more network and operating system tuning.

avatar image prabhat ♦ Kristopher Kleva · Jan 20, 2016 at 10:45 PM 1
Link

This is awesome. If you can, pls share network and os tuning you had to done for others benefits.

avatar image Benjamin Goldman Kristopher Kleva · Feb 05, 2016 at 02:05 AM 0
Link

Share please.

avatar image
1

Answer by Benjamin Goldman · Feb 18, 2016 at 07:21 PM

Thanks Kristopher - that means that you didnt see 1000 tps on one instance then? To get this you load balanced multiple instances. Did you put these instances on one compute instance or did you spread it out across multiple?

Comment
Add comment Show 3 · Link
10 |5000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by Apigeeks only
  • Viewable by the original poster
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image Benjamin Goldman · Feb 17, 2016 at 09:49 PM 0
Link
@Kristopher Kleva

could you share the network tuning you did? im curious what i might be missing.

avatar image Kristopher Kleva Benjamin Goldman · Feb 18, 2016 at 07:17 PM 2
Link

Here are a couple things to consider right away.

#1 Ulimits - It appears that the number of open files when attempting high TPS is higher than most operating systems has set as a default

#2 Load balancing across micro gateways is a must. I found pretty quickly we ran into issue with a single node and didn't get great stability till I was was load-balancing across 4-5 nodes.

avatar image Kristopher Kleva Benjamin Goldman · Feb 18, 2016 at 09:11 PM 2
Link

For networking looking into increasing your TCP buffer maximum and limits. Google 'linux TCP tuning' and there are all sorts of articles that can help guide you through the basics.

I was not able to get to 1000 TPS with a single instance. However, I didn't see this as a limitation of the Edge Micro-gateway but more able the hosting system and my API Target. I was finding that depending on the response time of the target I'd run into issues as the throughput was increased. I would not recommend using Edge Micro in production without load-balancing but in dev, qa it should be fine.

avatar image
1

Answer by prabhat · Feb 18, 2016 at 09:19 PM

We did find a performance related bug that we now have fixed. It will be available in next release.

Comment
Add comment Show 3 · Link
10 |5000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by Apigeeks only
  • Viewable by the original poster
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image Benjamin Goldman · Feb 22, 2016 at 10:47 PM 0
Link

@prabhat I want to share what we are seeing now w/ micro gateway. for the record we are testing w/ the fixes both in the edge micro code, and in one of the policies in the gateway its self - and we are testing on premise.

avatar image Benjamin Goldman · Feb 22, 2016 at 11:02 PM 0
Link

we found that we still had what appeared to be a bottleneck where the systems would start to fail after about 30 seconds of work - but we were pretty sure that this was where linux network tuning would come in handy.

We are running 2 sets of tests:

1) gatling

2) jmeter w/ slave traffic generators

I will focus on the gatling test settings first - because they are the ones that i have at hand.

On the traffic generator server (the gatling box) i made the following changes:

sudo sysctl net.ipv4.ip_local_port_range="15000 61000"
sudo sysctl net.ipv4.tcp_fin_timeout=10
sudo sysctl net.ipv4.tcp_tw_recycle=1
sudo sysctl net.ipv4.tcp_tw_reuse=1 

on the server running the edge micro installation we made these changes:

sudo sysctl net.ipv4.ip_local_port_range="15000 61000" 
sudo sysctl net.ipv4.tcp_fin_timeout=10
sudo sysctl net.ipv4.tcp_tw_recycle=1
sudo sysctl net.ipv4.tcp_tw_reuse=1 

sudo sysctl net.core.somaxconn=1024


ifconfig eth0 txqueuelen 5000
echo "/sbin/ifconfig eth0 txqueuelen 5000" >> /etc/rc.local
sudo sysctl net.core.netdev_max_backlog=2000
sysctl net.ipv4.tcp_max_syn_backlog=2048

If you want the WHYs go read this wonderful stackoverflow article:

http://stackoverflow.com/questions/410616/increasing-the-maximum-number-of-tcp-ip-connections-in-linux

Note - i am in the process of running a test for 30 minutes against a single core node.js edgemicro install. It did pretty well (not perfectly) - but this was also in our internal cloud which is not as performant as AWS is.

I will be re-running the tests against 1000, and 1200 req/second tonight to see where our hardware gives up.

avatar image Benjamin Goldman · Feb 22, 2016 at 11:12 PM 0
Link

one further note: our openfiles are set to unlimited :) so that was not part of our problem. In the process of pushing a single instance of edgemicro i did change it to 800k or something just to make sure that the unlimited wasnt some misunderstanding. With the rest of the settings this didnt make any difference (obviously..)

Follow this Question

Answers Answers and Comments

34 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

Can I use a Node.js based API in Microgateway? 1 Answer

Preferred Pattern for Micro Gateways with unique plugin and sequences on conditional resource paths. 0 Answers

Node.js Version for Edge Microgateway 1 Answer

Step Debugging Edge Micro Proxies 2 Answers

Streaming of large files with microgateway 4 Answers

  • Products
    • Edge - APIs
    • Insights - Big Data
    • Plans
  • Developers
    • Overview
    • Documentation
  • Resources
    • Overview
    • Blog
    • Apigee Institute
    • Academy
    • Documentation
  • Company
    • Overview
    • Press
    • Customers
    • Partners
    • Team
    • Events
    • Careers
    • Contact Us
  • Support
    • Support Overview
    • Documentation
    • Status
    • Edge Support Portal
    • Privacy Policy
    • Terms & Conditions
© 2019 Apigee Corp. All rights reserved. - Apigee Community Terms of Use - Powered by AnswerHub
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Create an article
  • Post an idea
  • Spaces
  • Product Announcements
  • General
  • Edge/API Management
  • Developer Portal (Drupal-based)
  • Developer Portal (Integrated)
  • API Design
  • APIM on Istio
  • Extensions
  • Business of APIs
  • Academy/Certification
  • Analytics
  • Events
  • Hybrid
  • Integration (AWS, PCF, Etc.)
  • Microgateway
  • Monetization
  • Private Cloud Deployment
  • Insights
  • IoT Apigee Link
  • BaaS/Usergrid
  • BaaS Transition/Migration
  • Apigee-127
  • New Customers
  • Explore
  • Topics
  • Questions
  • Articles
  • Ideas
  • Members
  • Badges