Hi All,
I'm trying to use this example with my own application :
Example: Using instance assignment to scale based on a Pub/Sub queue
https://cloud.google.com/compute/docs/autoscaler/scaling-stackdriver-monitoring-metrics#example_usin...
I have built a managed instance group with streaming pull subscription application in it. I have set a autoscaling to scale based on Pub/Sub subscription volume.
I currently also have a pub/sub subscription linked to this MIG that will receive roughly 50 message in a short amount of time (less than a min) daily.
My MIG would process 2 message per pull (Using flow control) that would work over 7 min before acknowledge the message to Pub/Sub.
My concern comes with that because the job takes really long time to process, If MIG happens to scale down, would it accidentally scale down the machine that is still taking the job (My Job is I/O and Network Heavy but not CPU heavy)?
I have an understanding that there is a 10 min gap before the machine actually scale down but what if a new pulling comes in during the 10 min gap (ex: new pulling comes in on 8 min within the gap). I'm concerned because the job running inside is quite heavy to some of my resource.
Are there any good strategy on managing the scale in(down) and would it really kill my MIG in mid Job?
Thanks for the response in advance!
LinkedIn
Twitter
