Blog

AWS Autoscaling & Queues

In GPSN we have this task stereotype of spinning up a group of VMs to process task items in an SQS queue. While reading through the autoscaling docs two things jumped out at me: The minimum size of size of an autoscale group is zero, and autoscaling can be triggered off of the size of a queue. In other words, we could have a fifteen-step process based on fifteen stacks and fifteen queues, and we dont have to manually start any of them – we can just let the stacks autoscale down to zero instances when their queue is empty.

nn

A great side-effect of this is that each stack will start worker VMs when items start popping up in the queue theyre watching. While not as fine-grained as a full-blown Actor Model framework, this still lets multiple types of tasks through the system in parallel and at their own pace.

n