Operating Lambda: Application design — Scaling and concurrency

Scaling and concurrency in Lambda

AWS Lambda provides scaling as part of the service i.e. as traffic increases, Lambda increases the number of concurrent executions of our functions without the need of threading or any custom engineering in our code.

Request and Concurrency

One Lambda instance can handle only one request at a time. However, for asynchronous invocation like SQS, we can batch the messages in a single Lambda request that is processed together.

OnDemand scaling

Concurrent instances of Lambda are created based on the volume of requests received. For each new request, either an existing free environment is reused or a new environment is provisioned that will experience a cold start.

Provisioned Concurrency

For asynchronous workloads, the default scaling and concurrency limits provide a reasonable tradeoff between throughput and configuration management overhead. However, for synchronous workload with expected traffic more than the default burst capacity or for a use case where double-digit latency is desirable, Provisioned concurrency is recommended.

Using service integrations and asynchronous processing

Integration of Lambda with Synchronous Services can be improved by providing an asynchronous experience especially for the case when the service can tolerate latency, the expected traffic pattern is unpredictable and throttling customer requests need to be avoided.

Reserved concurrency

All the Lambda functions in your account share the same bucket of concurrency limits. This creates a coupling between the functions and if one function consumes all the limits, other functions will fail. This can be prevented by setting a reserved concurrency limit on high volume and critical Lambdas making sure they have the required limits.

Conclusion

This post discusses the different approaches and designs that can be used to better architect your service in serverless. Also, refers to this article to understand the limits of Lambdas and how to architect a service around it.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store