Cloud Computing
AWS Lambda GPU Support Enables Model Sharing Across 48 Concurrent Requests
Lambda's new GPU capabilities on managed instances let event-driven functions share large language models efficiently, with up to 48 invocations running from a single GPU copy.
InfoQ
AWS has introduced GPU acceleration for Lambda Event Functions running on managed instances, allowing developers to run inference workloads on EC2 G-series processors without managing drivers, clusters, or device infrastructure. The feature is now generally available across commercial regions where both managed instances and GPU hardware coexist.
How It Works#
The GPU model leverages Lambda's managed instances architecture, where multiple invocations share a single execution environment. With the new AcceleratorConfig parameter, functions declare GPU memory requirements—ranging from 3 to 48 GB—alongside traditional configuration like vCPU allocation and storage. A single model loaded in GPU memory can serve up to 48 concurrent invocations, with scaling driven by GPU utilization, CPU demand, and concurrency metrics.
Developers deploy container images (up to 32 GB) using one of five AWS-provided base images: CUDA, CuPy, PyTorch, vLLM, or SGLang. The vLLM and SGLang variants require no handler code—just a MODEL_ID environment variable pointing to a Hugging Face repository, ModelScope, an S3 path, or local storage. Swapping models becomes a simple configuration update followed by publishing a new version.
Cold Start and Optimization#
Publishing a version pre-warms the environment by downloading model weights before the first event arrives. Seekable OCI lazy loading can reduce startup time by up to 70 percent, addressing a traditional serverless cold-start limitation. Instance types are selected automatically, with Lambda preferring newer generations and falling back to older ones when capacity is constrained.
Pricing and Trade-offs#
The service charges EC2 on-demand rates plus a 9.75 percent management fee (lower than the 15 percent CPU overhead), plus the standard Lambda request fee. Savings Plans and Reserved Instances apply to the underlying instances, with no separate duration charges—billing continues only while minimum environment counts remain above zero.
Early testing revealed practical constraints: Terraform and Cloud Control API support for AcceleratorConfig remain pending, and new AWS accounts start with zero GPU vCPU quota. One benchmark using PyTorch showed roughly 66× speedup on an L4 GPU versus CPU execution. A production test running Gemma 4 E2B on a single g6.2xlarge through vLLM achieved 3,400–3,600 tokens per second across 48 simultaneous requests, though the 484-second publish-to-active time was a bottleneck.

