AWS from Zero to Production — Phase 4, Part 5 of 6 · See the full series roadmap
Imagine a feature that resizes every image users upload.
With servers, you’d provision instances, install software, patch them, scale them for peak load, and pay for them all night while nobody uploads anything.
With serverless, you write one function. It runs when a file lands in S3, scales to thousands of uploads at once, and costs nothing when nobody’s uploading.
That’s the appeal. But serverless also has limits, quirks, and failure modes that surprise people in production. This article covers both.
What you’ll walk away with
- ⚡ How Lambda works: functions, execution roles, environment variables, and layers
- 🏷️ Versions and aliases for safe releases
- 🚦 Concurrency, cold starts, and throttling
- 🔌 Event source mappings — Lambda reading from queues and streams
- 🌐 API Gateway, Step Functions, and EventBridge
- 📬 SNS, SQS, and dead-letter queues
- 🧰 The Serverless Application Model (SAM)
What “serverless” really means
There are still servers. You just don’t manage them. Serverless services share four traits:
- No infrastructure to manage — no instances, patching, or capacity planning.
- Automatic scaling, including to zero.
- Pay for use, not for idle capacity.
- Built-in availability across AZs.
AWS Lambda
AWS Lambda runs your code in response to events. You upload a function; AWS runs it when triggered.
import json
def handler(event, context):
name = event.get("queryStringParameters", {}).get("name", "world")
return {
"statusCode": 200,
"headers": {"Content-Type": "application/json"},
"body": json.dumps({"message": f"Hello, {name}!"}),
}
Key limits and settings
- Runtimes: Python, Node.js, Java, .NET, Ruby, and custom runtimes — or a container image up to 10 GB.
- Memory: 128 MB to 10,240 MB. CPU scales with memory, so more memory often means faster and sometimes cheaper.
- Timeout: up to 15 minutes. Longer jobs belong elsewhere — Fargate, Step Functions, or AWS Batch.
- Ephemeral storage:
/tmpfrom 512 MB up to 10 GB. - Pricing: per request plus compute time in GB-seconds. Check current Lambda pricing and Free Tier.
Execution roles
Every function has an execution role — the IAM role it assumes when it runs. At minimum, it needs permission to write logs to CloudWatch. Add exactly what the function calls, nothing more.
Environment variables
Use them for configuration: table names, feature flags, and endpoints. For secrets, store a reference and fetch the value from Secrets Manager or Parameter Store at runtime — or use the AWS Parameters and Secrets Lambda Extension, which caches them.
Layers
A layer is a ZIP archive of libraries or shared code that multiple functions can use. A function can use up to five layers. They keep deployment packages small and share common dependencies.
Versions and aliases
- Publishing a version creates an immutable snapshot of your function’s code and configuration:
orders-fn:1,orders-fn:2. - An alias is a named pointer to a version, like
prod -> 7.
Triggers and clients call the alias. To release, point the alias at a new version. To roll back, point it back.
Aliases also support weighted routing, for example 90% to version 7 and 10% to version 8 — a built-in canary release.
Concurrency: how Lambda scales
Concurrency is the number of requests your functions are handling at the same moment. Each concurrent request runs in its own execution environment.
- Your account has a Regional concurrency quota, shared by all functions. It can be raised.
- When the quota is reached, new requests are throttled.
- Reserved concurrency guarantees capacity for a critical function — and also caps it, which protects downstream systems like a database.
- Provisioned concurrency keeps environments initialized and ready, removing cold starts for latency-sensitive functions, at extra cost.
Cold starts
When Lambda creates a new execution environment, it downloads your code and runs initialization code. That’s a cold start. It adds latency to that request.
Reduce it by:
- Keeping deployment packages small
- Initializing SDK clients outside the handler, so they’re reused
- Using provisioned concurrency or SnapStart for supported runtimes
Watch out: Lambda can scale far faster than a relational database can accept connections. A burst of 1,000 concurrent functions each opening a connection can overwhelm the database. Use RDS Proxy (Phase 5), reserved concurrency, or a queue in between.
How Lambda gets invoked
- Synchronous: the caller waits for a response. API Gateway, ALB, and direct SDK calls.
- Asynchronous: the event is queued, and Lambda retries on failure. S3 event notifications, SNS, and EventBridge.
- Event source mappings: Lambda polls a queue or stream and invokes your function with batches of records. Used for SQS, Kinesis, DynamoDB Streams, Amazon MQ, and Kafka.
Event source mappings in practice
SQS queue ---> Lambda polls ---> function gets a batch of up to N messages
success -> messages deleted
failure -> messages return to the queue
Enable partial batch responses, so one bad message doesn’t force the whole batch to be retried.
Amazon API Gateway
API Gateway puts an HTTPS API in front of Lambda functions and other backends.
- HTTP APIs: simpler, faster, and cheaper. JWT authorizers, CORS, and Lambda or HTTP integrations. The default choice for most new APIs.
- REST APIs: more features: API keys and usage plans, request validation, caching, AWS WAF, and private APIs.
- WebSocket APIs: real-time, two-way communication, like chat.
API Gateway handles throttling, authorization, and custom domains with ACM certificates.
AWS Step Functions
Step Functions coordinates multi-step workflows as state machines: run this, then that; retry on failure; branch; wait; run in parallel.
Order received
|
v
Validate payment ----(fails)----> Notify customer
|
v
Reserve stock (retry 3x with backoff)
|
v
Parallel: send email | update analytics
|
v
Done
- Standard workflows: long-running (up to a year), exactly-once execution, with full history. Good for business processes.
- Express workflows: high-volume and short (up to five minutes), at-least-once, cheaper per execution. Good for event processing.
Step Functions can call over 200 AWS services directly, often with no Lambda in between.
Amazon EventBridge
EventBridge is a serverless event bus. Services publish events; rules match them and route them to targets.
{
"source": ["myapp.orders"],
"detail-type": ["OrderPlaced"],
"detail": { "total": [{ "numeric": [">", 1000] }] }
}
This rule matches only orders over 1,000 — and can send them to Lambda, Step Functions, SQS, or many other targets.
- Receives events from AWS services, your apps, and SaaS partners.
- EventBridge Scheduler runs one-time or recurring tasks at scale — a serverless cron.
Phase 10 goes deep on event-driven design.
Amazon SNS and Amazon SQS
SNS — publish and subscribe
Amazon Simple Notification Service pushes each message to all subscribers: Lambda functions, SQS queues, HTTP endpoints, email, or SMS. One event, many reactions.
SQS — queues
Amazon Simple Queue Service stores messages until a consumer processes them. It decouples producers from consumers and absorbs bursts.
- Standard queues: nearly unlimited throughput, at-least-once delivery, best-effort ordering.
- FIFO queues: strict ordering and exactly-once processing within a message group, with lower throughput limits.
- Visibility timeout: while a consumer processes a message, it’s hidden from others. Set it longer than your processing time.
The fan-out pattern
+--> SQS queue --> billing service
SNS topic -----+--> SQS queue --> email service
+--> SQS queue --> analytics service
Each service gets its own queue, processes at its own pace, and fails independently.
Dead-letter queues
Some messages will always fail: bad data, a bug, a missing record. Without a plan, they retry forever or vanish.
A dead-letter queue (DLQ) catches messages that fail too many times:
- On an SQS queue, set a redrive policy with
maxReceiveCount. After that many failed attempts, the message moves to the DLQ. - For asynchronous Lambda invocations, configure a DLQ or an on-failure destination.
- Then alarm on DLQ depth, investigate, fix, and redrive messages back to the source queue.
Rule: Every queue and asynchronous function in production should have a DLQ — and an alarm on it.
AWS Serverless Application Model (SAM)
SAM is an extension of CloudFormation with short syntax for serverless resources, plus a CLI for building, testing locally, and deploying.
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Resources:
HelloFunction:
Type: AWS::Serverless::Function
Properties:
Runtime: python3.12
Handler: app.handler
CodeUri: src/
MemorySize: 256
Timeout: 10
Events:
Api:
Type: HttpApi
Properties:
Path: /hello
Method: get
sam build
sam local invoke HelloFunction
sam deploy --guided
Phase 6 covers CloudFormation, which SAM builds on.
When serverless is — and isn’t — a good fit
Great for
- Event-driven processing: files, queues, streams, and schedules
- APIs with spiky or unpredictable traffic
- Glue code between services
- Workloads with long idle periods
Think twice for
- Long-running jobs over 15 minutes
- Steady, high-throughput workloads, where containers may be cheaper
- Very latency-sensitive paths that can’t tolerate cold starts
- Apps that need many persistent connections to a relational database
Common mistakes
- Initializing SDK clients inside the handler, paying setup cost on every call.
- No DLQ or failure destination on asynchronous work.
- Visibility timeout shorter than processing time, causing duplicate processing.
- Unbounded concurrency overwhelming a database.
- Assuming exactly-once delivery from standard queues. Make consumers idempotent.
- Secrets in plain environment variables.
- Wildcard permissions on execution roles.
Key takeaways
- Lambda runs code on events, scales automatically, and bills per request and duration.
- Use versions and aliases for safe releases and instant rollback.
- Concurrency drives scaling; reserved concurrency protects downstream systems.
- API Gateway fronts APIs, Step Functions orchestrates workflows, and EventBridge routes events.
- SNS fans out; SQS buffers; DLQs catch what fails.
- SAM makes serverless infrastructure as code short and testable.
Quick self-check
Test yourself before moving on:
- What’s the maximum Lambda timeout?
- How do you release a new Lambda version with instant rollback?
- A message keeps getting processed twice. What SQS setting should you check first?
- Where do messages go after failing
maxReceiveCounttimes?
Answers
- 15 minutes.
- Publish a version and point an alias at it; roll back by pointing the alias back.
- The visibility timeout — it may be shorter than the processing time. And make the consumer idempotent.
- To the dead-letter queue.
What’s next
In Part 6, the Phase 4 lab: containerize an API, push it to Amazon ECR, and run it on ECS Fargate in private subnets behind an Application Load Balancer — then ship a second version with zero downtime.
Serverless fan or skeptic? Make your case in the comments.