The Backend Ladder · Deep dive · Pillar C of three
Cloud fluency is knowing what happens between a user typing a hostname and your handler running — every hop, who authorised it, what it costs, and what breaks it. This page follows that path all the way down: IP ranges, IAM decisions, failover clocks, the scheduler, and the bill.
Start with the map. Every later section is one box on this diagram, opened up.
Nine components sit between a user and your handler. Knowing the order lets you bisect an incident in one pass instead of guessing — and knowing what each hop fails like tells you which alert fired for a reason.
AWS publishes well over two hundred services. You will meet about twenty-five. This part is the working set — what each one is for, and the handful of facts about each that decide whether your design works — followed by the two databases you will spend the most time with.
Grouped by the job they do, not by the console's menu. If you can place a service in this grid and say one true thing about its limits, you know enough to start designing with it.
| Service | The one thing to know | The gotcha that bites |
|---|---|---|
| S3 | Effectively unlimited object storage, strongly consistent for reads after write. | Request cost at high QPS, and a bucket policy that quietly makes it public. |
| Lambda | Scales to zero, pays per millisecond, 15-minute maximum. | Cold starts, and one function's concurrency starving every other function in the account. |
| ECS Fargate | Containers with no nodes to patch — the low-ops default for a steady service. | Slower to start than Lambda, and no bin-packing, so you pay per task. |
| EKS | Real Kubernetes, portable, every knob available. | You own upgrades, the CNI, IP planning, and a platform team's worth of work. |
| SQS | Durable work queue with retries and a DLQ, no capacity to manage. | Visibility timeout shorter than the handler means duplicate processing. |
| EventBridge | Content-based routing between services, plus schedules and SaaS events. | At-least-once with no ordering; latency is fine but not sub-millisecond. |
| ElastiCache | Managed Redis or Valkey, sub-millisecond in the same AZ. | A failover drops connections; the client must reconnect and the origin must survive the miss storm. |
| KMS | Keys that never leave the service; every encrypt/decrypt is an API call. | Request rate limits, and a key policy separate from IAM that must also allow you. |
| CloudWatch | Metrics, log groups, alarms, dashboards — the default for everything. | Ingestion cost, and custom metric cardinality that grows without limit. |
RDS is a normal Postgres or MySQL server with the operations attached: backups, failover, patching, monitoring. Everything below is a switch you turn on — and most production incidents come from one that nobody turned on.
| Feature | What it gives you | What to actually do |
|---|---|---|
| Automated backups + PITR | Restore to any second within a 1–35 day window. | Set retention to at least 7 days, and rehearse a restore — an untested backup is a hypothesis. |
| Manual snapshots | Kept until you delete them; survive instance deletion. | Take one before every risky migration; automated backups die with the instance. |
| Multi-AZ | Automatic failover to a synchronous standby. | On for anything that matters. Budget 60–120s of failed writes and retry them. |
| Read replicas | Extra read capacity; cross-region copies; promotable. | Route only stale-tolerant reads there, and alert on replica lag. |
| RDS Proxy | Connection pooling in front of the database; faster failover; IAM auth. | Use it when many pods or Lambdas each hold a pool — it is the standard fix for connection exhaustion. |
| Parameter groups | Server configuration: max_connections, timeouts, logging. | Never use the default group — you cannot edit it. Static parameters need a reboot. |
| Performance Insights | Which queries consume database time, by wait event. | Turn it on before you need it; it is the fastest route from "the DB is slow" to a query. |
| Storage autoscaling | Grows the volume automatically up to a cap. | Set it. A full disk takes the database read-only, and storage cannot be shrunk afterwards. |
| Encryption at rest (KMS) | Transparent encryption of volume, snapshots and replicas. | Enable at creation — you cannot turn it on later without a snapshot-restore migration. |
| Blue/green deployments | A synced copy for major version upgrades, switched over in about a minute. | The sane path for a major engine upgrade; the in-place alternative is a long outage. |
max_connections scales with instance memory, not with your pod count. Forty pods × a pool of twenty is 800 connections against a db.t3.medium that allows roughly 450 — and Postgres spends real memory per connection. Size pools from Little's Law (concurrency = throughput × query time), not from a default, and put RDS Proxy in front once the fleet is large.
DynamoDB gives you single-digit-millisecond reads at any scale, in exchange for deciding your queries before you create the table. Its features are mostly ways of buying back the flexibility that constraint costs you.
| Feature | What it gives you | What to actually do |
|---|---|---|
| Partition key + sort key | One query returns an ordered range of items under one key. | Design keys from your access patterns first. Prefix sort keys (ORDER#, PROFILE#) to hold several entity types in one table. |
| GSI | A second key on the same data — a different way to look things up. | Project only the attributes you need; a GSI is a full copy, and every write to the base table writes here too. |
| LSI | Same partition key, alternative sort key, strongly consistent. | Must be created with the table and caps that partition at 10GB. Prefer a GSI unless you specifically need strong consistency. |
| Read consistency | Eventually consistent by default; strongly consistent on request (double cost). | Use strong reads only where read-your-writes matters. GSIs cannot do strong reads at all. |
| Conditional writes | attribute_not_exists or a version check, evaluated atomically. | This is your optimistic lock and your idempotency guard — the same ledger trick as the foundations page. |
| Transactions | All-or-nothing across up to 100 items, including across tables. | Real, and twice the cost. Use for the few writes that genuinely must be atomic. |
| Streams | An ordered change log per key, retained 24 hours, triggers Lambda. | The native outbox: update the item and let the stream publish the event. No dual write. |
| TTL | Items delete themselves after a timestamp attribute passes. | Free expiry for sessions and carts — but deletion is best-effort within ~48 hours, so filter on read too. |
| PITR | Restore the table to any second in the last 35 days. | Turn it on. It restores to a new table, so plan the cutover. |
| DAX | A DynamoDB-aware cache with microsecond reads. | Only for read-heavy hot keys; it adds a cache-invalidation problem you did not previously have. |
| Global tables | Multi-region, active-active replication. | Last-writer-wins conflict resolution — fine for profiles, wrong for balances. |
The real question is not scale — both scale further than most products need. It is whether you know your queries in advance, and whether you need to ask new ones later without a migration.
S3 is the one service almost every design touches, and the one most people know least about. It is not a filesystem: it is a keyed object store with a price list, and nearly every S3 surprise — a bill, a leak, a stale read — comes from treating it as a disk.
| Feature | What it gives you | What to actually do |
|---|---|---|
| Storage classes | The same object at a tenth or a twentieth of the price, at the cost of retrieval time and fees. | Standard for anything served to users. Intelligent-Tiering when access is unpredictable and objects are over 128KB. Glacier tiers only for data you can wait for. |
| Lifecycle rules | Automatic transitions and expiries by prefix or tag. | Write one on day one: transition at 30 days, expire noncurrent versions at 30, abort incomplete multipart uploads at 7. That last rule is pure money. |
| Versioning | Every overwrite and delete is recoverable; a delete writes a marker rather than removing bytes. | On for anything you would cry about. Then pair it with an expiry rule — without one, storage only grows, and deletions never reduce the bill. |
| Multipart upload | Parallel parts, resumable, required above 5GB and sensible above ~100MB. | Let the SDK’s transfer manager do it. Failed uploads leave orphaned parts you are billed for until a lifecycle rule aborts them. |
| Presigned URLs | A time-limited, signed URL for one operation on one key. | Use for both upload and download. The signature carries the signer’s permissions, so sign with a narrowly scoped role, and keep expiry in minutes. |
| Event notifications | ObjectCreated / ObjectRemoved events to Lambda, SQS or EventBridge, filtered by prefix and suffix. | Prefer SQS or EventBridge over direct Lambda for anything bursty — the queue is your buffer and your retry story. |
| SSE-S3 vs SSE-KMS | Both encrypt at rest. SSE-S3 is free and invisible; SSE-KMS gives you a key policy and an audit trail. | SSE-KMS where you must prove who can decrypt. Turn on S3 Bucket Keys with it, or a high-volume prefix generates a KMS call per object and a real bill. |
| Block Public Access | An account and bucket-level override that beats any policy or ACL. | On, at the account level, everywhere. Serve public content through CloudFront with an origin access control instead of a public bucket. |
| Strong consistency | Read-after-write and read-after-list, in all regions, at no cost. | Stop writing the retry loops that older guides teach. What still goes stale is your own cache and any CloudFront copy. |
| Replication (CRR/SRR) | Asynchronous copies to another bucket or region, optionally with RTC. | For DR and data residency. It replicates forward only — existing objects need a batch copy, and deletes do not propagate unless you say so. |
S3 scales to about 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second per prefix, and it partitions on the key prefix — so a million small objects under one prefix throttle while the same million spread over ab/, cd/, ef/… do not. Cost follows the same shape: a terabyte sitting still is a few tens of dollars a month, but a million PUTs is roughly $5 and a million GETs roughly $0.40, so a chatty job that does one request per row will out-spend the storage it produces. Batch small objects, and never put a per-request S3 call inside a per-row loop.
Three services that all “send messages” and are not interchangeable. SQS is a work queue with one winner per message. SNS is a fan-out with no memory. EventBridge is a router that reads the message body. Picking the wrong one shows up months later as a reliability problem.
| SQS | SNS | EventBridge | |
|---|---|---|---|
| Delivery model | Pull. One consumer per message. | Push. Every subscriber, in parallel. | Push. Every matching rule’s targets. |
| Ordering | None (standard); per MessageGroupId (FIFO). | None (standard); FIFO topics to FIFO queues. | None. Do not rely on event order. |
| Retries | Yours, driven by visibility timeout; DLQ after maxReceiveCount. | Per subscription policy; a failing HTTPS endpoint just retries and gives up. | 24 hours of retries per target, then a DLQ if you configured one. |
| Filtering | None — consumers read everything on the queue. | Filter policy on message attributes or body, evaluated at the topic. | Event patterns over the whole JSON, including prefixes, ranges and negation. |
| Latency | Single-digit ms to receive; long polling waits up to 20 s for a message. | Tens of milliseconds. | Typically half a second — a router, not a hot path. |
| Replay | No. Once deleted, it is gone. | No. | Yes: archive with a retention period, then replay a time window. |
| Message size | 256KB (extended client pattern for more). | 256KB. | 256KB. |
| Costs | Per request, in 64KB chunks; the cheapest of the three at volume. | Per publish plus per delivery. | Per event published; targets billed separately. |
| Use it for | Work: jobs, tasks, anything that must be done once. | Notification: the same fact to several places, fast. | Integration: routing, schedules, other services’ events. |
Set ReceiveMessageWaitTimeSeconds to 20. Short polling returns empty responses against a sample of servers, which costs money and hides messages that are plainly there. For payloads over 256KB, do not compress harder — write the object to S3 and put the key in the message; the queue then carries a pointer and the consumer fetches the bytes it needs. And configure the DLQ with redrive from the start: a DLQ you cannot replay from is a folder of failures, not a recovery tool.
Serverless is not “no servers”; it is a different set of limits. The three that decide your design are the execution environment’s lifecycle, the concurrency pool everything shares, and the fact that retry behaviour depends entirely on how the function was invoked.
| Front door | What it is good at | What it costs you |
|---|---|---|
| HTTP API (API Gateway v2) | Lambda and HTTP proxying, JWT authorisers, CORS. Roughly a third the price of REST and lower latency. | No request/response transformation, no API keys with usage plans, no WAF integration on its own. |
| REST API (API Gateway v1) | Request validation and mapping templates, usage plans and API keys, WAF, private endpoints, canary stages. | More expensive per million requests, more configuration surface, and mapping templates written in VTL. |
| Application Load Balancer | Cheapest at sustained high volume; the same front door for containers and Lambda targets. | An hourly charge whether or not traffic arrives, no throttling per client, no request validation. |
| Lambda Function URL | One HTTPS endpoint on the function, IAM or no auth, zero extra service. | No custom domain without CloudFront, no throttling, no authoriser beyond IAM. Fine for webhooks, not for a public API. |
| Knob | What it does | What to actually do |
|---|---|---|
| Memory | Sets memory and, proportionally, CPU and network. 128MB to 10,240MB. | Tune it: doubling memory often halves duration for no extra cost, and sometimes less. Measure, do not guess. |
| Timeout | Up to 900 seconds; the caller’s own timeout is usually shorter. | Set it just above p99, not at the maximum. A 15-minute timeout on a 200 ms function turns a hang into a bill. |
| Reserved concurrency | Guarantees a slice of the account pool and caps the function at that number. | Reserve for the function that must never be starved; cap the one that talks to a fragile database. |
| Provisioned concurrency | Keeps N environments initialised, so those invokes have no cold start. | Only for user-facing p99 that a cold start would break. Combine with scheduled scaling; it bills by the hour. |
| Event source mapping | A managed poller for SQS, DynamoDB Streams, Kinesis and Kafka, with batch size and window. | Set functionResponseTypes: ReportBatchItemFailures so a single bad record does not replay the whole batch. |
| Destinations | On-success and on-failure targets for asynchronous invokes. | Always set an on-failure destination. Without it, an exhausted async retry is silently dropped. |
| Layers and container images | Shared dependencies, or a 10GB image instead of a 250MB zip. | Prefer a small zip. Large artefacts lengthen init, which is exactly the number you are trying to reduce. |
| VPC attachment | Puts the ENI in your subnets so the function can reach RDS or ElastiCache. | Only when you must. It removes public internet access unless you add a NAT gateway or VPC endpoints. |
Since the Hyperplane ENI change the per-invoke networking penalty is gone, so attachment is cheap — but the connection arithmetic is not. Each concurrent execution is its own process holding its own connection, so 400 concurrent invokes means 400 connections against an instance that allows perhaps 450, and there is no pool to share because there is no shared process. The fixes are all structural: put RDS Proxy in front, cap the function with reserved concurrency, or use a data store that speaks HTTP rather than sessions. Reaching for a bigger instance only moves the cliff.
The last four services in the daily kit: the CDN that decides how much traffic your origin ever sees, the cache that sits in front of your database, the two places secrets live, and the registry your images come from.
| Thing | What it gives you | What to actually do |
|---|---|---|
| Origins and behaviours | One distribution, several origins, routed by path pattern with its own cache policy each. | /static/* to the bucket with a long TTL, /api/* to the load balancer with caching disabled but compression on. |
| TTLs | The origin’s Cache-Control decides, bounded by the behaviour’s min, default and max TTL. | Set Cache-Control at the origin and let CloudFront obey it. Overriding TTL in the console hides the real rule from everyone who reads the code. |
| Origin access control | CloudFront signs origin requests with SigV4, so the bucket can refuse everyone else. | Use OAC, not the legacy OAI, and keep Block Public Access on. This is the correct way to serve a static site. |
| Signed URLs and cookies | Time-limited access to private content at the edge, with an optional IP restriction. | Signed cookies for a whole path (a video library); signed URLs for one object. Not the same thing as an S3 presigned URL — these are checked at the POP. |
| Invalidation | Purges paths from every POP. | A break-glass tool. Version your asset filenames and you will rarely need it. |
| ElastiCache | Managed Redis or Valkey, sub-millisecond in-AZ, replicas and automatic failover. | Set a TTL on every key, add jitter to it, and single-flight the origin fetch. A cache with no TTL policy becomes a second database with no backups. |
| Parameter Store | Hierarchical config, standard tier free, versioned, KMS for SecureString. | Everything non-secret, plus secrets you rotate manually. Cache values in the process — the throughput limit is real. |
| Secrets Manager | Rotation via a Lambda, cross-account access, native RDS integration. | Database credentials and third-party API keys. Pay the per-secret fee only where rotation actually earns it. |
| ECR | Private image registry with IAM auth, lifecycle policies and image scanning. | A lifecycle policy that keeps the last 20 tagged images and expires untagged ones after a day; otherwise storage grows forever. |
Both CloudFront and ElastiCache fail the same way. Everything is fine while the hit rate is high, so the origin is sized for the 5% that gets through — then a deploy changes the cache key, or a failover empties the cluster, and 100% of traffic arrives at a database that has never seen it. Size the origin for a plausible miss rate, not the happy one; add jitter so a thousand keys do not expire in the same second; and use a single-flight lock so one miss fetches from the origin while the rest wait for that result.
Three things you configure once and live with for years: the address space, the permissions, and the traffic paths that generate the bill.
A VPC CIDR cannot be shrunk and awkwardly extended. On EKS the surprise is that pods take VPC addresses — so a subnet sized for instances runs out of IPs while the nodes still have free CPU.
Compute is the line item people optimise and rarely the surprise. The surprises are all data movement — charged per gigabyte, invisible in code review, and driven by architecture choices made months earlier.
A pod has no access keys and should never have any. The chain that turns a Kubernetes service account into temporary AWS credentials is five steps, and knowing it turns "AccessDenied" from a mystery into a lookup.
Managed databases move the operational work, not the physics. Two things stay yours: what the application experiences during a failover, and whether your key distributes.
"Multi-AZ" does not mean "no interruption". It means a standby is promoted automatically, and for roughly a minute your writes fail. What your service does in that minute is a design decision.
Throughput is provisioned for the table and delivered per partition. A key that concentrates traffic gets throttled while the table as a whole looks half idle — the hot-key problem from Pillar A, with a hard published ceiling.
Three mechanisms explain most cluster behaviour: what actually happens on apply, how the scheduler chooses a node, and how packets find a pod.
Seven components touch your YAML before a container runs, and each is a separate place for it to stop. Knowing the sequence turns "the pod isn't starting" into a specific question with a specific command.
Two phases: filter removes nodes that cannot host the pod, score ranks the survivors. Every scheduling complaint — Pending pods, lopsided placement, all three replicas in one zone — is one of these two phases doing exactly what it was told.
kubectl describe pod prints the count per reason — "3 Insufficient cpu, 2 node(s) had taint" — which is the answer, spelled out, in the place people least often look.How packets reach a pod, how new code reaches production, and how the thing you declared stops matching the thing that is running.
Every pod gets a real routable address and can talk to every other pod in the cluster — across namespaces, by default, with nothing in the way. Kubernetes networking is open until you close it.
Rolling, blue/green and canary differ in one variable: how much traffic sees the new version before you find out whether it works. That choice buys either cost efficiency or a smaller blast radius.
The value of Terraform is not automation — it is that the intended state is reviewable and diffable. That value evaporates the first time someone fixes production in the console and does not tell the code.
The last mile: knowing what to measure, what deserves to wake someone, and where to look first when it does.
Two standard methods: RED for things that serve requests, USE for things that are resources. Together they cover a system without a hundred bespoke dashboards nobody reads.
A lookup table for the ten things that actually happen. Each row is a symptom you will see on a dashboard, the check that discriminates fastest, and the cause it usually turns out to be.
| Symptom | Check first | Usual cause |
|---|---|---|
| 502 from the load balancer | Target group health, readiness probes | All pods failing readiness — often a dependency the probe touches. |
| 504 from the load balancer | Your p99 vs the LB idle timeout | A handler with no deadline, waiting on something slow. |
| Pods Pending | describe pod → Events | Requests do not fit, a taint, or no node in the required zone. |
| ContainerCreating forever | Free IPs in the subnet; Secret exists | CNI address exhaustion, or a missing volume/secret. |
| CrashLoopBackOff | logs --previous | Config or migration failure at start-up — not Kubernetes. |
| Latency up, CPU flat | CPU throttling metric, pool waits | A CPU limit throttling you, or a saturated connection pool. |
| OOMKilled, exit 137 | Memory limit vs actual working set | Limit set from a guess, or a genuine leak — take a heap profile. |
| Intermittent timeouts to one service | NetworkPolicy, security group, DNS | A policy dropping packets silently, or DNS throttling under load. |
| AccessDenied from the SDK | CloudTrail for the exact denied call | Trust policy, permission policy, or a resource policy — in that order. |
| Bill up, traffic flat | Cost Explorer by usage type | NAT processing, cross-AZ transfer, or log ingestion. |
The phrase in the job description is doing a lot of work. Here is a concrete reading of it — things you have done, not things you have configured once from a tutorial.
The foundations page said every system is queues connected by contracts. The Go page said every goroutine needs an owner and an exit. This page says every hop needs an identity, a bound and a bill. They are the same discipline at three altitudes — and the reason the job description asks for all three is that an incident never stays in one of them.