AWS

The Portable Server: Design for the Exit

The workload in this book is deliberately unglamorous: one uptime monitor — Uptime Kuma, in the build this book is based on — running on a small VPS at a budget hoster, watching a homelab from outside the house. The job is to move it to AWS EC2. Not because the old hoster failed. As a proof of concept: the migration is the lesson, and the monitor is just important enough that doing it wrong would sting.

Honest framing up front, because AWS deserves it: this move does not save money. The AWS equivalent of a bargain VPS lands a few dollars a month higher, not lower. What you're buying is the skill set — EC2, security groups, EBS, key pairs, user data — learned on a workload where a mistake costs you an afternoon, not a business.

Why the move was boring (by design)

The whole migration kit fits in one directory, and none of it mentions a provider:

• A Docker Compose file declaring the service: image, restart policy, port binding, volume mount.
• A named volume holding all the app's data — every monitor, every heartbeat, the admin account.
• A backup script that tars the volume into a timestamped file, and a matching restore script.
• A cloud-init file that bootstraps any fresh host into "the server" (chapter 4).

Here's the heart of it — the compose file that ran on the old hoster, unchanged on AWS:

services:
  uptime-kuma:
    image: louislam/uptime-kuma:2
    container_name: uptime-kuma
    restart: unless-stopped
    ports:
      # localhost only -- exposure is someone else's job (chapter 5)
      - "127.0.0.1:3001:3001"
    volumes:
      - kuma-data:/app/data

volumes:
  kuma-data:
    name: kuma-data   # explicit name: scripts find it on any host

Two decisions in that file do the heavy lifting. The app binds to 127.0.0.1 only — it never faces a network directly; a separate layer decides who may reach it. And the volume's name is pinned explicitly instead of accepting Compose's auto-generated project-prefixed name, so backup.sh and restore.sh find kuma-data on any machine, in any directory.

The backup itself is one idea: mount the volume read-only into a throwaway container and tar it up.

docker run --rm \
  -v kuma-data:/data:ro \
  -v "$PWD/backups:/backup" \
  alpine:3 tar czf "/backup/kuma-data-$(date +%Y%m%d-%H%M%S).tgz" -C /data .

One analogy, then the principle: this is the difference between moving house and furnishing a hotel room. If your possessions live in labeled boxes (the volume) and the furniture list is written down (the compose file), the moving truck does not care which city it's driving to. The cheapest day to design your exit from a provider is the day you deploy — every shortcut you take on day one ("I'll just configure it in their panel") is a hostage you negotiate with on moving day.

• The migration kit is four provider-agnostic files: compose, named volume, backup/restore scripts, cloud-init.
• Bind apps to localhost and pin volume names explicitly — portability is a day-one decision.
• Design for the exit when you deploy; it's the cheapest that decision will ever be.

EC2 Anatomy: Reading the Instance Menu

The EC2 launch wizard is the first boss fight: hundreds of instance types with names like c7gn.2xlarge. The trick is that the name is a compressed spec sheet — once you can read it, the menu stops being intimidating. This migration's pick, decoded:

t4g.small
t  = family: burstable general purpose
4  = generation (newer = faster silicon, usually cheaper per unit of work)
g  = Graviton: AWS's own ARM processors (no letter = Intel, a = AMD)
.small = size step; each step up roughly doubles the RAM

The families worth knowing cold: t burstable general purpose, m balanced general purpose, c compute-heavy, r memory-heavy. For a homelab-scale service, you'll live in t and visit the others when a workload actually demands it.

Nitro: the offload you don't choose

Under nearly every modern instance family sits the Nitro system: networking, storage, and virtualization work is offloaded from the main CPUs onto dedicated Nitro cards. If you've sold datacenter gear, you already know this shape — it's the DPU/SmartNIC idea. The hypervisor tax mostly disappears, the host silicon spends its cycles on your workload, and it's why a two-vCPU cloud instance feels closer to bare metal than it has any right to. You never select Nitro — it comes with the family and generation you pick.

Burstable: honest sizing for lazy workloads

The t family is cheap because it's burstable: each instance is guaranteed a baseline slice of CPU and banks CPU credits while it idles, spending them when load spikes — a capped entitlement with a savings account. An uptime monitor asleep 99% of the time between checks is the perfect burstable tenant; a sustained render farm would be the worst. Match the family to the duty cycle, not to your hopes.

ARM without fear, and storage without regret

The g raises the only real question: will my software run on ARM? For containers, usually yes — a modern image tag points at a manifest list covering several architectures, and the engine pulls the build that matches the host. Verify before migration day, not during:

docker buildx imagetools inspect louislam/uptime-kuma:2
# check the platform list: you want linux/arm64 next to linux/amd64

Storage is an EBS volume — network block storage, in hypervisor terms a VMDK that lives on the storage fabric instead of the host. The default choice, gp3, prices simply: a flat per-GB monthly rate (around $0.08/GB-month) with baseline performance included, so at this scale you're paying for gigabytes, not IOPS. One asymmetry rules every sizing decision: EBS volumes can be grown online, but never shrunk. Choose like you're pouring concrete — 20 GB with room to grow beats 40 GB of ego, and beating the 8 GB default matters because Docker image churn eats small disks. If you undersize, growing is a console edit plus a filesystem extend. If you oversize, shrinking is a rebuild.

• Instance names are spec sheets: family.generation+architecture.size.
• Nitro is SmartNIC-style offload under everything modern; burstable t-family suits idle-mostly workloads.
• Multi-arch images make ARM safe — verify platforms before you migrate, not during.
• EBS grows but never shrinks; size for the workload you measured.

The Front Door: Keys, Security Groups, and the Bill

Three things get settled before the launch button gets pressed, because each is roughly ten times more annoying to retrofit afterward: how you'll authenticate, what the firewall allows, and how you'll know what it costs. In that order.

Key pairs: download once, gone forever

EC2 key pairs are asymmetric SSH keys where AWS keeps the public half and hands you the private half — a .pem file — exactly once, at creation. AWS does not retain a copy it can show you later. Lose the file and that door is simply gone. (This is less catastrophic than it sounds if your cloud-init installs your own key for your admin user, chapter 4 — two independent doors is an accident worth engineering on purpose.) Treat the .pem like any other secret: into the password manager, never into git, never into a chat window.

Security groups: a stateful allow-list

A security group is a firewall policy wrapped around the instance. Three properties define it. It's an allow-list only — there are no deny rules; anything not explicitly allowed is dropped. It's stateful — if your box initiates a connection outbound, the replies are allowed back automatically, unlike a stateless ACL where you'd have to write the return rule yourself. And it evaluates before traffic reaches the host, so it protects the box even from its own misconfigurations. For bootstrap, exactly one rule — SSH, from your current public IP, expressed as a /32:

aws ec2 authorize-security-group-ingress \
  --group-id sg-0123456789abcdef0 \
  --protocol tcp --port 22 --cidr 192.0.2.10/32

Read that rule the paranoid way: port 22, one source address on Earth. Not 0.0.0.0/0 "just for setup" — in production and homelabs alike, temporary rules are the most permanent kind.

The bill: IPv4 rent and the tripwire

Two facts about AWS billing surprise everyone exactly once. First: since February 2024, every public IPv4 address costs about $0.005/hour — roughly $3.65 a month — whether it's attached to a running instance or sitting idle. The era of free spare addresses is over; an address is a metered resource, which will matter again in chapter 6. Second: your account type changes the math. Brand-new accounts may sit on a free plan with starter credits — genuinely useful, but read the terms: free-plan accounts can auto-close when the credits or the plan window run out unless upgraded. An account you've held for years is a plain paid account with a card on file and no credits to burn. Either way, open the Billing console and know which one you have before the first launch — it decides whether your POC is effectively free or honestly metered.

Then install the tripwire: an AWS Budget. Set a monthly cost budget slightly above the expected run-rate (expected ≈ $19/month, budget = $20), with email alerts at 80% and 100% of it. Five minutes in the console, and the worst case for a forgotten instance or a pricing misunderstanding becomes an email instead of a statement surprise. A proof of concept without a budget alarm isn't an experiment — it's an open tab.

• The .pem downloads once; AWS can't re-show it. Vault it like a password.
• Security groups: allow-list only, stateful, evaluated before the host. Bootstrap rule = 22/tcp from your IP/32, nothing else.
• Public IPv4 costs ~$3.65/month each since Feb 2024 — addresses are metered now.
• Check your plan type in Billing, and set a budget alarm before launching anything.

cloud-init: The All-or-Nothing Bootstrap

cloud-init is the bootstrap contract between you and any cloud. At launch you hand the provider a block of user data — a YAML file starting with #cloud-config — and on first boot the instance reads it and builds itself: users, packages, files, commands, firewall. It's what turns a VM from a pet you raise by hand into cattle you can recreate from a file in your repo.

Anatomy of the file

#cloud-config
hostname: monitor
manage_etc_hosts: true

users:
  - name: labuser
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
      - ssh-ed25519 TODO_REPLACE_WITH_YOUR_PUBLIC_KEY monitor

package_update: true
package_upgrade: true

packages:
  - ca-certificates
  - curl
  - gnupg
  - ufw
  - jq

runcmd:
  # Docker from the official apt repo (keyring + source lines omitted here)
  - apt-get update -y
  - apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
  - usermod -aG docker labuser
  # Tailscale
  - curl -fsSL https://tailscale.com/install.sh | sh
  # Host firewall: deny inbound by default
  - ufw default deny incoming
  - ufw allow 22/tcp
  - ufw allow in on tailscale0
  - ufw --force enable

Read it top-down: identity first (hostname, the admin user with your public key — note this is independent of the EC2 key pair from chapter 3), then packages, then runcmd for everything that's really a shell script in a trench coat: install Docker, install Tailscale, raise the host firewall. When it works, you SSH in a few minutes after launch to a finished machine. Verify by outcomes, not vibes:

id labuser            # the user exists
docker --version      # Docker answers
tailscale version     # Tailscale answers
sudo ufw status       # active, default deny, 22 + tailscale0 allowed
hostname              # says monitor -- remember this one

The launch where nothing happened

First launch of this migration: the instance booted, SSH answered, cloud-init status cheerfully reported done — and none of the config had applied. No labuser. No Docker, no Tailscale. And the tell, hiding in the shell prompt: the hostname was still the provider default, not monitor. cloud-init had finished — it just hadn't run our file.

Diagnosis rule: debug from the ground truth on the box, not the file on your laptop. The instance keeps exactly what it received:

sudo head -25 /var/lib/cloud/instance/user-data.txt

The file was all there — right user, right key, right packages. But two comment lines showed mojibake where em-dashes belonged. The file had been edited and saved on a Windows workstation, and the editor saved it in legacy ANSI encoding instead of UTF-8. The em-dash characters became invalid UTF-8 bytes. cloud-init's YAML loader decodes the entire stream before parsing — one bad byte, even inside a harmless comment, and the whole document is rejected. cloud-init shrugs, falls back to defaults, and still reports done, because from its perspective it did its job: it ran the (empty) config it could read.

The fix was boring on purpose: regenerate the user data as pure ASCII (em-dashes typed as --, no smart quotes, nothing an editor can corrupt), with the real SSH key pre-baked so the launch takes zero hand edits — every hand edit at paste time is a fresh chance to break the file. Then terminate the instance and relaunch from scratch rather than hand-repairing the broken one. A hand-fixed box is a snowflake that proves nothing about your bootstrap; the entire point of the POC was the bootstrap. Ten minutes lost, and the relaunched box passed every checker in the list above.

• User data is a first-boot contract: users, packages, files, runcmd, firewall — the host as a file.
• cloud-init is all-or-nothing: one invalid byte rejects the whole config, and status still says "done".
• Debug from /var/lib/cloud/instance/user-data.txt — what the instance received beats what you think you sent.
• Fix by regenerating clean ASCII with the key pre-baked, then relaunch. Cattle, not pets.

Moving Day: Backup, Restore, Rename

Moving day is a sequence, and the order is the safety:

# on the OLD box, inside the project directory
./backup.sh
# on your workstation: pull the tarball down (this copy IS the rollback)
scp labuser@<old-box>:~/homelab/backups/kuma-data-<stamp>.tgz .
# on the NEW box (after chapter 4's bootstrap checks pass)
git clone https://github.com/<your-org>/homelab.git
# push the tarball up into the clone's backups/ folder, then:
cd homelab
./restore.sh backups/kuma-data-<stamp>.tgz
docker compose ps        # service up, bound to 127.0.0.1:3001

The restore script stops the service, empties the named volume, untars your data into it from a throwaway container, and starts the service again. Then comes the only verification that actually matters: open the dashboard and get a login prompt instead of a setup wizard. Log in with the admin credentials from the old box and there it all is — every monitor, every setting, the full heartbeat history, with one gap in the timeline covering the move itself. The gap is the migration, fossilized. This works because chapter 1 did its job: the data never lived in the container, so swapping the entire machine under it is just plumbing.

What does not migrate: Tailscale

Everything about the app moved in a tarball. Nothing about the network identity did, and forcing it to would be a mistake. A Tailscale node's identity lives in a state file (/var/lib/tailscale/tailscaled.state) holding its machine key — deliberately not copied, because two machines claiming one identity on the same tailnet is a conflict, not a shortcut. The new box enrolls as what it is: a new node, with a new tailnet IP, using a pre-authorized key generated in the admin console. (Auth keys are time-boxed — generate yours on moving-day morning, not last week.)

Two consequences follow from that, and both are one-liners once you know them. First, tailscale serve configuration is per-node state, so the command that exposes the app on the tailnet must be re-run on the new box:

sudo tailscale serve --bg http://127.0.0.1:3001

What does not need repeating: the HTTPS approval. Enabling HTTPS certificates is a tailnet-wide setting in the admin console — flip it once, ever, and each new node's certificate is issued automatically. Per-node config re-runs; tailnet-wide settings persist. Knowing which is which saves a confused half-hour.

The rename trick, and its two traps

Your bookmarks, scripts, and muscle memory all point at the node's MagicDNS name — so move the name, not the habits. In the admin console, rename the old node to monitor-old, then give the new node the name monitor. From your workstation, ssh labuser@monitor now lands on AWS. Same name, different continent of infrastructure, zero broken references.

Trap one lives on the old box: renaming a node does not move its serve registration. The old node's serve endpoint stays bound to its previous name, so its dashboard starts answering with an invalid-response error — the node is healthy, the nameplate on its web endpoint is stale. The fix is to re-register serve under the current name:

sudo tailscale serve reset
sudo tailscale serve --bg http://127.0.0.1:3001
tailscale serve status   # should now show the node's CURRENT name

Trap two lives on your machine. The first SSH to the reused name gets you the full-screen warning: REMOTE HOST IDENTIFICATION HAS CHANGED. It is not an attack — it's your own migration looking back at you; the name legitimately points at a new machine with a new host key. But "it's probably fine" is not a verification method. Do it properly: the warning prints the new key's SHA256 fingerprint, and the EC2 console's Get system log for the instance printed the host key fingerprints at first boot. Match them, then clear the stale entry and reconnect:

ssh-keygen -R monitor

• Backup off the old box first — the tarball on your workstation is the rollback.
• Login prompt, not setup wizard: that's the restore proving itself, history and all.
• Tailscale identity doesn't migrate: new node, fresh enrollment, serve re-run; HTTPS enablement is tailnet-wide and persists.
• Renames move names, not serve registrations — and the known_hosts alarm afterward is you, verified via the console system log.

Lock It Down, Kill the Old Box

The instance was born with one public door: the bootstrap SSH rule from chapter 3. That door existed so you could reach the box before Tailscale was up. Tailscale is up — you've been SSHing through it and the dashboard loads over it — so the door has finished its only job. Delete the security group's inbound rule. The group now contains zero inbound rules, and the box's posture changes category: not "well-firewalled," but unreachable.

Verify like you mean it, in a fresh terminal:

ssh labuser@<public-ip>   # expect: hangs, then times out -- nothing answers
ssh labuser@monitor         # over the tailnet: works
# and the monitor's checks are still landing green

Learn to read that timeout: a refusal would mean something received your packet and answered "no." A timeout means silence — the security group dropped it and the internet's scanners get exactly the same nothing. The host's own ufw default-deny (chapter 4) remains as a second layer underneath, so cloud firewall and host firewall agree: nobody knocks, because there's no door.

Why the public IP stays

Your next instinct will be to finish the job — release the public IP entirely. Resist it, because in the default VPC that address is doing double duty: it is also the box's only outbound path. There is no NAT gateway in this network, so every packet the box sends — the monitor's checks, package updates, and Tailscale's own connection to its control plane and relays — leaves through that public IP. Release it and the box doesn't become private; it becomes dark, tailnet included, and your only way back in is the provider console. The AWS-native alternative, a managed NAT gateway, runs roughly $32 a month — about ten times the $3.65 the address itself costs — to solve a problem the empty security group already solved. Exposure is a property of rules, not of addresses: an assigned IP behind zero inbound rules accepts connections from nobody.

Burn-in, then the kill

Both boxes are now running the same monitor independently, and that overlap is the burn-in: a chosen window — 24 to 48 hours covers full day/night check cycles for a workload like this — where the new box has to prove itself while the old one stands by as instant rollback and independent witness. How long is a judgment call about blast radius: a single uptime monitor earns a shorter fuse than a production database would. But make it a decision, written down, not an oversight you rationalize later.

When the window closes: delete the old server in the provider's console, remove the monitor-old node from the Tailscale admin console, and keep the final tarball archived on your workstation. One tail to tie off: deleting the server is not closing the account. Providers typically require every product cancelled and the final invoice — issued on their billing cycle, days later — paid before account deactivation is even offered. Set a reminder for that date now, while you still remember the sequence exists, instead of rediscovering a zombie account next year.

The dangerous checklist

• Read any EC2 instance type name as a spec sheet — family, generation, architecture, size — and know when burstable is the honest, cheap answer.
• Bootstrap a server with zero post-launch clicking: cloud-init user data that creates the user, installs Docker and Tailscale, and raises the host firewall — verified by outcomes, not status lines.
• Diagnose a silent bootstrap failure from ground truth: compare hostnames, read the user-data file the instance actually received, and recognize encoding as a failure class.
• Move a stateful service between providers with a tarball — data, admin account, and history intact — then move its tailnet name so no bookmark breaks.
• Take a cloud box completely off the public internet — zero-inbound security group, tailnet-only access — and explain why the public IP stays anyway.