{"schemaVersion":"1.0","type":"TechArticle","types":["Article","TechArticle"],"slug":"architecture-case-study-migrating-a-developer-saas-from-serverless-to-a-10-vps-with-docker-uph37","url":"https://zyvop.com/architecture-case-study-migrating-a-developer-saas-from-serverless-to-a-10-vps-with-docker-uph37","title":"Architecture Case Study: Migrating a Developer SaaS from Serverless to a $10 VPS with Docker","subtitle":"How we eliminated cold starts, solved database connection pool exhaustion, reduced P99 latency by 92%, and stabilized infrastructure costs using Docker, Caddy, and a $10 VPS.","tldr":"Serverless platforms like Vercel and AWS Lambda are the default choice for modern web applications.","keywords":["Docker","nextjs","Architecture","performance","DevOps","self-hosted","Case Studies"],"entities":["Sanju Singh","Docker","nextjs","Architecture","performance","DevOps","self-hosted","Case Studies","ZyVOP"],"keyTakeaways":["Serverless platforms like Vercel and AWS Lambda are the default choice for modern web applications.","For the first few months of building our developer platform, the serverless promise held up: push to git main, instant preview branches, and zero server maintenance.","Then real traffic arrived."],"headings":["1. The Serverless Wall: Why We Had to Move","The Three Breaking Points:","A. Connection Pool Exhaustion on Relational Databases","B. The Cold-Start & Memory Penalty","C. Bandwidth & Asset Markup","2. The Target Architecture: The $10 VPS Stack","The 4GB RAM Budget Allocation","3. The Production Multi-Stage Dockerfile","next.config.js","Production Dockerfile","4. docker-compose.yml with Hard Memory Limits","5. Reverse Proxy: Caddyfile vs. Nginx","Caddyfile","6. Zero-Downtime Blue/Green Deployments with Caddy","scripts/deploy.sh","7. The Database Win: From Leased Connections to Persistent Pool","The Result:","8. Hardening the $10 VPS for Production","A. The 4GB Swap Buffer (Prevent OOM Spikes)","B. Automatic Docker Log Rotation","C. Firewall (UFW)","D. Automated Encrypted Backups via S3/R2","9. Performance & Cost Post-Mortem","Benchmark Results"],"outboundLinks":[],"contentText":"Serverless platforms like Vercel and AWS Lambda are the default choice for modern web applications. For the first few months of building our developer platform, the serverless promise held up: push to git main, instant preview branches, and zero server maintenance. Then real traffic arrived. Within weeks, we ran headfirst into database connection pool exhaustion, unpredictable cold-start spikes up to 1.8 seconds, restrictive execution timeouts, and an unexpected bandwidth billing spike for serving static assets and dynamic OpenGraph images. We decided to migrate our entire production web application to a $10/month VPS (2 vCPU, 4GB RAM) running Docker and Caddy. Here is the complete post-mortem, the architecture shift, production Dockerfile configurations, zero-downtime deployment workflows, memory budget allocations, and real benchmark data. 1. The Serverless Wall: Why We Had to Move Serverless is marketed as \"infinitely scalable,\" but it forces applications into an ephemeral, stateless execution model that creates significant hidden friction for content-heavy or data-intensive developer tools. flowchart TD subgraph Serverless Architecture [The Serverless Bottleneck] Req1[Client Request 1] --> F1[Lambda Function 1] Req2[Client Request 2] --> F2[Lambda Function 2] Req3[Client Request 3] --> F3[Lambda Function 3] F1 -- New TCP Handshake --> DB[(PostgreSQL)] F2 -- New TCP Handshake --> DB F3 -- New TCP Handshake --> DB style DB fill:#ff6b6b,stroke:#333,stroke-width:2px,color:#fff endThe Three Breaking Points: A. Connection Pool Exhaustion on Relational Databases Every serverless invocation spins up an isolated Node.js container. Because instances don't share memory, they cannot share a traditional database connection pool. A burst of 100 concurrent requests created 100 simultaneous TCP + TLS handshakes to our PostgreSQL instance. Even with external poolers (like PgBouncer or Supabase/Neon connection poolers), HTTP-based pooling added 40–80ms of transaction overhead per request. B. The Cold-Start & Memory Penalty When an API route had to parse Markdown into an Abstract Syntax Tree (AST), sanitize HTML, and compute syntax highlighting, the cold start was brutal: Warm Serverless Invocation: ~85ms Cold Serverless Invocation (P99): 1,450ms – 1,820ms Memory Constraints: Increasing function memory from 1024MB to 3008MB trimmed cold-start times, but tripled invocation costs. C. Bandwidth & Asset Markup Serverless platforms typically charge $0.15 to $0.40 per GB for outbound bandwidth once you exceed initial tiers. For an app generating dynamic OG images (each 200KB–400KB) and syndicating rich technical content, bandwidth quickly became our largest single operational expense. 2. The Target Architecture: The $10 VPS Stack We selected a mid-tier VPS (such as a Hetzner Cloud CX22 or DigitalOcean Droplet: 2 vCPU, 4GB RAM, 40GB NVMe SSD) costing roughly €5 to $10/month. flowchart LR Client([Global Users]) --> CF[Cloudflare CDN / DNS] CF --> Caddy[Caddy Reverse Proxy\\nAutomatic TLS + HTTP/3] subgraph Docker Host [Docker Host: $10 VPS with 4GB Swap] Caddy -->|Internal HTTP Proxy| AppA[App Container: Blue\\nPort 3001] Caddy -.->|Atomic Config Reload| AppB[App Container: Green\\nPort 3002] AppA --> Redis[(Local Redis\\nCache & Queues)] AppA --> DB[(PostgreSQL\\nPooled Sockets)] endThe 4GB RAM Budget Allocation Running a full stack on a 4GB VPS requires explicit memory ceilings to prevent the Linux Out-Of-Memory (OOM) killer from terminating your process: Service Memory Limit (Hard Cap) Reserved / Typical Footprint Next.js Standalone App 1,536 MB 350 MB – 600 MB PostgreSQL (Local / Container) 1,024 MB 250 MB – 450 MB Redis (Cache & Queues) 384 MB 64 MB – 128 MB Caddy Reverse Proxy 128 MB 30 MB – 60 MB Linux OS & System Daemons Uncapped ~350 MB Swap Buffer (Disk NVMe) 4,096 MB Safety net for build spikes 3. The Production Multi-Stage Dockerfile The standard next build produces large build artifacts with development dependencies. By enabling output: 'standalone' in next.config.js, Next.js traces imports and bundles only the exact node_modules needed in production. next.config.js /** @type {import('next').NextConfig} */ const nextConfig = { output: 'standalone', poweredByHeader: false, compress: true, }; module.exports = nextConfig;Production Dockerfile This Dockerfile solves three common production pitfalls: Missing libc6-compat on Alpine (required by sharp for image optimization). Unprivileged user permissions for Next.js Incremental Static Regeneration (.next/cache). Internal health check probe for zero-downtime rolling deploys. # Stage 1: Dependency resolution FROM node:20-alpine AS deps RUN apk add --no-cache libc6-compat WORKDIR /app COPY package.json package-lock.json ./ RUN npm ci --ignore-scripts # Stage 2: Application builder FROM node:20-alpine AS builder RUN apk add --no-cache libc6-compat WORKDIR /app COPY --from=deps /app/node_modules ./node_modules COPY . . ENV NEXT_TELEMETRY_DISABLED=1 ENV NODE_ENV=production RUN npm run build # Stage 3: Minimal runner image (~85MB) FROM node:20-alpine AS runner # Install libc6-compat for sharp native bindings on Alpine RUN apk add --no-cache libc6-compat curl WORKDIR /app ENV NODE_ENV=production ENV NEXT_TELEMETRY_DISABLED=1 ENV PORT=3000 ENV HOSTNAME=\"0.0.0.0\" # Create unprivileged system user for security RUN addgroup --system --gid 1001 nodejs && \\ adduser --system --uid 1001 nextjs # Copy public static assets and standalone server bundle COPY --from=builder /app/public ./public COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./ COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static # Ensure nextjs user owns the cache directory for ISR updates RUN mkdir -p .next/cache && chown -R nextjs:nodejs .next/cache USER nextjs EXPOSE 3000 # Health check to ensure container is responding before traffic routing HEALTHCHECK --interval=5s --timeout=3s --start-period=5s --retries=3 \\ CMD curl -f http://127.0.0.1:3000/api/health || exit 1 CMD [\"node\", \"server.js\"]4. docker-compose.yml with Hard Memory Limits version: '3.8' services: caddy: image: caddy:2.8-alpine restart: unless-stopped ports: - \"80:80\" - \"443:443\" - \"443:443/udp\" # HTTP/3 QUIC volumes: - ./Caddyfile:/etc/caddy/Caddyfile:ro - caddy_data:/data - caddy_config:/config networks: - internal-net depends_on: - web-blue web-blue: image: app-web:latest build: context: . dockerfile: Dockerfile restart: unless-stopped env_file: - .env.production expose: - \"3000\" deploy: resources: limits: cpus: '1.50' memory: 1536M reservations: cpus: '0.25' memory: 512M networks: - internal-net web-green: image: app-web:latest restart: \"no\" env_file: - .env.production expose: - \"3000\" deploy: resources: limits: cpus: '1.50' memory: 1536M reservations: cpus: '0.25' memory: 512M networks: - internal-net redis: image: redis:7-alpine restart: unless-stopped command: [\"redis-server\", \"--appendonly\", \"yes\", \"--maxmemory\", \"256mb\", \"--maxmemory-policy\", \"allkeys-lru\"] volumes: - redis_data:/data networks: - internal-net deploy: resources: limits: memory: 384M volumes: caddy_data: caddy_config: redis_data: networks: internal-net: driver: bridge5. Reverse Proxy: Caddyfile vs. Nginx Nginx requires manual Certbot cron jobs, Diffie-Hellman parameter generation, and verbose SSL configuration. Caddy does this automatically out of the box with modern TLS 1.3 defaults and HTTP/3 support. Caddyfile yourdomain.com { # Modern compression encode zstd gzip # Essential Security Headers header { Strict-Transport-Security \"max-age=31536000; includeSubDomains; preload\" X-Content-Type-Options \"nosniff\" X-Frame-Options \"DENY\" Referrer-Policy \"strict-origin-when-cross-origin\" } # Reverse proxy to the active Next.js container slot reverse_proxy web-blue:3000 { # Active health checks health_uri /api/health health_interval 5s health_timeout 2s # Persistent HTTP keepalive to backend container transport http { keepalive 30s keepalive_idle_conns 100 } } }6. Zero-Downtime Blue/Green Deployments with Caddy A common mistake with docker compose --scale web=2 is that Docker Compose scales down from the highest numbered container, which can accidentally terminate the freshly built container rather than the old one. Instead, we use a simple Blue/Green deployment script. It boots the inactive container, waits for its health check to return HTTP 200, updates Caddy's upstream, issues an atomic caddy reload (which drops zero active connections), and stops the previous container. scripts/deploy.sh #!/usr/bin/env bash set -euo pipefail ACTIVE_TARGET=$(docker ps --format '{{.Names}}' | grep -E 'web-(blue|green)' | head -n 1 || true) if [[ \"$ACTIVE_TARGET\" == *\"web-blue\"* ]]; then NEW_TARGET=\"web-green\" OLD_TARGET=\"web-blue\" else NEW_TARGET=\"web-blue\" OLD_TARGET=\"web-green\" fi echo \"==> Currently active: ${OLD_TARGET:-none}\" echo \"==> Deploying new version to: ${NEW_TARGET}\" git pull origin main docker compose build \"${NEW_TARGET}\" docker compose up -d --no-deps \"${NEW_TARGET}\" echo \"==> Waiting for ${NEW_TARGET} to become healthy...\" for i in {1..20}; do STATUS=$(docker inspect --format='{{json .State.Health.Status}}' \"zyvop-${NEW_TARGET}-1\" 2>/dev/null || echo '\"starting\"') if [ \"$STATUS\" == '\"healthy\"' ]; then echo \"==> ${NEW_TARGET} is healthy!\" break fi if [ \"$i\" -eq 20 ]; then echo \"==> ERROR: Health check timed out on ${NEW_TARGET}. Aborting cutover.\" docker compose stop \"${NEW_TARGET}\" exit 1 fi sleep 2 done # Atomically switch Caddy upstream sed -i \"s/${OLD_TARGET}:3000/${NEW_TARGET}:3000/g\" Caddyfile docker compose exec caddy caddy reload --config /etc/caddy/Caddyfile echo \"==> Traffic switched to ${NEW_TARGET}. Stopping ${OLD_TARGET}...\" if [ -n \"$OLD_TARGET\" ]; then docker compose stop \"$OLD_TARGET\" fi docker image prune -f echo \"==> Deployment completed with zero downtime!\"7. The Database Win: From Leased Connections to Persistent Pool Under serverless, every database query required either an HTTP fetch to a serverless driver or opening a new TCP socket: // Old Serverless Approach (Ephemeral) // Created on every function cold start import { Pool } from 'pg'; const pool = new Pool({ connectionString: process.env.DATABASE_URL, max: 1, // Kept at 1 to prevent connection pool exhaustion across lambdas });On our persistent VPS, the Node.js runtime remains alive indefinitely. We configure a single persistent pool of 15–20 reusable connections with graceful shutdown handling: // New VPS Approach (Persistent Process) import { Pool } from 'pg'; declare global { var __dbPool: Pool | undefined; } export const db = global.__dbPool ?? new Pool({ connectionString: process.env.DATABASE_URL, max: 20, // 20 persistent, pre-warmed connections idleTimeoutMillis: 30000, connectionTimeoutMillis: 2000, }); if (process.env.NODE_ENV !== 'production') { global.__dbPool = db; } // Gracefully drain pool on container shutdown process.on('SIGTERM', async () => { console.log('SIGTERM received: Draining PostgreSQL connection pool...'); await db.end(); process.exit(0); });The Result: Database connection latency dropped from ~65ms (TCP + TLS negotiation per request) to < 1.2ms (reused local socket connection). 8. Hardening the $10 VPS for Production A common hesitation with self-hosting is maintenance. By following these four steps, our maintenance overhead is less than 15 minutes per month. A. The 4GB Swap Buffer (Prevent OOM Spikes) Never run a 4GB VPS without swap. If npm run build or traffic spikes exceed 4GB, swap prevents the kernel from panicking: sudo fallocate -l 4G /swapfile sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstabB. Automatic Docker Log Rotation Unbounded container logs will quietly fill your disk. Configure the Docker daemon globally: /etc/docker/daemon.json: { \"log-driver\": \"json-file\", \"log-opts\": { \"max-size\": \"50m\", \"max-file\": \"3\" } }C. Firewall (UFW) Only expose ports 80, 443, and your SSH port: sudo ufw default deny incoming sudo ufw default allow outgoing sudo ufw allow 22/tcp sudo ufw allow 80/tcp sudo ufw allow 443/tcp sudo ufw allow 443/udp sudo ufw enableD. Automated Encrypted Backups via S3/R2 A simple cron job runs pg_dump, encrypts the archive, and synchronizes to Cloudflare R2 / AWS S3: scripts/backup.sh: #!/usr/bin/env bash set -euo pipefail BACKUP_FILE=\"/tmp/backup_$(date +%Y%m%d_%H%M%S).sql.gz\" docker compose exec -T postgres pg_dump -U postgres app_production | gzip > \"$BACKUP_FILE\" # Upload to S3-compatible storage (e.g. Cloudflare R2 via rclone) rclone copy \"$BACKUP_FILE\" remote:backups-bucket/db/ rm -f \"$BACKUP_FILE\" echo \"==> Backup successfully uploaded to offsite storage.\"9. Performance & Cost Post-Mortem We ran a load test using wrk simulating 50 concurrent connections over 60 seconds hitting an authenticated database-backed endpoint. Benchmark Results Metric Serverless (Previous Stack) $10 VPS + Docker (Current Stack) Improvement P50 Latency 114 ms 38 ms 3.0x faster (66% drop) P95 Latency 412 ms 82 ms 5.0x faster (80% drop) P99 Latency (Cold Start Spike) 1,740 ms 124 ms 14.0x faster (92.8% drop) Max Throughput ~280 req/sec (hit concurrency limits) 1,150 req/sec 4.1x capacity Failed Requests (504 / 500) 1.8% during spikes 0.00% Zero errors Monthly Cost Comparison Expense Item Serverless Stack $10 VPS Stack Compute / Invocations $45.00 (Base Pro + Overage) $10.00 (Fixed VPS) Bandwidth / Egress $38.50 ($0.15/GB overage) $0.00 (20TB included) External DB Connection Pooler $15.00 $0.00 (Native socket pooling) Static Asset Hosting / Preview URLs Included $0.00 (Cloudflare Free Tier) Total Monthly Spend $98.50+ $10.00 flat 10. The Decision Matrix: When Should You Stay Serverless? Self-hosting isn't a silver bullet for every project. Here is our practical decision framework: Stay on Serverless if: Your traffic is completely sporadic (e.g., 0 requests for 8 hours, then a 5-minute spike). You do not have a dedicated engineer with basic Linux terminal comfort. Your application architecture is entirely stateless (no relational database or using third-party managed HTTP APIs like DynamoDB/Firestore). Migrate to a VPS + Docker if: You use a relational database (PostgreSQL, MySQL) and suffer connection bottlenecks. You need predictable sub-100ms response times without cold-start jitter. You run background jobs, WebSockets, or long-running worker processes. You want a fixed, predictable monthly infrastructure bill regardless of traffic surges. Conclusion Serverless got our project off the ground in a weekend. But moving to a modest $10 VPS simplified our database connectivity, eliminated cold starts, and delivered a snappier, more predictable experience for our users. Modern tools like Docker standalone builds, Caddy's automated SSL, and Blue/Green atomic config reloads have removed the traditional friction of managing servers. You don't need Kubernetes to scale a developer SaaS—a single well-tuned Linux box can take you much further than you think.","contentHash":"sha256:9a809f184f83d9d54f92394782fdeaf8243b1f057f60192c992ac0f9a41c113d","authorName":"Sanju Singh","authorUrl":"https://zyvop.com/author/sanjay687","authorSameAs":[],"category":"Case Studies","tags":["Docker","nextjs","Architecture","performance","DevOps","self-hosted"],"audience":"Senior software engineers, systems architects, and technical leads working with Case Studies","tone":"Instructional, practical, code-first","readingTimeMinutes":10,"wordCount":2126,"faqs":null,"primaryTopic":"Case Studies","publishedAt":"2026-09-26T08:19:40.271Z","updatedAt":"2026-09-26T08:19:40.271Z","canonicalUrl":"https://zyvop.com/architecture-case-study-migrating-a-developer-saas-from-serverless-to-a-10-vps-with-docker-uph37"}