Why Your App Is Slow (And It's Not the Database)

The reflex when an API is slow is to check the database. You add an index. You look at EXPLAIN ANALYZE. You scale up the instance. Nothing changes. This happens because the slow query log only tells you about query execution time — it says nothing about the other four places where your request can bleed latency before a query even runs, or after it finishes.

This post systematically covers all five root causes, how to instrument each one, and what to do when you find it. The order matters: start from the top.

The Request Lifecycle

Before diagnosing, it helps to see the full path a database-backed request takes. Latency can accumulate at every transition.

Most engineers profile the green box. Production slowdowns live in the red ones.

Cause 1: Connection Pool Exhaustion

Your pool has 10 connections. Under normal load, each request borrows one, runs a query in ~20ms, and returns it. But under load, or when requests hold connections longer than usual, the pool drains. New requests don't fail — they queue, waiting silently for a slot to open.

With the default connectionTimeoutMillis: 0 in node-postgres, that queue has no timeout. Requests wait indefinitely. The endpoint appears to hang. The database is idle.

This is the most common misdiagnosis in backend engineering. It looks exactly like a database problem.

How to see it

import { Pool } from 'pg'

const pool = new Pool({ max: 10 })

setInterval(() => {
  console.log({
    total:   pool.totalCount,    // connections created
    idle:    pool.idleCount,     // available right now
    waiting: pool.waitingCount,  // requests queued for a connection ← this is your signal
  })
}, 2000)

waitingCount > 0 under normal load means the pool is the bottleneck. If it's 0 at rest and climbs under traffic, you're undersized or connections are being held too long.

The fix: configure the pool correctly

const pool = new Pool({
  max: 10,
  connectionTimeoutMillis: 5_000,   // fail after 5s — never leave at 0
  idleTimeoutMillis: 30_000,
  statement_timeout: 10_000,        // kill runaway queries that hold connections
})

connectionTimeoutMillis: 5_000 is the most important line. When the pool is exhausted and a request has waited 5 seconds, it gets a clear error instead of hanging forever. That error surfaces as a 503, which is the correct behavior — fail fast, let the client retry.

The leak pattern

A pool that slowly drains over hours (watch totalCount dropping without restarts) has a connection leak. The cause is almost always release() being skipped on error:

// LEAKS: if the query throws, release() is never called
async function getUser(id: string) {
  const client = await pool.connect()
  const result = await client.query('SELECT * FROM users WHERE id = $1', [id])
  client.release()  // ← skipped when the line above throws
  return result.rows[0]
}

// CORRECT: finally() always runs
async function getUser(id: string) {
  const client = await pool.connect()
  try {
    const result = await client.query('SELECT * FROM users WHERE id = $1', [id])
    return result.rows[0]
  } finally {
    client.release()  // runs on success, error, and everything in between
  }
}

// SIMPLEST: pool.query() manages the lifecycle for you
async function getUser(id: string) {
  const { rows } = await pool.query('SELECT * FROM users WHERE id = $1', [id])
  return rows[0]
}

Use pool.query() for single statements. Reach for pool.connect() only when you need a transaction or multiple statements on the same connection.

Pool sizing formula

Running multiple service instances? Each one creates its own pool. The math:

max_per_instance = floor((pg_max_connections × 0.8) / num_instances)

For 100 max_connections and 3 API replicas: floor(80 / 3) = 26. Keep 20% headroom because max_connections includes superuser connections, replication, and monitoring tools. Above 90%, PostgreSQL starts refusing connections entirely — not gracefully, just hard refusals.

At higher scale, run PgBouncer in front of PostgreSQL in transaction pooling mode. It multiplexes hundreds of application connections onto a small pool of actual server connections, letting your instances set max: 5 while PostgreSQL still only sees 20–25 connections total.

Cause 2: Queries Blocked on Locks (Not Running Slowly)

The slow query log records execution time. A query waiting on a lock isn't executing — it's waiting. Those seconds don't appear anywhere in the log, and the query duration from the application's side includes all of them.

What causes lock chains

idle in transaction sessions — a connection that ran BEGIN and never committed holds its row and table locks indefinitely. A long-running query, an idle-in-transaction session, or an ALTER TABLE waiting for its AccessExclusiveLock can each create a chain reaction where everything queues behind them. One forgotten transaction from a batch job or migration script can queue dozens of application requests.

DDL during peak traffic — ALTER TABLE takes an AccessExclusiveLock that blocks every read and write on the table. It also waits for current queries to finish before acquiring the lock — meaning it queues behind slow queries, and every new query queues behind it. The result is a complete stall that lasts as long as the migration.

Find what's blocking what, right now

SELECT
  blocked.pid                           AS blocked_pid,
  left(blocked.query, 80)               AS blocked_query,
  blocking.pid                          AS blocking_pid,
  blocking.state                        AS blocking_state,  -- 'idle in transaction' is the red flag
  left(blocking.query, 80)              AS blocking_query,
  now() - blocking.query_start          AS blocking_duration
FROM pg_stat_activity blocked
JOIN pg_locks blocked_locks
  ON blocked_locks.pid = blocked.pid AND NOT blocked_locks.granted
JOIN pg_locks blocking_locks
  ON  blocking_locks.locktype  IS NOT DISTINCT FROM blocked_locks.locktype
  AND blocking_locks.relation  IS NOT DISTINCT FROM blocked_locks.relation
  AND blocking_locks.pid      != blocked_locks.pid
  AND blocking_locks.granted
JOIN pg_stat_activity blocking
  ON blocking.pid = blocking_locks.pid;

If blocking_state is idle in transaction with a duration measured in minutes, that connection forgot to commit. You can terminate it:

SELECT pg_terminate_backend(blocking_pid);

Lock-wait aggregate view

For recurring contention you need to understand over time rather than catch in the moment:

SELECT wait_event_type, wait_event, count(*) AS backends
FROM pg_stat_activity
WHERE state != 'idle'
GROUP BY wait_event_type, wait_event
ORDER BY backends DESC;

wait_event_type = 'Lock' appearing regularly means lock contention is structural, not a one-off.

Prevention: configure PostgreSQL defensively

-- postgresql.conf

-- Kill idle-in-transaction connections after 30 seconds
idle_in_transaction_session_timeout = '30s'

-- Don't let queries wait on locks indefinitely (especially in migrations)
lock_timeout = '5s'

-- Log any time a query waits on a lock (near-zero overhead, safe in production)
log_lock_waits = on
deadlock_timeout = '1s'

idle_in_transaction_session_timeout is the single most impactful setting most teams aren't running. It makes the category of "forgot to commit" operationally impossible to sustain.

For DDL in production, always use CREATE INDEX CONCURRENTLY instead of CREATE INDEX, and set lock_timeout in your migration scripts so they bail out rather than creating a prolonged stall.

Cause 3: Sequential Awaits (Serialized Independent Operations)

await is sequential by default. Two await calls that don't depend on each other still run one after the other. The total time is their sum. This is the simplest of the five causes and the one that accumulates silently over time as handlers grow.

The problem

// Total time: ~300ms (100ms + 100ms + 100ms)
const user    = await db.users.findById(userId)
const orders  = await db.orders.recent(userId)
const reviews = await db.reviews.byUser(userId)

None of these depend on each other. The user query doesn't need to finish for the orders query to start. Running them sequentially is a pure accident of default syntax.

The fix

// Total time: ~100ms (slowest of the three)
const [user, orders, reviews] = await Promise.all([
  db.users.findById(userId),
  db.orders.recent(userId),
  db.reviews.byUser(userId),
])

Promise.all fires all three concurrently and resolves when the slowest finishes.

Promise.all vs Promise.allSettled

Promise.all rejects the entire batch if any one promise rejects. That's correct when all results are required. For supplementary data (recommendations, notifications, enrichment) where partial failure is acceptable, use Promise.allSettled:

const [profileResult, recsResult, notifResult] = await Promise.allSettled([
  db.users.findById(userId),           // required
  recommendations.fetch(userId),       // optional — degrades gracefully
  notifications.unread(userId),        // optional — degrades gracefully
])

const user          = profileResult.status === 'fulfilled' ? profileResult.value : null
const recs          = recsResult.status    === 'fulfilled' ? recsResult.value    : []
const notifications = notifResult.status   === 'fulfilled' ? notifResult.value   : []

A down recommendations service doesn't take the page with it.

The large-batch edge case

Promise.all over a large array hammers your connection pool — 1,000 concurrent queries against a pool of 10 connections will queue 990 of them. Use concurrency limiting for large batches:

import pLimit from 'p-limit'

const limit = pLimit(5)  // max 5 concurrent DB queries

const results = await Promise.all(
  userIds.map(id => limit(() => db.users.findById(id)))
)

p-limit keeps the concurrency bounded regardless of input size.

How to find these in your codebase

Search your handlers for any function that makes more than one database or external call. Any two await statements without a data dependency between them are candidates. Profile endpoints and dashboard loaders are the most common offenders — they start with two calls, accumulate to six over a year of feature development, and none of the additions were ever parallelized.

Cause 4: Event Loop Blocking

Node.js runs JavaScript on a single thread. When that thread is occupied by synchronous work — a large sort, a file read, a CPU-heavy parse — every request queued behind it waits. Not in the database. Not in the network. In your process, unable to advance.

The signature: uniform latency increase across all endpoints simultaneously. One slow endpoint usually means one slow dependency. All-endpoint degradation means the thread itself is blocked.

The metric that matters: Event Loop Utilization

CPU utilization doesn't catch this well. A process doing CPU-heavy synchronous work can spike ELU to 1.0 while showing moderate CPU — because ELU measures the proportion of time the event loop spends active (doing work) vs. idle (waiting for I/O). A healthy server sits around 0.5–0.7. Sustained above 0.85 means you're out of headroom for bursts and p99 latency degrades.

ELU is one of the cleanest saturation signals for Node.js services. It pairs with event loop delay to explain tail latency without guessing — when the loop gets starved by synchronous work, everything queues up: timers fire late, responses wait, and p99 gets ugly.

import { performance, monitorEventLoopDelay } from 'perf_hooks'

let lastELU = performance.eventLoopUtilization()

// p99 event loop delay — how long does the worst tick take?
const histogram = monitorEventLoopDelay({ resolution: 10 })
histogram.enable()

setInterval(() => {
  const elu    = performance.eventLoopUtilization(lastELU)
  lastELU      = performance.eventLoopUtilization()
  const eldP99 = histogram.percentile(99) / 1e6  // nanoseconds → ms
  histogram.reset()

  if (elu.utilization > 0.85) {
    console.warn(`[loop] ELU=${elu.utilization.toFixed(2)} — saturation risk`)
  }
  if (eldP99 > 100) {
    console.warn(`[loop] p99 delay=${eldP99.toFixed(1)}ms — blocking detected`)
  }
}, 5000)

Note: monitorEventLoopDelay underreports large spikes — it can't take measurements while the loop is blocked. Use ELU as the primary signal. Note also from the Node.js source that h.mean is skewed toward small values over time; use h.percentile(99) instead.

Common blockers

Sync APIs in request handlers:

// Blocks the thread for the entire read
const config = JSON.parse(fs.readFileSync('./config.json', 'utf8'))

// Yields while waiting
const raw    = await fs.promises.readFile('./config.json', 'utf8')
const config = JSON.parse(raw)

Any Node.js API with a Sync suffix — fs.readFileSync, crypto.pbkdf2Sync, zlib.gzipSync — runs on the main thread. They're fine at startup. They're expensive in request handlers.

In-memory sorts on large result sets:

// This sort blocks while sorting — for 50k rows that's measurable
const rows   = await db.scores.findAll()
const sorted = rows.sort((a, b) => b.score - a.score)

// The database can sort in microseconds with an index
const top100 = await db.query('SELECT * FROM scores ORDER BY score DESC LIMIT 100')

Push work into the database when you can. When you can't (complex business logic), move it to a worker thread so it doesn't block the event loop.

Catastrophic regex backtracking:

// Works on normal input. On a crafted string, backtracks exponentially.
const vulnerable = /^(a+)+$/
vulnerable.test(userInput)  // can freeze the thread for seconds

Validate and sanitize external inputs before running them through complex patterns. Use safe-regex to audit patterns in your codebase.

Finding the source in production

Measure event loop delays with node --trace-event-categories v8,node,node.async_hooks. Pinpoint asynchronous code blocking the main thread for rapid fixes. For deeper profiling, Clinic.js generates flame graphs and event loop analysis without requiring a production deploy:

npx clinic doctor -- node server.js
# Run load against it with autocannon or k6
# Generates a flame graph showing exactly where time is spent

Cause 5: External Calls Without Timeouts or Circuit Breaking

Any call to an external service imports that service's reliability and latency into yours. Average latency might be 50ms. At p99 it might be 800ms. During degradation, it might be 30 seconds or infinity.

Without a timeout, your thread waits for the full duration. Under load, multiple threads pile up waiting on the same degraded service. The application stalls — not from anything in your code, but from a dependency you didn't guard.

Always set explicit timeouts

// axios: timeout in ms
const response = await axios.get(url, { timeout: 2_000 })

// native fetch / undici: AbortSignal
const response = await fetch(url, {
  signal: AbortSignal.timeout(2_000),
})

No outbound HTTP call should ever be made without a timeout. There is no correct value for "wait forever."

Circuit breaking for repeated failures

A timeout protects individual requests. A circuit breaker protects all requests once a service starts failing repeatedly.

The circuit breaker operates as a three-state machine: CLOSED (normal — all calls pass through), OPEN (tripped — all calls immediately return a fallback, no traffic reaches the failing service), and HALF-OPEN (one probe request allowed through; success resets to CLOSED, failure stays OPEN).

opossum is the standard circuit breaker library for Node.js:

import CircuitBreaker from 'opossum'

const breaker = new CircuitBreaker(externalApi.fetch, {
  timeout: 2_000,                  // individual call timeout
  errorThresholdPercentage: 50,    // open after 50% failure rate
  resetTimeout: 10_000,            // try again after 10s
  volumeThreshold: 5,              // need at least 5 calls before tripping
})

breaker.fallback(() => null)       // return null when circuit is open

// Wire events to your observability stack
breaker.on('open',     () => metrics.increment('circuit.open',     { service: 'partner-api' }))
breaker.on('close',    () => metrics.increment('circuit.close',    { service: 'partner-api' }))
breaker.on('halfOpen', () => metrics.increment('circuit.halfopen', { service: 'partner-api' }))

// Use exactly like the original call
const data = await breaker.fire(userId)

When the circuit is open, breaker.fire() returns the fallback immediately — no network call, no latency, no cascading load. The external service gets time to recover.

The critical path question

For every external call in your codebase, ask: if this service is down, should my endpoint fail?

If yes — it's in your critical path. Protect it with a tight timeout and alert on failure rate. If no — wrap it in try/catch, return a fallback, and don't let its failure mode propagate to the user.

// Non-critical enrichment: always returns something
async function getEnrichment(userId: string) {
  try {
    const { data } = await axios.get(`https://partner.api/users/${userId}`, {
      timeout: 1_500,
    })
    return data
  } catch {
    return null  // enrichment failed — null is an acceptable degraded response
  }
}

// Critical path: let errors propagate so the caller can handle them
async function verifyPayment(paymentId: string) {
  const { data } = await axios.post(`https://payments.api/verify`, { paymentId }, {
    timeout: 5_000,
  })
  return data  // caller gets the error if this fails
}

Cause 0: The Query IS Slow (But The Log Isn't Configured Right)

Before concluding the database is innocent, make sure you've actually checked it properly.

log_min_duration_statement logs queries slower than a threshold. But if it's set to 1000ms (a common default), queries taking 400ms every time — which will destroy a dashboard endpoint — never appear. And pg_stat_statements gives you aggregate view of all queries, which surfaces patterns the per-request log can miss.

-- Enable pg_stat_statements (add to shared_preload_libraries + restart)
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

-- Queries with highest total execution time
SELECT
  left(query, 100) AS query,
  calls,
  round(mean_exec_time::numeric, 2)  AS avg_ms,
  round(total_exec_time::numeric, 2) AS total_ms,
  round(stddev_exec_time::numeric, 2) AS stddev_ms,  -- high stddev = inconsistent
  rows
FROM pg_stat_statements
ORDER BY total_exec_time DESC
LIMIT 20;

mean_exec_time shows average query speed. stddev_exec_time is the one most engineers miss: high standard deviation means the query is fast most of the time and catastrophically slow occasionally — which is exactly the profile of a query that's fast on warm cache and slow on cold read, or one that behaves well on small result sets and falls apart on large ones. The wait_event_type and wait_event columns in pg_stat_activity show if a query is waiting on a lock, I/O, or another resource — essential for diagnosing why a query appears stuck rather than slow.

Configure your postgresql.conf for production visibility:

log_min_duration_statement = 200    # log anything over 200ms — not 1000ms
log_lock_waits = on                 # log whenever a query waits for a lock
log_checkpoints = on
log_temp_files = 0                  # log any temp file creation (sort/hash spills)

Putting It All Together: The Diagnosis Order

When an endpoint is slow and the obvious explanations aren't obvious, run through this in sequence:

1. Check pool waiting count. Add the setInterval monitor. Is waitingCount > 0? If yes: the pool is the bottleneck. Check for leaks or size the pool correctly.

2. Check for blocked queries. Run the locking query against pg_stat_activity. Any idle in transaction connections with duration in the minutes? Kill them and add idle_in_transaction_session_timeout.

3. Check event loop utilization. Add the ELU monitor. Is it consistently above 0.85? If yes: something is blocking the thread. Use Clinic.js to find it.

4. Check for sequential awaits. Read the slow handler. Do two or more await calls have no data dependency between them? Move them into Promise.all.

5. Check external call durations. Every outbound HTTP call should log its response time. A p99 that's 10× the mean means one in a hundred requests is waiting on a degraded service. Add timeouts and circuit breakers.

6. Check pg_stat_statements. Look at mean_exec_time and stddev_exec_time. High stddev on a frequently-called query is a silent killer.

The slow query log is one instrument in a larger set. It catches slow execution — not waiting for connections, not waiting for locks, not blocked threads, not external service degradation. Production latency can live in any of these places, and the only way to find it is to measure all of them.

Further reading:

• Node.js perf_hooks documentation — official reference for ELU and event loop delay APIs

• Clinic.js — flame graphs and event loop analysis for Node.js

• PgBouncer documentation — connection pooler configuration reference

• opossum circuit breaker — the standard circuit breaker for Node.js

• p-limit — concurrency limiting for Promise.all at scale

• pg_stat_statements — PostgreSQL extension for aggregate query statistics

• PostgreSQL lock monitoring — official wiki with locking queries and explanations