TTFB | ScrapeRouter

TTFB stands for Time to First Byte: the time between making a request and receiving the first byte of the response. It tells you how long the network, server, TLS handshake, redirects, and backend work took before anything actually started coming back, so when it is bad, everything after it starts late too.

Examples

A scraper can have a perfectly fast parser and still feel slow because the real delay is upstream in TTFB.

Example: target returns HTML in 200 ms once it starts sending, but first byte does not arrive for 2.8 seconds : your bottleneck is connection setup, blocking, redirects, or server-side delay, not parsing.
Example: one proxy pool gives 600 ms TTFB and another gives 3.5 s TTFB : both may eventually return 200, but one will kill throughput in production.

import time
import requests

url = "https://example.com"
start = time.time()
resp = requests.get(url, timeout=30, stream=True)
first_chunk = next(resp.iter_content(chunk_size=1))
ttfb_ms = (time.time() - start) * 1000

print({
    "status": resp.status_code,
    "ttfb_ms": round(ttfb_ms, 2),
    "first_byte": first_chunk
})

curl -o /dev/null -s -w 'dns=%{time_namelookup}\nconnect=%{time_connect}\ntls=%{time_appconnect}\nttfb=%{time_starttransfer}\ntotal=%{time_total}\n' https://example.com

Practical tips

Measure TTFB separately from total request time. If you only watch total duration, you miss whether the delay is before the response starts or during download.
Watch for the boring production causes : cold connections, bad proxy routes, extra redirects, slow TLS negotiation, overloaded origin servers, bot challenges.
Compare TTFB by provider, region, and target domain. A setup that looks fine in local testing can collapse at scale because one route adds 2 to 4 seconds before first byte.
Reuse connections where possible. Opening a fresh TCP and TLS session for every request adds avoidable latency.
Log request phases when debugging : DNS, connect, TLS, first byte, total. Without that breakdown, people guess, and they usually guess wrong.
In scraping, high TTFB often means lower throughput even when success rate looks acceptable. You are still paying in wall-clock time, worker capacity, and retries.
If you are routing requests through ScrapeRouter, track TTFB alongside success rate and cost. A request that succeeds with lower first-byte latency is often the one that actually scales better in production.

Use cases

Scraper performance debugging: figuring out whether a job is slow because the target is slow to respond, the proxy path is bad, or your code is doing something dumb after the response arrives.
Provider comparison: testing two proxy or unblocker providers where both return valid HTML, but one has much worse TTFB and cuts overall throughput.
Bot defense detection: spotting cases where the origin stalls, redirects, or challenge-pages requests before returning content.
Capacity planning: estimating how many workers you need when each request spends most of its life waiting for first byte.
Regression monitoring: catching when a target site, proxy pool, or routing change adds 1 to 2 seconds before response start and quietly wrecks scrape speed.