Examples
A normal browser and a basic HTTP client can send very different TLS handshakes, even if they use the same headers and cookies. That is why requests that look fine at the HTTP layer still get blocked.
curl -I https://example.com
The request above is simple, but the server sees the TLS handshake before it sees the User-Agent. If your client hello, cipher suites, extensions, or protocol behavior do not match what the site expects, you can get challenged or blocked before headers matter.
In scraping, this usually shows up like this:
- Works in the browser, fails in code
- Works for a while, then gets blocked after a browser or TLS stack change
- Looks like a proxy issue, but is really a client fingerprint issue
Practical tips
- Treat TLS as part of your scraper fingerprint, not just transport plumbing.
- If a target blocks requests that look fine on paper: same headers, same cookies, same IP quality, check whether the TLS client fingerprint is the real problem.
- Do not assume rotating proxies fixes handshake-level detection.
- Browser-based automation usually inherits a more realistic TLS handshake than a bare HTTP client, but it costs more and is slower.
- If you are routing traffic across providers, watch for handshake differences between stacks. This is one of those annoying production details that breaks "identical" requests.
- With ScrapeRouter, this is part of the point: you are not hand-maintaining provider quirks and transport-layer behavior across a pile of integrations.
Useful things to compare when debugging:
- TLS version
- Cipher suites
- Extensions
- ALPN negotiation
- HTTP/2 support
- JA3 or similar TLS fingerprints
Use cases
- Debugging mystery blocks: Your scraper sends the right headers but still gets denied. The site is likely looking at the TLS handshake before it cares about your request payload.
- Provider evaluation: Two proxy or scraping vendors can have very different TLS behavior. One gets through, the other burns time and money.
- Browser vs raw client decisions: If a target is sensitive to TLS fingerprinting, a lightweight HTTP client may stop being worth the operational pain.
- Production stability: A library update can change handshake behavior and quietly tank success rates. This is the kind of thing that does not show up in a happy-path demo but absolutely shows up in production.