Bearer token

Examples

A typical API request with a bearer token looks like this:

curl https://api.example.com/data \
  -H "Authorization: Bearer eyJhbGciOi..."

In Python:

import requests

headers = {
    "Authorization": "Bearer eyJhbGciOi..."
}

response = requests.get("https://api.example.com/data", headers=headers, timeout=30)
print(response.status_code)
print(response.text)

In scraping work, you often find a bearer token by watching the site's network requests and seeing what the frontend sends to its internal API. If that token expires, requests start failing with 401 or 403 and the whole thing quietly falls over until you refresh the auth flow.

Practical tips

• Do not hardcode bearer tokens in scripts you commit: put them in environment variables or a secret manager.
• Check token lifetime early: some last for hours, some for minutes, and that changes whether your scraper is stable or annoying to maintain.
• Watch for the full auth flow, not just the header: token refresh endpoints, CSRF protection, cookies, and session state often matter.
• Treat 401 and 403 differently: 401 often means the token is missing or expired, 403 often means the token is valid but blocked from that action.
• If you are calling ScrapeRouter, this is one place where things are simpler: ScrapeRouter uses an API key header, not a bearer token.

Example with an environment variable:

export API_TOKEN="eyJhbGciOi..."
curl https://api.example.com/data \
  -H "Authorization: Bearer $API_TOKEN"

import os
import requests

token = os.environ["API_TOKEN"]
headers = {"Authorization": f"Bearer {token}"}
response = requests.get("https://api.example.com/data", headers=headers, timeout=30)

Use cases

• Accessing private API endpoints behind a logged-in app.
• Calling internal JSON endpoints a frontend uses after user authentication.
• Running scheduled data collection jobs where the scraper first logs in, gets a token, then uses that token for API requests.
• Integrating with third-party APIs that use OAuth2 access tokens in bearer format.
• Scraping authenticated dashboards where using the API directly is more reliable than trying to parse the rendered HTML every time.