Examples
A profile is what lets a browser stay logged in across runs instead of redoing auth every time.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--user-data-dir=/tmp/my-scraper-profile")
driver = webdriver.Chrome(options=options)
driver.get("https://example.com/account")
print(driver.title)
driver.quit()
In production, teams often keep separate profiles per account.
- Example split: one profile for account A, one for account B, one clean profile for testing
- What persists: cookies, local storage, session tokens, site preferences
Practical tips
- Treat profiles as stateful infrastructure, not a convenience feature
- Use one profile per account or identity: sharing a profile across multiple accounts is how you create weird cross-session bugs
- Don’t assume a saved profile will live forever: cookies expire, sessions get revoked, sites rotate checks
- Keep profiles isolated by job type: login flows, checkout flows, and general scraping runs often behave better when separated
- Be careful with concurrency: two workers writing to the same profile at once can corrupt it or produce flaky behavior
- If a profile starts failing, test with a fresh one before blaming proxies or selectors
- For simple public-page scraping, skip profiles entirely: they add storage, state, and debugging overhead you may not need
Use cases
- Staying logged into a site across scheduled scraping runs
- Reusing a warm session to reduce repeated login, MFA, or consent flows
- Managing multiple seller, buyer, or customer accounts with separate browser state
- Preserving site preferences like region, language, currency, or dismissed popups
- Browser automation flows where stateless runs keep getting flagged because every session looks brand new