Getting blocked is rarely about one mistake — it's the sum of signals that mark traffic as automated: a low-trust IP, an unnatural request rate, a mismatched fingerprint. Fix them in order and most blocks disappear. Here is the checklist that matters, roughly by impact.
1. Use the right IP type
IP reputation is the biggest single factor. If a target blocks you, the first move is almost always to switch from datacenter to residential — or to mobile for the very strictest sites. No amount of header tuning saves a low-trust IP on a protected target.
Match geography too: request from the country (and ideally city) your traffic should plausibly come from. A US storefront expects US visitors.
2. Rotate and pace requests
Rotate IPs so no single address accumulates suspicious volume, and throttle your request rate to something a human-scale audience could produce. Hammering a site from one IP is the clearest automation signal there is.
For stateful flows, hold a sticky session for the steps that need continuity, then rotate. Add small random delays rather than firing requests at a fixed interval.
3. Look like a real client
Send a complete, consistent set of headers — a real User-Agent, Accept and language values that match your target geography. Mismatches (a US IP with a non-US language header, or a missing User-Agent) are easy tells.
For JavaScript-heavy sites, a headless browser that renders the page and carries cookies between steps blends in better than a bare HTTP client. Reuse cookies within a sticky session so the target sees a coherent visitor.
4. Respect the target
Honor robots directives and rate limits where they apply, cache aggressively so you don't refetch the same page, and back off when you see a surge of CAPTCHAs — that's the site asking you to slow down. Sustainable scraping beats a fast run that gets your whole pool flagged.