Static vs Rotating Proxies for Web Scraping: Decision Table, Costs and Code

Static vs rotating proxies for scraping comes down to one question: can a single IP keep carrying your traffic to this site? If a site lets one address make steady requests (product pages you re-check daily, your own sites, public catalogs), static ISP proxies win: a fixed, dedicated IP with unlimited bandwidth is cheaper per page and faster than paying per GB. If a site limits each IP hard, or you need many countries and cities, rotating residential proxies win: every request (or every sticky session) gets a fresh IP from a large pool, and you pay only for the data you use. This guide gives you a decision table by target type, a break-even formula between per-IP and per-GB pricing, and a tested Python router that sends each site to the right pool. If you are choosing proxies for web scraping right now, CleanProxies sells both: static ISP proxies from $0.75/IP and rotating residential proxies from $0.63/GB, the cheapest pricing in the market.

TL;DR

• Static ISP proxies are the default for repeat crawls of the same sites: dedicated IPs on Tier-1 US carrier ASNs, unlimited bandwidth and threads, sub-20 ms latency, 0 fraud score. Cost scales with IP count, not traffic.

• Rotating residential proxies are the default when per-IP limits are strict, when you need 190+ countries or city/state targeting, or when traffic per IP would be tiny. Cost scales with GB, and CleanProxies data never expires.

• The break-even: at CleanProxies' as-low-as rates, one ISP IP per month costs the same as 1.19 GB of residential data. If each IP in your plan would carry more than that, static is cheaper. If it would carry less, rotate.

• Most serious scrapers use both. Our tested router (code below) sends each site to the pool that fits it.

• Test before you buy for $4.99: the trial gives you 5 dedicated ISP IPs for 1 hour, or 100 MB of residential data (one trial per account, you pick the product).

Static vs rotating proxies at a glance

What static and rotating actually mean for a scraper

A static ISP proxy is one IP address that belongs to a consumer internet provider (an ISP such as AT&T or Comcast) but is hosted in a data center. You get the reputation of a home connection with data-center speed, and the address is yours for the month. CleanProxies ISP IPs are 100% dedicated, so nobody else's scraper burns their reputation. For a scraper, that means you control the request rate per IP completely: if you pace each IP politely, it stays clean for months. For a deeper look at the type itself, read what is a static residential proxy.

A rotating residential proxy is a gateway (resipro.cleanproxies.com:5555) in front of a pool of real home devices. Each new connection exits from a different residential IP unless you use a sticky line, which holds one IP for a set time. For a scraper, that means you never have to manage IPs: you just send requests and let the pool spread them. The full mechanics of session modes are in rotating vs sticky sessions; here we only care about which one to use per target.

The difference that matters is who holds the per-IP budget. Most sites rate-limit per IP address (an HTTP 429 "Too Many Requests" answer, defined in RFC 6585, often with a Retry-After header). With static IPs, you spend that budget deliberately, IP by IP. With rotating IPs, the pool spreads it across millions of addresses for you.

Two separate choices, not one

"Static vs rotating" mixes two decisions. The first is the IP source: dedicated ISP addresses or a residential peer pool. The second is rotation behaviour: a fixed IP, a new IP per request, or a sticky IP for a while. You can rotate across your own static ISP IPs (the router below does), and you can hold one residential IP with a sticky line. Sticky residential sessions are a target length, not a guarantee: the IP comes from a real home device, so a session can end early if that peer goes offline, and your code should handle a changed IP.

Also check what the site rate-limits on. RFC 6585 lets a server count requests per user, per session or per cookie, not only per IP. If a site limits your account or cookie, a new IP changes nothing: slow down instead. Rotation solves per-IP limits, not every 429.

The decision table: which proxy for which target

Work through it in this order for each target site: first what identity and geo the job needs, then what the site accepts per IP, then the measured cost.

The cost model: per IP vs per GB

Static ISP proxies are billed per IP per month with unlimited bandwidth. Rotating residential proxies are billed per GB. That gives a clean break-even:

break-even GB per IP per month = (price per IP per month) / (price per GB)

At CleanProxies' as-low-as rates, that is $0.75 / $0.63 = 1.19 GB per IP per month. Those are the top volume tiers, and AT&T IPs have their own price ladder, so plug in the tier you would actually buy from /pricing: the ratio is what matters, and the script below takes both prices as arguments. Minimum orders are 5 ISP IPs or 5 GB of residential data.

Then work out how many GB each static IP would carry in your plan:

1. Traffic per month = pages per day x 30 x page size x overhead (retries, headers and TLS; we use 15%).

2. IPs needed = pages per hour / safe requests per IP per hour (one set per country).

3. GB per IP = traffic / IPs. Above break-even, static is cheaper. Below it, rotate.

The model assumes traffic is spread evenly over 24 hours and that both pools reach a similar success rate on the target. If your crawl runs in a 4-hour window, or one pool gets more blocks, adjust: what you really pay for is cost per validated, successful page.

Page sizes we measured (6 Nov 2026)

We fetched five public pages once each, directly from a server (no proxy), with a Chrome user agent, using curl -s -o /dev/null --compressed -w "%{size_download}" URL for the compressed response body and the same request without --compressed for the raw HTML. Billed proxy traffic is a bit higher than the body size: request and response headers, TLS handshakes and retries add to it, which is what the 15% overhead in the script stands for.

Two lessons. First, always request compression (Accept-Encoding: gzip, br): it cut these pages by 3x to 10x, and on per-GB plans that is money. Second, plain HTML is tiny next to a rendered browser page: the median desktop page weighs about 2.6 MB with all its resources (HTTP Archive Web Almanac 2024). If you render in a browser, block images, fonts and media you don't need, or move that job to static IPs.

The break-even script

#!/usr/bin/env python3
"""Static ISP (per IP, unmetered) vs rotating residential (per GB) for a scraping job.
Usage: python3 breakeven.py [price_per_ip_month] [price_per_gb]"""
import math, sys

ISP_PER_IP_MONTH = float(sys.argv[1]) if len(sys.argv) > 1 else 0.75
RESI_PER_GB = float(sys.argv[2]) if len(sys.argv) > 2 else 0.63
ISP_COUNTRIES = {"US", "CA", "GB", "FR"}            # static ISP geo, Nov 2026
BREAK_EVEN_GB = ISP_PER_IP_MONTH / RESI_PER_GB

def job(name, pages_per_day, kb_per_page, safe_req_per_ip_per_hour, countries=("US",), overhead=1.15):
    gb_month = pages_per_day * 30 * kb_per_page * overhead / 1e6
    ips = max(1, math.ceil(pages_per_day / len(countries) / 24 / safe_req_per_ip_per_hour)) * len(countries)
    gb_per_ip = gb_month / ips
    if not set(countries) <= ISP_COUNTRIES:
        verdict = "rotating residential (geo: static ISP not sold there)"
    elif gb_per_ip > BREAK_EVEN_GB:
        verdict = "static ISP (each IP carries more than the break-even GB)"
    else:
        verdict = "rotating residential (too few GB per IP)"
    print(f"{name:<36} {gb_month:>8.1f} GB/mo  {ips:>5} IPs  {gb_per_ip:>7.2f} GB/IP  -> {verdict}")

print(f"ISP ${ISP_PER_IP_MONTH}/IP/month vs residential ${RESI_PER_GB}/GB: "
      f"break-even = {BREAK_EVEN_GB:.2f} GB per IP per month\n")
job("A 50k product pages/day, HTML",        50_000,   50, 120)
job("B 50k pages/day, browser-rendered",    50_000, 2000, 120)
job("C 5k SERPs/day, strict 20 req/IP/h",    5_000,   40,  20)
job("D 200k pages/day, strict 10 req/IP/h", 200_000,  20,  10)
job("E 2k pages/day in each of 40 countries", 2_000 * 40, 50, 120,
    countries=tuple(f"C{i}" for i in range(40)))

Output (Python 3.8, run 6 Nov 2026):

ISP $0.75/IP/month vs residential $0.63/GB: break-even = 1.19 GB per IP per month

A 50k product pages/day, HTML            86.2 GB/mo     18 IPs     4.79 GB/IP  -> static ISP (each IP carries more than the break-even GB)
B 50k pages/day, browser-rendered      3450.0 GB/mo     18 IPs   191.67 GB/IP  -> static ISP (each IP carries more than the break-even GB)
C 5k SERPs/day, strict 20 req/IP/h        6.9 GB/mo     11 IPs     0.63 GB/IP  -> rotating residential (too few GB per IP)
D 200k pages/day, strict 10 req/IP/h    138.0 GB/mo    834 IPs     0.17 GB/IP  -> rotating residential (too few GB per IP)
E 2k pages/day in each of 40 countries    138.0 GB/mo     40 IPs     3.45 GB/IP  -> rotating residential (geo: static ISP not sold there)

What the numbers mean

• Job A and B are the classic daily price crawl. Each IP carries 4.8 GB (HTML) to 192 GB (browser) a month, far above 1.19 GB. Static ISP proxies with unlimited bandwidth are the clear pick, and the gap grows with page weight: browser rendering through per-GB proxies costs 40 times more data than HTML.

• Job C and D hit sites that allow only 10 to 20 requests per IP per hour. Job C needs 11 static IPs carrying 0.63 GB each, and job D needs 834 IPs carrying 0.17 GB each: below break-even in both cases, so the rotating pool is cheaper and simpler.

• Job E has enough traffic per IP for static to win on cost, but static ISP is not sold in 40 countries. Geo decides it: rotating residential.

• The safe rate per IP is your most important input. Measure it on each target (start slow, raise until 429s appear, then back off by half) instead of guessing. Sizing formulas for IP counts are in how many proxies do I need.

A tested router: send each site to the right pool

Most teams end up with a mixed setup. This router keeps that simple: a static pool that paces each IP and honours Retry-After, a rotating gateway that opens a new connection per request (so each request gets a new exit IP), and a routing table per site.

#!/usr/bin/env python3
"""Route each scraping target to static ISP IPs or a rotating residential gateway.
Static: round-robin over dedicated IPs with a per-IP cooldown that honours Retry-After.
Rotating: one gateway line, new connection per request (new exit IP per request).
Sticky: one sticky generator line per multi-step task."""
import time, requests
from urllib.parse import urlparse
from email.utils import parsedate_to_datetime

def proxy_url(line):                      # generator line host:port:user:pass
    host, port, user, pw = line.strip().split(":", 3)
    u = f"http://{user}:{pw}@{host}:{port}"
    return {"http": u, "https": u}

def retry_after_seconds(resp, default=30.0):
    v = resp.headers.get("Retry-After")
    if not v:
        return default
    try:
        return max(0.0, float(v))
    except ValueError:                    # HTTP-date form
        try:
            return max(0.0, parsedate_to_datetime(v).timestamp() - time.time())
        except (TypeError, ValueError):
            return default

class StaticPool:
    def __init__(self, lines, min_gap):
        self.ips = [{"proxy": proxy_url(l), "free_at": 0.0} for l in lines]
        self.min_gap = min_gap            # seconds between requests on one IP for one site

    def get(self, url, tries=4, **kw):
        for _ in range(tries):
            ip = min(self.ips, key=lambda i: i["free_at"])   # least-recently-limited IP
            wait = ip["free_at"] - time.time()
            if wait > 0:
                time.sleep(wait)
            r = requests.get(url, proxies=ip["proxy"], timeout=20, **kw)
            if r.status_code in (429, 503):
                ip["free_at"] = time.time() + retry_after_seconds(r)
                continue
            ip["free_at"] = time.time() + self.min_gap
            return r
        return r

class RotatingGateway:
    """Residential gateway. Rotating line: no connection reuse, so each request can get a new exit.
    Sticky line: pass sticky=True to keep one session (cookies + connection) for a multi-step task."""
    def __init__(self, line, sticky=False):
        self.proxy = proxy_url(line)
        self.session = requests.Session() if sticky else None

    def get(self, url, tries=4, **kw):
        for attempt in range(tries):
            if self.session:
                r = self.session.get(url, proxies=self.proxy, timeout=30, **kw)
            else:
                r = requests.get(url, proxies=self.proxy, timeout=30,
                                 headers={"Connection": "close"}, **kw)
            if r.status_code not in (403, 429, 503):
                return r
            time.sleep(min(retry_after_seconds(r, default=2 ** attempt), 30))
        return r

# Your decision table, per site host (see the guide for how to fill it)
ROUTES = {
    "static":   {"shop-a.example", "your-own-site.example"},   # repeat crawls, same pages daily
    "rotating": {"serp.example", "listings.example"},          # strict per-IP limits, many geos
}

class Router:
    def __init__(self, static_pool, gateway, routes=ROUTES):
        self.static, self.gateway, self.routes = static_pool, gateway, routes

    def get(self, url, **kw):
        host = urlparse(url).hostname or ""
        if host in self.routes["static"]:
            return self.static.get(url, **kw)
        if host in self.routes["rotating"]:
            return self.gateway.get(url, **kw)
        raise ValueError(f"no route for {host}: add it to ROUTES")

# Example
# router = Router(StaticPool(isp_lines, min_gap=2.0), RotatingGateway(rotating_line))
# html = router.get("https://shop-a.example/item/123").text

Use your ISP lines (host:port:user:pass) from the dashboard for StaticPool, and one rotating line from the residential generator for RotatingGateway. For multi-step flows, create one RotatingGateway(sticky_line, sticky=True) per task: a sticky line from the generator keeps one IP, and the requests.Session keeps cookies and the connection for the whole flow. Copy generator lines exactly; the username carries your targeting and session settings. If a pool keeps answering with errors, the fixes are in how to fix proxy errors, and more rotation patterns (httpx, Scrapy) are in how to rotate proxies in Python.

How we tested it (6 Nov 2026)

We did not want to hammer a real website to prove a point, so we built a local simulation: a test site that allows 5 requests per IP per 10 seconds and then answers 429 with Retry-After: 3, three local proxies that each act as one static IP, and a local gateway that gives every request a new exit address. Nothing left the machine. Python 3.8.10, requests 2.22.0, 30 requests per run:

1 static IP, no pacing, no retry             200= 5 429=25  target saw 30 requests from  1 exits    0.2s
StaticPool: 3 ISP IPs, 2 s gap per IP        200=30 429= 0  target saw 30 requests from  3 exits   18.1s
StaticPool: 1 IP, no gap, honours Retry-After 200= 9 429= 3  target saw 21 requests from  1 exits   33.2s
RotatingGateway: new exit per request        200=30 429= 0  target saw 30 requests from 30 exits    0.1s

(The third run made 12 fetches instead of 30, with up to 4 tries each.)

• One static IP, flat out: 25 of 30 requests rate-limited. This is the most common scraper failure, and it is a pacing problem, not a proxy problem.

• Three static IPs, paced at one request per 2 s each: 30 of 30 succeeded with zero 429s. Same IPs, same site, just paced.

• One IP that honours Retry-After but has no pacing: it recovers, but slowly (9 successes, 3 failures after 4 tries). Waiting is not a substitute for enough IPs.

• Rotating gateway: 30 of 30, each from a different exit, with no pacing logic at all. On a site that limits per IP, this is the easy path.

• Router: 9 of 9 requests for the "static" host went through the 3-IP pool, and 9 of 9 for the "rotating" host went through the gateway.

This is a simulation: the mock gateway hands out a made-up exit per request, so it proves the code's logic (pacing, Retry-After, routing), not a real site's limits, real pool size or speed. Sticky mode is not exercised by it. Before you scale, measure each target's safe rate through your own proxies. The harness, so you can rerun it:

# mock_test.py: target allows 5 req per exit per 10 s, then 429 + Retry-After: 3. Stays on 127.0.0.1.
import base64, collections, random, socketserver, threading, time, http.server, urllib.request, urllib.error
import requests
from router import StaticPool, RotatingGateway

LIMIT, WINDOW = 5, 10.0
hits, seen = collections.defaultdict(list), collections.Counter()

class Target(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        ip, now = self.headers.get("X-Exit", "direct"), time.time()
        hits[ip] = [t for t in hits[ip] if now - t < WINDOW] + [now]; seen[ip] += 1
        code = 429 if len(hits[ip]) > LIMIT else 200
        self.send_response(code)
        if code == 429: self.send_header("Retry-After", "3")
        self.send_header("Content-Length", "2"); self.end_headers(); self.wfile.write(b"ok")
    def log_message(self, *a): pass

def make_proxy(exit_fn):                  # plain-HTTP forward proxy that stamps its exit name
    class P(http.server.BaseHTTPRequestHandler):
        def do_GET(self):
            want = "Basic " + base64.b64encode(b"user:pass").decode()
            if self.headers.get("Proxy-Authorization") != want:
                self.send_response(407); self.send_header("Content-Length", "0"); self.end_headers(); return
            req = urllib.request.Request(self.path, headers={"X-Exit": exit_fn()})
            try: r = urllib.request.urlopen(req); code, h, body = r.status, r.headers, r.read()
            except urllib.error.HTTPError as e: code, h, body = e.code, e.headers, e.read()
            self.send_response(code)
            if h.get("Retry-After"): self.send_header("Retry-After", h["Retry-After"])
            self.send_header("Content-Length", str(len(body))); self.end_headers(); self.wfile.write(body)
        def log_message(self, *a): pass
    return P

def serve(port, handler):
    socketserver.ThreadingTCPServer.allow_reuse_address = True
    s = socketserver.ThreadingTCPServer(("127.0.0.1", port), handler); s.daemon_threads = True
    threading.Thread(target=s.serve_forever, daemon=True).start()

serve(28080, Target)
for i in range(3): serve(28101 + i, make_proxy(lambda i=i: f"isp-{i+1}"))
serve(28200, make_proxy(lambda: f"resi-{random.randint(1, 50_000_000)}"))
time.sleep(0.3)

def run(name, fetch, n=30):
    seen.clear(); hits.clear(); codes = collections.Counter(fetch("http://127.0.0.1:28080/item").status_code for _ in range(n))
    print(f"{name:<40} 200={codes[200]:>2} 429={codes[429]:>2} exits={len(seen)}")

run("StaticPool: 3 ISP IPs, 2 s gap", StaticPool([f"127.0.0.1:{28101+i}:user:pass" for i in range(3)], 2.0).get)
run("RotatingGateway", RotatingGateway("127.0.0.1:28200:user:pass").get)

Example setups that work

Whatever the job, scrape public data, respect each site's terms and robots.txt, and keep request rates polite. Proxies spread load; they do not make abusive traffic acceptable.

Key stats

• 51% of all web traffic in 2024 was automated, and bad bots made up 37% (Imperva 2025 Bad Bot Report). That is why sites rate-limit per IP, and why your per-IP rate is the key input.

• The median desktop page weighs about 2.6 MB with all resources (HTTP Archive Web Almanac 2024), while the compressed HTML of the pages we measured was 2.9 to 47 KB.

• HTTP 429 with Retry-After is the standard "slow down" answer (RFC 6585); our router waits exactly as long as the server asks.

• 1.19 GB per IP per month is the break-even between CleanProxies static ISP ($0.75/IP) and rotating residential ($0.63/GB) at as-low-as rates (our calculation above).

Which should you pick?

Start with the question at the top: can one IP keep carrying your traffic to this site?

• Yes, and you crawl the same sites repeatedly: CleanProxies static ISP proxies. Dedicated IPs on Tier-1 carrier ASNs, 0 fraud score, unlimited bandwidth and threads, sub-20 ms latency, 99.9% uptime and instant delivery, from $0.75/IP.

• No, the site limits each IP hard, or you need many geos: CleanProxies rotating residential proxies. 50M+ IPs, 190+ countries, city and state targeting, sticky sessions up to 24 hours, from $0.63/GB, and your data never expires.

• Mobile-only targets (sites that serve different content to phone networks): CleanProxies mobile proxies from $0.79/GB in 150+ countries.

• Both kinds of target: run the router above with an ISP pool and a residential line.

Not sure where CleanProxies stands against other providers? The comparison page puts the numbers side by side, and our ranking of the best residential proxies for web scraping covers the rotating side in depth.

Testing checklist before you scale

1. List your target sites and fill in the decision table for each.

2. Measure each site's compressed page size (curl -s -o /dev/null --compressed -w "%{size_download}\n" URL).

3. Find the safe rate per IP: start slow, increase until 429s or 403s appear, then halve it.

4. Run the break-even script with your real tier prices from /pricing.

5. Check your exit IPs with the proxy checker: location, ASN and that each static IP stays the same.

6. Confirm the content you get back is the real page (not a block page or an empty shell) before counting a request as a success.

7. Test the pool your table points to on your real targets with the $4.99 trial: 5 dedicated ISP IPs for 1 hour, or 100 MB of residential data (one trial per account). Start the $4.99 trial.

CleanProxies pricing

Full tiers are on the pricing page. Most US scraping setups start with US proxies on static ISP and add a residential GB balance for the geo-spread jobs.

Get started

Pick the right pool per site and you stop paying for blocks and wasted IPs. Compare plans on /pricing, get static ISP proxies for your repeat crawls, add rotating residential proxies for strict and multi-geo targets, and see every scraping setup on our proxies for web scraping page. CleanProxies support is available 24/7 if you want help sizing your pool.

Frequently Asked Questions

Team CleanProxies

Proxy Infrastructure Team

Share