Does Screaming Frog support proxies with a username and password?

Yes. The proxy settings take the address, port and credentials. With a CleanProxies ISP proxy you can also whitelist your IP and skip credentials.

Do I need to restart Screaming Frog after setting a proxy?

Yes. Proxy changes only take effect after the SEO Spider restarts.

Can I use several proxies or rotate IPs in Screaming Frog?

Only one proxy server can be configured. For a new IP per connection, use a rotating residential line, which rotates on the CleanProxies side; for audits a single static IP is usually better.

Can I exclude some sites from the proxy?

No. Either all HTTP and HTTPS traffic from the SEO Spider goes through the proxy, or none of it does.

Which proxy type works best for Screaming Frog?

Static ISP proxies: one dedicated US IP for the whole crawl that clients can allowlist, with unlimited bandwidth. Residential proxies when you need to check a site from another country.

4.7/5

Excellent

Trustpilot

How to use a proxy in Screaming Frog

In the Screaming Frog SEO Spider, open File → Settings → Proxy (Screaming Frog SEO Spider → Settings → Proxy on macOS), tick Use Proxy Server, enter your CleanProxies host and port plus your username and password, click OK and restart the SEO Spider. All HTTP and HTTPS traffic then goes through the proxy.

Try Now for $4.99

Continue with Google

Session Control
High Speed
Privacy First

What you need

SEO audits often need to see a site the way a US visitor does: which country redirect fires, which prices and hreflang versions load, and whether a CDN treats your crawl differently. Running the SEO Spider through a US CleanProxies IP shows exactly that for your own or your clients' sites.

An active CleanProxies plan

Static ISP proxies for fixed US IPs, or rotating residential / mobile proxies for pools. The $4.99 trial works too.

Your credential line

From the dashboard's proxy manager: host (or IP), port, username and password. Convert lists with the proxy list formatter.

Screaming Frog SEO Spider ready

The Screaming Frog SEO Spider on Windows, macOS or Linux.
Permission to crawl the site: your own, or a client's.

Which proxy fits Screaming Frog SEO Spider?

Both products use the same setup in Screaming Frog SEO Spider. What changes is how long an IP stays with you.

Static ISP Proxies

Best fit

Recommended. One 100% dedicated US ISP IP holds for the whole crawl and all month, so a client can allowlist it in their CDN or firewall, and unlimited bandwidth covers large crawls.

starting from$0.75/IP

Dedicated static IPs registered to US and Canadian ISPs. The address stays the same for as long as you hold it, with unlimited bandwidth and HTTP(S) or SOCKS5 access.

Buy Now See Static ISP Proxies features

Rotating Residential Proxies

Also works

Pick a residential line in the country you want to check, from 190+ countries. Use a sticky line (up to 24 hours) for full crawls so redirects and cookies stay consistent; a rotating line changes IP between connections.

starting from$0.63/GB

Residential IPs in 190+ countries with country, state and city targeting. Rotate on every request or hold a sticky session for up to 90 minutes; GB never expire.

Buy Now See Rotating Residential Proxies features

credentials

Your CleanProxies credentials

Every product connects over HTTP(S); static ISP and ISP subnet proxies also run SOCKS5 and IP whitelisting. This is what the dashboard gives you to paste into Screaming Frog SEO Spider.

Static ISP and ISP subnetHTTP(S)SOCKS5IP whitelist

IP:PORT:USER:PASS

One line per dedicated IP. With IP whitelisting on, the line is just IP:PORT and no password is sent.

Rotating residential and mobileHTTP(S)User:pass

HOST:PORT:USER:PASS

Generated in the proxy manager. The username carries the location and session you pick, so generate a new list to change them.

As a URL, for code and most toolshttp://USERNAME:PASSWORD@HOST:PORTsocks5://USERNAME:PASSWORD@HOST:PORT

Use http:// even for HTTPS websites; the TLS session still runs end to end. The socks5:// form is for static ISP and ISP subnet proxies.

How to set up CleanProxies in Screaming Frog SEO Spider

Four steps, then check the exit IP with the free proxy checker.

1
Open the proxy settings
File → Settings → Proxy on Windows and Linux, or Screaming Frog SEO Spider → Settings → Proxy on macOS. It sits under Settings, not under the crawl Configuration menu.
2
Tick Use Proxy Server
Enter your CleanProxies HOST as the address and PORT as the port.
3
Add your credentials
Enter your proxy username and password. With an ISP proxy on IP whitelisting, add your computer's public IP in the dashboard and leave the credentials empty.
4
Click OK and restart
The SEO Spider only applies proxy changes after a restart. Then start your crawl as usual.

settings

Screaming Frog SEO Spider proxy settings

Replace the highlighted placeholders with the values from your dashboard.

Screaming Frog SEO Spider + CleanProxiesText

From your dashboard line to the Proxy settings

Split the line you copy from the CleanProxies dashboard across the fields:

Field mappingText

Dashboard line   HOST:PORT:USERNAME:PASSWORD

Use Proxy Server   ticked
Address            HOST
Port               PORT
Credentials        USERNAME / PASSWORD

To go back to direct crawling, untick Use Proxy Server and restart again.

Check geo-redirects and US content

Switch to Mode → List and paste the URLs you want to check (homepage, country folders, key product pages) instead of crawling the whole site.
Open the Response Codes tab and filter on Redirection (3xx) to see which URLs redirect a US visitor and where to.
If the site serves bots differently, set Configuration → User-Agent to Chrome, as Screaming Frog's own FAQ suggests for blocked crawls.
Repeat the crawl with a residential line in another country to compare versions; the SEO Spider holds one proxy at a time.

What the proxy setting covers

One proxy server only. There is no proxy list or rotation inside the SEO Spider.
No exceptions: all HTTP and HTTPS traffic goes through the proxy, including API connections such as Search Console or PageSpeed Insights.
Keep crawl speed sensible under Configuration → Speed, especially on a client's production site.

fix it fast

Troubleshooting Screaming Frog SEO Spider proxy errors

Verify your exit IP

Crawl a page on a server you control in List mode and look for the CleanProxies IP in that server's access log. A normal 200 status in the Internal tab means the proxy connection works; if every URL shows Connection Error, the proxy details are wrong.

Open What is my IP? through the proxy to see the address and headers a website receives, or paste up to ten lines into the free proxy checker to test connectivity, latency and exit IP.

Still stuck? Our 24/7 support team answers fast: contact us with the error and the line you used (leave the password out).

Screaming Frog SEO Spider error fixes

Every URL returns Connection ErrorWhy: Wrong proxy address, port or credentials, or the proxy left switched on by mistake.Check File → Settings → Proxy, copy the details again from the dashboard, and restart.
The crawl still uses your own IPWhy: The SEO Spider wasn't restarted after the change.Click OK, close and reopen the SEO Spider, then crawl again.
403 Forbidden from the client's siteWhy: Their CDN or firewall blocks unknown crawlers.Ask the client to allowlist your static ISP IP and the SEO Spider user-agent, and lower the speed under Configuration → Speed.
Search Console or licence checks fail after enabling the proxyWhy: All traffic, including these connections, goes through the proxy, so a wrong credential breaks them too.Fix the proxy details, or untick Use Proxy Server and restart while you connect accounts.