Proxy Rotation & Session Management

This guide is designed for developers, data engineers, and automation professionals who need to manage IP rotation and session persistence efficiently using Squid Proxies. It assumes you're working at scale, possibly across multiple threads or domains, and want to avoid bans, CAPTCHAs, and session resets—without needing to contact support.

Proxy Types Overview

Squid Proxies supports two proxy types:

What Is Proxy Rotation?

Proxy rotation is the practice of changing your IP address on a regular basis or per request to avoid:

• IP-based rate limiting
• CAPTCHA triggers
• Session drops
• Blacklist flagging

Rotation can be:

• Per request – maximizes anonymity
• Per session – preserves state while avoiding overuse
• Per time interval or task – balances anonymity and stability

Using Squid Proxies for Rotation

Datacenter Proxies

• Each assigned proxy is static
• Rotation is manual: switch IPs per request, thread, or domain
• Use your full pool to distribute traffic

Sample list:

proxies = [
    "http://username:password@123.45.67.89:3128",
    "http://username:password@98.76.54.32:3128"
]

Use round-robin or random assignment based on your concurrency model.

Residential Proxies

• Use a rotating hostname and port (e.g., p102.squidproxies.com:8907)
• Rotation occurs approximately every 10 minutes
• You don’t control individual IPs, but can maintain stickiness per session by reusing the port or connection

Example configuration:

proxies = {
    "http": "http://p102.squidproxies.com:8907",
    "https": "http://p102.squidproxies.com:8907"
}

Maintaining Proxy Session State

Many workflows require IP consistency during a session:

• Logging in
• Filling multi-step forms
• Navigating dashboards
• Adding items to a cart

If the IP changes mid-session, servers may terminate the session or flag it as suspicious.

How to Maintain Sessions

• Datacenter proxies: Use the same IP for the full session
• Residential proxies: Stick to the same connection or port for under 10 minutes

Session Map Pattern:

session_map = {}

def get_session(user_id):
    if user_id not in session_map:
        proxy = select_proxy()
        session = requests.Session()
        session.headers.update({"User-Agent": get_random_user_agent()})
        session.proxies.update({"http": proxy, "https": proxy})
        session_map[user_id] = session
    return session_map[user_id]

Proxy Rotation Logic Patterns

Scaling Proxy Rotation Across Threads

When running concurrent threads:

• Assign a dedicated proxy (or proxy + user-agent) per thread
• Never share the same IP across threads at the same time
• Use thread-safe proxy pools or task queues

Example with a queue:

from queue import Queue

proxy_queue = Queue()
for proxy in proxy_list:
    proxy_queue.put(proxy)

def worker():
    proxy = proxy_queue.get()
    try:
        # perform request
        pass
    finally:
        proxy_queue.put(proxy)

Proxy Authentication Methods

Squid Proxies supports:

• IP whitelisting: Preferred for both proxy types. Configure allowed IPs via dashboard.
• Username/password: Used when IP whitelisting isn't feasible (e.g., dynamic IP environments or CI/CD pipelines).

Auth examples:

• Datacenter with IP whitelist:

proxies = {"http": "http://123.45.67.89:3128"}

• Datacenter with username/password:

proxies = {"http": "http://user:pass@123.45.67.89:3128"}

• Residential (always hostname:port):

proxies = {"http": "http://p102.squidproxies.com:8907"}

Troubleshooting Proxy Rotation Issues

Testing Proxy Rotation Setup

Use these tools to verify IP rotation and session behavior:

• https://httpbin.org/ip – confirms IP address
• https://ip-api.com/json – checks geolocation
• https://httpbin.org/headers – inspects request headers
• Developer tools – monitor cookies and session continuity

curl -x http://proxy:port https://httpbin.org/ip

Final Recommendations for Proxy Rotation & Session Management

---

More Guides:

• Test Your Proxies Before Integrating
• Use Proxies in Your Code