Server notes

Dedicated servers for proxy work: read the logs first

Buy hardware only after the logs blame the host, not the proxy route.

Dedicated servers Queue age, browser workers, proxy timing Updated 2026-06-12

Proxy problems do not respond to more hardware

A dedicated server helps when the machine running the scraper is the bottleneck. It does not help when the slow part is proxy latency, target response time, retries, or challenge pages. Those are proxy or pacing problems, and adding CPU makes the retry loop louder, not faster.

Do not tune the machine and the proxy at the same time. Change one variable, keep the log, then decide whether the bottleneck was CPU, memory, network, or the exit route.

The log row that settles the argument

A run was falling behind. CPU looked busy at a glance, and the first guess was a bigger box. After lining up worker logs with proxy timing, the actual problem was visible: retries had doubled and two targets were returning challenge pages.

The row that settled it: worker=br-07 queue_age=18m cpu=41 ram=79 proxy_p95=420ms status=403 provider_bytes=1.8gb app_bytes=612mb retries=9 note=challenge_page. Queue age was bad, but the host was not pinned. Status codes and provider-metered bytes explained the day better than the processor graph did.

HTTP workers and browser workers fail differently

HTTP collectors usually want many light workers, hard retry limits, and clean accounting by target host. Browser collectors stress RAM, process cleanup, context isolation, temp files, screenshots, and stuck child processes. A browser run can look stable for an hour, then slowly degrade because old contexts never close.

Before moving scrapers to dedicated hardware, put these numbers side by side: worker start and finish times, queue age, CPU, RAM, disk growth, proxy latency p95, status codes, provider-metered bytes, and the app's own byte counter. The app counter can mislead by omission: it may count completed pages while leaving out retries, blocked responses, partial bodies, or bytes that moved outside the measured path.

Signs the host is actually the limit

Dedicated hardware is the right answer when queues grow while proxy timing stays steady, workers wait on local parsing or rendering, browser contexts crash from memory pressure, or disk writes fall behind logs and output.

It is the wrong answer when 403 or 429 rates climb, proxy latency rises, or target responses slow down while local workers sit mostly idle. Those are target or routing problems.

Reading the server catalog after the evidence

Proxynade's bare-metal catalog is available at /products/servers. For browser-heavy work, RAM headroom matters more than the small monthly price gap between tiers. For HTTP workers, either entry-level box may exceed what the job actually needs if the real limit is target behavior.

A smaller worker pool run cleanly will outperform a bigger server with bad retry rules burning proxy traffic on blocked responses.

A two-run test before committing to hardware

Run the same target set twice: once at current concurrency and once with tighter worker limits. Keep retry rules fixed across both runs.

If lower concurrency barely hurts completed output and makes the run stable, the previous setup was thrashing. If more local capacity drains queues while proxy timing and status codes stay normal, dedicated hardware is doing real work. If queues empty quickly and targets are still slow, leave the server catalog closed.