Label failed rows before touching anything
After one public catalog run where every bad line was called "blocked," the word stopped meaning anything. Some URLs failed on the first request. Some loaded twice then hit 429. Some returned a challenge page before any product text. A few loaded assets but the parser returned empty results. Those are different problems.
Run one route at small scale first. Scaling before the failure mode is labeled only hides which part failed and makes the usage logs harder to read.
| Signal | What it indicates | First check |
|---|---|---|
| 403 on first request | ASN or IP reputation block at the edge | Check exit network ownership before touching headers. |
| Challenge page before content | Bot score or fingerprint check | TLS fingerprint, header order, or request pacing — not the proxy tier alone. |
| 429 after two requests | Rate limit at the target | Pacing and retry logic. Changing routes will not fix this. |
| Assets load, parser returns empty | Client-side rendering or selector drift | Proxy is fine. The parser or the client is the problem. |
First-request 403 means check the ASN
A 403 that arrives before any authentication exchange means the target classified the exit IP's ASN and rejected the connection at the edge. Changing headers will not help. The exit network is the variable.
Datacenter is the right starting point for simple public targets. When every hosting network dies immediately on a given host, that host is scoring by network type. Residential routing gets tested there — not applied to everything by default.
Volume Residential is $0.89/GB for broad country, region, or city rotation via proxynade.net:2555. Move one host or one URL group first. If that slice improves, the route was the problem. Premium Residential at $5.00/GB covers cases where the target scores by ISP or carrier fit rather than just network class.
TLS fingerprint and header shape are client problems
TLS fingerprinting identifies the client library from the ClientHello handshake: cipher order, extensions, GREASE values. A target can reject a connection at the TLS layer before reading a single header. Switching proxy tiers does not change this — the client library does.
HTTP/2 frame order, header capitalization, and the sequence of standard headers also produce a fingerprint. A Python httpx client looks different from a Chromium browser regardless of which proxy it exits through.
Request timing produces a third signal. Bursts of identical requests at machine speed, or retrying at exact fixed intervals, are anomalous. Pacing and jitter belong in the client, not the proxy config.
When a failure follows every proxy tier, at least one of these client signals is the issue. Record that as a client or pacing problem, not a proxy problem.
Usage logs show cost; app counters do not
App-level counters record accepted rows, saved HTML, and sometimes screenshots. They skip retries, redirects, blocked pages, failed pages, image loads, discarded responses, and challenge pages that were downloaded but not parsed. The Proxynade dashboard usage logs record every byte transferred, host by host, with outcome and latency. That is the accurate cost number.
Read the usage logs before calling a tier expensive or declaring a route bad. A Premium Residential byte count that looks high often traces to retries against a host that was never going to respond differently, or to assets the app silently loaded and discarded.
Block domains after confirming the parser still works
Chat widgets, review panels, video hosts, and tracking pixels can consume bandwidth without contributing to the parsed output. Blocking those domains cuts cost but can also change the page shape in ways that break the parser.
Block one small set. Rerun a few URLs. Confirm the parser returns the same fields. If the page shape changes, undo it. Do not block speculatively across a whole domain list.
Change one variable per test run
Same URL list. Same client. Same pace. One route changed. If the failure disappears only on residential, the host scored the datacenter network. If the failure follows every route, the proxy tier was not the main variable.
Static ISP is a per-IP line with unlimited bandwidth — useful for comparing a fixed public route against rotating residential on a specific host. It does not fix every blocked row.
When a target requires authentication, a paywall, a private API contract, a robots-restricted path, or any other form of authorization, the path forward is getting that authorization, not changing the proxy tier.
Detection signals FAQ
Why does a first-request 403 point to ASN rather than credentials? The 403 arrives before any authentication exchange. The target classified the exit IP's ASN and rejected the connection at the edge, so changing headers will not help — the exit network is the variable.
When should I switch from datacenter to residential? Move one host or URL group first. If that slice improves, the route was the problem. Volume Residential is $0.89/GB for broad country rotation; Premium Residential is $5.00/GB when ASN or ISP fit is the variable.
Why does my app show fewer failures than the proxy usage logs? App-level counters record accepted rows and saved HTML. The Proxynade dashboard usage logs record every byte transferred — retries, redirects, blocked pages, and discarded responses included. The usage logs are the accurate cost number.
What does blocking by TLS fingerprint mean? TLS fingerprinting identifies the client library from the ClientHello handshake — cipher order, extensions, GREASE values. A target can reject a connection at the TLS layer before reading headers or cookies. Switching proxy tiers does not change this; the client library does.
When does blocking a domain in the request list help? Only when the blocked domain is consuming bandwidth without contributing to the parsed result. Block one small set, rerun a few URLs, and confirm the parser returns the same fields. If the page shape changes, undo it.
Diagnostic checklist
- Label each failure mode before changing anything.
- Run one route at small scale before scaling up.
- Check ASN before changing headers on a first-request 403.
- Read usage logs, not app counters, for cost.
- Change one variable per test: route, client, pace, or scope.
- Verify parser output after blocking any domain group.