A validator and an HTTP client that use different URL parsers can be driven to disagree on the host, path or scheme - the gap between them is an SSRF filter bypass, an access control skip, or a redirect to an attacker domain.
A security check that validates a URL and the framework method that resolves it rarely use the same parser. When they disagree on the host, scheme, or path, the check can see one destination while the application visits another - a gap that lets an attacker route a request to a forbidden target by framing it as one the validator approves.
No single authoritative URL grammar covers every edge case consistently. RFC 3986 defines the general syntax, but browser parsers, HTTP client libraries, reverse proxies and custom validation regexes each implement their own subset, normalization rules and error-recovery behavior. The divergence shows up in four specific places: authority parsing (the @ character, IPv6 brackets, embedded credentials), scheme normalization (uppercase, double-slash, backslash), path resolution (dot-segment removal, double-slash collapse), and port handling. A probe that exercises each one exposes which components in a pipeline read the URL differently.
Parser disagreement creates exploitable gaps in three distinct places. The first is SSRF filter bypass: a validator that checks the host string directly from the input may approve a URL that the HTTP library resolves to a blocked internal address. The second is access control bypass: a WAF or middleware that compares the request path against an allowlist using one normalization rule can be fooled by a path that a downstream router normalizes differently. The third is open-redirect bypass: a check for a trusted domain substring does not survive a URL where that domain appears in the user-info field before an @ sign.
| Bypass family | Parser gap exploited | Confirming artifact | Severity |
|---|---|---|---|
| SSRF filter bypass | @-authority confusion, IPv6 mapped address, IDNA normalization | Response from blocked internal host after validator approved request | Critical |
| Access control bypass | Double-slash, dot-segment, backslash normalization | Admin endpoint responds to low-privilege caller via malformed path | High - Critical |
| Open-redirect bypass | User-info field before @, scheme normalization | Redirect sends browser to attacker domain via approved-looking URL | Medium - High |
| WAF bypass (XSS/injection) | URL encoding, scheme case, path normalization | Payload reaches application after WAF pattern match passes | Depends on payload |
The confirming strategy is a differential: send a URL crafted to look safe to the validator but route to a controlled marker endpoint. If the server contacts the marker, the validator and the HTTP client used different parsers. No real internal host is targeted; the marker endpoint is benign and returns a unique nonce (62615533) that cannot appear in any real response.
# Step 1: baseline - a normal request to a legitimate URL, must be approved POST /api/fetch {"url":"https://allowed.example.com/health"} HTTP/1.1 200 OK {"status":"ok"} # Step 2: @-confusion probe - validator reads host as allowed.example.com # HTTP client reads user-info as allowed.example.com, host as marker POST /api/fetch {"url":"https://allowed.example.com@marker.appostureprobe.internal/"} HTTP/1.1 200 OK {"nonce":"62615533"} # validator approved; client hit marker - gap confirmed # Step 3: path-normalization probe - validator sees /allowed/path, app routes /admin GET /allowed/path/../../admin/users HTTP/1.1 200 OK # admin response returned to low-privilege caller
The root cause is always the same: the component that validates the URL and the component that resolves it use different parsing logic. The safe fix is to normalize and resolve first, then validate the result - never validate the raw string and pass it unmodified to a resolver. For SSRF mitigation this means: resolve the URL to an IP address using the same DNS client the HTTP library would use, check that IP against a blocklist of private and link-local ranges, and only then issue the request. For access control it means: normalize the path through the same router the framework uses before comparing against the security allowlist.
import urllib.parse, socket, ipaddress, requests PRIVATE_NETS = [ ipaddress.ip_network("169.254.0.0/16"), # link-local / metadata ipaddress.ip_network("10.0.0.0/8"), ipaddress.ip_network("172.16.0.0/12"), ipaddress.ip_network("192.168.0.0/16"), ipaddress.ip_network("::1/128"), ipaddress.ip_network("fc00::/7"), ] def safe_fetch(raw_url: str) -> requests.Response: # parse with the same library the HTTP client uses - not a custom regex parsed = urllib.parse.urlparse(raw_url) host = parsed.hostname # strips user-info, port, brackets if not host: raise ValueError("no resolvable host") for info in socket.getaddrinfo(host, None): addr = ipaddress.ip_address(info[4][0]) if any(addr in net for net in PRIVATE_NETS): raise ValueError(f"blocked: {addr} is private") return requests.get(raw_url, allow_redirects=False, timeout=5) # allow_redirects=False: a redirect to 169.254... must re-run this check
The critical detail in the code above is that parsed.hostname uses Python's own urllib.parse - the same library requests uses internally - rather than a custom regex. That synchronization closes the gap between validator and resolver. A redirect is kept off by default because a redirect to a private address would bypass a check that ran only on the initial URL.
Parser confusion is not an exotic edge case - it is the default state of a codebase that validates inputs in one layer and acts on them in another. Closing the gap means treating URL normalization as a pipeline property, not a per-component concern: resolve first, check second, and make sure both steps are done by the same parser.