Skip to content

Persistent HTTP 400 Errors After Failed CONNECT Request #2071

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jasonjoo2010 opened this issue Mar 9, 2025 · 0 comments
Closed

Persistent HTTP 400 Errors After Failed CONNECT Request #2071

jasonjoo2010 opened this issue Mar 9, 2025 · 0 comments

Comments

@jasonjoo2010
Copy link
Contributor

Hi folks,

We rely heavily on AHC for its performance and async support in many internal projects. However, we've recently encountered multiple escalations regarding unusual HTTP 400 errors.

Context

Our applications operate under strict network restrictions and must use an HTTP proxy to access external servers. This setup has worked reliably for years.

Issue Details

We've identified that when initializing an HTTPS tunnel via a proxy:

  1. If the initial CONNECT request fails (e.g., 503/403), the connection is pooled rather than abandoned.
  2. All subsequent requests on this connection fail with HTTP 400 (INVALID_URL) because only the HTTP path is sent instead of a fully qualified URL (as required by the proxy).
  3. Although the scheme is HTTPS, the payload is sent in cleartext after the failure.
  4. Once this issue starts, it persists indefinitely.
  5. Furthermore, as the proxy responds HTTP400 after receiving the client request immediately, the "bad" connections have higher chance of being selected for the requests.

Expected Behavior

  • The client should either abandon the connection or retry the CONNECT request to recover.
  • Failed connections should not be reused in a way that leads to invalid HTTP requests.

Actual Behavior

  • The client neither abandons nor reattempts the CONNECT request.
  • Instead, it continues sending requests in cleartext, leading to persistent failures.

This appears to be a bug in the error handling logic.

Thanks
Jason
@jasonjoo2010 jasonjoo2010 mentioned this issue Mar 9, 2025
Merged
hyperxpro pushed a commit that referenced this issue Mar 9, 2025
@jasonjoo2010
fix: inappropriate connection reuse when using HTTP proxy if the init…
f194152 
…ial CONNECT failed (#2072)

# What This MR Resolves

A CONNECT request is needed to sent to the HTTP proxy first before the
actual client request to establish the tunnel on the proxy. A `HTTP/1.1
200 Connection established` is expected for the initial CONNECT request.
Only when the CONNECT is successful, the client continues sending the
actual request through the "tunnel". And when CONNECT failed, the
connection remains the initial state `unconnected`.

There are following circumstances that a CONNECT fails under but not
limited to following situations:

- The destination is not whitelisted.
- The dest domain can't be resolved(timeout/SERVFAIL/NX/etc.).
- The dest IP can't be connected(timeout/unreachable/etc.).

There could be 2 following strategies to deal with CONNECT failures on
the client side:

1. Close the connection before return to the caller.
2. Mark this connection "unconnected" and put it into the pool. Then
retry the CONNECT next time it's picked out of the pool.

The 2nd one needs to add extra state to Channel in the manager which
brings bigger change to the code.
This MR employs the 1st strategy to resolve it. The issue is described
in #2071 .

# Readings

The CONNECT is documented in `Section 5.3` in RFC2871:
https://www.ietf.org/rfc/rfc2817.txt

The proxy won't actively terminate the connection if the CONNECT failed
if keep-alive is enabled. Unless the tunnel is established and there is
any communication failures in the middle. Therefore the client needs to
deal with this error by its own.

Signed-off-by: Jason Joo <[email protected]>
This was referenced Mar 10, 2025
Closed
Merged
hyperxpro pushed a commit that referenced this issue Mar 14, 2025
@jasonjoo2010
fix: send CONNECT first when recovering a HTTPS request (#2077)
a9a3a7e 
# Issue description

AHC has retry mechanism enabled with up to 5 attempts by default. But
the initial CONNECT is omitted when recovering the HTTPS requests with
IO exceptions. This MR fixes this issue and guarantees the proper
workflow in retries.

It's related to #2071 and fixes a different failing case.

# How the issue is fixed

* For any new connections, make sure there is an initial CONNECT for
WebSocket/HTTPS request.
* For the condition check that a CONNECT has been sent, make sure the
connection the current future attaches is reusable/active.

# Unit test

IOException has various reasons but in the unit test, we emulate it by
closing the connection after receiving the CONNECT request. The internal
recovery process will retry another 4 times, and through an IOException
eventually.

Signed-off-by: Jason Joo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jasonjoo2010 @hyperxpro