Summary

This PR fixes SOCKS5 CONNECT reply timing for both forward and reverse tunnel paths. Reported as an issue in #485.

Before this change, SOCKS5 success could be returned before the upstream target connect result was known. In practice, proxy tooling (proxychains + nc / nmap -sT) could observe false-positive connect success for unreachable ports.

After this change, SOCKS5 success/failure is sent only when the real upstream connect outcome is known.

Problem

In affected paths, the SOCKS5 listener acknowledged CONNECT too early.
This breaks expected SOCKS5 semantics and can produce incorrect scan/probe results.

Root Cause

• The TCP SOCKS5 path replied REP=Succeeded immediately after SOCKS handshake acceptance, instead of waiting for upstream tunnel connect outcome.
• Reverse SOCKS5 flow did not have an explicit connect-result handshake across WS/HTTP2 transport.
• On server side, replying to SOCKS clients required downcasting tunnel writers; we needed a safe and object-safe mechanism for that in the mixed writer pipeline.

What Changed

1. Deferred SOCKS5 reply handling in the SOCKS server write half.
2. Forward tunnel path now sends SOCKS5 success/failure based on actual server connect outcome.
3. Reverse SOCKS5 adds explicit one-byte connect-result handshake:
   • client sends connected/failed outcome
   • server waits for that before replying to SOCKS client
4. WS/HTTP2 read paths now support reading one handshake byte without losing first payload bytes (prefetch buffer).
5. Added configurable connect timeout via CLI/env on both client and server:
   • --timeout-connect (alias --timeout-connect-sec)
   • WSTUNNEL_TIMEOUT_CONNECT
   • default set to 3s
6. Added unit + integration tests for SOCKS reply timing and connect success/failure behavior.
7. Extended both fixes to WebTransport, so all three transports behave identically:
   • reverse path: WebTransportTunnelRead::read_handshake_byte() plus the same deferred reply in the webtransport handler
   • forward path: the wts arm now sends the SOCKS failure reply when connecting to the server fails, as the ws and http2 arms already did

Behavioral Outcomes

• Forward SOCKS5:
  • success reply only after upstream connect succeeds
  • failure reply when upstream connect fails
• Reverse SOCKS5:
  • server SOCKS reply synchronized with local connector result over WS, HTTP2 and WebTransport
• Timeout behavior:
  • connect/handshake timeout is now operator-tunable on both client and server
  • for reverse SOCKS5, client/server timeout values can differ; the shorter side effectively limits wait time
• proxychains-based TCP connect probing now reflects target state more accurately.

Rebase onto WebTransport

This branch is rebased onto 14168c5. WebTransport landed upstream in 05a779e after this PR was opened, adding a third transport to the same paths this PR touches, which surfaced two gaps.

On the reverse path, the connect handshake is a single byte the client writes into the tunnel and the server reads before replying to the SOCKS client. Only the websocket and http2 server handlers implemented that read, while the client wrote the byte on any transport. Over wts:// it was therefore never consumed as a handshake and arrived as the first byte of the payload, corrupting the stream.

On the forward path, a failed connection to the server sends the SOCKS client a failure reply, but the wts arm returned early instead, so the SOCKS client waited until the socket closed rather than being told the connect had failed.

Both are now closed rather than worked around. WebTransportTunnelRead::read_handshake_byte() mirrors the http2 read half, and the webtransport handler gates its deferred reply the same way the other two do. A QUIC stream is a byte stream, so taking one byte leaves the remainder queued and no prefetch buffer is needed — the webtransport read half is the simplest of the three. Reverse and forward SOCKS5 now behave identically across websocket, http2 and WebTransport, with integration coverage for connect success and failure over the new transport.

Validation

The checks CI now runs on every pull request (per CONTRIBUTING.md) were run locally, for both crypto providers:

• cargo fmt --all -- --check
• taplo fmt --check
• cargo clippy --all --all-features --locked -- -D warnings
• cargo nextest run --locked
• cargo nextest run --locked --no-default-features --features ring

All pass. test_proxy_connection needs a Docker daemon and was skipped locally; it is covered by the CI run below.

The full workflow was also run against this branch on a fork and is green: 15/15 jobs, with Release skipped as tag-gated. That covers Check - format, lint & tests and the entire build matrix — Linux x86_64/x86/aarch64/armv7hf/armv6, macOS x86_64/aarch64, Windows x86_64/x86/aarch64, FreeBSD x86_64/x86, and Android aarch64/armv7.

End-to-end, against the built binary

Beyond the test suite, the release binary was driven directly over every transport that carries
SOCKS5 — ws, wss, http, https, wts — in both forward (-L socks5://) and reverse
(-R socks5://) mode. Each combination was checked twice: CONNECT to a listening target, which
must reply 0x00 and pass bytes both ways, and CONNECT to a port nothing is bound to, which must
reply 0x01. Twenty cases in total, run identically against 14168c5 and against this branch:

So the reported behavior reproduces on every transport in both directions, is fixed on every
transport in both directions, and the success path is unaffected.

The harness that produced this table is here, if you want to reproduce it yourself:
https://gist.github.com/0typos/3e823b2541470cb969c8100c9b187c3a — one stdlib-only Python file, run
as python3 e2e_socks5.py ./target/release/wstunnel. It is deliberately not part of this PR: it
drives the built binary and spawns real listeners, so it does not belong in the crate's test suite.

Notes

• Added configurable connect timeout for both client and server via --timeout-connect (alias --timeout-connect-sec) and WSTUNNEL_TIMEOUT_CONNECT.
• This timeout now governs upstream connect attempts and reverse SOCKS5 connect-handshake wait, which allows tuning scan speed vs accuracy tradeoff for tools like nmap/proxychains.
• The branch is clean under the new clippy --all --all-features -- -D warnings gate; no allow attributes were added to get there.

Reviewer-Oriented Change Summary (File-by-File)

1. wstunnel/src/protocols/socks5/tcp_server.rs

   • Changed Socks5WriteHalf::Tcp to hold pending_reply state.
   • Stopped immediate TCP CONNECT success reply in accept loop.
   • Added send_reply_if_needed() to emit a one-shot SOCKS reply only when connect outcome is known.
   • Updated AsyncWrite impl for new enum shape.
   • Added unit tests for deferred-reply behavior (sent-once, skipped-when-not-pending).
   • Reason: this is the core fix for premature SOCKS success in forward/reverse paths.

2. wstunnel/src/tunnel/client/client.rs

   • Added forward-path logic to send SOCKS success/failure based on actual connect() result to server transport.
   • Added reverse-path handshake writer (HANDSHAKE_CONNECT_OK/FAIL) to report local connector outcome back to server.
   • Forward path: the Wts arm now sends the SOCKS failure reply when connecting to the server fails, matching the Ws and Http2 arms instead of returning early.
   • Reason: makes client side the authoritative source of real upstream connect status.

3. wstunnel/src/tunnel/server/server.rs

   • handle_tunnel_request() now carries whether request is reverse-socks5 and uses a writer type that can be downcast safely.
   • Replaced hardcoded connect timeout usage with self.config.timeout_connect in connector paths.
   • Reason: server handlers need this context to wait for reverse handshake and then reply to SOCKS client, and connector timeout behavior should be configurable.

4. wstunnel/src/tunnel/server/handler_websocket.rs

   • Added reverse-socks5 handshake read/wait on WS path.
   • Sends SOCKS success/failure reply only after handshake result.
   • Closes WS on failure paths to avoid dangling connection state.
   • Reason: synchronize server SOCKS reply with real connect outcome over websocket transport.

5. wstunnel/src/tunnel/server/handler_http2.rs

   • Same behavior as websocket handler, for HTTP2 transport.
   • Reason: keep reverse-socks5 correctness consistent across both transports.

6. wstunnel/src/tunnel/transport/websocket.rs

   • Added read_handshake_byte() and prefetched buffer in WebsocketTunnelRead.
   • Preserves payload bytes after consuming handshake byte.
   • Reason: handshake framing without data loss/corruption.

7. wstunnel/src/tunnel/transport/http2.rs

   • Added read_handshake_byte() and prefetched buffer in Http2TunnelRead.
   • Reason: same as websocket, for HTTP2 stream body frames.

8. wstunnel/src/tunnel/reverse_socks5.rs (new)

   • Defines reverse handshake constants and timeout-aware decoder utility.
   • Includes focused unit tests (connected, failed, timeout, IO error).
   • Reason: isolate reverse handshake protocol logic and keep handlers simple.

9. wstunnel/src/tunnel/server/socks5_reply.rs (new)

   • Introduces AnyAsyncWrite trait and helper to downcast writer to Socks5WriteHalf safely and send reply if needed.
   • Reason: object-safe writer abstraction needed in server pipeline to trigger SOCKS reply correctly.

10. wstunnel/src/tunnel/server/mod.rs

    • Wires in new socks5_reply module exports.
    • Reason: shared access for server handlers.

11. wstunnel/src/tunnel/mod.rs

    • Exposes reverse_socks5 module internally.
    • Reason: shared handshake logic between client and server handlers.

12. wstunnel/src/test_integrations.rs

    • Added real SOCKS5 integration tests for CONNECT success and failure over wstunnel.
    • Includes minimal SOCKS5 handshake helper used by tests.
    • Added the same two tests over WebTransport, covering the reverse handshake and the forward failure reply on that transport.
    • Uses the free_addr() helper and the client_ws(server_port, dns_resolver) form introduced in ci: run tests, formatting and lint checks on pull requests #529, rather than fixed ports.
    • Reason: verifies end-to-end behavior that specifically regressed with proxychains-like usage.

13. wstunnel/src/config.rs

    • Added --timeout-connect (alias --timeout-connect-sec) and WSTUNNEL_TIMEOUT_CONNECT to both client and server options.
    • Set default connect timeout to 3s.
    • Added help text clarifying reverse SOCKS5 client/server timeout interaction.
    • Reason: expose operator control over connect/handshake latency and scanning behavior.

14. wstunnel/src/lib.rs

    • Replaced hardcoded timeout_connect defaults with CLI-configured values for both client and server runtime config.
    • Reason: ensure new timeout setting is actually applied to runtime behavior.

15. wstunnel/src/tunnel/server/handler_webtransport.rs

    • Reads the reverse connect handshake and defers the SOCKS reply on it, mirroring the websocket and http2 handlers.
    • Closes the session with a matching status on connect failure, handshake timeout, and reply failure.
    • Reason: reverse SOCKS5 correctness on the third transport.

16. wstunnel/src/tunnel/transport/webtransport.rs

    • Added read_handshake_byte() to WebTransportTunnelRead, via RecvStream::read_exact.
    • No prefetch buffer, unlike the websocket and http2 read halves: a QUIC stream is a byte stream, so taking one byte leaves the remainder queued.
    • Reason: handshake framing on the WebTransport read half.

Disclosure

The original code changes, repro script, issue draft, and first version of this description were prepared with assistance from ChatGPT 5.3 Codex.

The rebase onto 14168c5 was prepared with assistance from Claude Opus 5, via Claude Code: conflict resolution against #529, the commit carrying the connect handshake and the forward failure reply over WebTransport, and the updates to this description.

I manually reviewed the code and manually tested the behavior of all changes in this PR, including the rebase, before submitting.