[gulldan]

Fixes #1114.

What

The BoringSSL/OpenSSL upstream connector did a full TLS handshake for every new upstream connection: the SslConnector was built without a client session cache and connect() never offered a previous session, so the session IDs and tickets the upstream issued were dropped. Whenever the keepalive pool could not satisfy a request (concurrent streaming requests over HTTP/1.1, mutual TLS, post-quantum key shares) the upstream paid a full handshake, where nginx as a client (proxy_ssl_session_reuse on, its default) resumes.

PeerOptions::tls_session_reuse (default off, so nothing changes for existing users) makes the connector keep the latest session the server issued and offer it on the next connection to a peer with the same reuse_hash and ALPN:

• the connector context runs with SSL_SESS_CACHE_CLIENT | SSL_SESS_CACHE_NO_INTERNAL_STORE and a new-session callback, the only way to see TLS 1.3 tickets, which arrive after the handshake (usually with the first read); the library's own session cache is for servers and stays off;
• the sessions live in the context's ex data as a 1024-entry LRU keyed by hash(peer.reuse_hash(), alpn, peer CA bundle), so peers that may share a pooled connection, offer the same ALPN and trust the same CAs share sessions and nothing else does (the CA bundle is not in reuse_hash, and a resumed handshake does not verify the server's certificate again); the key of the connection being made travels in its Ssl's ex data to the callback;
• the sessions are stored serialized (i2d/d2i), not as SslSession objects: OpenSSL marks a session object not resumable when a connection holding it is freed without a TLS shutdown (ssl_clear_bad_session() in SSL_free()), which is what happens to a pooled upstream connection when it is dropped. A copy per connection is immune to that; BoringSSL does not do this, and the serialized form costs the same on both;
• connect() calls SSL_set_session before the handshake when there is a session for the key. An expired or refused session falls back to a full handshake inside the library, and the new ticket replaces it.

The docs of upstream_tls_verify_callback already say it does not run for resumed sessions. Reporting resumption to callers needs no new field: upstream_tls_handshake_complete_hook receives the SslRef and can call session_reused().

lru becomes a dependency of the openssl_derived feature (it already is one of s2n), and pingora-boringssl/pingora-openssl re-export ex_data for the Index type.

Tests

In pingora-core/src/connectors/tls/boringssl_openssl/mod.rs, each against a local SslAcceptor kept across handshakes:

• tls_sessions_are_not_reused_by_default: two handshakes with one connector, neither resumed; this is also what main does whatever the peer says;
• tls13_sessions_are_resumed_when_the_peer_enables_it: the ticket of the first handshake (delivered by the read after it) resumes the second, whose ticket resumes the third;
• tls12_sessions_are_resumed_when_the_peer_enables_it;
• tls_sessions_are_kept_per_peer_alpn_and_ca: another SNI, another ALPN or another CA bundle is another entry, and the first entry survives them.

They pass with --features boringssl and with --features openssl; the OpenSSL run is what found the SSL_free() behaviour above.

Measured

A 32-core Linux x86_64 host running nginx 1.31.8 (OpenSSL 3.5.9) as a mutual-TLS upstream (TLS 1.3, ECDSA P-256 certificates, ssl_verify_client on, $ssl_session_reused logged, and keepalive_timeout 0 so that the pool never has a connection to reuse and every request is a new upstream connection), a pingora-proxy on this branch (boringssl feature, 8 threads) in front of it, and oha -c 64 -z 20s over HTTP/1.1 as the client. nginx CPU is its cgroup's usage_usec, the proxy's is from /proc/<pid>/stat; both runs of a pair saturate the host, so the request rate follows the handshake cost.

key shares	tls_session_reuse	upstream connections in 20 s	resumed	nginx CPU / connection	proxy CPU / connection	requests/s	p50 / p99
X25519	off	388 977	0	0.68 ms	0.37 ms	19 445	3 / 6 ms
X25519	on	542 974	542 910	0.41 ms	0.24 ms	27 143	2 / 4 ms
X25519MLKEM768	off	339 570	0	0.76 ms	0.43 ms	16 975	3 / 7 ms
X25519MLKEM768	on	455 673	455 609	0.49 ms	0.30 ms	22 780	2 / 5 ms

Per upstream connection nginx spends 40 % (X25519) and 36 % (X25519MLKEM768) less CPU on the handshake, the proxy 33 % and 29 % less, and the same client load gets 40 % and 34 % more requests through. The 64 connections per run that are not resumed are the first upstream connection of each of the 64 concurrent client connections, made before the first ticket had arrived.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-09T04:32:47.366784Z	b6a6c7b	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b6a6c7b47d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[gulldan]

254de8e makes the session key include the peer's CA bundle (Codex's finding above); the PR description is updated accordingly.

The pingora (1.97.1) job on b6a6c7b failed in test_cache::test_cache_max_file_size_range (assert!(epoch1 != epoch2), pingora-proxy/tests/test_upstream.rs:5591), a cache test against a plain HTTP upstream; it passed on 1.85.0 and nightly in the same run and does not touch the TLS connector.