Skip to content

Support net-ssh 8 - #1974

Open
excid3 wants to merge 4 commits into
basecamp:mainfrom
excid3:net-ssh-8
Open

excid3 wants to merge 4 commits into
basecamp:mainfrom
excid3:net-ssh-8

Conversation

@excid3

@excid3 excid3 commented Oct 20, 2026 •

Copy link
Copy Markdown

net-ssh 8 is out, and net-scp 4.1.1 and net-sftp 4.0.1 (released 2026-10-22) raise their net-ssh bound to < 9, so Kamal can run on net-ssh 8 now. This widens Kamal's dependency from ~> 7.3 to >= 7.3, < 9.0, locks 8.0.2, and keeps 7.x working.

What changed

  • kamal.gemspec: allow net-ssh 8. Floor net-scp at 4.1 and net-sftp at 4.0, because without a floor Bundler would satisfy net-ssh 8 by downgrading those to net-scp 1.2.1 and net-sftp 2.1.2 from 2013, the last releases with no upper bound on net-ssh. CI deletes the lockfile and resolves fresh, so it would have done exactly that. ed25519 stays while net-ssh 7 is supported; bcrypt_pbkdf is still used by net-ssh 8 for passphrase-protected OpenSSH keys.
  • lib/kamal/sshkit_with_ext.rb: net-ssh 8 fixed remote forwarding printing diagnostics to stdout (Stop remote forwarding from logging to stdout net-ssh/net-ssh#953), so the NetSshForwardingNoPuts prepend is now applied only when Net::SSH::Version::MAJOR < 8.
  • Gemfile.lock: net-ssh 8.0.2, net-scp 4.1.1, net-sftp 4.0.1.
  • CI: every job now resolves to net-ssh 8, so gemfiles/net_ssh_7.gemfile plus one matrix job (Ruby 3.4) keep the net-ssh 7 path covered until 7 support is dropped.

Nothing else in Kamal needed to change. The SSH option hash is .compacted, so net-ssh 8's ArgumentError on nil option values doesn't apply, and the Proxy::Jump, Proxy::Command and Forward#remote/cancel_remote APIs Kamal calls are unchanged. net-ssh 8.0.1 also made control_master: :auto tolerate the forward_agent, keys, known_hosts and keepalive options sshkit always passes.

Verification

  • Unit suite: 841 runs, 0 failures on net-ssh 8.0.2 (default Gemfile) and on net-ssh 7.3.6 (gemfiles/net_ssh_7.gemfile).
  • Docker integration suite on net-ssh 8.0.2: the deployer image builds the gem from the gemspec with gem install, so it resolved 8.0.2. Every SSH step against the test VMs succeeded. The remaining local failures (exec /bin/sh: invalid argument inside nested containers) reproduce identically on main and are a local Docker issue, not SSH.
  • Earlier, against a local sshd on 8: SSHKit command execution, scp upload, and a remote port forward in the same shape as Kamal::Cli::Build::PortForwarding all work.

If we go net-ssh 8 only later

Supporting both stays cheap, but when 7 is dropped:

  • spec.add_dependency "net-ssh", ">= 7.3", "< 9.0" → "~> 8.0", and the net-scp/net-sftp floors become >= 4.1.1 / >= 4.0.1.
  • Drop ed25519: net-ssh 8 does Ed25519 through OpenSSL and no longer loads the gem.
  • Remove NetSshForwardingNoPuts and its prepend, plus gemfiles/net_ssh_7.gemfile and its CI job.

Worth a changelog note either way: net-ssh 8 requires the openssl gem 3.2+, which Ruby 3.2 doesn't ship, so Bundler installs it as a native gem on 3.2. net-ssh 8 also stops offering SHA-1 RSA host keys and diffie-hellman-group14-sha1 by default, so very old servers need those re-enabled in ssh_config.

Widen the net-ssh dependency from ~> 7.3 to >= 7.3, < 9.0 so Kamal can
run on net-ssh 8.0.0.rc5 while still working on 7.x.

net-ssh 8 merged the fix for remote forwarding printing diagnostics to
stdout (net-ssh/net-ssh#953), so the NetSshForwardingNoPuts prepend is
now only applied on net-ssh 7. Nothing else needed to change: Kamal's
SSH option hashes are already free of nils, and the Proxy::Jump and
Forward APIs it calls are unchanged in 8.

Claude-Session: https://claude.ai/code/session_01EkpXJSoszCrBejRLUVUpjg
…eases

With net-ssh 8.0.0 released, a fresh resolve of the widened constraint
picked net-ssh 8.0.0 together with net-scp 1.2.1 and net-sftp 2.1.2,
the last releases of each without an upper bound on net-ssh. The
current net-scp 4.1.0 and net-sftp 4.0.0 both cap net-ssh below 8.

Floor both so Bundler stays on net-ssh 7.x until net-scp and net-sftp
publish releases that admit 8, at which point 8 resolves on its own.
@excid3

excid3 commented Oct 21, 2026

Copy link
Copy Markdown
Author

net-ssh 8.0.0 final is out (identical to rc5 apart from the version file), so I went to finish this as 8-only and hit two things worth flagging.

Going ~> 8.0 isn't possible yet. sshkit depends on net-scp and net-sftp, and their current releases (net-scp 4.1.0, net-sftp 4.0.0) both cap net-ssh at < 8.0.0. Neither has relaxed that on master, net-sftp hasn't had a commit since 2022, and there's no issue or PR tracking it in either repo. Until they release, nobody can resolve net-ssh 8.0.0 with Kamal at all.

The widened constraint alone was a trap. With 8.0.0 published, a fresh resolve of the first commit's gemspec picked net-ssh 8.0.0 together with net-scp 1.2.1 and net-sftp 2.1.2 from 2013, the last releases of each with no upper bound on net-ssh. CI deletes the lockfile, so it would have done exactly that. The second commit adds net-scp >= 4.1, < 5.0 and net-sftp >= 4.0, < 5.0 to close that route. Kamal already requires net/scp directly; the net-sftp floor is there purely so Bundler can't downgrade it either.

Resolution with the floors in place:

Gemfile Result
Plain gemspec, fresh resolve net-ssh 7.3.6, net-scp 4.1.0, net-sftp 4.0.0
net-ssh 8.0.0.rc5 pinned resolves (rc5 sorts below the < 8.0.0 caps)
net-ssh 8.0.0 pinned clean conflict naming net-sftp, no silent downgrade

So the PR is mergeable as-is: it stays on 7.x today and net-ssh 8 lights up on its own once net-scp and net-sftp publish compatible releases. The 8-only cleanup (~> 8.0, drop ed25519, drop the NetSshForwardingNoPuts prepend) can follow then.

@excid3

excid3 commented Oct 21, 2026

Copy link
Copy Markdown
Author

Opened the upstream bumps: net-ssh/net-scp#101 and net-ssh/net-sftp#181. Both are one-line < 9.0.0 changes; the live transfer checks and each gem's own test suite give the same results on net-ssh 8.0.0 as on 7.3.3, so no code changes needed on their side. Once those release, net-ssh 8 resolves under this PR with no further change here.

net-scp 4.1.1 and net-sftp 4.0.1 lift their net-ssh upper bound to < 9, so Bundler can resolve net-ssh 8 without the floors having to do any work.
CI resolves dependencies fresh, so every job now runs on net-ssh 8. Add a gemfile pinning net-ssh 7 and one matrix job for it, so the version-guarded forwarding patch in sshkit_with_ext.rb stays covered until net-ssh 7 support is dropped.
@excid3 excid3 changed the title Allow net-ssh 8 Support net-ssh 8 Oct 22, 2026
@excid3

excid3 commented Oct 22, 2026

Copy link
Copy Markdown
Author

net-scp 4.1.1 and net-sftp 4.0.1 shipped today with < 9.0.0 bounds on net-ssh, so this is unblocked. Pushed two commits:

  • Lock net-ssh 8.0.2 with net-scp 4.1.1 and net-sftp 4.0.1. The gemspec floors are unchanged and still do their job of keeping Bundler off the 2013 releases.
  • Add gemfiles/net_ssh_7.gemfile and one CI matrix job for it, since every other job now resolves to net-ssh 8 and the Net::SSH::Version::MAJOR < 8 guard in sshkit_with_ext.rb would otherwise go untested.

Unit suite passes on both 8.0.2 and 7.3.6. The Docker integration suite ran on 8.0.2 this time (the deployer image resolves the gemspec fresh), and all SSH steps against the VMs succeeded. PR description updated and marked ready for review.

@excid3
excid3 marked this pull request as ready for review October 22, 2026 18:15
Copilot AI balanced review requested due to automatic review settings October 22, 2026 18:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The newly locked OpenSSL gem needs development headers missing from the production Alpine Dockerfile, blocking image builds.

No open issues 🎉

What changed in this PR

This PR adds net-ssh 8 support while preserving compatibility with net-ssh 7.

Changes:

  • Widens SSH dependency constraints and locks net-ssh 8.0.2 with compatible transfer libraries.
  • Limits the forwarding-output workaround to net-ssh 7.
  • Adds a Ruby 3.4 CI job using a net-ssh 7-specific Gemfile.

[!TIP]
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

File Description
lib/​kamal/​sshkit_with_ext.rb Applies the forwarding workaround only before net-ssh 8.
kamal.gemspec Allows net-ssh 8 and sets transfer-library version bounds.
gemfiles/​net_ssh_7.gemfile Pins net-ssh 7 for compatibility testing.
Gemfile.lock Locks updated SSH dependencies and OpenSSL.
.github/​workflows/​ci.yml Adds the net-ssh 7 compatibility job.

Review effort: Balanced

@mfazekas

Copy link
Copy Markdown

Hi, net-ssh maintainer here. Thanks for doing this. I benchmarked Kamal on net-ssh 8 to see what it changes for deploy times, in case it's useful here.

Setup: 3 Docker hosts running OpenSSH, with tc netem adding 50 ms RTT (and a 0 ms run), Kamal main (e3ec7b6) with SSHKit 1.25.1, gems from RubyGems. "Remote registry" is kamal deploy --skip-push from a registry the hosts pull from; "full deploy" also builds and pushes to a local registry over the port forward. Images and layers were already cached, so pull time is minimal and these numbers mostly show SSH overhead. Medians of 5 runs (7 at 0 ms):

50 ms, remote registry 50 ms, full deploy 0 ms, remote registry 0 ms, full deploy
net-ssh 7.3.7 (current pin) 9.65 s 10.39 s 2.86 s 3.39 s
net-ssh 8.0.3 6.64 s (-31%) 7.21 s (-31%) 1.63 s (-43%) 2.22 s (-34%)
8.0.3 + preopen_channels: true 4.87 s (-50%) 5.37 s (-48%) 1.62 s 2.34 s (noise)

Most of the 7.3 → 8.0 gain is TCP_NODELAY: 7.3.x doesn't set it, so small command packets wait on Nagle plus delayed ACK, about one stall per command (roughly 3.3 s per deploy at 50 ms). So it may be worth requiring net-ssh 8 at some point rather than only allowing it, but that's your call.

Kamal's unit suite and the Docker integration suite passed on 8.0.3 with and without preopen_channels.

preopen_channels is new in net-ssh 8.0.3 (net-ssh/net-ssh#1071): it keeps one spare session channel open, so each sequential command costs one round trip instead of two (78 → 46 waits per host here). It works through ControlMaster, and if the server refuses the spare because of MaxSessions it falls back and stops pre-opening. Kamal's config validator only passes documented ssh: keys through, so it can't be enabled today. Would you be open to a follow-up PR (after this one) adding an ssh/preopen_channels setting, applied only on net-ssh >= 8.0.3? Happy to open it.

@eimantas

Copy link
Copy Markdown

It'd be great to have this merged sooner than later. My bin/ci is failing on GitHub due to vulns in net-ssh 7.3 and I can't pin to 8.0.0 in my Gemfile because of Kamal's dependency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants