Skip to content

feat(db): add SeekLT and Last to iterators, rename Seek to SeekGE - #4268

Open
EgeCaner wants to merge 2 commits into
mainfrom
feat/db/iterator-seek-lt
Open

EgeCaner wants to merge 2 commits into
mainfrom
feat/db/iterator-seek-lt

Conversation

@EgeCaner

@EgeCaner EgeCaner commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Seek was always pebble's SeekGE, so callers that wanted the last key below a bound had to seek forward and step back with Prev. This exposes pebble's SeekLT and Last directly and names Seek after what it does.

Changes

  • db.Iterator: Seek → SeekGE; new SeekLT and Last.
  • pebble / pebblev2: thin wrappers over pebble's SeekGE, SeekLT, Last.
  • memory: seeks use binary search over the sorted key snapshot (previously a linear scan).
  • remote: proto SEEK → SEEK_GE (same wire number), new SEEK_LT and LAST ops; client positioning methods share one helper; the server's iterator ops move into moveCursor, and CLOSE returns early.
  • All existing callers renamed to SeekGE, no behavior change.

Tests

  • Shared suite (memory, pebble, pebblev2): SeekGE/SeekLT table (exact, between, out of range, prefix bounds), Last on bounded and empty ranges, reverse iteration from Last, direction changes after SeekLT/Last, panics after close.
  • Remote: SEEK_LT and LAST over a real gRPC server.
  • gRPC handler: every cursor op against data, plus unknown-op and unknown-cursor errors.

@EgeCaner
EgeCaner requested a review from infrmtcs November 5, 2026 09:51
@EgeCaner
EgeCaner force-pushed the feat/db/iterator-seek-lt branch from bee344c to df05720 Compare November 5, 2026 09:52
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Title

(Describe updated until commit 75607c6)

feat(db): add SeekLT and Last to iterators, rename Seek to SeekGE


PR Type

Enhancement, Bug fix


Description

  • Iterator Interface Changes: Renamed Seek to SeekGE in the db.Iterator interface to clarify its behavior, and added SeekLT and Last operations.

  • Backend Implementations: Implemented SeekGE, SeekLT, and Last across all database backends (memory, pebble, pebblev2, and remote).

  • Memory Iterator Fix: Fixed the memory backend iterator to remain exhausted when moving past either end, matching Pebble's behavior.

  • Remote Protocol Updates: Updated the gRPC protocol (kv.proto) and server handlers to support the new SEEK_GE, SEEK_LT, and LAST cursor operations.

  • Usage Updates: Replaced all occurrences of Seek with SeekGE throughout the codebase, including in state, migrations, and pruners.


File Walkthrough

Relevant files
Miscellaneous
9 files
state.go
Rename Seek to SeekGE in deprecated state                               
+2/-2     
state_reader.go
Rename Seek to SeekGE in state reader                                       
+2/-2     
batch.go
Rename Seek to SeekGE in memory batch                                       
+1/-1     
blockeventsbloom.go
Rename Seek to SeekGE                                                                       
+1/-1     
history.go
Rename Seek to SeekGE                                                                       
+1/-1     
stager.go
Rename Seek to SeekGE                                                                       
+1/-1     
migrator.go
Rename Seek to SeekGE                                                                       
+1/-1     
storage_ingestor.go
Rename Seek to SeekGE                                                                       
+1/-1     
accessors.go
Rename Seek to SeekGE                                                                       
+1/-1     
Enhancement
7 files
iterator.go
Update db.Iterator interface with SeekGE, SeekLT, and Last
+12/-3   
iterator.go
Add Last, SeekLT and rename Seek in pebble iterator           
+13/-1   
iterator.go
Add Last, SeekLT and rename Seek in pebblev2 iterator       
+13/-1   
iterator.go
Implement new operations and refactor with move helper     
+19/-15 
kv.pb.go
Regenerate gRPC code with new Op enums                                     
+15/-7   
handlers.go
Update gRPC handler to support new cursor operations         
+32/-36 
kv.proto
Add SEEK_GE, LAST, and SEEK_LT to Op enum                               
+3/-1     
Bug fix
1 files
iterator.go
Implement SeekGE, SeekLT, Last and fix exhaustion in memory iterator
+45/-26 
Tests
4 files
db_test.go
Add tests for remote iterator bounds and new operations   
+13/-2   
testutil.go
Add comprehensive tests for iterator operations and exhaustion
+211/-10
handlers_test.go
Add tests for gRPC cursor moves and errors                             
+73/-1   
migrator_test.go
Rename Seek to SeekGE in tests                                                     
+2/-2     

@EgeCaner
EgeCaner requested a review from rodrodros November 5, 2026 09:53
@codecov

codecov Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.55357% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 81.57%. Comparing base (4c0aaf8) to head (75607c6).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
db/remote/iterator.go 87.50% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4268      +/-   ##
==========================================
+ Coverage   81.50%   81.57%   +0.07%     
==========================================
  Files         500      501       +1     
  Lines       36844    37034     +190     
==========================================
+ Hits        30029    30211     +182     
- Misses       6809     6817       +8     
  Partials        6        6              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Reviewed PR #4268 (feat(db): add SeekLT and Last to iterators, rename Seek to SeekGE). This is a clean, well-scoped refactor: Seek renamed to SeekGE everywhere, new SeekLT/Last added across all db.Iterator implementations (pebble, pebblev2, memory, remote), with the gRPC wire protocol updated consistently (SEEK→SEEK_GE same wire number, new SEEK_LT/LAST ops added) and the server-side cursor handling split into a moveCursor helper that correctly short-circuits on errors.

Checked carefully for:

  • All call sites renamed consistently (no leftover .Seek( calls on db.Iterator; only unrelated os.File.Seek calls remain).
  • memory iterator's new binary-search-based SeekGE/SeekLT/Last correctly operate over the prefix/upper-bound-filtered, sorted key slice, matching pebble's SeekGE/SeekLT/Last semantics (including positioning one past the end / at index-1 for Prev-after-seek behavior) — well covered by the new SeekLT/Last/direction-change table tests in db/testutil.go.
  • grpc/handlers.go moveCursor correctly returns (false, err) only for unknown op, and the caller doesn't clobber a real error when valid is false.
  • No new goroutine/iterator leaks in the grpc Handler.Tx path (Op_CLOSE now returns early via tx.closeCursor; error paths still joined with tx.cleanup()).
  • Test files are properly external test packages, no time.Sleep-based synchronization used in new tests.

I found no correctness bugs, race conditions, panics, or resource leaks. The PR is clean — no inline comments posted.

@EgeCaner
EgeCaner added this pull request to stack #4272 November 6, 2026 11:43
@EgeCaner
EgeCaner force-pushed the feat/db/iterator-seek-lt branch from df05720 to 61075d7 Compare November 6, 2026 11:44
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Reviewed the PR (db iterator rename Seek→SeekGE, new SeekLT/Last across memory/pebble/pebblev2/remote backends, plus the gRPC proto/handler changes). The change is clean: memory's binary-search reimplementation of SeekGE/SeekLT matches pebble's semantics, all call sites were mechanically renamed (verified via repo-wide grep — no leftover .Seek( callers), the grpc handler refactor into moveCursor is behavior-preserving, and the new proto enum values reuse unused wire numbers safely. Test coverage is thorough (table-driven seek/bounds/prefix cases, close-after-panic, gRPC cursor moves and error paths). No correctness, concurrency, or resource-leak issues found. PR is clean — no inline comments posted.

Comment thread db/memory/iterator.go
return i.Valid()
}

func (i *iterator) SeekLT(key []byte) bool {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Failed SeekLT + Prev behaves differently from pebble here. Pebble returns false, while memory turns it into First().

Probably, there are other differences in memory and pebble behaviour on such "out-of-range" cases, like SeekGE + Next etc.

There are no places in production where we use similar combinations of moves, but at least worth checking

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thank you! I will go over it and see the places behavior deviates between DBs and try to make all behave same

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, we can add something similar to pebble's positioned flag to track this or strictly limit curInd

And different behaviour existed before this PR on other scenarios, SeekLT + Prev is the new one. So maybe we can fix it in other PR or leave it as it is. What do you think?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. I added a positioned flag as you suggested, and curInd can now only go one step past either end, so moving further keeps the iterator exhausted. That covers the SeekLT + Prev case and an older one where SeekGE past the end + Next + Prev returned true with an empty key. A new iterator still moves to the first key on Next/Prev, same as pebble.

To check there weren't more, I ran every move sequence up to 3 steps long against memory, pebble, pebblev2 and remote (3 keys, 1 key, empty DB). They all match now. I also added a table of these cases to the shared suite in db/testutil.go, so all backends are tested against them.

While checking the rest of the API I found three other ways memory differs from pebble, none related to iterator moves:

  1. Batch DeleteRange deletes the keys that exist when it's called, while pebble applies the range when the batch is written. Keys written to the DB in between survive in memory.
  2. NewIterator(prefix, false) still stops at the prefix in memory, while pebble uses the prefix only as the starting point. No production code passes a prefix with false today.
  3. A prefix made only of 0xFF bytes returns nothing in memory, because its upper bound is nil and memory treats that as an empty bound instead of no bound.

These are older than this PR and unrelated to SeekLT/Last, so I'll fix them in a separate PR.

Expose pebble's SeekLT and Last on every backend and name Seek after
what it does. The remote protocol renames SEEK to SEEK_GE and adds
SEEK_LT and LAST.
@EgeCaner
EgeCaner force-pushed the feat/db/iterator-seek-lt branch from 61075d7 to 75607c6 Compare November 13, 2026 10:41
@github-actions

Copy link
Copy Markdown
Contributor

Reviewed PR #4268 (db iterator SeekGE/SeekLT/Last refactor) across pebble, pebblev2, memory, and remote/gRPC implementations plus all call-site renames. Traced the memory iterator's new exhaustion-clamping logic against pebble semantics and verified all new edge-case tests. No correctness, concurrency, or resource-leak issues found; the change is a clean, mechanical, well-tested refactor. No inline comments posted — PR is clean.

@EgeCaner
EgeCaner deployed to Development November 13, 2026 11:02 — with GitHub Actions Active

This branch was successfully deployed

1 active deployment
Development — 75607c64 Deployed Nov 13, 2026 by EgeCaner via dev_deploy_test / deploy #6520
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants