Skip to content

txn: fix scan missing endKey in V2 API#794

Open
lilei-jd wants to merge 1 commit into
tikv:masterfrom
lilei-jd:fix/txn-scan-v2-endkey
Open

txn: fix scan missing endKey in V2 API#794
lilei-jd wants to merge 1 commit into
tikv:masterfrom
lilei-jd:fix/txn-scan-v2-endkey

Conversation

@lilei-jd

@lilei-jd lilei-jd commented Jul 9, 2026

Copy link
Copy Markdown
  • Add scan(backOffer, startKey, endKey, version, keyOnly) overload
  • Use codec.encodeRange() to set both startKey and endKey
  • Decode response KvPairs via codec.decodeKvPairs()
  • Pass rangeEndKey through ScanIterator

What problem does this PR solve?

Issue Number: close #660

Problem Description: In V2 API mode, RegionStoreClient.scan() only sets startKey in the Scan request without encoding or passing endKey, causing TiKV to receive an incomplete scan range and return InvalidReqRange errors. Additionally, the response KvPairs are not decoded (V2 key prefix not stripped), and the ScanIterator/ConcreteScanIterator have incorrect logic when endKey is empty or startKey reaches +∞, causing premature iteration termination.

What is changed and how does it work?

  • Add scan(backOffer, startKey, endKey, version, keyOnly) overload in RegionStoreClient: uses codec.encodeRange() to encode both startKey and endKey together (V2 adds key prefix x\0\0\0; empty endKey is encoded as x\0\0\1 representing the keyspace upper bound). The old 4-param scan() delegates to the new 5-param version with endKey=EMPTY.
  • Decode response KvPairs via codec.decodeKvPairs(): strips the V2 key prefix from returned pairs so callers receive keys in user space.
  • Pass rangeEndKey through ScanIterator/ConcreteScanIterator: fixes hasNext() logic — when there is no endKey (scan to +∞), !hasEndKey is added as a condition instead of the previous (hasEndKey && compareTo < 0) which short-circuited to false. Fixes cacheLoadFails() — splits the hasEndKey and startKey.isEmpty() conditions into separate if-else branches so that reaching +∞ only marks processingLastBatch without clearing startKey to null.

Code changes

  • Has exported function/method change: RegionStoreClient.scan() adds new overload with endKey parameter
  • Has methods of interface change: ScanIterator constructor now accepts and propagates rangeEndKey

Check List for Tests

This PR has been tested by at least one of the following methods:

  • Manual test: verified against TiKV cluste with V2 API
    • scan("a", "z", startTs) — range scan with both start and end
    • scan("a", startTs) — scan without endKey (scan to +∞)
    • scan(EMPTY, EMPTY, startTs) — full table scan
    • RegionStoreClient.scan(backOffer, startKey, EMPTY, startTs, false) — direct client scan without endKey
    • All combinations return correct results without InvalidReqRange errors

Side effects

  • NO side effects: the old 4-param scan() method signature is preserved and delegates to the new 5-param version with endKey=EMPTY

Related changes

  • NO related changes

Summary by CodeRabbit

  • New Features
    • Scan operations now support explicit key ranges, allowing more precise reads over a defined start and end key.
  • Bug Fixes
    • Improved scan continuation logic so scans stop and resume more reliably at range boundaries.
    • Fixed result handling to return decoded key-value data consistently after successful scans.

- Add scan(backOffer, startKey, endKey, version, keyOnly) overload
- Use codec.encodeRange() to set both startKey and endKey
- Decode response KvPairs via codec.decodeKvPairs()
- Pass rangeEndKey through ScanIterator

Signed-off-by: lilei <lilei592@jd.com>
@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6322bbef-4f04-4d03-8b00-aa775879c350

📥 Commits

Reviewing files that changed from the base of the PR and between c12047a and f1aeb35.

📒 Files selected for processing (3)
  • src/main/java/org/tikv/common/operation/iterator/ConcreteScanIterator.java
  • src/main/java/org/tikv/common/operation/iterator/ScanIterator.java
  • src/main/java/org/tikv/common/region/RegionStoreClient.java

📝 Walkthrough

Walkthrough

RegionStoreClient.scan now supports scanning with both start and end keys, encoding the range and decoding returned kv pairs. ScanIterator stores a rangeEndKey field and adjusts end-of-scan branching in cacheLoadFails. ConcreteScanIterator passes rangeEndKey/keyOnly to scan and updates its hasNext end-key check.

Changes

Scan end key propagation

Layer / File(s) Summary
RegionStoreClient scan request/response updates
src/main/java/org/tikv/common/region/RegionStoreClient.java
Adds a startKey/endKey scan overload that encodes the key range into the request via codec.encodeRange, and returns codec.decodeKvPairs(resp.getPairsList()) on success instead of raw pairs.
ScanIterator rangeEndKey field and end-of-scan logic
src/main/java/org/tikv/common/operation/iterator/ScanIterator.java
Adds a rangeEndKey field initialized in the constructor and used to derive endKey; splits cacheLoadFails end-of-scan handling into a branch that nulls startKey when endKey is reached and a separate branch for an already-empty startKey.
ConcreteScanIterator scan invocation and hasNext condition
src/main/java/org/tikv/common/operation/iterator/ConcreteScanIterator.java
Passes rangeEndKey and keyOnly to client.scan; reworks the hasNext end-key check to return true when there is no end key, otherwise comparing against endKey.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ConcreteScanIterator
  participant ScanIterator
  participant RegionStoreClient
  participant Codec

  ConcreteScanIterator->>RegionStoreClient: scan(backOffer, startKey, rangeEndKey, version, keyOnly)
  RegionStoreClient->>Codec: encodeRange(startKey, endKey)
  Codec-->>RegionStoreClient: encoded key range
  RegionStoreClient->>RegionStoreClient: build ScanRequest with startKey/endKey
  RegionStoreClient->>Codec: decodeKvPairs(resp.getPairsList())
  Codec-->>RegionStoreClient: decoded kv pairs
  RegionStoreClient-->>ConcreteScanIterator: decoded kv pairs
  ConcreteScanIterator->>ScanIterator: cacheLoadFails(lastKey)
  ScanIterator->>ScanIterator: check lastKey against endKey, update processingLastBatch/startKey
  ConcreteScanIterator->>ConcreteScanIterator: hasNext() checks !hasEndKey or key < endKey
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main fix: scan requests now include endKey in the V2 API path.
Description check ✅ Passed The description follows the template closely and includes the problem, changes, tests, side effects, and related changes.
Linked Issues check ✅ Passed [#660] The changes implement ranged scan support, encode both bounds, decode returned pairs, and fix iterator behavior as requested.
Out of Scope Changes check ✅ Passed All changes stay within the scan endKey/V2 API fix and related iterator/client adjustments.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ti-chi-bot

ti-chi-bot Bot commented Jul 9, 2026

Copy link
Copy Markdown

Welcome @lilei-jd! It looks like this is your first PR to tikv/client-java 🎉

@lilei-jd lilei-jd marked this pull request as ready for review July 9, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Txn Client scan with endKey not working

1 participant