G
YouTube TikTok Games
Skip to content

fix(exec): preserve fragment scope in top-k late materialization - #11097

Open
zhangyue19921010 wants to merge 2 commits into
lance-format:mainfrom
zhangyue19921010:fix/topk-row-stream-fragment-scope
Open

zhangyue19921010 wants to merge 2 commits into
lance-format:mainfrom
zhangyue19921010:fix/topk-row-stream-fragment-scope

Conversation

@zhangyue19921010

Copy link
Copy Markdown
Collaborator

Problem

Top-k late materialization can remove a fragment-scoped row-stream read and return rows from excluded fragments.

Solution

Keep scoped row-stream reads below the sort, alongside the existing physical-selection guard. Unscoped plans retain the optimization.

Result

All 61 top-k tests pass, including 16 cases covering single, multiple and empty fragment scopes, the unscoped control, both row-ID modes and both partitioning shapes. The same test code has 12 scoped failures on unchanged upstream production code. cargo fmt --all and full-workspace cargo clippy --locked --profile ci --all --tests --benches -- -D warnings pass.

Fixes #11096

Problem
Top-k late materialization can remove a fragment-scoped row-stream read
and return rows from excluded fragments.

Solution
Keep scoped row-stream reads below the sort, alongside the existing
physical-selection guard.

Result
Add optimized-versus-unoptimized result checks for single, multiple and
empty fragment scopes, plus the unscoped optimization control, across
both row-ID modes and top-k partitioning shapes.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@github-actions github-actions Bot added the bug Something isn't working label Nov 15, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Gate recommendation: approve.

The guard preserves fragment restrictions before top-k selection. All 65 targeted tests pass; restoring the original production code makes all 12 scoped regression cases fail while the four unscoped controls pass.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Nov 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working K-approved Latest Gatekeeper recommendation permits acceptance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: top-k late materialization drops row-stream fragment scope

1 participant