Skip to content

perf(core): avoid O(K^2) duplicate and missing-rank scans in RankGPTRerank - #23672

Open
deep1blue wants to merge 1 commit into
run-llama:mainfrom
deep1blue:perf/rankgpt-rerank-dedup
Open

deep1blue wants to merge 1 commit into
run-llama:mainfrom
deep1blue:perf/rankgpt-rerank-dedup

Conversation

@deep1blue

Copy link
Copy Markdown

Description

In RankGPTRerank (llama-index-core/llama_index/core/postprocessor/rankGPT_rerank.py):

  1. _remove_duplicate checks if c not in new_response: against a list on every element ($O(K^2)$). Using list(dict.fromkeys(response)) preserves first-seen order in $O(K)$.
  2. _receive_permutation appends unranked candidates via [tt for tt in range(rank_end) if tt not in response_list], performing an $O(K)$ list membership scan per candidate ($O(K^2)$ total). Precomputing seen = set(response_list) makes this $O(K)$.
  3. _clean_response builds new_response via repeated += character concatenation in a loop instead of "".join(c if c.isdigit() else " " for c in response).strip().

On a 2,000-hit permutation (20 calls), _receive_permutation drops from 610.49 ms to 38.13 ms (16.0x faster).

Type of Change

  • Bug fix / performance improvement (non-breaking change)

How Has This Been Tested?

  • Added unit test test_rankgpt_rerank_receive_permutation_duplicates_and_missing in llama-index-core/tests/postprocessor/test_rankgpt_rerank.py
  • Ran pytest llama-index-core/tests/postprocessor/test_rankgpt_rerank.py (3 passed)

…erank

Signed-off-by: Parth Vadera <parthvadera@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant