[wguo7]

Description

llama-index-vector-stores-lancedb cannot be imported in a clean environment:

>>> import llama_index.vector_stores.lancedb
...
  File ".../llama_index/vector_stores/lancedb/base.py", line 33, in <module>
    from pandas import DataFrame
ModuleNotFoundError: No module named 'pandas'

Root cause: base.py used pandas at runtime (module-level from pandas import DataFrame, .to_pandas() in get_nodes()/query(), DataFrame-based _to_llama_similarities), but pyproject.toml never declared pandas in [project].dependencies — and none of the declared deps (lancedb, pylance, tantivy, llama-index-core) requires pandas at runtime (lancedb carries it only in its tests extra across the whole supported range; verified 0.21.1 → 0.36.0). CI never caught it because the dev dependency group installs pandas>=2.2.3 into the uv-locked test env.

Why it surfaces now: until llama-index 0.14.16 the metapackage masked the bug by transitively pulling llama-index-readers-file (which hard-requires pandas). The slimmed metapackage (0.14.18+) dropped it, so since March even pip install llama-index llama-index-vector-stores-lancedb fails at import in a clean env. Minimal installs (llama-index-core + this integration) have been broken the whole time.

Fix: per the discussion on #22619, remove the pandas dependency entirely rather than declare it:

• both result fetches use LanceDB's native .to_list() (list of row dicts; implemented upstream as to_arrow().to_pylist()) instead of .to_pandas() — to_list() exists across the entire supported range (present in lancedb 0.21.1, the version floor)
• row loops iterate dicts directly (item["metadata"], item["id"]) instead of DataFrame.iterrows() attribute access
• _to_llama_similarities operates on the list of dicts with the same normalization math (softmax-style over score, exp(-_distance), linspace fallback), plus an explicit [] return for empty input
• ids come from a list comprehension instead of results["id"].tolist()

Struct columns (metadata) arrive as dicts and vector columns as lists from to_pylist(), so metadata_dict_to_node(...) and the embedding extraction behave identically; on a row missing the metadata key the legacy-fallback branch raises KeyError where it previously raised AttributeError — same uncaught crash class, no silent behavior change.

This drops a heavy transitive dependency from the integration and fixes the clean-env import break at the root.

Fixes #22619

New Package?

• Yes
• No

Version Bump?

• Yes (0.5.0 -> 0.5.1)
• No

Type of Change

• Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

• I ran the package's full test suite with pandas absent from the environment

All runs on Windows 11 / Python 3.13.9, package installed editable, pandas verifiably absent (import pandas → ModuleNotFoundError) in both environments:

Latest deps (fresh resolve: lancedb 0.36.0, pylance 10.0.0, llama-index-core 0.14.23):

26 passed, 529 warnings in 8.25s

Lockfile-pinned deps (lancedb 0.30.2, pylance 6.0.0, llama-index-core 0.14.21):

26 passed, 65 warnings in 11.26s

LanceDB is embedded, so this is the complete suite — no server-gated skips. Version floor compatibility: to_list() is present in lancedb 0.21.1 (lancedb/query.py, sync and async builders).

Pre-fix, the import failure reproduces in any clean venv as described in #22619 (also reproduced with deps pinned to the uv.lock versions — the break is version-independent, an undeclared-dependency bug rather than a lancedb regression).

Not covered: LanceDB Cloud/remote-table (db://) paths (no credentials available); to_list() is part of the same query-builder surface remote tables expose, but CI/reviewers with cloud access may want to confirm.

Suggested Checklist:

• I have performed a self-review of my own code
• My changes generate no new warnings
• New and existing unit tests pass locally with my changes
• I ran ruff check / ruff format (pinned 0.11.11) on the changed files

[wguo7]

Bump - this one has no reviewer assigned yet.

It fixes #22619: llama-index-vector-stores-lancedb imports pandas at module level but never declares it as a runtime dependency, so a fresh pip install of the package fails at import with ModuleNotFoundError: No module named 'pandas'. Rather than adding the dependency, this removes it - .to_pandas() becomes .to_list(), which is LanceDB's own API - matching what Dosu suggested on the issue.

@logan-markewich