Skip to content

fix(core): handle pre-epoch file timestamps in SimpleDirectoryReader - #23789

Open
NANA6719 wants to merge 2 commits into
run-llama:mainfrom
cnu-ossp-2026:fix/readers-pre-epoch-timestamp
Open

NANA6719 wants to merge 2 commits into
run-llama:mainfrom
cnu-ossp-2026:fix/readers-pre-epoch-timestamp

Conversation

@NANA6719

@NANA6719 NANA6719 commented Oct 29, 2026 •

Copy link
Copy Markdown

Description

On Windows, SimpleDirectoryReader.load_data() failed with OSError: [Errno 22] Invalid argument when any file had a modification time before 1970, because datetime.fromtimestamp() does not accept negative timestamps there. The metadata is built outside the raise_on_error try block, so a single file made the whole call fail.

_format_file_timestamp() now catches OSError, OverflowError and ValueError and returns None. This follows the function's existing contract (it already returns None for a missing timestamp), and default_file_metadata_func() already drops None values, so only that date field is omitted and the file still loads. Other behavior is unchanged.

Fixes #23787

New Package?

  • Yes
  • No

Version Bump?

  • Yes
  • No (llama-index-core is exempt)

Type of Change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

  • I added new unit tests to cover this change

  • test__format_file_timestamp_out_of_range: inf, 1e20 and nan return None instead of raising. These values fail on every OS, so this test catches the bug on Linux CI too.

  • test_SimpleDirectoryReader_pre_epoch_mtime: loads a real file whose mtime is set to 1969-12-31 and checks that the text and file_name metadata are kept. Before the fix it failed with OSError on Windows.

I confirmed that both new tests fail without the fix and pass with it. uv run -- pytest tests/readers/ passes (33 passed) on Windows 11 / Python 3.11.15, and pre-commit passes on the changed files.

AI Usage

I used Claude (an AI assistant) to help find this edge case, draft the fix and tests, and polish the English in this description. I reproduced the bug myself on Windows, checked for existing issues/PRs, reviewed every changed line, and ran the tests locally before and after the fix.

Suggested Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I ran uv run make format; uv run make lint to appease the lint gods

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: SimpleDirectoryReader.load_data() fails on Windows if any file has a pre-1970 modification time

1 participant