[Chaewon1000]

Description

This PR fixes a Pydantic ValidationError that occurs when backup_separators=None is passed to TokenTextSplitter.

The constructor accepts Optional[List[str]], but the corresponding Pydantic field requires a list. This change normalizes None to an empty list before validation, preserving the existing behavior for valid separator lists.

Changes:

• Normalize backup_separators=None to [] before calling super().__init__().
• Add a regression test to verify that None is handled correctly.
• No additional dependencies are required.

Fixes #23790

New Package?

Did I fill in the tool.llamahub section in the pyproject.toml and provide a detailed README.md for my new integration or package?

• Yes
• No

Version Bump?

Did I bump the version in the pyproject.toml file of the package I am updating? (Except for the llama-index-core package)

• Yes
• No

Type of Change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Your pull-request will likely not be merged unless it is covered by some form of impactful unit testing.

• I added new unit tests to cover this change
• I believe this change is already covered by existing unit tests

Test Results

• The regression test failed with ValidationError before the fix and passed afterward.
• All tests in test_token_splitter.py passed.
• Full llama-index-core test suite: 1,498 passed, 2 failed, 22 skipped, 6 xfailed.

The two failing tests were:

• tests/callbacks/test_llama_debug.py::test_get_event_stats
• tests/schema/test_media_resource.py::test_hash

These failures appear unrelated to the changes in this PR, but have not been verified against an unmodified baseline.

Suggested Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added Google Colab support for the newly added notebooks.
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I ran uv run make format; uv run make lint to appease the lint gods