[deep1blue]

Summary

Eliminates $O(N^2)$ front-list insertions/deletions (list.insert(0, ...) and list.pop(0)) and redundant per-message tokenization in LongContextReorder and ChatSummaryMemoryBuffer:

1. LongContextReorder._postprocess_nodes (llama-index-core/llama_index/core/postprocessor/node.py):

   • Previously iterated over ordered_nodes and called reordered_nodes.insert(0, node) on every even index (i % 2 == 0), shifting all preceding elements ($O(N^2)$).
   • Replaced with ordered_nodes[::2][::-1] + ordered_nodes[1::2], producing the exact same lost-in-the-middle node ordering in $O(N)$ slice operations.

2. ChatSummaryMemoryBuffer._split_messages_summary_or_full_text & _handle_assistant_and_tool_messages (llama-index-core/llama_index/core/memory/chat_summary_memory_buffer.py):

   • _split_messages_summary_or_full_text previously called self._token_count_for_messages([chat_history[-1]]) twice per retained message (once in the while condition and once when incrementing self._token_count), invoking tokenizer_fn twice on every message that fits within token_limit, and called chat_history_full_text.insert(0, chat_history.pop()) ($O(M^2)$).
   • Cache msg_tokens = self._token_count_for_messages([chat_history[-1]]) once per iteration, append to chat_history_full_text, and reverse once at the end ($O(M)$).
   • In _handle_assistant_and_tool_messages, replace repeated chat_history_full_text.pop(0) calls with a single slice transfer (chat_history_to_be_summarized.extend(chat_history_full_text[:idx]); del chat_history_full_text[:idx]).

Testing

• Added test_long_context_reorder in llama-index-core/tests/postprocessor/test_base.py.
• Added test_split_messages_single_tokenization_per_message in llama-index-core/tests/memory/test_chat_summary_memory_buffer.py.
• Verified all test_chat_summary_memory_buffer.py and test_base.py tests pass.

Acknowledgments

Credit to @logan-markewich, @shadyelgewily-slimstock (#13155), @Junxiao-Zhao (#14563), @pablonm3 (#17845), and @xpbowler (#16127) for the design and maintenance of LongContextReorder and ChatSummaryMemoryBuffer.