如何使用本地下载的C-mteb的T2reranks数据集对本地下载的reranker模型进行测试？

[chaoStart]

from sentence_transformers import SentenceTransformer
model = SentenceTransformer(model_name_or_path="/home/sciyonadmin/xinference/modelscope/hub/Xorbits/bge-reranker-large")
from mteb import MTEB
import mteb
tasks = mteb.get_tasks(tasks=["MMarcoReranking"],languages=["zh"])

evaluation = MTEB(tasks=tasks)
evaluation.run(
model,
eval_splits=["dev"],
verbosity=3,
output_folder="results/stage1",
encode_kwargs={"batch_size": 32},
path='/home/bqc/.cache/evalscope/dataset/Mmarco-reranking',

data_files='/home/bqc/.cache/evalscope/dataset/Mmarco-reranking/data/dev-00000-of-00001-3df8a0d7f3506d33.parquet',

)
运行上面的代码会报错，常常报错有缺少eval_split的key等

[chaoStart]

排查了几天下来，发现python的版本对于使用C-mteb数据集测试reranker模型有关键的作用，同样的代码，当我使用虚拟环境版本为3.11.9直接成功运行起来，至于3.10.15为何不能成功运行上面代码的原因还在排查中。

[KennethEnevoldsen]

Title translated says:
“How to test a locally downloaded reranker model using the locally downloaded T2reranks dataset from C-mteb?”

Have you tried:
https://github.com/embeddings-benchmark/mteb?tab=readme-ov-file#using-a-cross-encoder-for-reranking

Using a local sentencetransformer/cross-encoder here should be as usual - I just want to make sure we can get that to work before we work on the local dataset.

[KennethEnevoldsen]

(moving this to discussion as it is a usage question)