Skip to content

Issues: embeddings-benchmark/mteb

Merge v2.0.0: Overview issue
#1791 opened Jan 13, 2025 by KennethEnevoldsen
Open 2
Overview: Leaderboard release
#1867 opened Jan 24, 2025 by x-tabdeveloping
Open 26
MAEB Overview Issue
#2072 opened Feb 16, 2025 by isaac-chung
Open 6
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

244 Open 711 Closed
244 Open 711 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

How to Support Task Specific Instructions in HIT-TMG/KaLM-embedding Models
#2482 opened Apr 3, 2025 by ayush1298
Dataset navjordj/SNL_summarization no longer available
#2477 opened Apr 2, 2025 by KennethEnevoldsen
[v2] Define encoding kwargs v2 Issues and PRs related to `v2` branch
#2476 opened Apr 2, 2025 by Samoed
1
Jasper model is not being evaluated well by mteb.get_model() bug Something isn't working
#2474 opened Apr 2, 2025 by yjoonjang
8
Add datasets for audio pair classification
#2461 opened Mar 30, 2025 by kkaitlyn111
@kkaitlyn111
Add ESC50 (pair classification task)
#2459 opened Mar 30, 2025 by switchpiggy
1
@switchpiggy
Potential major duplication bug in COIR benchmark
#2455 opened Mar 29, 2025 by Siddharth-Gandhi
2
Integrate Vidore v2 mieb The image extension of MTEB new benchmark Issues related to adding a new benchmark
#2453 opened Mar 28, 2025 by Samoed
Integrate ColPali, ColQwen mieb The image extension of MTEB new model Questions related to adding a new model to the benchmark
#2452 opened Mar 28, 2025 by Samoed
Reduce imports of full packages in MTEB to reduce import time
#2450 opened Mar 27, 2025 by KennethEnevoldsen
Move validation of metadata object to tests instead of import v2 Issues and PRs related to `v2` branch
#2449 opened Mar 27, 2025 by KennethEnevoldsen
Allow numpy output in Image2Text v2 Issues and PRs related to `v2` branch
#2441 opened Mar 26, 2025 by Samoed
Update to latest version of gradio enhancement New feature or request leaderboard issues related to the leaderboard
#2438 opened Mar 26, 2025 by KennethEnevoldsen
3
Refactor MIEB retrieval v2 Issues and PRs related to `v2` branch
#2435 opened Mar 25, 2025 by Samoed v2.0.0
@Samoed
8
Refactor MIEB Clustering v2 Issues and PRs related to `v2` branch
#2434 opened Mar 25, 2025 by Samoed v2.0.0
@Samoed
2
Refactor MIEB ZeroShotClassification v2 Issues and PRs related to `v2` branch
#2433 opened Mar 25, 2025 by Samoed v2.0.0
Refactor MIEB classification v2 Issues and PRs related to `v2` branch
#2432 opened Mar 25, 2025 by Samoed v2.0.0
STS tasks do not support prompt types more information needed More information is needed before this issue can be resolved
#2431 opened Mar 25, 2025 by malteos
1
[v2] Remove SpeedTask v2 Issues and PRs related to `v2` branch
#2425 opened Mar 24, 2025 by Samoed
5
Number of Parameters is in alphabetical order and not numerical order. leaderboard issues related to the leaderboard
#2418 opened Mar 23, 2025 by Hobstarr
2
[v2] Update ruff rules v2 Issues and PRs related to `v2` branch
#2416 opened Mar 23, 2025 by Samoed
21 tasks
1
CovidRetrieval is missing if select Retrieval task type bug Something isn't working leaderboard issues related to the leaderboard
#2411 opened Mar 22, 2025 by Samoed
1
Refactor CLIP models to SentenceTransformer mieb The image extension of MTEB v2 Issues and PRs related to `v2` branch
#2407 opened Mar 21, 2025 by Samoed
Custom retrieval dataset will be Error with the model has prompts
#2394 opened Mar 19, 2025 by RobinChiu
5
Integrate rank1 new model Questions related to adding a new model to the benchmark
#2378 opened Mar 15, 2025 by Samoed
1
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.