Skip to content

feat: Integration or regression tests - #336

Open
stephantul wants to merge 9 commits into
mainfrom
integration-tests
Open

feat: Integration or regression tests#336
stephantul wants to merge 9 commits into
mainfrom
integration-tests

Conversation

@stephantul

@stephantul stephantul commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

This PR adds a couple of integration/regression test baselines. These are mostly meant to guard against unforeseen changes in distillation. They are not run in CI, because they download full models. I've added three models to start:

  1. minilm
  2. bge-base
  3. embeddinggemma
  4. e5-multilingual
  5. gte-modernbert

These are added as defaults. You can add new models by running make test_integration_update. You should only run this after changing the code.

Comparing the results without regenerating is done by running make test_integration. This will fail when the new results deviate significantly from the stored ones.

The integration tests surfaced a bug with BPE models and added vocabulary. An edge case, also fixed in this PR.

@codecov

codecov Bot commented Aug 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
model2vec/tokenizer/tokenizer.py 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@stephantul
stephantul requested a review from Pringled August 1, 2026 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant