Mistral Small 24B Instruct 2501 Model Card

Mistral-Small-24B-Instruct-2501, marketed as "Mistral Small 3," is a 24-billion-parameter instruction-fine-tuned language model released by Mistral AI in January 2025. It is built on the Mistral-Small-24B-Base-2501 base model and positioned as state-of-the-art in the sub-70B category. Its stated design goals are local deployment, fast conversational response, low-latency function calling, and fine-tuning for domain experts.

The model scores 0.663 on MMLU Pro 5-shot CoT, 0.453 on GPQA Main 5-shot CoT, 0.848 on HumanEval pass@1, and 0.706 on MATH instruct. Instruction-following benchmarks show 8.35 on MT-Bench, 52.27 on WildBench, and 0.873 on Arena Hard. The model supports a 32k context window, eleven or more languages including English, French, German, Spanish, Italian, Chinese, Japanese, and Korean, and provides native function calling and JSON output via a Tekken tokenizer with a 131k vocabulary.

Public benchmarks were run through a single internal evaluation pipeline; Mistral notes that numbers "may vary slightly from previously reported performance" for comparison models. Judge-based evals — WildBench, Arena Hard, and MT-Bench — used GPT-4o-2024-05-13 as the judge. Human preference evaluations were conducted side-by-side with an external third-party vendor on over 1,000 proprietary coding and generalist prompts, with model identity anonymized; Mistral states it took "extra caution in verifying a fair evaluation" and is "confident that the above benchmarks are valid."

Not disclosed in this document.

Not disclosed in this document.

The model is released under the Apache 2.0 license, permitting commercial and non-commercial use and modification. It is available via vLLM (requiring approximately 55 GB GPU RAM in bf16 or fp16), Hugging Face Transformers, and Ollama for local inference. Quantized versions (4-bit, 8-bit) can run on a single RTX 4090 or a 32 GB RAM MacBook. The card notes that enterprises needing extended context, specific modalities, or domain knowledge will be served by separate commercial models beyond this open release.

The model's knowledge base has a cutoff of October 1, 2023, as stated in the recommended system prompt. The card includes no explicit limitations section and does not discuss failure modes, bias evaluations, or out-of-distribution behavior.

This is the January 2025 release of Mistral Small, designated "Mistral Small 3 (2501)," superseding prior Mistral Small releases. No version changelog or explicit list of changes from a prior version is included in the card.