Mistral AI has introduced Mistral OCR, a new Optical Character Recognition (OCR) API designed to enhance document analysis by accurately extracting structured text, media, tables, and equations from images and PDFs. According to Mistral AI, approximately 90% of organizational data worldwide is stored as documents, and Mistral OCR aims to unlock this potential. The API integrates with Retrieval-Augmented Generation (RAG) systems, making it suitable for processing multimodal documents like slides and complex PDFs.
Table of Contents
Key Features and Capabilities
- Superior Document Understanding: Mistral OCR excels in understanding complex document elements, including interleaved imagery, mathematical expressions, tables, and advanced layouts like LaTeX formatting. It enables a deeper understanding of rich documents such as scientific papers with charts, graphs, equations, and figures. The model extracts both text and embedded images from documents.
- Multilingual Support: Mistral OCR can parse, understand, and transcribe thousands of scripts, fonts, and languages across all continents. This versatility is crucial for global organizations handling documents from diverse linguistic backgrounds and hyperlocal businesses serving niche markets. Benchmarks reveal its high accuracy in languages such as Russian, French, Hindi, Chinese, Portuguese, German, Spanish, Turkish, Ukrainian, and Italian.
- Top-Tier Benchmarks: Mistral OCR consistently outperforms other leading OCR models in rigorous benchmark tests. It achieved an overall score of 94.89 in tests against Google Document AI, Azure OCR, Gemini models, and GPT-4o. Its high performance is noted in mathematical expressions, scanned documents, and tables.
- Speed and Efficiency: Mistral OCR processes up to 2000 pages per minute on a single node. Its rapid document processing ensures continuous learning and improvement, even in high-throughput environments.
- Doc-as-Prompt Functionality: Mistral OCR introduces the use of documents as prompts, enabling more powerful and precise instructions. This allows users to extract specific information from documents and format it in structured outputs like JSON, which can be chained into downstream function calls to build agents.
- Self-Hosting Option: For organizations with strict data privacy requirements, Mistral OCR offers a self-hosting option, ensuring that sensitive or classified information remains secure within their own infrastructure.
Performance Highlights
Mistral OCR’s performance has been rigorously tested against other leading OCR models. The results of these benchmarks are as follows:
| Model | Overall | Math | Multilingual | Scanned | Tables |
|---|---|---|---|---|---|
| Google Document AI | 83.42 | 80.29 | 86.42 | 92.77 | 78.16 |
| Azure OCR | 89.52 | 85.72 | 87.52 | 94.65 | 89.52 |
| Gemini-1.5-Flash-002 | 90.23 | 89.11 | 86.76 | 94.87 | 90.48 |
| Gemini-1.5-Pro-002 | 89.92 | 88.48 | 86.33 | 96.15 | 89.71 |
| Gemini-2.0-Flash-001 | 88.69 | 84.18 | 85.80 | 95.11 | 91.46 |
| GPT-4o-2024-11-20 | 89.77 | 87.55 | 86.00 | 94.58 | 91.70 |
| Mistral OCR 2503 | 94.89 | 94.29 | 89.55 | 98.96 | 96.12 |
The table demonstrates that Mistral OCR outperforms its competitors across various metrics, including overall accuracy, mathematical expression recognition, multilingual support, scanned document processing, and table extraction.
Use Cases
Beta customers are utilizing Mistral OCR across various sectors:
- Scientific Research: Converting scientific papers and journals into AI-ready formats to accelerate collaboration and scientific workflows.
- Historical Preservation: Digitizing historical documents and artifacts to ensure their preservation and broader accessibility.
- Customer Service: Transforming documentation and manuals into indexed knowledge bases to reduce response times and improve customer satisfaction.
- Technical Literature: Converting technical literature, engineering drawings, lecture notes, presentations, and regulatory filings into indexed, answer-ready formats.
Availability
Mistral OCR is the default model for document understanding on Le Chat. The API, named mistral-ocr-latest, is available on Mistral’s developer suite, La Plateforme, at a cost of $1 per 1000 pages, with approximately double the efficiency using batch inference. It will soon be available through cloud and inference partners, as well as on-premises deployments.
