v1.4.0

datalab-to/markerv1.4.0Feb 11, 2025by VikParuchuri

AI Summary

Introduces benchmarks against competitors and updates the Gemini model to version 2.0.

Key Highlights

New benchmarks (llamaparse, docling, mathpix)
Gemini model update (Flash 2.0)
OCR heuristics fixes

New Features

Benchmarking suite
Gemini Flash 2.0 integration
LLM text support for references

Full Release Notes

# New benchmarks

## Overall

Benchmark against llamaparse, docling, mathpix (see README for how to run benchmarks).  Marker performs favorably against alternatives in speed, llm as judge scoring, and heuristic scoring.

![image](https://github.com/user-attachments/assets/acda34d0-3dab-4a1e-a677-a10a64830699)

## Table

Benchmark tables against gemini flash:

![image](https://github.com/user-attachments/assets/e46744e5-d853-43fa-9069-990a67c32b9f)

# Update gemini model

- Use the new genai library
- Update to gemini flash 2.0

# Misc bugfixes

- Fix bug with OCR heuristics not being aggressive enough
- Fix bug with empty tables
- Ensure references get passed through in llm processors

## What's Changed
* Add llm text support for references, superscripts etc by @iammosespaulr in https://github.com/VikParuchuri/marker/pull/523
* Update overall benchmark by @VikParuchuri in https://github.com/VikParuchuri/marker/pull/515
* Benchmarks by @VikParuchuri in https://github.com/VikParuchuri/marker/pull/531


**Full Changelog**: https://github.com/VikParuchuri/marker/compare/v1.3.5...v1.4.0