v0.0.1

NVIDIA/NeMo-Speech.cppv0.0.1Jun 27, 2026by chonknick

AI Summary

This is the first public release of pulpie, a fast and CPU-friendly tool for extracting main content from HTML. It leverages encoder models to return clean HTML or Markdown, achieving exact parity with MinerU-HTML in accuracy benchmarks.

Key Highlights

Fast and CPU-friendly web content extraction using encoder models.
Achieves MinerU-HTML parity with identical ROUGE-5 scores (0.8625).
Byte-faithful simplification verified across 65/65 tests.
Supports three model variants: Small (210M), Base (610M), and Large (2.1B).

New Features

Core extraction functionality via the `Extractor` class.
Markdown output support via the `pulpie[markdown]` extra.
Byte-faithful HTML simplification matching MinerU-HTML.
Automatic download of pulpie-orange-small model on first use.

Full Release Notes

First public release of **pulpie** — fast, CPU-friendly web content extraction from HTML using encoder models.

## Install

```bash
pip install pulpie            # core
pip install pulpie[markdown]  # + markdown output
```

## What it does

pulpie extracts the main content from raw HTML and returns clean HTML or markdown. It classifies HTML blocks with small encoder models (the **Orange** family), reaching autoregressive-extractor quality at a fraction of the cost.

```python
from pulpie import Extractor

extractor = Extractor()             # downloads pulpie-orange-small (210M) on first use
result = extractor.extract(html)
print(result.markdown)              # clean markdown
print(result.html)                  # clean HTML
print(result.n_main, result.n_other)
```

## MinerU-HTML parity

pulpie's `simplify`/`reconstruct` faithfully reproduce MinerU-HTML's `simplify_html` / `map_to_main` (Apache-2.0), the segmentation format the Orange models were distilled on. Verified two ways:

- **Offline byte-parity:** 65/65 tests reproduce MinerU's exact output across 13 real-page fixtures.
- **End-to-end ROUGE-5** on full WebMainBench English (6,647 pages, orange-small):

  | Simplification | ROUGE-5 F1 |
  |----------------|-----------|
  | MinerU-HTML (reference) | 0.8625 |
  | **pulpie (this release)** | **0.8625** |

  Byte-faithful end-to-end (Δ vs MinerU = +0.0000), a +0.14 improvement over the prior simplification.

## Model zoo

| Name | HF repo | Size | Notes |
|------|---------|------|-------|
| Orange Small | `chonkie-ai/pulpie-orange-small` | 210M | Best value; default |
| Orange Base | `chonkie-ai/pulpie-orange-base` | 610M | Distilled |
| Orange Large | `chonkie-ai/pulpie-orange-large` | 2.1B | Highest quality |

## Credits

Built on MinerU-HTML / Dripper (Ma et al., 2025) — their `simplify_html` preprocessing, block-annotation scheme, and the WebMainBench benchmark are foundational. Thank you for releasing your tools and data.