v0.0.1

cloudflare/pingorav0.0.1Jun 27, 2026by chonknick

AI Summary

The initial public release introduces pulpie, a fast, CPU-friendly tool for extracting main content from HTML using the Orange encoder models.

Key Highlights

First public release of pulpie
Achieves MinerU-HTML parity with verified byte-parity and ROUGE-5 benchmarks
Supports both clean HTML and Markdown output
Three model sizes available (Small, Base, Large)
Built on MinerU-HTML and Dripper foundations

New Features

Core Extractor class
Markdown output support via pulpie[markdown]
Simplify and reconstruct methods
Model Zoo with Orange Small, Base, and Large variants

Full Release Notes

First public release of **pulpie** — fast, CPU-friendly web content extraction from HTML using encoder models.

## Install

```bash
pip install pulpie            # core
pip install pulpie[markdown]  # + markdown output
```

## What it does

pulpie extracts the main content from raw HTML and returns clean HTML or markdown. It classifies HTML blocks with small encoder models (the **Orange** family), reaching autoregressive-extractor quality at a fraction of the cost.

```python
from pulpie import Extractor

extractor = Extractor()             # downloads pulpie-orange-small (210M) on first use
result = extractor.extract(html)
print(result.markdown)              # clean markdown
print(result.html)                  # clean HTML
print(result.n_main, result.n_other)
```

## MinerU-HTML parity

pulpie's `simplify`/`reconstruct` faithfully reproduce MinerU-HTML's `simplify_html` / `map_to_main` (Apache-2.0), the segmentation format the Orange models were distilled on. Verified two ways:

- **Offline byte-parity:** 65/65 tests reproduce MinerU's exact output across 13 real-page fixtures.
- **End-to-end ROUGE-5** on full WebMainBench English (6,647 pages, orange-small):

  | Simplification | ROUGE-5 F1 |
  |----------------|-----------|
  | MinerU-HTML (reference) | 0.8625 |
  | **pulpie (this release)** | **0.8625** |

  Byte-faithful end-to-end (Δ vs MinerU = +0.0000), a +0.14 improvement over the prior simplification.

## Model zoo

| Name | HF repo | Size | Notes |
|------|---------|------|-------|
| Orange Small | `chonkie-ai/pulpie-orange-small` | 210M | Best value; default |
| Orange Base | `chonkie-ai/pulpie-orange-base` | 610M | Distilled |
| Orange Large | `chonkie-ai/pulpie-orange-large` | 2.1B | Highest quality |

## Credits

Built on MinerU-HTML / Dripper (Ma et al., 2025) — their `simplify_html` preprocessing, block-annotation scheme, and the WebMainBench benchmark are foundational. Thank you for releasing your tools and data.