v1.12.0

adbar/trafilaturav1.12.0Jul 30, 2024by adbar

AI Summary

Breaking release enforcing fixed output formats and improving extraction accuracy with better justext fallback and XPath rules.

Key Highlights

Enforced fixed list of output formats, deprecated -out CLI option
Improved justext fallback mechanism
Prevented LXML error in JSON-LD baseline
Refined XPath rules for undesirable content
Fixed markdown formatting with newlines
Replaced RelaxNG with DTD in XML-TEI

Breaking Changes

Fixed output formats list, deprecated -out on CLI

New Features

Fixed output formats enforcement
Improved justext fallback
JSON-LD error prevention
Enhanced XPath content filtering
XML-TEI DTD replacement
MemoryError and ValueError fixes for tables

Full Release Notes

Breaking change:
- enforce fixed list of output formats, deprecate `-out` on the CLI (#647)

Faster, more accurate extraction:
- review link and structure checks (#653)
- improve justext fallback (#652)
- baseline: prevent LXML error in JSON-LD (#643), do not use as backup extraction (#646)
- review XPaths for undesirable content (#645)

Bugfixes and maintenance:
- CLI fix: markdown format should trigger `include_formatting` (#649)
- images fix: use a length threshold on src attribute (#654)
- XML-TEI: replace RelaxNG by DTD, remove pickle, and update (#655)
- formatting & markdown fix: add newlines (#656)
- table fix: prevent `MemoryError` & `ValueError` during conversion to text (#658)

Documentation:
- update `crawls.rst`: `known` is an unexpected argument, by @tommytyc in #638