v1.4.0

adbar/trafilaturav1.4.0Oct 18, 2022by adbar

AI Summary

Improves extraction quality and XML output format while optimizing performance and adding utility functions.

Key Highlights

XML: preserve list type as attribute
XML TEI: better conformity to standards
Faster text cleaning and shorter code
Metadata: add language when detector is activated
Cache clearing function added

New Features

Cache clearing function
CLI exit code changes for download failures

Full Release Notes

Impact on extraction and output format:
- better extraction (#233, #243 & #250 with @knit-bee, #246 with @mrienstra, #258)
- XML: preserve list type as attribute (#229)
- XML TEI: better conformity with @knit-bee (#238, #242, #253, #254)
- faster text cleaning and shorter code (#237 with @deedy5, #245)
- metadata: add language when detector is activated (#224)
- metadata: extend fallbacks and test coverage for json_metadata functions by @felipehertzer (#235)
- TXT: change markdown formatting of headers by @LaundroMat (#257)

Smaller changes in convenience functions:
- add function to clear caches (#219)
- CLI: change exit code if download fails (#223)
- settings: use "\n" for multiple user agents by @k-sareen (#241)

Updates:
- docs updated (and #244 by @dsgibbons)
- package dependencies updated

**Full Changelog**: https://github.com/adbar/trafilatura/compare/v1.3.0...v1.4.0