Starstruk Logo MarkStarstruk

v0.5.2

adbar/trafilaturav0.5.2Sep 22, 2020by adbar

AI Summary

This release updates the language detector, improves URL handling, and introduces a new extraction helper function.

Key Highlights

  • Optional language detector changed from langid to pycld3
  • New helper function bare_extraction()
  • Optional deduplication (disabled by default)
  • Better URL handling via courlan
  • Code consolidation for cleaner and shorter code

New Features

  • Helper function bare_extraction()
  • Optional deduplication
  • Better URL handling using courlan

Full Release Notes

- optional language detector changed: `langid` → `pycld3`
- helper function `bare_extraction()`
- optional deduplication off by default
- better URL handling (`courlan`), more complete metadata
- code consolidation (cleaner and shorter)