Starstruk Logo MarkStarstruk

v1.1.0

projectdiscovery/katanav1.1.0Mar 26, 2024by ehsandeep

AI Summary

Introduced passive crawling capabilities using external sources like Wayback and Common Crawl, alongside options to exclude targets from crawling.

Key Highlights

  • Introduced passive crawling with `-passive` and `-passive-source` flags.
  • Introduced `-exclude` flag to filter out specific targets.
  • Fixed headless redirect issues.
  • Added conditional support for tree-sitter.

New Features

  • Passive crawling
  • Exclude targets to crawl
  • Conditional tree-sitter support

Full Release Notes

<!-- Release notes generated using configuration in .github/release.yml at main -->

## What's Changed
### 🎉 New Features
* **Introduced passive crawling** by @dogancanbakir in https://github.com/projectdiscovery/katana/pull/781
```
   -ps, -passive  enable passive sources to discover target endpoints
   -pss, -passive-source string[]  passive source to use for url discovery (waybackarchive,commoncrawl,alienvault)
```
#### Example:
```console
katana -u tesla.com -passive -f qurl
...
https://static-assets-pay.tesla.com/api/payment-schema/creditcard?countrycode=dk
https://static-assets-pay.tesla.com/api/paymenttypes/?clientlibrary=payment-website
https://tradepartnertickets.tesla.com/dist/main.css?029b26e9be3aef4fc82c
https://tradepartnertickets.tesla.com/dist/vendors.vendors.css?029b26e9be3aef4fc82c
https://workforce.tesla.com/auth/callback?code=na_921f11c72db1d416c2fb624ea94ab5e1bad5f803
...
...
...
[INF] Found 208114 endpoints for https://tesla.com in 2m11.65937825s (commoncrawl: 128, alienvault: 1615, waybackarchive: 206371)
```

* Introduced option to exclude targets to crawl by @dogancanbakir in https://github.com/projectdiscovery/katana/pull/731
```
   -e, -exclude string[]  exclude host matching specified filter ('cdn', 'private-ips', cidr, ip, regex)
```


### 🐞 Bug Fixes
* Fixed issue with headless redirect by @Mzack9999 in https://github.com/projectdiscovery/katana/pull/823
* Fixed issue with filter and match functionality (SDK) by @dogancanbakir in https://github.com/projectdiscovery/katana/pull/775
* Fixed issue with `-kf` option to skip if the maximum depth is exceeded by @dogancanbakir in https://github.com/projectdiscovery/katana/pull/799

### Other Changes
* Added conditional support to include go-tree-sitter in build process by @RamanaReddy0M in https://github.com/projectdiscovery/katana/pull/814
* Fixed nil pointer check by @dogancanbakir in https://github.com/projectdiscovery/katana/pull/773
* Fixed error check on hybrid request parsing by @niudaii in https://github.com/projectdiscovery/katana/pull/804
* Updated jsluice to new version with more results by @Ice3man543 in https://github.com/projectdiscovery/katana/pull/695
* Updated jsluice by @geeknik in https://github.com/projectdiscovery/katana/pull/800

## New Contributors
* @beac0n5 made their first contribution in https://github.com/projectdiscovery/katana/pull/732
* @toufik-airane made their first contribution in https://github.com/projectdiscovery/katana/pull/736
* @niudaii made their first contribution in https://github.com/projectdiscovery/katana/pull/804
* @geeknik made their first contribution in https://github.com/projectdiscovery/katana/pull/800

**Full Changelog**: https://github.com/projectdiscovery/katana/compare/v1.0.5...v1.1.0