v2.5.1

![Swirl Metasearch Logo](https://raw.githubusercontent.com/wiki/swirlai/swirl-search/images/swirl-metasearch-logo.png)

Team Swirl is announcing General Availability of Swirl 2.5.1! 

This release features three new SearchProviders ([arXiv.org](https://arxiv.org/), [EuropePMC.org](https://europepmc.org/), and [LinkedIn](https://www.linkedin.com/) profiles via Google PSE), enhanced result processing to handle XML responses from source APIs, plus updates to the ChatGPT `Connector` and `QueryProcessor` to support the latest GPT-3.5-Turbo and GPT-4 models.

![Release 2.5.1 New SearchProviders](https://raw.githubusercontent.com/wiki/swirlai/swirl-search/images/2_5_1-New_SPs.png)

🥇 We would also like to give a shout out to GitHub user [Rajpratik71](https://github.com/Rajpratik71) for [his contribution](https://github.com/swirlai/swirl-search/pull/511) to this release which improved our `Dockerfile` configuration.  Many thanks for contributing to Swirl!

### PLEASE STAR OUR REPO: [https://github.com/swirlai/swirl-search](https://github.com/swirlai/swirl-search)

<br/>

# New Features

:small_blue_diamond:  Both the ChatGPT [`Connector`](https://github.com/swirlai/swirl-search/blob/main/swirl/connectors/chatgpt.py) and [`QueryProcessor`](https://github.com/swirlai/swirl-search/blob/main/swirl/processors/chatgpt_query.py) were updated to use OpenAI's [`ChatCompletion`](https://platform.openai.com/docs/api-reference/chat) method which supports the latest GPT models -  including GPT-4 - and a much greater range of interactivity.

- Three new Tags and one new  `query_mapping` option are available for the ChatGPT SearchProvider to help shape the Prompt or Default Role passed to ChatGPT along with the user's query.
- The ChatGPT SearchProvder now queries the GPT-3.5-Turbo model by default.

:small_blue_diamond:  Search LinkedIn profiles with the new LinkedIn Google PSE and SearchProvider.

:small_blue_diamond:  Search the academic paper collections of [arXiv.org](https://arxiv.org/) and [EuropePMC.org](https://europepmc.org/) with new SearchProivders for each.

:small_blue_diamond:  New result processing to handle XML responses from source APIs; see this in action with the [arXiv.org](https://arxiv.org/) SearchProvider.

:small_blue_diamond:  Swirl's `Dockerfile` now uses the `--no-cache-dir` flag with `pip` to save space.
Thanks to GitHub user [Rajpratik71](https://github.com/Rajpratik71) for [this contribution](https://github.com/swirlai/swirl-search/pull/511)!


# Improvements

:small_blue_diamond:  The Subscribe functionality was reworked to better handle Microsoft Authentication for [subscribe runs that include M365 sources](https://github.com/swirlai/swirl-search/wiki/5.-Developer-Guide#subscribe-to-a-search-with-m365-sources).

:small_blue_diamond:  Some default SearchProviders have additional Tags:  `Internal` or `Develop`.  See the documentation for details about targeting specific groups of sources using [SearchProvider Tags](https://github.com/swirlai/swirl-search/wiki/2.-User-Guide#organizing-searchproviders-with-active-default-and-tags).

:small_blue_diamond:  The `install.sh` script was further enhanced to synchronize spaCy model and code version updates.


# Resolved Issues

:small_blue_diamond:  Addressed an issue where Swirl could hang due to very long source repsonse times. Swirl's default timeout value in [`swirl_server/settings.py`](https://github.com/swirlai/swirl-search/blob/main/swirl_server/settings.py) is now: `SWIRL_TIMEOUT_DEFAULT = 10`

:small_blue_diamond:  Corrected a count inconsistency in the `CosineRelevancyResultProcessor` 

:small_blue_diamond:  Removed duplicate entries from `urls.py`

:small_blue_diamond:  Fixed a regression where unmatched Tags were being dropped in searches

:small_blue_diamond:  ChatGPT no longer returns a 500 error when enabled for Subscribed searches

:small_blue_diamond:  After authenticating to Mircosoft in the Galaxy UI, the first search run now correctly includes M365 resutls

:small_blue_diamond: Addressed an issue with date-sorting in the M365 sources:

- The default date-sort for M365 sources is:  `DEFAULT_DATESORT_X = "createdDateTime desc"`
- This can be overridden by setting `DATE_SORT` in the SearchProvider `query_mapping`
- Owners of a Search object or Swirl's Subscribe feature can set the `sort` attribute on a Search


# Known Issues

:small_blue_diamond: Clicking a Microsoft Teams result sometimes results in this error from the Teams app:
```
We cannot take you to that message because it's in a chat you're not in.
```
For Teams links to resolve correctly, you must have the Microsoft Teams app open *before* clicking the result link.

:small_blue_diamond: [Creating searches from a browser with q=](https://github.com/swirlai/swirl-search/wiki/5.-Developer-Guide#create-a-search-object-with-the-q-url-parameter) can sometimes create two Search objects. 

This is because of browser prefetch AKA predictive service. [Turn off Chrome prediction service](https://www.ghacks.net/2019/04/23/missing-chromes-use-a-prediction-service-setting/). [Turn off Safari prefetch](https://stackoverflow.com/questions/29214246/how-to-turn-off-safaris-prefetch-feature).

Please report any issues with this to [support](#support).


# Upgrading

:warning:  Version 2.5.1 requires database migration. See [Upgrading Swirl](https://github.com/swirlai/swirl-search/wiki/3.-Admin-Guide#upgrading-swirl).

:warning: Version 2.5.0 replaced RabbitMQ with Redis.  If you are upgrading from any Swirl 1.x version to Release 2.5.1, please see the [Quick Start - Local Installation](https://github.com/swirlai/swirl-search/wiki/1.-Quick-Start#local-installation) section for details on how to install Redis with Swirl on MacOS or Linux.


# Documentation

[Overview](https://docs.swirl.today/) | [Quick Start](https://docs.swirl.today/1.-Quick-Start) | [User Guide](https://docs.swirl.today/2.-User-Guide) | [Admin Guide](https://docs.swirl.today/3.-Admin-Guide) | [M365 Guide](https://docs.swirl.today/4.-M365-Guide) | [Developer Guide](https://docs.swirl.today/5.-Developer-Guide) | [Developer Reference](https://docs.swirl.today/6.-Developer-Reference)
_(updated links on 13-October-2023)_

# Support

:small_blue_diamond: [Join the Swirl Metasearch Community on Slack!](https://join.slack.com/t/swirlmetasearch/shared_invite/zt-1qk7q02eo-kpqFAbiZJGOdqgYVvR1sfw)

:small_blue_diamond: Email: [support@swirl.today](mailto:support@swirl.today) with issues, requests, questions, etc - we'd love to hear from you!

<br/>