v2.0.0

plausible/analyticsv2.0.0Jul 12, 2023by ruslandoga

AI Summary

A major release introducing new visitor metrics, period comparisons, ability to change domain names, and automatic geolocation.

Key Highlights

  • New visitor metrics: total visits and views per visit
  • Period comparisons feature
  • Ability to change domain names
  • Automatic geolocation database downloads
  • New email adapters: Mailgun, Mandrill, SendGrid

Breaking Changes

  • Page filter now treated as entry page filter for bounce_rate
  • Rejection of events with long URIs and data URIs
  • Registration disabled by default for self-hosted setups

New Features

  • with_imported=true support in Stats API
  • tagged-events script extension
  • LOG_FAILED_LOGIN_ATTEMPTS environment variable
  • MAILER_NAME and MAILGUN_BASE_URI support
  • Landing page for self-hosters
  • IPv6 support for clickhouse repo
  • conversion_rate metric in visitors.csv
  • views_per_visit metric
  • entry_page and exit_page updates
  • bounce_rate percentage change calculation update
  • Matomo referrer spam list integration
  • Goals API (PUT/DELETE)
  • Data export to zip file
  • Custom styles for embedded dashboards

Full Release Notes

The highlights of this release are:

- more visitor metrics like total visits and views per visit
- period comparisons [[docs]](https://plausible.io/docs/guided-tour#compare-your-stats-over-time) [[example]](https://plausible.io/plausible.io?comparison=previous_period)
- ability to change domains names https://github.com/plausible/analytics/pull/2803
- automatic geolocation database downloads  plausible/analytics#2362
- more email adapters support (Mailgun, Mandrill, SendGrid) plausible/analytics#2649

### Upgrading Plausible Analytics to v2.0

>  [!WARNING]
> 
> This guide assumes you are running v1.5.1.
> If you are upgrading from an earlier version, you might encounter the error reported (and resolved) in https://github.com/plausible/analytics/issues/4779

>  [!WARNING]
> 
> Upgrading to `v2.0` requires performing a data migration.
> Please read these notes until the end before deploying `v2.0.0`

#### Ensure you are using a new ClickHouse version

The steps below have been tested with [`clickhouse/clickhouse-server:22.6-alpine`](https://github.com/plausible/hosting/blob/3e1462eabd510cc9320a6cfed0ae45355d86bb05/docker-compose.yml#L16-L26) please make sure to upgrade ClickHouse to at least this version.

<details>
<summary>Here's the excerpt from <a href="https://github.com/plausible/analytics/discussions/2507"><code>v1.5 release discussion</code></a> regarding the ClickHouse upgrade</summary>

---

In your docker-compose.yml update the image used for plausible_events_db to a newer ClickHouse version:

### Upgrading ClickHouse to 22.6

```diff
plausible_events_db:
- image: yandex/clickhouse-server:21.3.2.5
+ image: clickhouse/clickhouse-server:22.6-alpine
```

#### Restart the container

```console
$ docker compose stop plausible plausible_events_db
$ docker compose rm plausible_events_db
$ docker compose up -d
```

This will boot up the new version of ClickHouse.

Related PR: https://github.com/plausible/hosting/pull/45

---

</details>

#### Update image tag

In your `docker-compose.yml` update the image used for `plausible` to `v2.0.0`

```diff
plausible:
- image: plausible/analytics:v1.5.1
+ image: plausible/analytics:v2.0.0
```

and restart the container

```console
$ cd hosting
$ docker compose stop plausible
$ docker compose rm plausible
$ docker compose up -d
```

This will boot up the new version of the app.

If you open the dashboards now, you wouldn't see any past metrics. This is expected as `v2.0` uses the new `events_v2` and `sessions_v2` tables to store analytics data. We need to perform data migration to copy the data into the new tables.

#### Run data migration

> ⚠️ ⚠️ ⚠️
> 
> You need enough free disk space available for x2 of the current `plausible_events_db`'s `event-data` volume size. You can use something like `docker system df -v | grep hosting_event-data` to check how much space the current volume is occupying.

Connect to the running `plausible` container and start the migration flow

```console
$ cd hosting
$ docker compose exec plausible bin/plausible rpc Plausible.DataMigration.NumericIDs.run
```

You can attempt this migration multiple times unless you drop `v1` tables.

#### Drop v1 tables (optional)

Once you **verify** the migration went **well,** the old tables can be dropped. It's easiest to use `clickhouse-client` for this

```console
$ ch hosting
$ docker compose exec plausible_events_db clickhouse-client
```
```sql
:) \c plausible_events_db
:) show tables;
:) drop table events;
:) drop table sessions;
```

See https://clickhouse.com/docs/en/operations/server-configuration-parameters/settings#max-table-size-to-drop for how to drop tables with more than 50GB of data.

#### Enable automatic MaxMind GeoLite2 updates (optional)

In your `plausible-config.env` set [`MAXMIND_LICENSE_KEY`](https://www.maxmind.com/en/accounts/current/license-key) environment variable and get an automatically updated [GeoLite2 City](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data) geolocation database. The database edition is configurable with `MAXMIND_EDITION` environment variable and defaults to `GeoLite2-City`.

```diff
- IP_GEOLOCATION_DB=...
+ MAXMIND_LICENSE_KEY=LNpsJCCKPis6XvBP
+ MAXMIND_EDITION=GeoLite2-City
```

Note that for the changes in `plausible-config.env` to propagate to the `plausible`, the container needs to be recreated:

```console
$ docker compose stop plausible
$ docker compose rm plausible
$ docker compose up -d
```

Also note that using `GeoLite2-City` edition requires more RAM than using `GeoLite2-Country`.

Now you can remove any other volumes and services used to download, store, and update geolocation databases.

## Changelog

Following [changes](https://github.com/plausible/analytics/compare/v1.5.1...v2.0.0) have been made since v1.5:

### Added
- Add support for `with_imported=true` in Stats API aggregate endpoint
- Ability to use '--' instead of '=' sign in the `tagged-events` classnames
- 'Last updated X seconds ago' info to 'current visitors' tooltips
- Add support for more Bamboo adapters, i.e. `Bamboo.MailgunAdapter`, `Bamboo.MandrillAdapter`, `Bamboo.SendGridAdapter` plausible/analytics#2649
- Ability to change domain for existing site (requires numeric IDs data migration, instructions will be provided separately) UI + API (`PUT /api/v1/sites`)
- Add `LOG_FAILED_LOGIN_ATTEMPTS` environment variable to enable failed login attempts logs plausible/analytics#2936
- Add `MAILER_NAME` environment variable support plausible/analytics#2937
- Add `MAILGUN_BASE_URI` support for `Bamboo.MailgunAdapter` plausible/analytics#2935
- Add a landing page for self-hosters plausible/analytics#2989
- Allow optional IPv6 for clickhouse repo plausible/analytics#2970

### Fixed
- Fix tracker bug - call callback function even when event is ignored
- Make goal-filtered CSV export return only unique_conversions timeseries in the 'visitors.csv' file
- Stop treating page filter as an entry page filter
- City report showing N/A instead of city names with imported data plausible/analytics#2675
- Empty values for Screen Size, OS and Browser are uniformly replaced with "(not set)"
- Fix [more pageviews with session prop filter than with no filters](https://github.com/plausible/analytics/issues/1666)
- Cascade delete sent_renewal_notifications table when user is deleted plausible/analytics#2549
- Show appropriate top-stat metric labels on the realtime dashboard when filtering by a goal
- Fix breakdown API pagination when using event metrics plausible/analytics#2562
- Automatically update all visible dashboard reports in the realtime view
- Connect via TLS when using HTTPS scheme in ClickHouse URL plausible/analytics#2570
- Add error message in case a transfer to an invited (but not joined) user is requested plausible/analytics#2651
- Fix bug with [showing property breakdown with a prop filter](https://github.com/plausible/analytics/issues/1789)
- Fix bug when combining goal and prop filters plausible/analytics#2654
- Fix broken favicons when domain includes a slash
- Fix bug when using multiple [wildcard goal filters](https://github.com/plausible/analytics/pull/3015)
- Fix a bug where realtime would fail with imported data
- Fix a bug where the country name was not shown when [filtering through the map](https://github.com/plausible/analytics/issues/3086)

### Changed
- Treat page filter as entry page filter for `bounce_rate`
- Reject events with long URIs and data URIs plausible/analytics#2536
- Always show direct traffic in sources reports plausible/analytics#2531
- Stop recording XX and T1 country codes plausible/analytics#2556
- Device type is now determined from the User-Agent instead of window.innerWidth plausible/analytics#2711
- Add padding by default to embedded dashboards so that shadows are not cut off plausible/analytics#2744
- Update the User Agents database (https://github.com/matomo-org/device-detector/releases/tag/6.1.1)
- Disable registration in self-hosted setups by default plausible/analytics#3014

### Removed
- Remove Firewall plug and `IP_BLOCKLIST` environment variable
- Remove the ability to collapse the main graph plausible/analytics#2627
- Remove `custom_dimension_filter` feature flag plausible/analytics#2996