Skip to content

Analytics: Track feature adoption with setting values and a config snapshot - #1370

Draft
gabrielcld2 wants to merge 4 commits into
developfrom
feature/WPP-1242-feature-adoption-analytics
Draft

gabrielcld2 wants to merge 4 commits into
developfrom
feature/WPP-1242-feature-adoption-analytics

Conversation

@gabrielcld2

@gabrielcld2 gabrielcld2 commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Makes feature adoption measurable: which delivery features each install runs, and with which values.

The problem

  • settings_saved only carries changed_keys, not the new values.
  • Features left at their defaults are never saved, so they are never reported.
  • As a result there is no way to tell what share of installs use lazy loading, responsive images, AVIF or adaptive streaming.

Approach

Values on settings_saved

  • A new changed_values object sits next to changed_keys, e.g. { "image_quality": "80", "image_format": "avif" }.
  • Values are only sent for an allowlist, Analytics::FEATURE_SETTINGS, keyed by settings page: image, video, lazy loading and responsive images. Everything else, including the connection string, is never sent. changed_values is left out when no allowlisted setting changed.
  • Values are read back after the save, so they reflect any cloudinary_settings_save_setting filters rather than what was submitted.
  • Values are normalised: toggles to booleans, numbers to ints, dropdowns to strings. A missing value is left out rather than reported as off.
  • The lazy-loading threshold is a free-text field, so it is only sent when it is a CSS length such as 100px.
  • The free transformation fields (image_freeform, video_freeform) can contain public IDs, overlay text or URLs, so they are only sent as a presence boolean.
  • crop_sizes is sent as the number of image sizes with a custom crop.

One-time feature_config_snapshot event (category features)

  • Sends the normalised value of every allowlisted setting, including defaults that were never saved.
  • Fires on the first admin page load by an administrator once the site is connected, never from admin-ajax requests. An autoloaded option stores the cloud name it was sent for, so it fires once per account and checking it costs no query.
  • The first send is gated with add_option(), so concurrent first requests can't both send it.
  • That covers wizard setups, sites connected through the CLOUDINARY_CONNECTION_STRING constant, account switches, and existing installs upgrading to this version (the baseline backfill).
  • It doesn't fire while analytics are disabled through cloudinary_analytics_enabled, so it is still sent if they are turned on later.
  • The one-time analytics flags (this one and the existing first_api_consumption one) are now removed by the uninstall clean-up, so a reinstall starts fresh.

Open questions

These may still change the payload. Keeping this as a draft until they are answered:

  1. Can the collector and dashboard read the nested changed_values object, or should it be flat prefixed fields, or one event per changed setting?
  2. Should the field names be the plugin's setting names (enable_breakpoints, adaptive_streaming_mode) as now, or the names proposed in the request (responsive, "mode")?
  3. Are the presence flag for freeform fields and the custom-crop count enough?
  4. Should the snapshot say whether it came from a new setup or an upgrade?
  5. Sites connected through the constant may send the snapshot before the wizard's choices are saved. Is that acceptable?
  6. "Max images" (breakpoints) has no default, so it is missing from the snapshot until a site sets it. Should it get a default?
  7. Should the snapshot report what is configured, or what is actually in effect? They differ when a parent feature is off (e.g. image optimisation settings while image delivery is off) or when code turns a feature off through a filter.

QA notes

  • npm run test:unit -- --filter Test_Analytics_Feature covers the allowlist, each kind of normalisation, and the snapshot rules: once per account, re-sent after an account switch, not sent while disconnected, with analytics off, for non-administrators or from admin-ajax, and only one of several concurrent first requests sending it.
  • tests/e2e/settings-analytics.spec.js checks changed_values on a save of each feature page: image, video, lazy loading and responsive.
  • tests/e2e/features-analytics.spec.js checks the snapshot's admin wiring. It is tagged @serial because its one-time flag is shared by the whole site.
  • Manual check:
    1. On a connected site, run wp option delete _cloudinary_feature_snapshot_emitted.
    2. Load any admin page twice. There should be exactly one feature_config_snapshot event, with the current feature values.
    3. Change settings on the Image, Video, Lazy loading and Responsive pages. Each settings_saved event should carry the new values under changed_values.
    4. Save the connection page. No connection details should appear in any event: if a settings_saved event is sent, it has no changed_values.

…fig snapshot

`settings_saved` only reported which keys changed, and features left at
their defaults were never reported at all, so adoption of lazy loading,
responsive images, formats or adaptive streaming couldn't be measured.

- `settings_saved` now carries `changed_values` for an allowlist of image,
  lazy loading, responsive and video settings. Anything outside the list,
  including the connection string, is never sent.
- Values are normalised: toggles to booleans, numbers to ints, free-text
  transformations to a presence flag, and crop sizes to a count of custom
  crops.
- A one-time `feature_config_snapshot` sends the full feature config on
  the first admin load once connected, once per cloud name. It covers
  wizard setups, constant-based connections, account switches and
  existing installs upgrading.
- Send the snapshot only on admin page loads by administrators, never from
  admin-ajax requests or for other roles.
- Autoload the one-time flag so checking it costs no query, and gate the
  first send with add_option() so concurrent requests can't both send it.
- Leave missing setting values out instead of reporting them as off.
- Send the lazy-loading threshold only when it is a CSS length; it is a
  free-text field.
- Allowlist feature settings per settings page.
- Report `changed_values` as read back after the save, so they reflect
  any save filters.
- Remove the analytics one-time flags on uninstall.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants