feat: what a real application needed and the gem did not give it

.rubocop.yml

@@ -33,6 +33,7 @@ Metrics/ParameterLists:
   CountKeywordArgs: false
 
 Metrics/ClassLength:
+  Max: 100
   Exclude:
     # Test classes are mostly tables of cases.
     - test/**/*

CHANGELOG.md

@@ -18,8 +18,76 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
   `config.validate_languages = false` globally. See
   [Languages](docs/languages.md).
 
+- **Cache keys change for any document containing a `pre` or `code`
+  element.** `pre` and `code` are now opaque (see Added, below), so what
+  gets sent to the provider changed, and what gets keyed changed with it;
+  an entry cached under the old behaviour keeps serving what the old
+  behaviour produced. Give the configuration a new `cache_namespace`, or
+  let `cache_ttl` lapse, to get every such document retranslated. See
+  [Caching](docs/caching.md#what-a-cache-key-is-made-of).
+
+- **A runtime `cache_namespace` change now moves the rate limiter too.**
+  `cache_namespace` names the limiter's own bookkeeping namespace as well
+  as the cache store's; it used to move only the store, leaving the
+  limiter counting silently under the old namespace. `active_record_base`
+  behaves the same way and for the same reason -- the SQL-backed limiter
+  builds its model from that class just as the store does. See
+  [Configuration](docs/configuration.md#changing-configuration-at-runtime).
+
+- **A provider with a blank `cache_key` now raises
+  `TranslationDiff::InvalidProviderError`, not
+  `TranslationDiff::Translator::Error`.** The two are siblings under
+  `TranslationDiff::Error`, not parent and child, so an application
+  rescuing the old class specifically stops catching this failure.
+  Rescue `TranslationDiff::Error` to catch both. See
+  [Errors](docs/errors.md).
+
 ### Added
 
+- **Every event from one `translate` call now shares a `call_id`.** Generated
+  once per call, opaque, and never derived from the text, it lands in
+  `translate`, `cache`, `request`, `rate_limit`, `usage` and `cache_error`
+  alike. Before it, a subscriber receiving `cache` or `request` events had no
+  way to tell which `translate` call they belonged to, short of tagging
+  `Thread.current` itself -- a workaround that breaks the moment two
+  translations share a thread. See
+  [Instrumentation](docs/instrumentation.md).
+
+- **`translate` now carries `characters`: the total this call considered,
+  hit or miss.** A call served entirely from cache never fires a `request`
+  event and used to report nothing about its size; it now reports a number
+  there instead. `request`'s own `characters` keeps its narrower meaning --
+  what one batch actually sent -- so the two fields share a name but not an
+  event: summing the wrong one produces a wrong bill. See
+  [Instrumentation](docs/instrumentation.md).
+
+- **`TranslationDiff.preview` predicts a `translate` call without making
+  it.** It answers how many sentences a call would send, how many the cache
+  already has, and how many characters that is -- without calling a
+  provider and without writing anything. A preview never pays for language
+  detection, so `from:` is required wherever there is anything to preview;
+  leaving it unset raises `TranslationDiff::Previewer::Error`. Built for an
+  editor that wants to show "this edit will send 1 sentence" before the
+  author saves. See
+  [Caching](docs/caching.md#asking-what-a-call-would-do-without-doing-it).
+
+- **`pre` and `code` are no longer sent for translation, and changing a
+  configuration option at runtime now rebuilds only what it actually
+  feeds.** `pre` and `code` join `script` and `style` in
+  `config.opaque_elements`, the set `TranslationDiff::Passage` never treats
+  as prose -- default `%i[script style pre code]`, widen or narrow it as
+  needed -- after a live `<pre><code>` block came back from Google with
+  `jq '.meters'` mangled into `jq '.metros'`, because nothing told the
+  pipeline that code holds language, not prose. Separately, `provider` and
+  the cache, pool, rate and segmenter options each rebuild only their own
+  collaborator now, instead of nothing: before this, switching `provider`
+  at runtime meant `TranslationDiff.reset!` and reconfiguring from scratch,
+  discarding a Redis pool that had no reason to go. See
+  [Configuration](docs/configuration.md#changing-configuration-at-runtime).
+  Two upgrade consequences of this are filed under Breaking, above:
+  cache keys changing for a document containing `pre` or `code`, and a
+  runtime `cache_namespace` change now moving the rate limiter too.
+
 - A `usage` instrumentation event, firing once per provider request, beside
   `translate`, `cache`, `request` and `rate_limit`. Its payload carries
   `provider`, `characters` (what this library sent, counted locally),
@@ -143,6 +211,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ### Fixed
 
+- **A `notranslate` span nested inside an opaque element (`pre`, `code`,
+  `script` or `style`) no longer silences every sentence after it.**
+  Closing the protected span used to leave the scanner's own opacity depth
+  one too high, so nothing past it was ever handed to the segmenter again.
+  Found while adding the `pre`/`code` opaque elements above, and fixed the
+  same way for all four. See [How it works](docs/how-it-works.md#html).
 - **Google and DeepL translations in HTML mode no longer come back
   double-escaped.** Both vendors return entity-escaped text -- an
   apostrophe as `&#39;`, a quote as `&quot;`, an ampersand as `&amp;` -- and

README.md

@@ -67,6 +67,13 @@ end
 formal.translate(contract, from: "en", to: "de", formality: :more)
 ```
 
+```ruby
+# See what a call would send and find cached, without calling the provider or writing anything
+preview = TranslationDiff.preview(contract, from: "en", to: "de")
+preview.sendable_sentences # => sentences not yet cached
+preview.cached_sentences   # => sentences already cached
+```
+
 ```ruby
 # Redis-backed cache, shared across processes
 TranslationDiff.configure { |config| config.redis_url = ENV["REDIS_URL"] }

docs/caching.md

@@ -58,6 +58,48 @@ you nothing.
 No options at all contributes no field to the key, which is the four-field
 key every already-warm cache is keyed on.
 
+## Asking what a call would do, without doing it
+
+`TranslationDiff.preview` answers what a `translate` call would send and
+find cached, without calling a provider and without writing anything: how
+many sentences it would send, how many the cache already has, and how many
+characters that is. It reads the same store, through the same
+`SentenceCache`, keyed the same way -- see [What a cache key is made
+of](#what-a-cache-key-is-made-of) above -- so a preview and the call it
+predicts always agree.
+
+```ruby
+preview = TranslationDiff.preview(article_body, from: "en", to: "es")
+preview.sendable_sentences   # => 1, not yet cached
+preview.cached_sentences     # => 4, already cached
+preview.sendable_characters  # => 23
+preview.characters           # => 412, the total this call would consider
+```
+
+`sendable_sentences` and `cached_sentences` are the same two counts the
+`cache` event reports as `misses` and `hits`; `characters` is the same total
+the `translate` event reports. A preview and the call it predicts are
+answering the same question through the same numbers, so "this edit will
+send 23 of 412 characters" and what the events for that call later report
+should agree.
+
+**`from:` is required wherever there is anything to preview.** `translate`
+can leave `from:` unset and pay for one `#detect` request to find it; a
+preview never calls the provider, so it cannot pay for that request either.
+Passing `to:` alone raises `TranslationDiff::Previewer::Error`, naming the
+provider and telling you to pass `from:` explicitly -- unless the document
+holds nothing translatable, or the source and target already match, in
+which case there is nothing to preview and an empty result comes back
+regardless of `from:`.
+
+This is the supported way to ask an editor's question before it becomes a
+bill -- show "this edit will send 1 sentence" before the author saves:
+
+```ruby
+preview = TranslationDiff.preview(edited_body, from: "en", to: "es")
+"This edit will send #{preview.sendable_sentences} sentence#{'s' unless preview.sendable_sentences == 1}."
+```
+
 ## The cache store contract
 
 `config.cache` accepts either a registered name (`:redis`, `:memory`,

docs/configuration.md

@@ -69,6 +69,7 @@ at all, so an unset environment variable never has to be special-cased.
 | `rate_interval` | `60` | Seconds over which `rate_limit` (or a limiter's own default threshold) is measured. **Actually enforced over roughly 5-600 seconds** -- see [The rate limiter contract](contracts.md#the-rate-limiter-contract). |
 | `rate_limiter` | `nil` | A registered name (`:redis`, `:active_record`) or an object satisfying the [rate limiter contract](contracts.md#the-rate-limiter-contract). `nil` with `rate_limit` also `nil` means no rate limiting; `nil` with `rate_limit` set resolves to `:redis`. Setting `rate_limiter` alone -- with `rate_limit` left unset -- is enough to turn rate limiting on, at the limiter's own default threshold; it no longer needs `rate_limit` set to avoid crashing. |
 | `segmenter` | `:pragmatic` | The sentence segmenter: a registered name or an object satisfying the [segmenter contract](contracts.md#the-segmenter-contract). |
+| `opaque_elements` | `%i[script style pre code]` | Element names `TranslationDiff::Passage` never treats as prose, whatever they contain, matched case-insensitively so `<PRE>` and `<STYLE>` count too. Read fresh on every passage rather than memoised, so a runtime change applies immediately, to the next translation -- and, since a call reads this from the configuration it is actually using, a context's own `opaque_elements` is honoured too, not the global value it was copied from. See [How it works](how-it-works.md#html). |
 | `instrumenter` | `nil` | Anything satisfying `ActiveSupport::Notifications`' `#instrument(name, payload) { }` interface. See [Instrumentation and logging](instrumentation.md). |
 | `logger` | `nil` | A standard `Logger` -- anything answering to `debug` and `warn` with a block. Receives one `debug` line per provider resolution, naming the provider class, and a `warn` line when a cache write fails; never content and never a credential. Note that `warn` must be a public method: a bare object inherits a private `Kernel#warn` and would raise instead of logging. See [Instrumentation and logging](instrumentation.md). |
 | `open_timeout` | `5` | Seconds an HTTP-backed provider waits to open a connection before raising `TranslationDiff::TransportError`. |
@@ -102,18 +103,59 @@ Every provider declares its own configuration options, registered the moment
 A provider you register yourself can declare its own options the same way --
 see [Writing a provider](providers.md#writing-a-provider) below.
 
-## Configure once, before the first translation
-
-**Configure once, before the first translation.** `provider`, `cache`,
-`segmenter` and `rate_limiter` each resolve to a collaborator on first use
-and that collaborator is memoised for the life of the configuration. Options
-stay writable afterwards, but changing one no longer reaches an object that
-has already been built: setting `cache_max_size` after something has
-translated leaves the store built with the old bound in place, and
-reassigning `provider` after a translation has run does not change the
-provider that configuration uses. `TranslationDiff.context` -- or
-`config.copy`, which it is built on -- is the way to get a configuration that
-resolves everything afresh from its own values.
+## Changing configuration at runtime
+
+`provider`, `cache`, `segmenter` and `rate_limiter` each resolve to a
+collaborator on first use, and that collaborator is memoised. Writing an
+option afterwards rebuilds only the memoised collaborator(s) that option
+actually feeds, not the whole configuration:
+
+- `provider`, any option a provider declares for itself (`deepl_api_key`
+  and the like), and the timeouts (`open_timeout`, `timeout`,
+  `max_retries`) rebuild the provider. An HTTP-backed provider memoises a
+  Faraday connection built from the three timeouts, so a change to any of
+  them has to reach the provider or it never reaches the connection.
+- The cache options (`cache`, `cache_ttl`, `cache_max_size`,
+  `cache_table_name`, `active_record_base`, `cache_prune_probability`,
+  `cache_namespace`) rebuild the cache store.
+- `redis_url`, `redis_pool_size` and `redis_pool_timeout` rebuild the
+  connection pool and everything holding it -- the cache store and the rate
+  limiter both.
+- The rate options (`rate_limit`, `rate_interval`, `rate_limiter`,
+  `rate_limit_table_name`), `cache_namespace` and `active_record_base`
+  rebuild the rate limiter.
+- `segmenter` rebuilds the segmenter.
+- `logger` and `instrumenter` rebuild nothing -- both are read live, on
+  every use, and nothing memoised reads either one.
+
+Before this, nothing was ever rebuilt: an application wanting to switch
+`provider` at runtime had no way to do it short of `TranslationDiff.reset!`
+and reconfiguring from scratch, which also threw away a Redis pool, and
+everything built from it, that had no reason to go.
+
+Two options carry further than their names suggest, and both rebuild the
+rate limiter as well as the store. `cache_namespace` names the limiter's own
+bookkeeping namespace, so changing it at runtime moves the limiter too --
+it counts under the new namespace from the next check on, rather than
+continuing silently under the old one. `active_record_base` is the class the
+SQL-backed limiter builds its model from just as the SQL-backed store does,
+so pointing it at another database moves both.
+
+**A write that leaves an option at the value it already holds rebuilds
+nothing.** Only a value that actually changes invalidates a memoised
+collaborator, even one the option is declared to invalidate. This is what
+makes a per-request `TranslationDiff.configure { |c| c.cache_namespace =
+current_tenant }` safe: writing the same tenant on every request no longer
+rebuilds the cache store on every request. Writing a genuinely *different*
+value still rebuilds the store exactly as before, though, and if that store
+is the default `MemoryCacheStore`, a rebuilt store is a fresh, empty Hash --
+its contents are gone, and whatever it held has to be paid for again at the
+provider.
+
+`TranslationDiff.context` -- or `config.copy`, which it is built on -- is
+still the way to get a configuration that resolves everything afresh from
+its own values, independently of whatever the configuration it was copied
+from has already built.
 
 ## Choosing the cache store
 
@@ -125,8 +167,8 @@ object) to override that choice.
 ## Contexts
 
 `TranslationDiff.context` returns a `TranslationDiff::Context`: an isolated
-configuration scope with the same `#translate` entry point as the
-`TranslationDiff` module itself, for multi-tenant applications and
+configuration scope with the same `#translate` and `#preview` entry points
+as the `TranslationDiff` module itself, for multi-tenant applications and
 per-request overrides. It starts from a copy of the global configuration, so
 it inherits every value already set, and changes made inside it never touch
 the global configuration:

docs/errors.md

@@ -22,13 +22,17 @@ TranslationDiff::Error
 │                                                # the wrong number of translations, or one
 │                                                # that returned no translation for an input
 ├── TranslationDiff::InvalidProviderError       # a class registered without inheriting
-│                                                # TranslationDiff::Provider
+│                                                # TranslationDiff::Provider, or any
+│                                                # provider -- registered or assigned --
+│                                                # whose cache_key is blank
 ├── TranslationDiff::UnsupportedLanguageError   # the shipped language data doesn't list
 │                                                # this source/target pair for this
 │                                                # provider -- see docs/languages.md
 ├── TranslationDiff::Translator::Error          # from: missing and the provider cannot
-│                                                # detect, cache_key missing on an
-│                                                # assigned provider object
+│                                                # detect
+├── TranslationDiff::Previewer::Error           # the same, asked of a preview: detecting
+│                                                # a language is a paid request, and a
+│                                                # preview never makes one
 ├── TranslationDiff::SentenceCache::Error       # provider options have no stable
 │                                                # serialisation for the cache key
 ├── TranslationDiff::Batch::Error               # one sentence, once escaped, is larger
@@ -82,9 +86,9 @@ even in a batch of its own. The message names a short prefix of the
 offending text and both numbers.
 
 `ArgumentError`, not a `TranslationDiff::Error`, is what
-`TranslationDiff.translate` and `Context#translate` raise when `to:` is
-missing or `nil`. It is a caller's mistake before it is a translation, and
-the message names the keyword.
+`TranslationDiff.translate`, `TranslationDiff.preview` and their `Context`
+counterparts raise when `to:` is missing or `nil`. It is a caller's mistake
+before it is a translation, and the message names the keyword.
 
 **Renamed in 3.1.0.** `TranslationDiff::Request::Error` is now
 `TranslationDiff::Translator::Error` and `TranslationDiff::Cache::Error` is