fix(web): mask the Config Editor's secrets; keep disabled plugins' rotation slot and Vegas exclusion; restore only missing plugins

CHANGELOG.md

@@ -540,6 +540,42 @@ policies are unchanged.
   notice is what shows next, and Vegas resumes after it; before, a rotation
   screen showed instead and the notice expired behind it. An active
   on-demand session still holds the panel until it ends.
+- The Config Editor tab no longer shows API keys and tokens in plain
+  text. Its `config_secrets.json` editor (`/partials/raw-json`) was filled
+  with the file as it is on disk, so while the web login is off (the
+  default) anyone who could reach the port could read every credential,
+  although `GET /api/v3/config/secrets` masks them. The editor now shows the
+  same masked values. Saving it unchanged changes nothing, because the save
+  drops the masks and merges onto the stored file; to change a secret,
+  replace its mask. A list of secrets still needs every entry's real value
+  to be changed. The `config.json` editor is unchanged: its save writes the
+  file as given, so a mask there would be stored.
+- A disabled plugin keeps its place in the rotation order and its Vegas
+  exclusion when the Display or Rotation & Durations tab is saved. The order
+  lists show enabled plugins only and rewrite their hidden inputs from those
+  rows as soon as they are drawn, so any save of either tab stored the lists
+  without the disabled plugin. Once re-enabled, it came back at the end of
+  the rotation and scrolling in Vegas again. A disabled plugin's saved id
+  now stays in its saved place (`widgets/plugin-order-list.js`); the id of
+  a plugin that is no longer installed is still dropped.
+- Restoring a backup with "Reinstall missing plugins" installs only the
+  plugins that are missing. Every plugin the backup listed was sent to the
+  store's install, which replaces an installed copy with a fresh download,
+  so a restore onto the same device re-downloaded all of them in one
+  request. A plugin installed from its own URL is not in the registry, so
+  its "reinstall" failed and the restore answered "Restore failed" while
+  the plugin sat there installed. An installed plugin, found by the store's
+  own lookup (registry aliases included), is now listed under Skipped as
+  `plugin:<id> (installed)`.
+- `POST /api/v3/config/main` answers a JSON body that does not parse with
+  400 `Invalid JSON in request body`, as `/config/raw/main` does, and an
+  empty JSON body with 400 `No data provided`. Both were a 500
+  `CONFIG_SAVE_FAILED` suggesting file permissions and disk space, with a
+  traceback logged at ERROR: `get_json()` raised inside the handler's
+  catch-all.
+- Fonts restored from a backup show up in the Fonts tab and the font
+  pickers straight away. The font catalog is cached for five minutes, and
+  upload and delete cleared it but a restore did not.
 - A game that goes live now takes over the panel within about a second.
   Live priority was only checked between screens, so a game that went live
   during a 30 s screen waited for that screen to end. The frame loops and the

src/backup_manager.py

@@ -213,8 +213,9 @@ def list_installed_plugins(project_root: Path) -> List[Dict[str, Any]]:
     The plugins are the ``manifest.json`` files in the configured plugin
     directory (see :func:`_plugins_directory`), with the manifest's version;
     ``enabled`` is config.json's flag by the display's rule (a missing flag
-    is disabled). A restore reinstalls every listed plugin and takes enabled
-    state from the restored config.json, so ``enabled`` is informational.
+    is disabled). A restore installs each listed plugin that is missing and
+    takes enabled state from the restored config.json, so ``enabled`` is
+    informational.
 
     ``data/plugin_state.json`` is not read: it only ever repeated config's
     enabled flags and the manifests' versions, and is retired (nothing

test/js/README.md

@@ -46,6 +46,7 @@ server has none.
 | `unit/test_list_filter.js` | no | `ListFilter` search/filter/sort/count/sticky, and the installed-plugins config **extracted verbatim** from `plugins_manager.js` so the test can't drift from it |
 | `unit/test_update_all.js` | no | `PluginInstallManager.updateAll` from `plugins/install_manager.js`: Check & Update All sends only plugin ids (never `starlark:` app entries), re-sends a request that got no HTTP answer (web service restarting) instead of skipping that plugin, never re-sends one that got any HTTP answer (the real `api_client.js` classifies a proxy 502 or a JSON error without `error_code` as `API_ERROR`), and counts a no-op update as already up to date in the summary. Also run by `test/web_interface/test_update_all_plugins.py` so CI covers it |
 | `unit/test_render_cards.js` | no | `renderInstalledCards` markup, both empty states, and HTML-escaping of hostile plugin metadata |
+| `unit/test_plugin_order_list.js` | no | `widgets/plugin-order-list.js` (the Vegas and rotation order lists): a disabled plugin, which gets no row, keeps its slot in the saved order and its Vegas exclusion when the list rewrites its hidden inputs, around reordering and include/exclude; an uninstalled plugin's id is dropped, a failed plugin list leaves the inputs as saved, and only string ids are carried over, once each |
 | `unit/test_style_editor_element_keys.js` | no | `elementKeys()`/`styleRows()`/`positionRows()` from `widgets/style-editor.js`: every `customization.layout` entry gets exactly one row -- paired with its style element through core's `x-layout-key` (so `score` belongs to `score_text`, not a second row), or a position row of its own, leaves included -- since the widget claims the whole `layout` block from the generic fallback renderer |
 | `unit/test_style_editor_layout_leaf_columns.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only key whose own value is a leaf (no x/y sub-object, e.g. a `show_logo` toggle) gets a self-keyed column instead of a blank, uneditable row |
 | `unit/test_style_editor_layout_leaf_collision.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only leaf key still gets its own column even when its name collides with an unrelated element's style sub-field or another layout axis's sub-field |

test/js/dom/test_durations_page.js

@@ -98,7 +98,11 @@ const ok = (l, c, x) => c ? (pass++, console.log('  ok   ' + l))
 
   const lists = () => requests.filter(r => r.url === '/api/v3/plugins/installed').length;
   const $ = id => doc.getElementById(id);
-  const order = () => JSON.parse($('rotation_plugin_order_value').value || '[]');
+  // The rows' ids, in order. The input also keeps saved ids that have no row
+  // (a disabled plugin's place, see test/js/unit/test_plugin_order_list.js),
+  // and the saved order comes from whatever config the server has.
+  const SHOWN = plugins.filter(p => p.enabled).map(p => p.id);
+  const order = () => JSON.parse($('rotation_plugin_order_value').value || '[]').filter(id => SHOWN.includes(id));
   async function swap() {
     panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
     panel.innerHTML = partial;

test/js/run_all.js

@@ -17,6 +17,7 @@ const fs = require('fs');
 
 const BASE = process.env.BASE || 'http://localhost:5000';
 const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
+              'unit/test_plugin_order_list.js',
               'unit/test_html_escaping.js', 'unit/test_style_editor_element_keys.js',
               'unit/test_style_editor_layout_leaf_columns.js',
               'unit/test_style_editor_layout_leaf_collision.js',

test/js/unit/test_plugin_order_list.js

@@ -0,0 +1,189 @@
+// The shared plugin order list (widgets/plugin-order-list.js) keeps what it
+// does not show.
+//
+// It lists enabled plugins only, and rewrites its hidden inputs from those
+// rows as soon as it has drawn them. A disabled plugin's place in the order
+// and its Vegas exclusion used to vanish from the inputs on that rewrite, so
+// any later save of the Display or Rotation & Durations tab stored them
+// without it: re-enabled, the plugin came back at the end of the rotation and
+// scrolling in Vegas again. An uninstalled plugin's id is still dropped, as
+// before, so the lists don't collect ids nothing can show. Runs the shipped
+// widget in a vm with a minimal fake DOM -- no jsdom and no server needed, so
+// it runs under test/test_js_unit_suites.py too.
+
+const fs = require('fs');
+const path = require('path');
+const vm = require('vm');
+const WIDGET = path.resolve(__dirname, '../../../web_interface/static/v3/js/widgets/plugin-order-list.js');
+
+let pass = 0, fail = 0;
+const ok = (label, cond, extra) => cond
+  ? (pass++, console.log('  ok   ' + label))
+  : (fail++, console.log('  FAIL ' + label + (extra !== undefined ? '  ' + JSON.stringify(extra) : '')));
+const same = (a, b) => JSON.stringify(a) === JSON.stringify(b);
+
+class FakeElement {
+  constructor(tag) {
+    this.tagName = tag.toUpperCase();
+    this.children = [];
+    this.parent = null;
+    this.dataset = {};
+    this.style = {};
+    this.className = '';
+    this.value = '';
+    this.checked = false;
+    this.listeners = {};
+    this._text = '';
+  }
+  appendChild(child) {
+    if (child.parent) child.parent.children = child.parent.children.filter(c => c !== child);
+    child.parent = this;
+    this.children.push(child);
+    return child;
+  }
+  insertBefore(child, ref) {
+    if (!ref) return this.appendChild(child);
+    if (child.parent) child.parent.children = child.parent.children.filter(c => c !== child);
+    child.parent = this;
+    this.children.splice(this.children.indexOf(ref), 0, child);
+    return child;
+  }
+  get previousElementSibling() {
+    const siblings = this.parent ? this.parent.children : [];
+    return siblings[siblings.indexOf(this) - 1] || null;
+  }
+  get nextElementSibling() {
+    const siblings = this.parent ? this.parent.children : [];
+    const i = siblings.indexOf(this);
+    return i < 0 ? null : siblings[i + 1] || null;
+  }
+  set textContent(value) { this._text = value; this.children = []; }
+  get textContent() { return this._text; }
+  setAttribute() {}
+  focus() {}
+  addEventListener(type, fn) { (this.listeners[type] ||= []).push(fn); }
+  fire(type, event) { (this.listeners[type] || []).forEach(fn => fn.call(this, event || {})); }
+  descendants() { return this.children.flatMap(c => [c, ...c.descendants()]); }
+  querySelectorAll(selector) {
+    const cls = selector.replace(/^\./, '');
+    return this.descendants().filter(e => e.className.split(/\s+/).includes(cls));
+  }
+  querySelector(selector) { return this.querySelectorAll(selector)[0] || null; }
+}
+
+/** Run the widget over `plugins` with the given saved inputs; resolves once it has drawn. */
+async function mount({ plugins, order, excluded, fetchFails }) {
+  const els = {
+    list: new FakeElement('div'),
+    order: Object.assign(new FakeElement('input'), { value: JSON.stringify(order) }),
+  };
+  if (excluded !== undefined) {
+    els.excluded = Object.assign(new FakeElement('input'), { value: JSON.stringify(excluded) });
+  }
+  const context = {
+    // The widget logs a failed list; expected there, so kept off the output.
+    console: fetchFails ? Object.assign({}, console, { error: () => {} }) : console,
+    window: {},
+    document: {
+      getElementById: (id) => els[id] || null,
+      createElement: (tag) => new FakeElement(tag),
+      createTextNode: (text) => new FakeElement('#text'),
+    },
+    fetch: () => (fetchFails ? Promise.reject(new Error('service restarting')) : Promise.resolve({
+      json: () => Promise.resolve({ status: 'success', data: { plugins } }),
+    })),
+  };
+  vm.createContext(context);
+  vm.runInContext(fs.readFileSync(WIDGET, 'utf8'), context);
+  context.window.PluginOrderList.init({
+    containerId: 'list', orderInputId: 'order',
+    excludedInputId: excluded !== undefined ? 'excluded' : undefined,
+  });
+  await new Promise(resolve => setTimeout(resolve, 0));
+  const rows = () => els.list.querySelectorAll('.plugin-order-item');
+  return {
+    rows,
+    rowIds: () => rows().map(r => r.dataset.pluginId),
+    order: () => JSON.parse(els.order.value),
+    excluded: () => JSON.parse(els.excluded.value),
+    row: (id) => rows().find(r => r.dataset.pluginId === id),
+  };
+}
+
+const PLUGINS = [
+  { id: 'weather', name: 'Weather', enabled: true },
+  { id: 'clock', name: 'Clock', enabled: false },
+  { id: 'stocks', name: 'Stocks', enabled: true },
+];
+
+(async () => {
+  console.log('\nVegas: a disabled plugin keeps its place and its exclusion');
+  {
+    const t = await mount({ plugins: PLUGINS, order: ['weather', 'clock', 'stocks'], excluded: ['clock'] });
+    ok('only enabled plugins get a row', same(t.rowIds(), ['weather', 'stocks']), t.rowIds());
+    ok('drawing the list keeps the disabled plugin in the order, in its place',
+       same(t.order(), ['weather', 'clock', 'stocks']), t.order());
+    ok('drawing the list keeps its exclusion', same(t.excluded(), ['clock']), t.excluded());
+
+    // Move Stocks up: the rows swap, and Clock stays in its saved slot.
+    const up = t.row('stocks').querySelectorAll('.plugin-order-move')[0];
+    up.fire('click');
+    ok('reordering the rows fills the other slots in the new order',
+       same(t.order(), ['stocks', 'clock', 'weather']), t.order());
+
+    const include = t.row('weather').querySelector('.plugin-order-include');
+    include.checked = false;
+    include.fire('change');
+    ok('unchecking a row adds it, and the disabled exclusion stays',
+       same([...t.excluded()].sort(), ['clock', 'weather']), t.excluded());
+    include.checked = true;
+    include.fire('change');
+    ok('checking it again removes only that one', same(t.excluded(), ['clock']), t.excluded());
+  }
+
+  console.log('\nRotation order: the same, without exclusions');
+  {
+    const plugins = [
+      { id: 'clock', enabled: true },
+      { id: 'off', enabled: false },
+      { id: 'weather', enabled: true },
+      { id: 'new', enabled: true },
+    ];
+    const t = await mount({ plugins, order: ['clock', 'off', 'weather'] });
+    ok('the disabled plugin keeps its slot; a plugin not in the saved order goes last',
+       same(t.order(), ['clock', 'off', 'weather', 'new']), t.order());
+  }
+
+  console.log('\nAn uninstalled plugin is dropped; a failed list keeps everything');
+  {
+    const t = await mount({ plugins: PLUGINS, order: ['weather', 'gone', 'clock', 'stocks'],
+                            excluded: ['gone', 'clock'] });
+    ok('the disabled plugin is kept and the uninstalled one dropped from the order',
+       same(t.order(), ['weather', 'clock', 'stocks']), t.order());
+    ok('and from the exclusions', same(t.excluded(), ['clock']), t.excluded());
+  }
+  {
+    const t = await mount({ plugins: PLUGINS, order: ['weather', 'gone', 'clock', 'stocks'],
+                            excluded: ['gone', 'clock'], fetchFails: true });
+    // No installed list, so nothing can be told apart: no rows, and the
+    // inputs keep what was saved, uninstalled ids included.
+    ok('a failed plugin list draws no rows', t.rowIds().length === 0, t.rowIds());
+    ok('and leaves the saved order as it was',
+       same(t.order(), ['weather', 'gone', 'clock', 'stocks']), t.order());
+    ok('and the saved exclusions', same(t.excluded(), ['gone', 'clock']), t.excluded());
+  }
+
+  console.log('\nOnly what the server would accept is carried over');
+  {
+    const t = await mount({ plugins: PLUGINS, order: ['weather', 7, 'clock', null, 'clock', 'stocks'],
+                            excluded: ['clock', 3, 'clock'] });
+    // /config/main refuses a list holding anything but strings, which would
+    // block every later Display save; a repeated id is kept once.
+    ok('non-string and repeated saved ids are dropped from the order',
+       same(t.order(), ['weather', 'clock', 'stocks']), t.order());
+    ok('and from the exclusions', same(t.excluded(), ['clock']), t.excluded());
+  }
+
+  console.log(`\n${pass} passed, ${fail} failed`);
+  process.exit(fail ? 1 : 0);
+})().catch(e => { console.error(e); process.exit(1); });

test/test_api_v3_partial_main_save.py

@@ -253,6 +253,32 @@ def test_per_mode_duration_saves_and_blank_clears_it(self, api_v3_client, saved,
         assert saved['config']['display']['display_durations'] == {'clock': 45}
 
 
+class TestMalformedBody:
+    """A JSON body that does not parse is the caller's mistake: a 400.
+
+    get_json() raised Werkzeug's BadRequest inside the handler's try, whose
+    catch-all answered 500 CONFIG_SAVE_FAILED with "check file permissions"
+    advice and logged a traceback at ERROR.
+    """
+
+    def test_is_a_400_in_the_raw_routes_shape(self, api_v3_client, saved, api_v3_module):
+        api_v3_module.api_v3.config_manager.get_raw_file_content.return_value = {}
+        resp = api_v3_client.post('/api/v3/config/main', data='{not json',
+                                  content_type='application/json')
+        assert resp.status_code == 400
+        assert resp.get_json() == {'status': 'error', 'message': 'Invalid JSON in request body'}
+        assert 'config' not in saved
+        raw = api_v3_client.post('/api/v3/config/raw/main', data='{not json',
+                                 content_type='application/json')
+        assert (raw.status_code, raw.get_json()) == (400, resp.get_json())
+
+    def test_an_empty_json_post_is_still_no_data(self, api_v3_client, saved):
+        resp = api_v3_client.post('/api/v3/config/main', data='',
+                                  content_type='application/json')
+        assert resp.status_code == 400
+        assert resp.get_json()['message'] == 'No data provided'
+
+
 class TestRawSaveStartsAutoUpdateSetup:
     @pytest.fixture
     def raw_env(self, api_v3_module, monkeypatch):