fix(web-ui): Plugin Manager - enable aliased installs, Update All, on-demand modes, long installs, categories, GitHub-URL install

CHANGELOG.md

@@ -560,6 +560,48 @@ policies are unchanged.
   being stopped, blanks the panel within about a second. It used to stay on
   until the next minute, because the once-a-minute schedule check had
   already run that minute and the session had overridden its answer.
+- Check & Update All updates what is installed now. A second run in the
+  same page sent the plugins the first run had seen, so a plugin uninstalled
+  since then failed with "plugin not found" and one installed since was
+  skipped. After a run the installed cards and the Updates badge show the
+  new versions; they kept offering "Update to vX" for what had just been
+  updated until the page was reloaded.
+- The Run On-Demand dialog lists a plugin's display modes, so a mode other
+  than the first can be started, and pinned. `/api/v3/plugins/installed`
+  never sent `display_modes`, which the dialog reads, so every plugin
+  offered only its own id under "This plugin exposes a single display
+  mode", and the display started its first mode. Each entry now carries
+  `display_modes`, the modes its manifest declares.
+- Installing Weather, Music, Stocks or Leaderboard from the Plugin Store
+  enables it, as installing any other plugin does. Each installs under the
+  id its manifest declares (`ledmatrix-weather` for the store's `weather`),
+  but the store enabled the store id, which `/api/v3/plugins/toggle`
+  answered with "Plugin not found": the plugin stayed disabled behind
+  "installed, but enabling it failed". `POST /api/v3/plugins/install` now
+  answers with the installed `plugin_id` (in the operation's result when it
+  is queued), and the store enables that.
+- Reinstalling a plugin from the Plugin Store leaves it enabled or disabled
+  as it was. Reinstall enabled it as a fresh install does, so a plugin the
+  user had switched off came back on.
+- A Plugin Store install that takes more than a minute is no longer
+  reported as failed. The store stopped waiting after 60 s and showed
+  "Install operation timed out" while the server, which allows the
+  plugin's dependency install 300 s on its own, carried on and usually
+  succeeded; the plugin was then neither enabled nor listed until the page
+  was reloaded. The store now waits up to 10 minutes, and if it still has
+  no answer it reloads the installed list and says the install may still
+  be running.
+- The Plugin Store's category filter lists every category its plugins
+  have. It offered a fixed seven while the registry uses about twenty, so
+  plugins filed under productivity, utility, transit and the rest could not
+  be filtered to, and "Financial" missed the plugin filed under "finance".
+  The choices are now built from the store's plugins, as the Starlark
+  section's are.
+- The Install button under Install Single Plugin (Plugin Manager > Install
+  from GitHub) runs one handler per click. It also had an inline `onclick`
+  whose handler threw a `ReferenceError` on every click; only the other
+  handler's request went out, and making the inline one work would have
+  sent every install twice. The inline handler is gone.
 - `/api/v3/plugins/installed` no longer reports the display's plugins as
   `live` while `/api/v3/health` says `display_loop: stalled`. The runtime
   snapshot is written from its own thread, which kept going while the render

test/js/README.md

@@ -45,6 +45,10 @@ server has none.
 |---|---|---|
 | `unit/test_list_filter.js` | no | `ListFilter` search/filter/sort/count/sticky, and the installed-plugins config **extracted verbatim** from `plugins_manager.js` so the test can't drift from it |
 | `unit/test_update_all.js` | no | `PluginInstallManager.updateAll` from `plugins/install_manager.js`: Check & Update All sends only plugin ids (never `starlark:` app entries), re-sends a request that got no HTTP answer (web service restarting) instead of skipping that plugin, never re-sends one that got any HTTP answer (the real `api_client.js` classifies a proxy 502 or a JSON error without `error_code` as `API_ERROR`), and counts a no-op update as already up to date in the summary. Also run by `test/web_interface/test_update_all_plugins.py` so CI covers it |
+| `unit/test_store_install.js` | no | The store's Install button, with the whole of `plugins_manager.js` run by `plugins_manager_sandbox.js` (a vm context, fake DOM and API): a fresh install reloads the list, then enables the id the plugin was installed as -- the answer's `plugin_id`, else the installed entry the store entry matches (Weather installs as `ledmatrix-weather`); a Reinstall leaves the enabled state alone |
+| `unit/test_install_polling.js` | no | How long Install waits for a queued install (sandbox): at least the server's 300 s dependency-install timeout; when it stops waiting it reloads the installed list and warns, rather than reporting a failure or enabling anything |
+| `unit/test_store_categories.js` | no | The store's category filter (sandbox): the template ships only All Categories, the rest come from the store's plugins (one per category whatever its case), choosing one filters to it, and a swapped-in select is refilled from the cache keeping the choice |
+| `unit/test_github_url_install.js` | no | Install Single Plugin (sandbox, the button as `plugins.html` ships it): no inline `onclick`, so a click or Enter sends exactly one `install-from-url` request and raises no error |
 | `unit/test_render_cards.js` | no | `renderInstalledCards` markup, both empty states, and HTML-escaping of hostile plugin metadata |
 | `unit/test_style_editor_element_keys.js` | no | `elementKeys()`/`styleRows()`/`positionRows()` from `widgets/style-editor.js`: every `customization.layout` entry gets exactly one row -- paired with its style element through core's `x-layout-key` (so `score` belongs to `score_text`, not a second row), or a position row of its own, leaves included -- since the widget claims the whole `layout` block from the generic fallback renderer |
 | `unit/test_style_editor_layout_leaf_columns.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only key whose own value is a leaf (no x/y sub-object, e.g. a `show_logo` toggle) gets a self-keyed column instead of a blank, uneditable row |

test/js/plugins_manager_sandbox.js

@@ -0,0 +1,212 @@
+// The whole of plugins_manager.js (and list_filter.js before it, as the page
+// loads them), evaluated in a node vm context against a small fake DOM.
+//
+// For suites that drive the plugin manager's real flows -- install, polling,
+// store filters, the GitHub-URL button -- rather than one function sliced
+// out of the file. Nothing is mocked inside the script: only what the page
+// gives it (document, fetch, timers, showNotification, LEDEscape).
+//
+//   const sb = create({ route: (method, url, body) => ({ status, json }) });
+//   sb.el('plugin-store-grid');          // make an element exist by id
+//   sb.window.installPlugin('weather');
+//   await sb.until(() => sb.requests.some(r => r.url.includes('/toggle')));
+//
+// Timers ignore their delays and run on the next turn, so a poll loop that
+// would take minutes in a browser finishes in milliseconds. The page is in
+// readyState "loading" with no #installed-plugins-grid, so the script's own
+// start-up does nothing until a suite asks for it (window.initPluginsPage()).
+const fs = require('fs');
+const path = require('path');
+const vm = require('vm');
+const ledEscape = require('./led_escape');
+
+const V3 = path.resolve(__dirname, '../../web_interface/static/v3');
+const PLUGINS_HTML = path.resolve(__dirname, '../../web_interface/templates/v3/partials/plugins.html');
+
+class FakeClassList {
+  constructor() { this.set = new Set(); }
+  add(...c) { c.forEach(x => this.set.add(x)); }
+  remove(...c) { c.forEach(x => this.set.delete(x)); }
+  contains(c) { return this.set.has(c); }
+  toggle(c, force) {
+    const on = force === undefined ? !this.set.has(c) : !!force;
+    if (on) this.set.add(c); else this.set.delete(c);
+    return on;
+  }
+}
+
+function create({ route } = {}) {
+  const elements = new Map();
+  const requests = [];
+  const toasts = [];
+  const errors = [];
+  const restartNotes = [];
+
+  class FakeElement {
+    constructor(id, tag = 'div', attributes = {}) {
+      this.id = id;
+      this.tagName = tag.toUpperCase();
+      this.attributes = { ...attributes };
+      this.listeners = {};
+      this.children = [];
+      this.classList = new FakeClassList();
+      this.style = { removeProperty() {} };
+      this.dataset = {};
+      this.value = '';
+      this.textContent = '';
+      this.disabled = false;
+      this.parentNode = null;
+      this._html = '';
+    }
+    get innerHTML() { return this._html; }
+    set innerHTML(v) { this._html = String(v); this.children = []; }
+    getAttribute(n) { return n in this.attributes ? this.attributes[n] : null; }
+    setAttribute(n, v) { this.attributes[n] = String(v); }
+    hasAttribute(n) { return n in this.attributes; }
+    removeAttribute(n) { delete this.attributes[n]; }
+    addEventListener(type, fn) { (this.listeners[type] = this.listeners[type] || []).push(fn); }
+    removeEventListener(type, fn) {
+      this.listeners[type] = (this.listeners[type] || []).filter(f => f !== fn);
+    }
+    appendChild(child) { this.children.push(child); child.parentNode = this; return child; }
+    querySelector() { return null; }
+    querySelectorAll() { return []; }
+    closest() { return null; }
+    cloneNode() {
+      const copy = new FakeElement(this.id, this.tagName, this.attributes);
+      copy._html = this._html;
+      copy.value = this.value;
+      return copy;
+    }
+    replaceChild(next, prev) {
+      next.parentNode = this;
+      prev.parentNode = null;
+      if (next.id) elements.set(next.id, next);
+      return prev;
+    }
+    replaceWith(next) { if (this.parentNode) this.parentNode.replaceChild(next, this); }
+    // A browser runs an inline on<type> attribute first (it was set before
+    // any listener was added), then the listeners, and an exception in one
+    // does not stop the next: it is reported, which is what `errors` holds.
+    dispatch(type, init = {}) {
+      const event = {
+        type, target: this, currentTarget: this, key: init.key,
+        defaultPrevented: false,
+        preventDefault() { this.defaultPrevented = true; },
+        stopPropagation() {}, stopImmediatePropagation() {},
+      };
+      const inline = this.getAttribute('on' + type);
+      const handlers = [];
+      if (inline !== null) {
+        handlers.push(vm.runInContext(`(function(event) {\n${inline}\n})`, ctx));
+      }
+      handlers.push(...(this.listeners[type] || []));
+      for (const h of handlers) {
+        try { h.call(this, event); } catch (e) { errors.push(e); }
+      }
+      return event;
+    }
+    click() { return this.dispatch('click'); }
+  }
+
+  function el(id, tag, attributes) {
+    if (!elements.has(id)) {
+      const parent = new FakeElement(null);
+      parent.appendChild(new FakeElement(id, tag, attributes));
+      elements.set(id, parent.children[0]);
+    }
+    return elements.get(id);
+  }
+
+  const timers = [];
+  const ctx = {
+    // Warnings are the script noting elements this fake page doesn't have.
+    console: { log: console.log.bind(console), error: console.error.bind(console),
+               warn: () => {}, info: () => {}, debug: () => {} },
+    debugLog: () => {},
+    addEventListener() {},
+    URL,
+    document: {
+      readyState: 'loading',
+      body: { addEventListener() {} },
+      getElementById: id => elements.get(id) || null,
+      querySelector: () => null,
+      querySelectorAll: () => [],
+      addEventListener() {},
+      dispatchEvent() { return true; },
+      createElement: tag => new FakeElement(null, tag),
+    },
+    CustomEvent: class { constructor(type, init) { this.type = type; this.detail = init && init.detail; } },
+    setTimeout: (fn, _ms, ...args) => { timers.push(setImmediate(() => fn(...args))); return timers.length; },
+    clearTimeout: () => {},
+    setInterval: () => 0,
+    clearInterval: () => {},
+    requestAnimationFrame: fn => setImmediate(fn),
+    getComputedStyle: () => ({ display: 'block' }),
+    scrollTo() {},
+    sessionStorage: { getItem: () => null, setItem() {}, removeItem() {} },
+    localStorage: { getItem: () => null, setItem() {}, removeItem() {} },
+    confirm: () => true,
+    alert: () => {},
+    showNotification: (message, type) => {
+      toasts.push({ message: String(message),
+                    type: type && typeof type === 'object' ? type.type : type });
+    },
+    noteRestartRequired: (body) => { restartNotes.push(body); },
+    fetch: async (url, opts = {}) => {
+      const method = (opts.method || 'GET').toUpperCase();
+      let body = null;
+      try { body = opts.body ? JSON.parse(opts.body) : null; } catch (e) { body = opts.body; }
+      requests.push({ method, url: String(url), body });
+      const answer = (route && route(method, String(url), body)) || { status: 200, json: { status: 'success' } };
+      const status = answer.status || 200;
+      return { ok: status < 400, status, json: async () => answer.json };
+    },
+  };
+  ctx.window = ctx;
+  vm.createContext(ctx);
+  ledEscape.install(ctx);
+  for (const file of ['js/plugins/list_filter.js', 'plugins_manager.js']) {
+    vm.runInContext(fs.readFileSync(path.join(V3, file), 'utf8'), ctx, { filename: file });
+  }
+
+  // Resolves once cond() is true, letting timers and promises run between
+  // checks; rejects if it never is.
+  async function until(cond, label = 'condition', turns = 20000) {
+    for (let i = 0; i < turns; i++) {
+      if (cond()) return;
+      await new Promise(r => setImmediate(r));
+    }
+    throw new Error('timed out waiting for ' + label);
+  }
+
+  // Lets every pending timer and promise run.
+  async function settle(turns = 50) {
+    for (let i = 0; i < turns; i++) await new Promise(r => setImmediate(r));
+  }
+
+  return { window: ctx, el, FakeElement, requests, toasts, errors, restartNotes, until, settle };
+}
+
+// The attributes of the element with this id in partials/plugins.html, as
+// the template ships them (no Jinja on the tags these suites read).
+function templateAttributes(id) {
+  const html = fs.readFileSync(PLUGINS_HTML, 'utf8');
+  const at = html.indexOf(`id="${id}"`);
+  if (at < 0) throw new Error(`no element with id ${id} in plugins.html`);
+  const start = html.lastIndexOf('<', at);
+  let end = start, quote = null;
+  for (; end < html.length; end++) {
+    const ch = html[end];
+    if (quote) { if (ch === quote) quote = null; } else if (ch === '"' || ch === "'") quote = ch;
+    else if (ch === '>') break;
+  }
+  const tag = html.slice(start, end + 1);
+  const attrs = {};
+  const re = /([\w:-]+)\s*=\s*("([^"]*)"|'([^']*)')/g;
+  let m;
+  while ((m = re.exec(tag))) attrs[m[1]] = m[3] !== undefined ? m[3] : m[4];
+  return { tag: tag.match(/^<(\w+)/)[1], attrs, source: tag };
+}
+
+module.exports = { create, templateAttributes };

test/js/run_all.js

@@ -20,7 +20,12 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
               'unit/test_html_escaping.js', 'unit/test_style_editor_element_keys.js',
               'unit/test_style_editor_layout_leaf_columns.js',
               'unit/test_style_editor_layout_leaf_collision.js',
-              'unit/test_update_all.js', 'unit/test_inline_handler_escaping.js',
+              'unit/test_update_all.js',
+              'unit/test_store_install.js',
+              'unit/test_install_polling.js',
+              'unit/test_store_categories.js',
+              'unit/test_github_url_install.js',
+              'unit/test_inline_handler_escaping.js',
               'unit/test_plugin_action_delegation.js', 'unit/test_file_upload_widget.js',
               'unit/test_store_registry_fields.js', 'unit/test_restart_banner.js',
               'unit/test_page_registry.js', 'unit/test_core_modules.js'];

test/js/unit/test_github_url_install.js

@@ -0,0 +1,93 @@
+// Plugin Manager > Install from GitHub > Install Single Plugin: one click,
+// one request, no errors.
+//
+// The Install button carried an inline onclick calling
+// window.handleGitHubPluginInstall, and attachInstallButtonHandler also gave
+// it a click listener that installs. Both ran on every click. The inline one
+// threw a ReferenceError (it called isGithubUrl, which lives inside the
+// plugin-manager IIFE, from outside it), so only the listener's request went
+// out -- and fixing that scope alone would have sent every install twice.
+// The button now has the listener only.
+//
+// Runs the whole of plugins_manager.js in the sandbox, with the button as
+// partials/plugins.html ships it.
+
+const { create, templateAttributes } = require('../plugins_manager_sandbox');
+
+let pass = 0, fail = 0;
+const ok = (label, cond, extra) => cond
+  ? (pass++, console.log('  ok   ' + label))
+  : (fail++, console.log('  FAIL ' + label + (extra !== undefined ? '  -> ' + JSON.stringify(extra).slice(0, 400) : '')));
+
+const URL = 'https://github.com/someone/ledmatrix-demo';
+
+function route(method, url) {
+  if (method === 'POST' && url === '/api/v3/plugins/install-from-url') {
+    return { json: { status: 'success', message: 'Plugin demo installed successfully', plugin_id: 'demo' } };
+  }
+  if (url.startsWith('/api/v3/plugins/installed')) return { json: { status: 'success', data: { plugins: [] } } };
+  return { json: { status: 'success' } };
+}
+
+function page() {
+  const sb = create({ route });
+  const button = templateAttributes('install-plugin-from-url');
+  sb.el('install-plugin-from-url', button.tag, button.attrs);
+  sb.el('github-plugin-url', 'input');
+  sb.el('github-plugin-status');
+  sb.el('plugin-branch-input', 'input');
+  return sb;
+}
+
+const installs = sb => sb.requests.filter(r => r.url === '/api/v3/plugins/install-from-url');
+
+(async () => {
+  console.log('\nthe template');
+  {
+    const { attrs } = templateAttributes('install-plugin-from-url');
+    ok('the Install button has no inline onclick', !('onclick' in attrs), attrs.onclick);
+  }
+
+  console.log('\na click');
+  {
+    const sb = page();
+    sb.window.attachInstallButtonHandler();
+    // htmx:afterSettle runs it again on every swap; that must not add a handler.
+    sb.window.attachInstallButtonHandler();
+    sb.el('github-plugin-url').value = URL;
+    sb.window.document.getElementById('install-plugin-from-url').click();
+    await sb.settle();
+    ok('raises no error', sb.errors.length === 0, sb.errors.map(String));
+    ok('sends exactly one install request', installs(sb).length === 1, installs(sb));
+    ok('for the URL typed', installs(sb)[0] && installs(sb)[0].body.repo_url === URL, installs(sb));
+    ok('and reports the result', /Successfully installed: demo/.test(sb.el('github-plugin-status').innerHTML),
+       sb.el('github-plugin-status').innerHTML);
+  }
+
+  console.log('\nEnter in the URL field');
+  {
+    const sb = page();
+    sb.window.attachInstallButtonHandler();
+    const input = sb.el('github-plugin-url');
+    input.value = URL;
+    input.dispatch('keypress', { key: 'Enter' });
+    await sb.settle();
+    ok('raises no error', sb.errors.length === 0, sb.errors.map(String));
+    ok('sends exactly one install request', installs(sb).length === 1, installs(sb));
+  }
+
+  console.log('\na URL that is not GitHub');
+  {
+    const sb = page();
+    sb.window.attachInstallButtonHandler();
+    sb.el('github-plugin-url').value = 'https://example.com/x';
+    sb.window.document.getElementById('install-plugin-from-url').click();
+    await sb.settle();
+    ok('is refused without a request or an error',
+       installs(sb).length === 0 && sb.errors.length === 0 && /valid GitHub URL/.test(sb.el('github-plugin-status').innerHTML),
+       { errors: sb.errors.map(String), status: sb.el('github-plugin-status').innerHTML });
+  }
+
+  console.log(`\n${pass} passed, ${fail} failed`);
+  process.exit(fail ? 1 : 0);
+})().catch(e => { console.error(e); process.exit(1); });