SOLR-8474: Add Selenium-based JUnit tests for the Admin UI

.github/workflows/admin-ui-test.yml

@@ -0,0 +1,48 @@
+name: Admin UI Tests
+
+on:
+  pull_request:
+    branches:
+      - '*'
+    paths:
+      - '.github/workflows/admin-ui-test.yml'
+      # the Admin UI itself, its browser tests and its build
+      - 'solr/webapp/**'
+      # the v2 API contract the UI (and its generated js-client) is built against
+      - 'solr/api/**'
+      # the v2 API implementations backing the UI screens
+      - 'solr/core/src/java/org/apache/solr/handler/admin/api/**'
+      # the embedded-Jetty harness and test-group annotation this suite runs on
+      - 'solr/test-framework/src/java/org/apache/solr/embedded/**'
+      - 'solr/test-framework/src/java/org/apache/solr/util/SeleniumTest.java'
+      - 'gradle/testing/randomization.gradle'
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  test:
+    name: Run Admin UI browser tests
+
+    # ubuntu-latest has Chrome preinstalled, which the Selenium tests require
+    runs-on: ubuntu-latest
+    timeout-minutes: 40
+
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+
+    - uses: ./.github/actions/prepare-for-build
+
+    - name: Run the Admin UI browser tests
+      run: ./gradlew :solr:webapp:test -Ptests.selenium=true
+
+    - name: Archive test output (screenshots, page sources, browser logs)
+      if: ${{ failure() }}
+      uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+      with:
+        name: test-output
+        path: |
+          solr/webapp/build/test-results/**
+          solr/webapp/build/tmp/tests-tmp/**

changelog/unreleased/admin-ui-selenium-tests.yml

@@ -0,0 +1,10 @@
+title: >
+  Added Selenium-based JUnit test coverage for the Admin UI, driving a headless Chrome against a test cluster.
+type: other
+authors:
+  - name: Jan Høydahl
+  - name: Eric Pugh
+  - name: David Smiley
+links:
+  - name: SOLR-8474
+    url: https://issues.apache.org/jira/browse/SOLR-8474

dev-docs/admin-ui-tests.md

@@ -0,0 +1,74 @@
+# Admin UI (AngularJS) Browser Tests
+
+Browser-based tests of the old AngularJS Admin UI (`solr/webapp/web/`), driven by
+Selenium WebDriver with headless Chrome. What each screen covers is documented in
+the test classes themselves; this page covers how the suite works and what it
+deliberately does not do.
+
+## How the tests work
+
+- Tests live in `solr/webapp/src/test/org/apache/solr/webapp/` and extend
+  `AdminUiTestBase`, which starts a 2-node `MiniSolrCloudCluster` whose Jetty
+  nodes also serve the Admin UI (opt-in `JettyConfig.Builder#enableAdminUi`),
+  then starts a headless Chrome via Selenium WebDriver.
+- The tests are opt-in: the suites carry the `@SeleniumTest` test group
+  annotation (disabled by default), enabled with `-Ptests.selenium=true`.
+- A locally installed Chrome/Chromium is required; since the tests only run
+  when explicitly enabled, a missing browser fails the tests rather than
+  skipping them. Override discovery with
+  `-Dtests.selenium.chrome.binary=/path/to/chrome`. The matching chromedriver is
+  provisioned (and cached) by Selenium Manager.
+- Display assertions compare UI text against live JSON from the same node's
+  admin APIs — never hardcoded values.
+- Tests are grouped per screen/feature, so each screen's display and write
+  tests live in the same class.
+- Most tests run against a 2-node cloud cluster; `AdminUiStandaloneTestBase`
+  additionally supports standalone (user-managed, no ZooKeeper) nodes, whose
+  UI differs (no Cloud/Collections/Schema Designer menus; the per-core menu
+  offers query/replication etc. directly).
+- On failure, a screenshot, the page source and the browser console log are
+  saved into the test temp dir.
+- Run with: `./gradlew :solr:webapp:test -Ptests.selenium=true`
+- From an IDE, set `-Dtests.selenium=true` on the run configuration; the
+  js-client bundle is picked up from the js-client build output, so run a
+  Gradle build once first.
+
+## Deliberately skipped (effort vs value)
+
+- **JWT/OAuth login flows**: require an external identity provider or heavy
+  mocking; BasicAuth covers the UI's login/session mechanics.
+- **Keystroke-level entry in the security dialogs**: native clicks/keystrokes
+  into the absolutely-positioned dialogs proved unreliable in headless Chrome;
+  the dialogs are driven via the Angular controller scope instead. Keyboard
+  entry is covered by the login form and the other screens' forms.
+
+## Known limitations
+
+- The generated js-client bundle (`libs/solr/index.js`) only exists inside the
+  built WAR, not in the source tree tests serve from, so the build hands its
+  location to the test JVM in `tests.ui.jsclient.bundle`. The bundle (and its
+  node/npm toolchain) is only built when `-Ptests.selenium=true` enables the
+  tests, keeping node off the default test build chain. With the js-client
+  build turned off (`-PdisableJsClient=true`) the bundle cannot be built, so
+  the build disables these tests with a warning — and fails with an error if
+  `-Ptests.selenium=true` was passed as well.
+- Every test cluster in the JVM registers a log-watcher appender under the same
+  name in the shared log4j config, so a later cluster's watcher can be blind;
+  the events-viewer test detects this via the API and skips itself.
+- The shared menu code logs a benign
+  `TypeError: Cannot read properties of null (reading 'name')` from
+  `$scope.showCore` while the per-collection menu resolves (filtered in the
+  console-error assertion; tracked in
+  [SOLR-18347](https://issues.apache.org/jira/browse/SOLR-18347)).
+- The core overview ping widget answers 503 when the configset has no
+  healthcheck file (allowed in the affected tests; tracked in
+  [SOLR-18347](https://issues.apache.org/jira/browse/SOLR-18347)).
+- The Schema Designer's backend transiently fails its own prep/analyze calls
+  with "version mismatch, retry" and recovers via its retry dialog; its API
+  errors are excluded from the console-error assertion.
+- ASF Jenkins jobs do not pass `-Ptests.selenium=true`, so these tests do
+  not run there (a nightly job could opt in if its build nodes have a
+  browser). In CI they run via the GitHub Actions workflow
+  `.github/workflows/admin-ui-test.yml`, on pull requests that touch the
+  webapp, the v2 API contract (`solr/api`) or the v2 API implementations
+  (`solr/core/.../handler/admin/api`).

dev-docs/gradle-help/tests.txt

@@ -68,7 +68,11 @@ Test groups
 -----------
 
 Tests can be filtered by an annotation they're marked with.
-Some test group annotations include: @AwaitsFix, @Nightly, @Monster
+Some test group annotations include: @AwaitsFix, @Nightly, @Monster, @SeleniumTest
+
+Disabled-by-default groups are enabled with their system property, for example
+-Ptests.nightly=true, or -Ptests.selenium=true for the browser-based Admin UI
+tests in solr/webapp (these need a local Chrome, see dev-docs/admin-ui-tests.md).
 
 This uses filtering infrastructure on the *runner* (randomizedtesting), 
 not gradle's built-in mechanisms (but it can be combined with "--tests").

gradle/libs.versions.toml

@@ -67,7 +67,7 @@ benmanes-versions = "0.54.0"
 bouncycastle = "1.84"
 # @keep Browserify version used in ref-guide
 browserify = "17.0.0"
-bytebuddy = "1.18.9"
+bytebuddy = "1.18.11"
 carrot2-core = "4.8.6"
 carrotsearch-hppc = "0.10.0"
 carrotsearch-randomizedtesting = "2.9.1"
@@ -187,6 +187,7 @@ owasp-dependencycheck = "13.0.0"
 perfmark = "0.27.0"
 prometheus-metrics = "1.8.0"
 quicktheories = "0.26"
+selenium = "4.47.0"
 semver4j = "6.0.0"
 slf4j = "2.0.17"
 spatial4j = "0.8"
@@ -512,6 +513,7 @@ perfmark-api = { module = "io.perfmark:perfmark-api", version.ref = "perfmark" }
 prometheus-metrics-expositionformats = { module = "io.prometheus:prometheus-metrics-exposition-formats", version.ref = "prometheus-metrics" }
 prometheus-metrics-model = { module = "io.prometheus:prometheus-metrics-model", version.ref = "prometheus-metrics" }
 quicktheories-quicktheories = { module = "org.quicktheories:quicktheories", version.ref = "quicktheories" }
+selenium-chromedriver = { module = "org.seleniumhq.selenium:selenium-chrome-driver", version.ref = "selenium" }
 semver4j-semver4j = { module = "org.semver4j:semver4j", version.ref = "semver4j" }
 slf4j-api = { module = "org.slf4j:slf4j-api", version.ref = "slf4j" }
 slf4j-jcloverslf4j = { module = "org.slf4j:jcl-over-slf4j", version.ref = "slf4j" }

gradle/testing/randomization.gradle

@@ -132,6 +132,7 @@ configure(allprojects.findAll {project -> project.path.startsWith(":solr")}) {
           [propName: 'tests.src.home', value: null, description: "See SOLR-14023."],
           [propName: 'solr.tests.use.numeric.points', value: null, description: "Point implementation to use (true=numerics, false=trie)."],
           [propName: 'tests.ssl', value: null, description: "Control SSL: true=force on, false=force off, unset=randomize (all respect @SuppressSSL)."],
+          [propName: 'tests.selenium', value: false, description: "Enables or disables @SeleniumTest tests (browser-based, need a local Chrome)."],
       ]
     }
   }

solr/core/gradle.lockfile

@@ -84,7 +84,7 @@ jakarta.ws.rs:jakarta.ws.rs-api:4.0.0=compileClasspath,compileClasspathCopy,jarV
 jakarta.xml.bind:jakarta.xml.bind-api:4.0.2=jarValidation,runtimeClasspath,runtimeClasspathCopy,runtimeLibs,testRuntimeClasspath,testRuntimeClasspathCopy
 javax.inject:javax.inject:1=annotationProcessor,errorprone,testAnnotationProcessor
 junit:junit:4.13.2=jarValidation,testCompileClasspath,testCompileClasspathCopy,testRuntimeClasspath,testRuntimeClasspathCopy
-net.bytebuddy:byte-buddy:1.18.9=jarValidation,testCompileClasspath,testCompileClasspathCopy,testRuntimeClasspath,testRuntimeClasspathCopy
+net.bytebuddy:byte-buddy:1.18.11=jarValidation,testCompileClasspath,testCompileClasspathCopy,testRuntimeClasspath,testRuntimeClasspathCopy
 org.antlr:antlr4-runtime:4.13.2=jarValidation,runtimeClasspath,runtimeClasspathCopy,runtimeLibs,testRuntimeClasspath,testRuntimeClasspathCopy
 org.apache.commons:commons-exec:1.6.0=compileClasspath,compileClasspathCopy,jarValidation,runtimeClasspath,runtimeClasspathCopy,runtimeLibs,testCompileClasspath,testCompileClasspathCopy,testRuntimeClasspath,testRuntimeClasspathCopy
 org.apache.commons:commons-lang3:3.20.0=compileClasspath,compileClasspathCopy,jarValidation,runtimeClasspath,runtimeClasspathCopy,runtimeLibs,testCompileClasspath,testCompileClasspathCopy,testRuntimeClasspath,testRuntimeClasspathCopy

solr/cross-dc-manager/gradle.lockfile

@@ -100,8 +100,8 @@ jakarta.ws.rs:jakarta.ws.rs-api:4.0.0=jarValidation,runtimeClasspath,runtimeLibs
 jakarta.xml.bind:jakarta.xml.bind-api:4.0.2=jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath
 javax.inject:javax.inject:1=annotationProcessor,errorprone,testAnnotationProcessor
 junit:junit:4.13.2=jarValidation,testCompileClasspath,testRuntimeClasspath
-net.bytebuddy:byte-buddy-agent:1.18.9=jarValidation,testRuntimeClasspath
-net.bytebuddy:byte-buddy:1.18.9=jarValidation,testRuntimeClasspath
+net.bytebuddy:byte-buddy-agent:1.18.11=jarValidation,testRuntimeClasspath
+net.bytebuddy:byte-buddy:1.18.11=jarValidation,testRuntimeClasspath
 net.sf.jopt-simple:jopt-simple:5.0.4=jarValidation,runtimeClasspath,runtimeLibs,testRuntimeClasspath
 net.sourceforge.argparse4j:argparse4j:0.7.0=jarValidation,runtimeClasspath,runtimeLibs,testRuntimeClasspath
 org.antlr:antlr4-runtime:4.13.2=jarValidation,runtimeClasspath,runtimeLibs,solrPlatformLibs,testRuntimeClasspath

solr/licenses/byte-buddy-1.18.11.jar.sha1

@@ -0,0 +1 @@
+8fcc3779ff85fae5164cd1b977798ca1af388e06

solr/licenses/byte-buddy-agent-1.18.11.jar.sha1

@@ -0,0 +1 @@
+82212e5b3633e7a65fca8bec525762cfb6c9bae4