Support AndroidJUnitRunner log-only discovery (-e log true) for JUnit5 parameterized tests

[GermanMalykh]

Summary

When running Android instrumentation tests with Marathon and
de.mannodermaus.junit5.AndroidJUnit5Builder, parameterized JUnit 5 tests are reported inconsistently.

Child invocations are executed and reported as passed, but Marathon cannot map them to discovered tests and marks the parent test as incomplete/failed.

It looks related to log-only discovery (-e log true) expectations used by external runners/tools.

Environment

• android-junit5 runner builder: de.mannodermaus.junit5.AndroidJUnit5Builder
• Marathon: 0.10.4
• testParserConfiguration = remote
• Android runner: androidx.test.runner.AndroidJUnitRunner

Observed behavior

• During execution, child entries like:
  • ...#postTokensResponseException[Pans, KClass] - [2] ... passed
• But Marathon logs:
  • No FSM registered for test ...
  • then marks parent as incomplete/failed

Expected behavior

Tools relying on instrumentation discovery/log protocol should receive a stable and complete test mapping for parameterized tests, so child results can be matched correctly.

Notes from code inspection

In ParsedSelectors.fromBundle(...), selector parsing is based on class and exact method matching.
Method filtering appears strict (it.name == methodName), while incoming method strings may include parameterized suffixes (#method[...]), which can cause fallback to class-level selector and downstream mismatches.

Repro (minimal)

1. Create a JUnit 5 parameterized instrumentation test.
2. Run through Marathon with remote parser.
3. Observe passed child invocations and No FSM registered / incomplete at final summary.

Question

Would you consider adding explicit support/compatibility for Android log-only discovery (-e log true) and/or more robust parsing/mapping for parameterized method selectors?

[mannodermaus]

Hey, thanks for raising the discussion. I'm not familiar with Marathon, but it seems like there is value in supporting log-only mode with the custom runner. To clarify, would it help your use case if -e log true was supported and you'd receive a test mapping that is consistent with the actual test execution?

[GermanMalykh]

@mannodermaus
Yes, that would directly address our use case.

We use Marathon with testParserConfiguration=remote: it relies on log-only discovery to build the initial test list, then maps execution events to those tests. With parameterized JUnit 5 tests, child invocations pass at runtime but the parent stays incomplete because the discovered list doesn’t line up with the executed/reported invocations (Marathon logs "No FSM registered").

If AndroidJUnit5Builder supported -e log true and the discovered test IDs matched what is actually executed/reported (including parameterized children), Marathon should be able to reconcile results without that mismatch.

I’m not claiming Marathon has no other edge cases, but for our parameterized tests this consistency between discovery and execution is what we’re missing.

Tagging @Malinskiy in case you want to align on what Marathon’s remote parser expects from log-only discovery for parameterized tests.

[Malinskiy]

There are 2 aspects for solving runtime-generated tests:

1. -e log true should report runtime-generated ids like the parameterized ones mentioned here
2. when executing the tests the runner should support those runtime-generated ids, i.e. -e class pkg.class#method[variation 1] should execute only this test

Just making sure it will actually work after -e log true