Repository navigation
Dependencies from a detected nested test project are not loadable #97
Description
Problem
A test item belongs to a package but uses a separate project in a nested test directory:
PackageA/
Project.toml
test/Project.toml
test/special/Project.toml
test/special/Manifest.toml
test/special/test_aqua.jl
The nested manifest path-develops PackageA. JuliaWorkspaces detects test/special as the project for the test item. TestItemControllers then activates a temporary environment based on the package test/Project.toml. Dependencies declared only in test/special/Project.toml are not loadable.
For example, a test item containing using Aqua fails with:
ArgumentError: Package Aqua not found in current path.
This reproduces with TestItemControllers 1.11.0.
Expected behavior
After the nested project and manifest are selected, their direct dependencies should be available to the test item.
Current workaround
QuantumClifford PR #820 adds the nested project to JULIA_LOAD_PATH for each test process:
Suggested solution
When projectUri differs from packageUri, preserve the selected project dependencies when creating the temporary test environment. This could merge the selected project dependencies into the temporary project or otherwise keep the selected project loadable. Callers should not need to overwrite JULIA_LOAD_PATH.
Activity
Krastanov commentedon Aug 25, 2026
This issue is discussed in more detail here: https://discourse.julialang.org/t/two-years-of-test-item-framework-updates/138792/23?u=krastanov
davidanthoff commentedon Aug 25, 2026
Ok, for a while I also thought this is a bug, and I think my answers in the discourse thread were actually incorrect.
But now I think the behavior that is implemented is actually correct, i.e. it just mirrors how Pkg.test handles things, which is the goal for how we handle test env stuff.
The basic story is that the project/manifest in the special folder can pin down versions that are used in the test env, but it can't add additional deps beyond what the packages test env specifies (be that either via an
[extra]section or atest/Project.toml).I think I kind of lost track why the current design is a problem. Why not just add all test dependencies as regular test deps? I know there were various problems with activation timings and timeouts etc, but I think (hope) they are resolved now, so helpful this is a strategy that could work?
Krastanov commentedon Aug 25, 2026
If I add JET.jl in test/Project.toml then PkgEval will constantly report my package as broken because JET frequently has issues even instantiating on nightly. Same for a bunch of other tools.
If I add CUDA.jl in test/Project.toml to just test a pkgext then all my test runners will download GB of libraries instead of just the CUDA-enabled runner (same for other GPU backends)
If I add Oscar.jl in test/Project.toml so that I can test the pgkext that depends on Oscar, then I can not run tests on Windows (Oscar does not install on Windows) even though the rest of my library supports Windows.
Krastanov commentedon Aug 25, 2026
In ParallelTestRunner.jl I can solve that issue because its testsets are not "extracted away" from the surrounding environment setup, so stuff like the following works fine:
https://github.com/QuantumSavory/QuantumSavory.jl/blob/464fffa9932ace7d489fe7223f7fe0e1d5218357/test/runtests.jl#L51
Krastanov commentedon Aug 25, 2026
Could you elaborate a bit. Say we are talking about Aqua. How should I structure the repo so that the "default" test runner does not try to install Aqua but there is a dedicated test runner that runs an environment in which Aqua is installed?
davidanthoff commentedon Aug 25, 2026
Sorry, Claude is running amok, I didn't write that response. I'm going to delete it.
Document what projectUri actually contributes