Skip to content

Add buildtool test to run Xcode unit-test targets with xctest - #24

Merged
gcasa merged 1 commit into
gnustep:masterfrom
danjboyd:buildtool-test
Oct 29, 2026
Merged

gcasa merged 1 commit into
gnustep:masterfrom
danjboyd:buildtool-test

Conversation

@danjboyd

Copy link
Copy Markdown
Contributor

buildtool test builds the project, like xcodebuild test, then runs each unit-test target's .xctest bundle with xctest from tools-xctest:

buildtool test                                  # every unit-test target
buildtool test -target MyTests                  # one target
buildtool test -scheme MyApp                    # a scheme's testables
buildtool test -only-testing:MyTests/FooTests/testBar

Targets come from -target, from a scheme (its testables, with its SkippedTests and SelectedTests), or are every unit-test target. For test, -target and -scheme choose the test targets rather than narrowing the build, so an application that tests are hosted in still gets built. XCTEST chooses the xctest program and XCTEST_FLAGS passes extra options to it. A failed build or a failed test fails the action, and so buildtool's exit status.

It works with tools-xctest as it is today: -only-testing: becomes an -XCTest Class[.method] selection. Two things need newer xctest options: skipping tests (-skip-testing: or a scheme's skipped tests) needs -skip-testing, and a test target whose TEST_HOST names an application in the project needs -host to run inside it. buildtool asks xctest --help and uses those options when they're listed; otherwise it fails that target with a message saying what's missing. (-skip-testing is in #5 on tools-xctest; -host would come later.)

The code is a new PBXProject (Testing) category (XCode/PBXProject+Testing.[hm]). PBXContainer gains -setTestOptions: and -test, and ToolDelegate gains the test action and its options.

Testing: Tests/run-buildtool-test-action.sh runs buildtool test on a new fixture project, Tests/Fixtures/Xcode/unit-tests (Calc.xcodeproj: a tool, its unit tests, an application, and unit tests hosted in it; generate-project.py writes its pbxproj). It covers:

  • the full run, plus checking the bundle's Info-gnustep.plist names its executable;
  • a failing test failing the action and buildtool exiting 1;
  • -only-testing: by method and by class;
  • -skip-testing:;
  • a scheme's testables and skipped tests;
  • the hosted target;
  • two mistakes: a non-test target and an unknown scheme.

It checks which test methods ran by name, so it doesn't depend on xctest's output format. Where the xctest lacks an option, it expects the "needs an xctest with …" failure instead. It passes with xctest built from tools-xctest master, from master plus #5, and from a build that also has -host. With current master's buildtool it fails at its first check, because test isn't an action there. The changed library and tool files and the fixture sources pass a syntax check with GCC's Objective-C front end. CalcTests.m passes it once the fixture's HEADER_SEARCH_PATHS (Sources) is supplied.

Run on Debian 13 amd64, clang 19.1.7, libobjc2 2.3, gnustep-base 1.31.1, gnustep-make 2.9.3.

AI assistance: this change, its test and this description were written with Claude (Anthropic's Claude Code).

`buildtool test` builds the project, then runs each unit-test target's
.xctest bundle with tools-xctest's xctest, like `xcodebuild test`.
Targets come from -target, a scheme (its testables, SkippedTests and
SelectedTests), or every unit-test target; xcodebuild's -only-testing
and -skip-testing select tests. For test, -target and -scheme choose
test targets instead of narrowing the build, so test hosts still get
built. XCTEST picks the xctest program and XCTEST_FLAGS adds options.
A failed build or test fails the action, and so buildtool's exit
status.

It works with tools-xctest as it is: -only-testing becomes an -XCTest
Class[.method] selection. Skipping tests and running a target inside
the application its TEST_HOST names need xctest's -skip-testing and
-host, which newer tools-xctest has; buildtool uses them when xctest
--help lists them, and otherwise fails that target with a message
saying what is missing.

Testing: Tests/run-buildtool-test-action.sh runs `buildtool test` on a
new fixture project (a tool, its tests, an application and tests hosted
in it): the full run, a failing test, -only-testing by method and by
class, -skip-testing, a scheme's testables and skipped tests, the
hosted target, and the mistakes (a non-test target, an unknown scheme).
It checks which test methods ran by name, so it doesn't depend on
xctest's output format, and expects the "needs an xctest with ..."
failure where the xctest lacks an option. It passes against xctest
from gnustep/tools-xctest master, from master plus the -only-testing
and -skip-testing patch, and from a build that also has -host. Against
master's buildtool it fails at the first check.

AI assistance: this change, its test and this description were written
with Claude (Anthropic's Claude Code).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@danjboyd
danjboyd requested a review from gcasa as a code owner October 29, 2026 18:59
@gcasa

gcasa commented Oct 29, 2026

Copy link
Copy Markdown
Member

I will accept this. I need to make sure have you signed a copyright assignment? Please write request-assign@gnu.org and ask for one for GNUstep.

@gcasa
gcasa merged commit ffb79f5 into gnustep:master Oct 29, 2026
0 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants