Skip to content

Handle the retry_total_tests configuration option in the runner #344

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
brasmusson wants to merge 1 commit into
base: main
Choose a base branch
from
+21 −39
Open

Handle the retry_total_tests configuration option in the runner #344

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 15 additions & 37 deletions .rubocop_todo.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# This configuration was generated by
# `rubocop --auto-gen-config`
# on 2026-07-13 08:18:00 UTC using RuboCop version 1.88.2.
# on 2026-08-30 17:09:30 UTC using RuboCop version 1.89.0.
# The point is for the user to remove these configuration records
# one by one as the offenses are removed from the code base.
# Note that changes in the inspected code, or installation of new
Expand All @@ -12,25 +12,7 @@
# TODO: [LH] v16.2 (Generic refactors / new events) -> 79 files inspected, 222 offenses detected, 10 offenses autocorrectable
# TODO: [LH] v17 prep -> 92 files inspected, 212 offenses detected, 10 offenses autocorrectable
# TODO: [LH] v18 prep -> 111 files inspected, 227 offenses detected, 16 offenses autocorrectable

# Offense count: 1
# This cop supports safe autocorrection (--autocorrect).
# Configuration parameters: AllowForAlignment, AllowBeforeTrailingComments, ForceEqualSignAlignment.
Layout/ExtraSpacing:
Exclude:
- 'spec/cucumber/core/test/result/summary_spec.rb'

# Offense count: 1
# This cop supports safe autocorrection (--autocorrect).
Layout/LeadingEmptyLines:
Exclude:
- 'spec/cucumber/core/test/result/skipped_spec.rb'

# Offense count: 1
# This cop supports safe autocorrection (--autocorrect).
Layout/SpaceAroundMethodCallOperator:
Exclude:
- 'spec/cucumber/core/test/result/summary_spec.rb'
# TODO: [BjR] v19 prep -> 109 files inspected, 211 offenses detected, 13 offenses autocorrectable

# Offense count: 1
# This cop supports safe autocorrection (--autocorrect).
Expand All @@ -46,22 +28,23 @@ Metrics/AbcSize:
# Offense count: 1
# Configuration parameters: CountComments, CountAsOne.
Metrics/ClassLength:
Max: 111
Max: 115

# Offense count: 3
# Configuration parameters: CountComments, CountAsOne, AllowedMethods, AllowedPatterns.
Metrics/MethodLength:
Max: 28

# Offense count: 3
# Offense count: 1
# Configuration parameters: CountComments, CountAsOne.
Metrics/ModuleLength:
Max: 110
Max: 103

# Offense count: 1
# Configuration parameters: CountKeywordArgs, MaxOptionalParameters.
# Offense count: 2
# Configuration parameters: CountKeywordArgs.
Metrics/ParameterLists:
Max: 8
MaxOptionalParameters: 4

# Offense count: 5
# This cop supports unsafe autocorrection (--autocorrect-all).
Expand All @@ -70,7 +53,7 @@ RSpec/EmptyExampleGroup:
- 'spec/cucumber/core/compiler_spec.rb'
- 'spec/cucumber/core/gherkin/writer_spec.rb'

# Offense count: 21
# Offense count: 22
# Configuration parameters: CountAsOne.
RSpec/ExampleLength:
Max: 11
Expand All @@ -87,7 +70,7 @@ RSpec/LeadingSubject:
Exclude:
- 'spec/cucumber/core/test/result/raisable_spec.rb'

# Offense count: 23
# Offense count: 22
RSpec/MissingExampleGroupArgument:
Exclude:
- 'spec/cucumber/core/compiler_spec.rb'
Expand All @@ -98,21 +81,21 @@ RSpec/MissingExampleGroupArgument:
- 'spec/cucumber/core/test/locations_filter_spec.rb'
- 'spec/cucumber/core_spec.rb'

# Offense count: 64
# Offense count: 62
RSpec/MultipleExpectations:
Max: 5

# Offense count: 59
# Offense count: 53
# Configuration parameters: AllowSubject.
RSpec/MultipleMemoizedHelpers:
Max: 11
Max: 10

# Offense count: 13
# Offense count: 8
# Configuration parameters: AllowedGroups.
RSpec/NestedGroups:
Max: 5

# Offense count: 22
# Offense count: 20
# Configuration parameters: AllowedPatterns.
# AllowedPatterns: ^expect_, ^assert_
RSpec/NoExpectationExample:
Expand All @@ -131,11 +114,6 @@ RSpec/RedundantPredicateMatcher:
Exclude:
- 'spec/cucumber/core/test/location_spec.rb'

# Offense count: 2
RSpec/RepeatedExample:
Exclude:
- 'spec/cucumber/core/gherkin/parser_spec.rb'

# Offense count: 1
# This cop supports unsafe autocorrection (--autocorrect-all).
# Configuration parameters: EnforcedStyle, AllowModifiersOnSymbols, AllowModifiersOnAttrs, AllowModifiersOnAliasMethod.
Expand Down
8 changes: 6 additions & 2 deletions lib/cucumber/core/test/runner.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,14 @@ class Runner
attr_reader :event_bus, :running_test_case, :running_test_step, :id_generator
private :event_bus, :running_test_case, :running_test_step, :id_generator

def initialize(event_bus, id_generator = Cucumber::Messages::Helpers::IdGenerator::UUID.new, backtrace_filter = nil, max_attempts = 1)
def initialize(event_bus, id_generator = Cucumber::Messages::Helpers::IdGenerator::UUID.new, backtrace_filter = nil, max_attempts = 1, max_total_retried_tests = Float::INFINITY)

@luke-hill luke-hill Aug 30, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this lean on the config option rather than the verbose value. As this value is a default for the config option (Question btw)

@luke-hill luke-hill Sep 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see now, this "will" lean on the config value specified.

@event_bus = event_bus
@id_generator = id_generator
@backtrace_filter = backtrace_filter
@max_attempts = max_attempts
@max_total_retried_tests = max_total_retried_tests
@current_test_case = nil
@total_permanently_failed = 0
end

def test_case(test_case, &descend)
Expand Down Expand Up @@ -83,11 +85,13 @@ def to_test_case_started_envelope(test_case)
end

def to_test_case_finished_envelope(result)
will_be_retried = result.failed? && (@attempt < @max_attempts) && (@total_permanently_failed < @max_total_retried_tests)
@total_permanently_failed += 1 if result.failed? && !will_be_retried
Cucumber::Messages::Envelope.new(
test_case_finished: Cucumber::Messages::TestCaseFinished.new(
test_case_started_id: @current_test_case_started_id,
timestamp: time_to_timestamp(Time.now),
will_be_retried: result.failed? && (@attempt < @max_attempts)
will_be_retried: will_be_retried
)
)
end
Expand Down