Avoid reporting crash results if it is like GPU problem#7
Conversation
There was a problem hiding this comment.
Pull request overview
This PR aims to reduce false “engine crash” reporting by detecting suspected GPU/hardware instability (via a base-engine benchmark check) and suppressing result reporting for affected game pairs.
Changes:
- Extends Cutechess result parsing to run a “GPU crash?” check on disconnect/stall adjudications and mark affected games as
hw_crash. - Suppresses pentanomial/trinomial updates for pairs containing
hw_crashby deleting the pair from the in-memory game map. - Threads
base_name/base_networkthrough the cutechess worker pipeline so the benchmark check has the required inputs.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| results['crashes' ] += 'disconnect' in reason or 'stalls' in reason | ||
| results['timelosses'] += 'on time' in reason | ||
| results['illegals' ] += 'illegal' in reason | ||
|
|
There was a problem hiding this comment.
I intentionally want to increment crash count for HW errors too. It makes sure we will notice to check Errors page.
| def is_gpu_crashed(config, engine, network): | ||
| print('[WARNING] Checking if crash was caused by a GPU problem...') | ||
| try: | ||
| safe_run_benchmarks(config, 'base', engine, network) | ||
| return False | ||
| except utils.OpenBenchBadBenchException: | ||
| print('[ERROR] GPU crash detected!') | ||
| return True |
There was a problem hiding this comment.
I want the server error report (and blacklisting) after benchmark fails. It was only about not reporting game results.
| def is_gpu_crashed(config, engine, network): | ||
| print('[WARNING] Checking if crash was caused by a GPU problem...') | ||
| try: | ||
| safe_run_benchmarks(config, 'base', engine, network) | ||
| return False | ||
| except utils.OpenBenchBadBenchException: | ||
| print('[ERROR] GPU crash detected!') |
There was a problem hiding this comment.
I don't think crash handling performance matters. Client is either going to stop working or test is about to fail.
| return | ||
|
|
||
| # Don't report results when we detect a GPU issue. | ||
| if results['games'][first] == 'hw_crash' or results['games'][second] == 'hw_crash': |
No description provided.