The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Repository	Step Name	Console log	Pull Request
1474677	dotnet/maui	Controls Layout	Log	dotnet/maui#34007
1474682	dotnet/maui	Controls CollectionView	Log	dotnet/maui#31672
1474646	dotnet/runtime	Build Tests	Log
3005157	dotnet-maui	Provision Android SDK - Common Packages	Log
3005107	dotnet-runtime	Send job to Helix (Unix)	Log
1474477	dotnet/runtime	Send to Helix	Log
1474470	dotnet/aspnetcore	Run Quarantined Tests	Log
1474392	dotnet/runtime	Send to Helix	Log	dotnet/runtime#129299
1474437	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#67321
1474425	dotnet/maui	Controls Layout	Log	dotnet/maui#29800
1474418	dotnet/maui	Controls CollectionView	Log	dotnet/maui#29922
1474395	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#67341
1474350	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#67345
1474327	dotnet/performance	Send job to Helix (Windows)	Log
1474264	dotnet/runtime	Send to Helix	Log
1474267	dotnet/runtime	Build Tests	Log
1474245	dotnet/runtime	Send to Helix	Log
1474225	dotnet/performance	Send job to Helix (Windows)	Log	dotnet/performance#5220
1474170	dotnet/runtime	Send to Helix	Log
3004895	dotnet-docker-tools	🟪 Clean public/dotnet/* (pruneEol > 15d)	Log
1474175	dotnet/aspnetcore	Run Quarantined Tests	Log
1474060	dotnet/runtime	Send tests to Helix (Unix)	Log
1474171	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log
1474114	dotnet/runtime	Publish	Log	dotnet/runtime#129661
1474110	dotnet/performance	Send job to Helix (Windows)	Log
1474084	dotnet/runtime	Send to Helix	Log	dotnet/runtime#129639
1474078	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128384
3004754	dotnet-runtime	Download Build Artifact LinuxMonoAOTarm64	Log
3004753	dotnet-runtime	Download Build Artifact BrowserWasm	Log
1474065	dotnet/runtime	Build Tests	Log
1473998	dotnet/runtime	Build product	Log	dotnet/runtime#129637
1473974	dotnet/runtime	Send to Helix	Log
1473883	dotnet/runtime	Send tests to Helix (Unix)	Log
1473897	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#67315
1473775	dotnet/runtime	Build Tests	Log
1473861	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#63642
1473844	dotnet/maui	Controls CollectionView	Log	dotnet/maui#35960
1473853	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#66692
1473840	dotnet/aspnetcore	Run Quarantined Tests	Log
1473841	dotnet/performance	Send job to Helix (Windows)	Log
1473798	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#66331
1473776	dotnet/runtime	Send to Helix	Log
1473726	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#66487
1473710	dotnet/runtime	Send to Helix	Log
1473689	dotnet/performance	Send job to Helix (Windows)	Log	dotnet/performance#5220
1473667	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log
3004476	dotnet-docker-tools	🟪 Clean public/dotnet/* (pruneEol > 15d)	Log
1473612	dotnet/runtime	Send to Helix	Log
3004466	dotnet-aspnetcore	Run build.sh	Log
1473643	dotnet/aspnetcore	Run build.sh	Log
1473515	dotnet/runtime	Send to Helix	Log	dotnet/runtime#129635
1473557	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log
3004379	dotnet-dotnet	Build	Log
1473529	dotnet/roslyn	Run Unit Tests	Log
3004373	dotnet-sdk	Run Tests	Log	#62266
3004375	dotnet-sdk	Run TestBuild Tests	Log	#62267
3004374	dotnet-sdk	Run AoT Tests	Log	#62266
1473455	dotnet/performance	Send job to Helix (Windows)	Log
1473433	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#67327
1473426	dotnet/roslyn	Run Unit Tests	Log
1473453	dotnet/aspnetcore	Build	Log
1470338	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67190
1473432	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67327
1473424	dotnet/aspnetcore	Run build.sh	Log
1473425	dotnet/aspnetcore	Build	Log
1473409	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67111
1473410	dotnet/aspnetcore	Build	Log	dotnet/aspnetcore#67111
3004194	dotnet-maui	Controls CollectionView	Log
1473389	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67325
1473180	dotnet/runtime	Publish	Log	dotnet/runtime#129475
3004263	dotnet-dotnet	Build (centOSStreamContainer)	Log
1473337	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67321
1473279	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#66487
1473308	dotnet/performance	Send job to Helix (Windows)	Log	dotnet/performance#5220
3004264	dotnet-aspnetcore	Build ARM64	Log	#62260
1473297	dotnet/aspnetcore	Run build.sh	Log
1473294	dotnet/performance	Send job to Helix (Windows)	Log	dotnet/performance#5244
3004213	dotnet-maui	Run Integration Tests - RunOniOS_BlazorRelease_CoreCLR	Log
1473278	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#66487
1473116	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#129555
1473256	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#63817
1473225	dotnet/maui	Controls CollectionView	Log
1473203	dotnet/maui	Controls Shell	Log	dotnet/maui#35626
3004261	dotnet-efcore	Build	Log
3004255	dotnet-aspnetcore	Run build.sh	Log
3004253	dotnet-sdk	Build	Log	#62259
3004252	dotnet-sdk	Check out refs/heads/internal/release/9.0.1xx	Log	#62259
3004193	dotnet-maui	Run Integration Tests - RunOniOS_BlazorRelease	Log
1473227	dotnet/aspnetcore	Build	Log
1473204	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log	dotnet/maui#35626
3004250	dotnet-installer	Build	Log	#62258
1473029	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#129553
1473194	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67325
1473163	dotnet/maui	Controls CollectionView	Log	dotnet/maui#34408
3004166	dotnet-installer	Build	Log	#62255
1473164	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log	dotnet/maui#34408
1472970	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#129605
1473153	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log
1473172	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67323
1473158	dotnet/aspnetcore	Run build.sh	Log	dotnet/aspnetcore#67324
Displaying 100 of 4567 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
21	1257	4567

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez