AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async fails in CI

[ericstj]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=976255
Build error leg or test failing: Microsoft.ML.AutoML.Test.AutoMLExperimentTests.AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async
Pull request: #7414

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Error:

Starting test: Microsoft.ML.AutoML.Test.AutoMLExperimentTests.AutoMLExperiment_Taxi_Fare_CV_5_Test
    Microsoft.ML.AutoML.Test.AutoMLExperimentTests.AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async [FAIL]
�[31;1m�[m�[37m      Expected value to be less than or equal to 2500L, but found 3236L (difference of 736).
�[m�[30;1m      Stack Trace:
�[m�[37m           at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message)
�[m�[37m           at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
�[m�[37m           at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message)
�[m�[37m           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
�[m�[37m           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
�[m�[37m           at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args)
�[m�[37m           at FluentAssertions.Numeric.NumericAssertions`2.BeLessThanOrEqualTo(T expected, String because, Object[] becauseArgs)
�[m�[37m           at FluentAssertions.Numeric.NumericAssertions`2.BeLessOrEqualTo(T expected, String because, Object[] becauseArgs)
�[m�[37m        /__w/1/s/test/Microsoft.ML.AutoML.Tests/AutoMLExperimentTests.cs(146,0): at Microsoft.ML.AutoML.Test.AutoMLExperimentTests.AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async()
�[m�[37m        --- End of stack trace from previous location ---

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=976255
Error message validated: [AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 3/12/2025 1:13:43 AM UTC

Report

Build	Definition	Test	Pull Request
1066689	dotnet/machinelearning	Microsoft.ML.AutoML.Test.AutoFitTests.AutoFitMultiClassification_Image_TrainTest
1062216	dotnet/machinelearning	Microsoft.ML.AutoML.Tests.WorkItemExecution	#7470
1046748	dotnet/machinelearning	Microsoft.ML.AutoML.Tests.WorkItemExecution	#7457
1045720	dotnet/machinelearning	Microsoft.ML.AutoML.Tests.WorkItemExecution	#7469
1045700	dotnet/machinelearning	Microsoft.ML.AutoML.Tests.WorkItemExecution	#7457
1045427	dotnet/machinelearning	Microsoft.ML.AutoML.Tests.WorkItemExecution	#7457

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
1	2	6