fix: disable profiler in some release tests #2261
Conversation
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
| @@ -91,6 +91,7 @@ def _create_and_fit_estimator(sagemaker_session, tf_version, py_version, instanc | |||
| py_version=py_version, | |||
| framework_version=tf_version, | |||
| distribution={"mpi": {"enabled": True}}, | |||
| disable_profiler=True, | |||
There was a problem hiding this comment.
Why is this change required?
There was a problem hiding this comment.
These three tests often fail in a few small regions with InternalServiceErrors in our release pipeline. Service log showed in one instance the error happened when the training job trying to upload profiler data to s3. We are trying to see if this will fix our tests while the service team tries to root cause the issue.
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
Co-authored-by: Rui Wang Napieralski <[email protected]>
Issue #, if available:
Description of changes:
Testing done:
Merge Checklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.General
Tests
unique_name_from_baseto create resource names in integ tests (if appropriate)By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.