increase creation time slack minutes #3606

danabens · 2023-01-20T20:51:44Z

Issue #, if available: flaky integration test, related to internal ticket

Description of changes: increase creation time verification window from 1 minute to 2 minutes

Testing done: ran locally

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2023-01-20T20:53:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 0133eb3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-20T20:55:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 0133eb3
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-20T20:57:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 0133eb3
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-20T21:06:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 84d26d9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-20T21:46:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 84d26d9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-20T21:50:22Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 84d26d9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-20T22:12:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 84d26d9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-01-20T22:12:21Z

Codecov Report

Merging #3606 (c7e17db) into master (5cb00a4) will decrease coverage by 0.80%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3606      +/-   ##
==========================================
- Coverage   89.59%   88.80%   -0.80%     
==========================================
  Files         968      227     -741     
  Lines       89584    22153   -67431     
==========================================
- Hits        80264    19673   -60591     
+ Misses       9320     2480    -6840

Impacted Files	Coverage Δ
...on3.10/site-packages/sagemaker/lineage/artifact.py
...e-packages/sagemaker/cli/compatibility/__init__.py
...3.9/site-packages/sagemaker/serverless/__init__.py
...s/sagemaker/tensorflow/training_compiler/config.py
...ython3.7/site-packages/sagemaker/workflow/retry.py
...b/python3.8/site-packages/sagemaker/mxnet/model.py
.../lib/python3.7/site-packages/sagemaker/__init__.py
...3.9/site-packages/sagemaker/jumpstart/artifacts.py
...on3.7/site-packages/sagemaker/amazon/object2vec.py
...3.10/site-packages/sagemaker/model_card/helpers.py
... and 1185 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

sagemaker-bot · 2023-01-20T22:38:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 84d26d9
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

danabens · 2023-01-20T23:01:01Z

slow test failure unrelated:

============================== slowest durations ===============================
5055.88s call     tests/integ/test_clarify_model_monitor.py::test_run_explainability_monitor
3986.16s call     tests/integ/test_model_quality_monitor.py::test_run_model_quality_monitor_baseline
3983.70s call     tests/integ/test_model_quality_monitor.py::test_run_model_quality_monitor
3742.86s call     tests/integ/test_clarify_model_monitor.py::test_run_bias_monitor
1824.01s call     tests/integ/test_auto_ml.py::test_auto_ml_fit
1701.90s setup    tests/integ/test_inference_recommender.py::test_default_right_size_and_deploy_registered_model_sklearn
1111.34s call     tests/integ/test_lda.py::test_lda
1074.15s setup    tests/integ/test_inference_recommender.py::test_advanced_right_size_and_deploy_registered_model_sklearn
899.92s call     tests/integ/test_multidatamodel.py::test_multi_data_model_deploy_train_model_from_amazon_first_party_estimator
580.79s call     tests/integ/test_lda.py::test_lda_serverless_inference
559.47s call     tests/integ/test_tuner.py::test_tuning_mxnet[1.9.0-py38]
552.38s call     tests/integ/test_mxnet.py::test_deploy_model_and_update_endpoint[1.9.0-py38-1.9.0-py38]
541.43s call     tests/integ/test_tuner.py::test_tuning_lda
494.64s call     tests/integ/test_inference_pipeline.py::test_inference_pipeline_model_deploy_and_update_endpoint
434.99s call     tests/integ/test_mxnet.py::test_deploy_estimator_with_different_instance_types[1.9.0-py38]
374.15s call     tests/integ/test_tf.py::test_mnist_async[2.11.0-2.11.0]
263.08s setup    tests/integ/test_mxnet.py::test_deploy_model_and_update_endpoint[1.9.0-py38-1.9.0-py38]
262.54s setup    tests/integ/test_mxnet.py::test_deploy_estimator_with_different_instance_types[1.9.0-py38]
214.83s call     tests/integ/test_inference_recommender.py::test_advanced_right_size_and_deploy_registered_model_sklearn
182.49s call     tests/integ/test_inference_recommender.py::test_default_right_size_and_deploy_registered_model_sklearn
181.85s setup    tests/integ/test_multidatamodel.py::test_multi_data_model_deploy_train_model_from_amazon_first_party_estimator
158.40s setup    tests/integ/test_model_quality_monitor.py::test_run_model_quality_monitor
128.62s setup    tests/integ/test_clarify_model_monitor.py::test_run_bias_monitor
128.56s setup    tests/integ/test_clarify_model_monitor.py::test_run_explainability_monitor
127.14s setup    tests/integ/test_model_quality_monitor.py::test_run_model_quality_monitor_baseline
57.44s teardown tests/integ/test_clarify_model_monitor.py::test_run_explainability_monitor
9.96s teardown tests/integ/test_model_quality_monitor.py::test_run_model_quality_monitor_baseline
8.57s teardown tests/integ/test_clarify_model_monitor.py::test_run_bias_monitor
8.38s teardown tests/integ/test_model_quality_monitor.py::test_run_model_quality_monitor
1.80s setup    tests/integ/test_inference_pipeline.py::test_inference_pipeline_model_deploy_and_update_endpoint
1.71s setup    tests/integ/test_tuner.py::test_tuning_lda
1.51s setup    tests/integ/test_lda.py::test_lda_serverless_inference
1.44s setup    tests/integ/test_tuner.py::test_tuning_mxnet[1.9.0-py38]
1.38s setup    tests/integ/test_lda.py::test_lda
1.37s setup    tests/integ/test_tf.py::test_mnist_async[2.11.0-2.11.0]
1.15s setup    tests/integ/test_auto_ml.py::test_auto_ml_fit
0.15s teardown tests/integ/test_multidatamodel.py::test_multi_data_model_deploy_train_model_from_amazon_first_party_estimator
0.01s teardown tests/integ/test_mxnet.py::test_deploy_estimator_with_different_instance_types[1.9.0-py38]

(10 durations < 0.005s hidden.  Use -vv to show these durations.)
=========================== short test summary info ============================
FAILED tests/integ/test_clarify_model_monitor.py::test_run_explainability_monitor
============= 1 failed, 15 passed, 1 warning in 5411.75s (1:30:11) =============
ERROR: InvocationError for command /codebuild/output/src494927799/src/github.com/aws/sagemaker-python-sdk/.tox/py310/bin/pytest --cov=sagemaker --cov-append tests/integ -m slow_test -n 16 --durations 0 (exited with code 1)
___________________________________ summary ____________________________________
ERROR:   py310: commands failed

[Container] 2023/01/20 22:38:08 Command did not exit successfully execute-command-if-has-matching-changes "tox $VERBOSITY -e py310 -- tests/integ -m slow_test -n 16 --durations 0" "tests/integ" "tests/data" "tests/conftest.py" "tests/__init__.py" "src/*.py" "setup.py" "setup.cfg" "buildspec-slowtests.yml" exit status 1
[Container] 2023/01/20 22:38:08 Phase complete: BUILD State: FAILED
[Container] 2023/01/20 22:38:08 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: execute-command-if-has-matching-changes "tox $VERBOSITY -e py310 -- tests/integ -m slow_test -n 16 --durations 0" "tests/integ" "tests/data" "tests/conftest.py" "tests/__init__.py" "src/*.py" "setup.py" "setup.cfg" "buildspec-slowtests.yml". Reason: exit status 1
[Container] 2023/01/20 22:38:08 Entering phase POST_BUILD
[Container] 2023/01/20 22:38:08 Phase complete: POST_BUILD State: SUCCEEDED
[Container] 2023/01/20 22:38:08 Phase context status code:  Message:

vikas0203

/bot run all

sagemaker-bot · 2023-01-23T20:22:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: c7e17db
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-23T20:26:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c7e17db
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-23T21:01:57Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: c7e17db
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-23T21:02:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: c7e17db
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-23T21:19:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: c7e17db
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

claytonparnell

/bot run slow-tests

sagemaker-bot · 2023-01-24T15:37:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c7e17db
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

claytonparnell

/bot run slow-tests

sagemaker-bot · 2023-01-24T16:23:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c7e17db
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-24T16:38:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c7e17db
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-01-24T19:22:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c7e17db
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

danabens requested a review from a team as a code owner January 20, 2023 20:51

danabens requested review from vikas0203 and removed request for a team January 20, 2023 20:51

jkasiraj approved these changes Jan 20, 2023

View reviewed changes

danabens force-pushed the increase-slack-minutes branch from 0133eb3 to 84d26d9 Compare January 20, 2023 21:05

danabens requested a review from a team January 20, 2023 22:37

jkasiraj approved these changes Jan 20, 2023

View reviewed changes

vikas0203 reviewed Jan 23, 2023

View reviewed changes

fix: increase creation time slack minutes

c7e17db

vikas0203 force-pushed the increase-slack-minutes branch from 84d26d9 to c7e17db Compare January 23, 2023 20:20

claytonparnell reviewed Jan 24, 2023

View reviewed changes

vikas0203 assigned claytonparnell Jan 24, 2023

claytonparnell approved these changes Jan 25, 2023

View reviewed changes

claytonparnell merged commit e4abc5c into aws:master Jan 25, 2023

JoseJuan98 pushed a commit to JoseJuan98/sagemaker-python-sdk that referenced this pull request Mar 4, 2023

fix: increase creation time slack minutes (aws#3606)

5a6e134

nmadan pushed a commit to nmadan/sagemaker-python-sdk that referenced this pull request Apr 18, 2023

fix: increase creation time slack minutes (aws#3606)

47b8d66

increase creation time slack minutes #3606

increase creation time slack minutes #3606

Uh oh!

Conversation

danabens commented Jan 20, 2023

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented Jan 20, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 20, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 20, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 20, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 20, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 20, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 20, 2023

AWS CodeBuild CI Report

Uh oh!

codecov-commenter commented Jan 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sagemaker-bot commented Jan 20, 2023

AWS CodeBuild CI Report

Uh oh!

danabens commented Jan 20, 2023

Uh oh!

vikas0203 left a comment

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Jan 23, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 23, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 23, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 23, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 23, 2023

AWS CodeBuild CI Report

Uh oh!

claytonparnell left a comment

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Jan 24, 2023

AWS CodeBuild CI Report

Uh oh!

claytonparnell left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Jan 24, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 24, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 24, 2023

AWS CodeBuild CI Report

Uh oh!

Uh oh!

codecov-commenter commented Jan 20, 2023 •

edited

Loading

claytonparnell left a comment •

edited

Loading