feature: modify `clarify.py` as necessary to support starting time series explainability jobs. #4214

rvasahu-amazon · 2023-10-18T15:56:30Z

Description of changes:

Added TimeSeriesDataConfig, TimeSeriesModelConfig, and AsymmetricSHAPConfig (child of ExplainabilityConfig classes. Added unit tests for these new components.
Modified DataConfig, and ModelConfig to take TimeSeriesDataConfig and TimeSeriesModelConfig respectively as parameters. Added unit tests for these changes.
Modified _AnalysisConfigGenerator to support them the modified DataConfig and ModelConfig plus the new AsymmetricSHAPConfig. Added unit tests for these changes.

Note that TimeSeries explainability as a feature is not complete yet on our end. Output format is subject to change and as a result integration tests for this feature are not present at the moment.

Testing done:

Clarify unit test results:

======================================= 117 passed in 7.36s =======================================

Full unit test results:

= 2 failed, 10327 passed, 3689 skipped, 3 xpassed, 3787 warnings in 5164.77s (1:26:04) =

The two failed tests are in test_serialization.py and don't appear to have anything to do with my changes.

Integration test results:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…Clarify. These components are TimeSeriesDataConfig, TimeSeriesModelConfig, and AsymmetricSHAPConfig, alomg with unit tests for them. fix: Modified DataConfig, ModelConfig, and _AnalysisConfigGenerator to support the new components and time series explainability. documentation: added docstrings for the new components and their tests.

sagemaker-bot · 2023-10-18T16:50:50Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: b620374
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T17:01:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: b620374
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T17:08:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: b620374
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T18:19:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: af8b7de
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T18:20:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: af8b7de
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T18:33:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: af8b7de
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T18:57:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: af8b7de
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T19:12:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: af8b7de
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-10-18T19:12:59Z

Codecov Report

Attention: 10 lines in your changes are missing coverage. Please review.

Comparison is base (45166be) 86.73% compared to head (e49daaf) 86.61%.
Report is 4 commits behind head on master.

Files	Patch %	Lines
src/sagemaker/clarify.py	91.89%	9 Missing ⚠️
src/sagemaker/serve/builder/model_builder.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4214      +/-   ##
==========================================
- Coverage   86.73%   86.61%   -0.12%     
==========================================
  Files        1197      380     -817     
  Lines      105873    34997   -70876     
==========================================
- Hits        91828    30313   -61515     
+ Misses      14045     4684    -9361

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…TimeSeriesModelConfig

sagemaker-bot · 2023-10-18T20:48:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: e7869cd
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T20:58:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e7869cd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T21:35:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: e7869cd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-18T21:42:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: e7869cd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-19T03:52:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: e7869cd
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-19T04:51:32Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 23cbd0b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-19T04:55:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 23cbd0b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-19T04:59:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 23cbd0b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-19T05:24:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 23cbd0b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-19T05:43:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 23cbd0b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

src/sagemaker/clarify.py

…ies_data_config``

change: renamed TimeSeriesDataConfig.predictor_config to time_series_model_config change: modified default value of forecast_horizon to 1

change: removed default value for num_samples change: changed default value for explanation_type change: added more explicit type hint for explanation_type documentation: added more information for parameters change: reworked unit tests for AsymmetricSHAPConfig

…and be more cx friendly

…d value

mufaddal-rohawala · 2023-12-13T20:33:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 467c679
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-12-13T20:40:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 467c679
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-12-13T20:43:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 467c679
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-12-13T21:32:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 467c679
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-12-13T22:14:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 467c679
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-12-14T08:58:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e49daaf
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-12-14T09:51:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: e49daaf
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-12-14T10:27:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: e49daaf
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-12-14T10:35:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: e49daaf
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-12-14T10:52:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: e49daaf
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

what i believed was a bug was actually intended behaviour

rvasahu-amazon · 2023-12-14T19:57:35Z

Closing this pull request as the feature is first going to a closed beta before a general release.

mufaddal-rohawala · 2023-12-14T20:55:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: c5d9edd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-12-14T20:57:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: c5d9edd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-12-14T21:00:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: c5d9edd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-12-14T21:23:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c5d9edd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-12-14T21:51:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: c5d9edd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Merge branch 'master' into master

af8b7de

mufaddal-rohawala self-assigned this Oct 18, 2023

rvasahu-amazon and others added 2 commits October 18, 2023 19:48

fix: removed field use_future_covariates and related unit tests from …

75204ad

…TimeSeriesModelConfig

Merge branch 'master' into master

e7869cd

Merge branch 'aws:master' into master

23cbd0b

aws-byeldos suggested changes Oct 19, 2023

View reviewed changes

rvasahu-amazon and others added 4 commits October 19, 2023 22:42

Merge branch 'master' into master

13fc3a8

change: rename TimeSeriesDataConfig.analysis_config to ``time_ser…

f885204

…ies_data_config``

change: validate DataConfig content_type and accept_type for TS exp.

30d4472

change: renamed TimeSeriesDataConfig.predictor_config to time_series_model_config change: modified default value of forecast_horizon to 1

rvasahu-amazon added 3 commits December 13, 2023 18:54

Merge branch 'master' of github.com:rvasahu-amazon/sagemaker-python-sdk

700ab47

refactor: change baseline config param names to keep with convention …

567a4db

…and be more cx friendly

refactor: baseline config from list to dictionary where key is item_i…

467c679

…d value

rvasahu-amazon and others added 2 commits December 14, 2023 08:55

fix: set dataset_uri from s3_data_input_path

66081d4

Merge branch 'master' into master

e49daaf

fix: undo previous bug fix

abafbdd

what i believed was a bug was actually intended behaviour

rvasahu-amazon marked this pull request as ready for review December 14, 2023 19:55

rvasahu-amazon requested a review from a team as a code owner December 14, 2023 19:55

rvasahu-amazon requested review from mohanasudhan and removed request for a team December 14, 2023 19:55

rvasahu-amazon marked this pull request as draft December 14, 2023 19:56

Merge branch 'master' into master

c5d9edd

rvasahu-amazon closed this Dec 14, 2023

rvasahu-amazon mentioned this pull request Mar 14, 2024

feature: add support to clarify.py for time series explainability jobs #4503

Merged

9 tasks

feature: modify clarify.py as necessary to support starting time series explainability jobs. #4214

feature: modify clarify.py as necessary to support starting time series explainability jobs. #4214

Uh oh!

Conversation

rvasahu-amazon commented Oct 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

Uh oh!

codecov-commenter commented Oct 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2023

AWS CodeBuild CI Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mufaddal-rohawala commented Dec 13, 2023

AWS CodeBuild CI Report

feature: modify `clarify.py` as necessary to support starting time series explainability jobs. #4214

feature: modify `clarify.py` as necessary to support starting time series explainability jobs. #4214

rvasahu-amazon commented Oct 18, 2023 •

edited

Loading

codecov-commenter commented Oct 18, 2023 •

edited

Loading