Skip to content

feat: add spark history server api #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 5, 2023
Merged

Conversation

subinpc
Copy link
Member

@subinpc subinpc commented Jan 5, 2023

PR summary

  1. Add api methods for managing spark history server.
  2. Add submission_time and auto_termination_time fields in spark application list and details responses.

BREAKING: The type of timestamp fields in Application response models has been changed to datetime.

Fields in Application and ApplicationGetResponse objects like start_time, end_time, finish_time will now have values of type datetime instead of plain strings.

Signed-off-by: Subin Shekhar [email protected]

Fixes: <! -- link to issue -->

PR Checklist

Please make sure that your PR fulfills the following requirements:

  • The commit message follows the Angular Commit Message Guidelines.
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)

PR Type

  • Bugfix
  • Feature
  • Code style update (formatting, local variables)
  • Refactoring (no functional changes, no api changes)
  • New tests
  • Build/CI related changes
  • Documentation content changes
  • Other (please describe)

What is the current behavior?

What is the new behavior?

Does this PR introduce a breaking change?

  • Yes
  • No

Fields in Application and ApplicationGetResponse objects like start_time, end_time, finish_time will now have values of type datetime instead of plain strings.
Review the code referencing these fields and update as required.

Other information

1. Add api methods for managing spark history server.
2. Add submission_time and auto_termination_time fields in spark application list and details responses.

BREAKING: The type of timestamp fields in Application response models has been changed to `datetime`.

Fields in Application and ApplicationGetResponse objects like start_time, end_time, finish_time will now have values of type `datetime` instead of plain strings.

Signed-off-by: Subin Shekhar <[email protected]>
@subinpc subinpc requested a review from ShyamalaGowri January 5, 2023 10:00
@subinpc
Copy link
Member Author

subinpc commented Jan 5, 2023

Unit test:

% make test-unit
python -m pytest --cov=iaesdk test/unit
=================================================================== test session starts ====================================================================
platform darwin -- Python 3.8.2, pytest-7.1.2, pluggy-1.0.0
rootdir: /Users/subin/Documents/gitworkspace/ibm-iae-python-sdk
plugins: cov-2.12.1, rerunfailures-10.2
collected 184 items                                                                                                                                        

test/unit/test_ibm_analytics_engine_api_v2.py ......................................................................                                 [ 38%]
test/unit/test_ibm_analytics_engine_api_v3.py ...................................................................................................... [ 93%]
............                                                                                                                                         [100%]

---------- coverage: platform darwin, python 3.8.2-final-0 -----------
Name                                    Stmts   Miss  Cover
-----------------------------------------------------------
iaesdk/__init__.py                          5      0   100%
iaesdk/common.py                           14      0   100%
iaesdk/ibm_analytics_engine_api_v2.py    1193     98    92%
iaesdk/ibm_analytics_engine_api_v3.py    1485     63    96%
iaesdk/version.py                           1      0   100%
-----------------------------------------------------------
TOTAL                                    2698    161    94%


=================================================================== 184 passed in 1.24s ====================================================================

Integration test:

% make test-int 
python -m pytest test/integration
=================================================================== test session starts ====================================================================
platform darwin -- Python 3.8.2, pytest-7.1.2, pluggy-1.0.0
rootdir: /Users/subin/Documents/gitworkspace/ibm-iae-python-sdk
plugins: cov-2.12.1, rerunfailures-10.2
collected 34 items                                                                                                                                         

test/integration/test_ibm_analytics_engine_api_v2.py ssssssssssss                                                                                    [ 35%]
test/integration/test_ibm_analytics_engine_api_v3.py ......................                                                                          [100%]

============================================================= 22 passed, 12 skipped in 48.64s ==============================================================

@ShyamalaGowri ShyamalaGowri merged commit e4ba0e1 into master Jan 5, 2023
@ShyamalaGowri ShyamalaGowri deleted the v3-sdk-spark-history branch January 5, 2023 11:51
@Surya-Penumatcha
Copy link
Collaborator

🎉 This PR is included in version 2.2.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants