Skip to content

config cachedatset for dataset_type_performance #1220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Feb 20, 2023

Conversation

wyli
Copy link
Contributor

@wyli wyli commented Feb 16, 2023

Fixes #1214

Description

update the cachedatset config to improve the performance

Checks

  • Avoid including large-size files in the PR.
  • Clean up long text outputs from code cells in the notebook.
  • For security purposes, please check the contents and remove any sensitive info such as user names and private key.
  • Ensure (1) hyperlinks and markdown anchors are working (2) use relative paths for tutorial repo files (3) put figure and graphs in the ./figure folder
  • Notebook runs automatically ./runner.sh -t <path to .ipynb file>

Signed-off-by: Wenqi Li <[email protected]>
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Signed-off-by: Wenqi Li <[email protected]>
@wyli wyli marked this pull request as ready for review February 16, 2023 22:34
@Nic-Ma
Copy link
Contributor

Nic-Ma commented Feb 17, 2023

Hi @wyli ,

May I know the root cause of the performance issue? Why image_only can help improve the CacheDataset than persistentDataset?
And BTW, if you want to show runtime_cache feature in this tutorial, please update the plot figure to remove the init cache part.

Thanks.

@wyli
Copy link
Contributor Author

wyli commented Feb 17, 2023

Yes, the root cause is the cache init being slow, and maybe related to Project-MONAI/MONAI#5930.

image_only will remove the additional unnecessary meta_dict and the warning message. It doesn't change the relative speed comparisons.

We should keep cache_init, because the cache dataset still have the option of precompute the cache, the user can experiment and plot.

@Nic-Ma
Copy link
Contributor

Nic-Ma commented Feb 17, 2023

OK, could you please help add some description or comment in the notebook to explain the "runtime_cache" usage and init_cache plot? I feel confused when I see the plot chart which has init_cache time = 0.
Others look good to me.

Thanks in advance.

@wyli wyli force-pushed the type_dataset_compare branch 2 times, most recently from 578fc59 to 70319f0 Compare February 17, 2023 12:54
@wyli wyli force-pushed the type_dataset_compare branch from 70319f0 to 1a808ea Compare February 17, 2023 12:54
@dongyang0122
Copy link
Collaborator

If image_only is important to showcase advantages of PersistentDataset, I would suggest to add few words under the section of PersistentDataset explaining it? The rest of the content looks great.

@dongyang0122 dongyang0122 merged commit de93605 into Project-MONAI:main Feb 20, 2023
@wyli wyli deleted the type_dataset_compare branch February 20, 2023 17:49
boneseva pushed a commit to boneseva/MONAI-tutorials that referenced this pull request Apr 21, 2024
Fixes Project-MONAI#1214

### Description
update the cachedatset config to improve the performance

### Checks
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Avoid including large-size files in the PR.
- [x] Clean up long text outputs from code cells in the notebook.
- [x] For security purposes, please check the contents and remove any
sensitive info such as user names and private key.
- [x] Ensure (1) hyperlinks and markdown anchors are working (2) use
relative paths for tutorial repo files (3) put figure and graphs in the
`./figure` folder
- [x] Notebook runs automatically `./runner.sh -t <path to .ipynb file>`

---------

Signed-off-by: Wenqi Li <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Compare results in dataset_type_performance.ipynb
3 participants