Skip to content
This repository was archived by the owner on Jun 2, 2025. It is now read-only.

new gfs loader #338

Merged
merged 29 commits into from
Jun 28, 2024
Merged

new gfs loader #338

merged 29 commits into from
Jun 28, 2024

Conversation

AUdaltsova
Copy link
Contributor

@AUdaltsova AUdaltsova commented Jun 27, 2024 •
edited
Loading

Pull Request

Description

New version of GFS loader added:

  • New GFS data has step 0 missing for irradiance (variables dswrf, dlwrf), currently defaulting to NaN.
  • Added a commented-out bit that extrapolates them in the loader
  • Added a warning if using irradiance
  • Added test data for gfs
  • Added standard test for gfs loader

How Has This Been Tested?

ran small batch creation on leonardo with and without irradiance variables

Checklist:

  • My code follows OCF's coding style guidelines
  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked my code and corrected any misspellings

AUdaltsova and others added 21 commits June 27, 2024 15:58
@AUdaltsova
new gfs loader
35a032b
@AUdaltsova
add warning to filter_channels.py
e3a002b
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
e20c917 
for more information, see https://pre-commit.ci
@AUdaltsova
pass provider label to filter_channels
5735f9f
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
ce4d82a 
for more information, see https://pre-commit.ci
@AUdaltsova
Update warning in filter_channels.py
4d9a132
@AUdaltsova
add gfs test data
da13cfd
@AUdaltsova
Added gfs load test to test_load_nwp.py
fcd6fe7
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
5cfcf9c 
for more information, see https://pre-commit.ci
@AUdaltsova
Update gfs.py
a61b8d4
@AUdaltsova
Update filter_channels.py
8beef57
@AUdaltsova
Update test_load_nwp.py
6d93665
@AUdaltsova
linting filter_channels.py
c19f4af
@AUdaltsova
linting gfs.py
6f3393b
@AUdaltsova
Delete tests/data/gfs.zarr.zip
5795881
@AUdaltsova
add gfs test data
50ac208
@AUdaltsova
Create .zmetadata
ab097f0
@AUdaltsova
Create .zgroup
7432995
@AUdaltsova
Create .zattrs
26d18c4
@AUdaltsova
Update gfs test data path test_load_nwp.py
32bfe1e
@AUdaltsova
Update provider label for gfs, cleaner type checks in test_load_nwp.py
d6f7a9f
@AUdaltsova AUdaltsova marked this pull request as ready for review June 28, 2024 11:25
@AUdaltsova AUdaltsova requested review from Sukh-P and peterdudfield and removed request for peterdudfield and Sukh-P June 28, 2024 11:25
@AUdaltsova
Copy link
Contributor Author

hang on I'll add mean and std here as well

Sukh-P
Sukh-P approved these changes Jun 28, 2024
View reviewed changes
Copy link
Member

@Sukh-P Sukh-P left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice work!

peterdudfield
peterdudfield reviewed Jun 28, 2024
View reviewed changes
Sukh-P
Sukh-P reviewed Jun 28, 2024
View reviewed changes
peterdudfield
peterdudfield reviewed Jun 28, 2024
View reviewed changes
peterdudfield
peterdudfield reviewed Jun 28, 2024
View reviewed changes
peterdudfield
peterdudfield approved these changes Jun 28, 2024
View reviewed changes
Copy link
Contributor

@peterdudfield peterdudfield left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ive left one tiny comment about a import, but otherwise all good

@AUdaltsova AUdaltsova merged commit 9effc61 into main Jun 28, 2024
3 checks passed
@AUdaltsova AUdaltsova deleted the gfs_loader branch June 28, 2024 15:36
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@Sukh-P Sukh-P Sukh-P approved these changes

@peterdudfield peterdudfield peterdudfield approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@AUdaltsova @peterdudfield @Sukh-P