Skip to content

Refactor TRPO and VPG with EpisodesSampler #952

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 29 commits into from
Aug 11, 2023
Merged

Refactor TRPO and VPG with EpisodesSampler #952

merged 29 commits into from
Aug 11, 2023

Conversation

HenriDeh
Copy link
Member

@HenriDeh HenriDeh commented Aug 9, 2023 •
edited
Loading

PR Checklist

  • Update NEWS.md?
  • Unit tests for all structs / functions?
  • Integration and correctness tests using a simple env?
  • PR Review?
  • Add or update documentation?
  • Write docstrings for new methods?

@HenriDeh
Copy link
Member Author

HenriDeh commented Aug 9, 2023

CI can't pass without 3.3 released

@HenriDeh HenriDeh linked an issue Aug 10, 2023 that may be closed by this pull request
EpisodeSampler in Trajectories #927
Closed
3 tasks
@HenriDeh
Copy link
Member Author

I can't get buildkite to rerun now that Traj 0.3.3 is released. But tests pass. Note that this PR does not make any promise on the correctness of the two algorithms.

@HenriDeh
Copy link
Member Author

Found some issues. Do not review yet.

@HenriDeh
Copy link
Member Author

I concluded that the algorithms are broken but due to additional reasons that are not tied to the refactor. Hence, I kept the algorithms commented but at least the optimise! functions are refactored and may serve as an example for other refactors in the future. It's all I can do in this PR. Further fixing of policy gradient algorithm require contributors knowledgeable about how they work.

@HenriDeh
Copy link
Member Author

need a new approval due to conflicts

jeremiahpslewis
jeremiahpslewis approved these changes Aug 11, 2023
View reviewed changes
@jeremiahpslewis jeremiahpslewis self-requested a review August 11, 2023 10:17
jeremiahpslewis
jeremiahpslewis requested changes Aug 11, 2023
View reviewed changes
Copy link
Member

@jeremiahpslewis jeremiahpslewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see above

@HenriDeh HenriDeh merged commit 208cfb7 into main Aug 11, 2023
@HenriDeh HenriDeh deleted the episodesampler branch August 11, 2023 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jeremiahpslewis jeremiahpslewis jeremiahpslewis approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

EpisodeSampler in Trajectories
2 participants
@HenriDeh @jeremiahpslewis