Skip to content

[CI] Add Python Script for Computing Projects/Runtimes to Test #132634

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

boomanaiden154
Copy link
Contributor

This patch adds a python script, compute_projects, and associated unit
tests for computing the projects and runtimes that need to be tested in
premerge. Rewriting in Python opens up a couple new
improvements/opportunities:

  1. I personally find python to be much easier to work with than shell
    scripts for tasks like this. Particularly it becomes a lot easier to
    work with paths with proper array support.
  2. Unit testing becomes easier which makes it a lot easier to reason
    about behavior changes, especially in review.
  3. Most of the configuration is now setup in some dictionaries, which
    makes changes much easier to apply for most of the common changes.

This preserves the behavior of the existing premerge scripts as much as
possible.

Created using spr 1.3.4
boomanaiden154 added a commit to boomanaiden154/llvm-project that referenced this pull request Mar 23, 2025
This patch adds a python script, compute_projects, and associated unit
tests for computing the projects and runtimes that need to be tested in
premerge. Rewriting in Python opens up a couple new
improvements/opportunities:
1. I personally find python to be much easier to work with than shell
   scripts for tasks like this. Particularly it becomes a lot easier to
   work with paths with proper array support.
2. Unit testing becomes easier which makes it a lot easier to reason
   about behavior changes, especially in review.
3. Most of the configuration is now setup in some dictionaries, which
   makes changes much easier to apply for most of the common changes.

This preserves the behavior of the existing premerge scripts as much as
possible.

Pull Request: llvm#132634
Created using spr 1.3.4
boomanaiden154 added a commit to boomanaiden154/llvm-project that referenced this pull request Mar 23, 2025
This patch adds a python script, compute_projects, and associated unit
tests for computing the projects and runtimes that need to be tested in
premerge. Rewriting in Python opens up a couple new
improvements/opportunities:
1. I personally find python to be much easier to work with than shell
   scripts for tasks like this. Particularly it becomes a lot easier to
   work with paths with proper array support.
2. Unit testing becomes easier which makes it a lot easier to reason
   about behavior changes, especially in review.
3. Most of the configuration is now setup in some dictionaries, which
   makes changes much easier to apply for most of the common changes.

This preserves the behavior of the existing premerge scripts as much as
possible.

Pull Request: llvm#132634
Created using spr 1.3.4
Copy link

github-actions bot commented Mar 23, 2025

✅ With the latest revision this PR passed the Python code formatter.

Copy link
Contributor

@lnihlen lnihlen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good idea and will allow more people to participate in these kinds of resource/coverage tradeoff discussions.

Copy link
Contributor

@Keenuts Keenuts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK with this change, one super-nit and one question, otherwise looks good to me

Copy link
Collaborator

@joker-eph joker-eph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! LGTM but you should get some other approvals as well.

import platform
import sys

PROJECT_DEPENDENCIES = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment about the content of this? What is this conveying and when you a project include others in its list?

Reading the usage of this, it seems like it is handling transitive dependencies for example, so I assume we should only list direct dependencies?

Also some dependencies are optional (depending on the CMake configuration), so worth giving some idea of what is the intent and the effect (to know when / whether to be conservative or how to exercise judgement).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a comment.

Yes, it handles transitive dependencies automatically (and I made sure to note that in the comment).

The dependencies should match what is required for the configuration int the premerge CI (also mentioned that in the comment).

"polly": {"llvm"},
}

DEPENDENTS_TO_TEST = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(same as above somehow)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment here as well on what exactly it does and the intention surrounding it.

boomanaiden154 added a commit to boomanaiden154/llvm-project that referenced this pull request Mar 24, 2025
This patch adds a python script, compute_projects, and associated unit
tests for computing the projects and runtimes that need to be tested in
premerge. Rewriting in Python opens up a couple new
improvements/opportunities:
1. I personally find python to be much easier to work with than shell
   scripts for tasks like this. Particularly it becomes a lot easier to
   work with paths with proper array support.
2. Unit testing becomes easier which makes it a lot easier to reason
   about behavior changes, especially in review.
3. Most of the configuration is now setup in some dictionaries, which
   makes changes much easier to apply for most of the common changes.

This preserves the behavior of the existing premerge scripts as much as
possible.

Pull Request: llvm#132634
Created using spr 1.3.4
Created using spr 1.3.4
boomanaiden154 added a commit to boomanaiden154/llvm-project that referenced this pull request Mar 26, 2025
This patch adds a python script, compute_projects, and associated unit
tests for computing the projects and runtimes that need to be tested in
premerge. Rewriting in Python opens up a couple new
improvements/opportunities:
1. I personally find python to be much easier to work with than shell
   scripts for tasks like this. Particularly it becomes a lot easier to
   work with paths with proper array support.
2. Unit testing becomes easier which makes it a lot easier to reason
   about behavior changes, especially in review.
3. Most of the configuration is now setup in some dictionaries, which
   makes changes much easier to apply for most of the common changes.

This preserves the behavior of the existing premerge scripts as much as
possible.

Pull Request: llvm#132634
Created using spr 1.3.4
@boomanaiden154 boomanaiden154 merged commit 9224165 into main Mar 26, 2025
7 of 9 checks passed
@boomanaiden154 boomanaiden154 deleted the users/boomanaiden154/ci-add-python-script-for-computing-projectsruntimes-to-test branch March 26, 2025 19:37
llvm-sync bot pushed a commit to arm/arm-toolchain that referenced this pull request Mar 26, 2025
…Test

This patch adds a python script, compute_projects, and associated unit
tests for computing the projects and runtimes that need to be tested in
premerge. Rewriting in Python opens up a couple new
improvements/opportunities:
1. I personally find python to be much easier to work with than shell
   scripts for tasks like this. Particularly it becomes a lot easier to
   work with paths with proper array support.
2. Unit testing becomes easier which makes it a lot easier to reason
   about behavior changes, especially in review.
3. Most of the configuration is now setup in some dictionaries, which
   makes changes much easier to apply for most of the common changes.

This preserves the behavior of the existing premerge scripts as much as
possible.

Reviewers: ldionne, lnihlen, Endilll, tstellar, Keenuts

Reviewed By: Keenuts

Pull Request: llvm/llvm-project#132634
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants