Skip to content

Test execution should target directory. #74

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 31, 2024
Merged

Test execution should target directory. #74

merged 1 commit into from
Oct 31, 2024

Conversation

michaelnebel
Copy link
Collaborator

@michaelnebel michaelnebel commented Oct 31, 2024

It turns out that the CI targets test files with the same name as the corresponding query files (just in a test location). This only works in case the test file has the exact same name as the query file in the source directory (this is for instance not the case for .qlref files).
In this PR we make a quick and dirty fix, which enables test execution of all tests in the corresponding test folder for a query.

We probably need to streamline the work for test execution, if we intend that this QL pack should contain many queries.
I think that with the current implementation of test execution, we

  • Might risk that tests are executed multiple times in case there are modifications to multiple files in the same src directory.
  • Might risk that not all relevant tests are executed in case they are not in the exact expected location and if the implicit assumption on dependencies change.
  • Might risk that test execution doesn't scale well (with this change the test execution on my draft PR for moving C# experimental queries takes around 35 minutes to execute).

A source of inspiration could be: https://github.com/github/codeql-coding-standards/blob/main/.github/workflows/codeql_unit_tests.yml

@michaelnebel michaelnebel marked this pull request as ready for review October 31, 2024 07:59
@pwntester pwntester merged commit 1014f8d into main Oct 31, 2024
12 checks passed
@pwntester pwntester deleted the executetests branch October 31, 2024 16:52
@pwntester
Copy link

Thanks @michaelnebel !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants