[ci] Diff against main when determining what files have changed for pre-commit CI

[ldionne]

Since we moved to Github PRs, the workflow has changed a bit and folks often merge main back into their PR branch. This is fine, except the previous way of determining modified files for pre-commit CI would use the content modified just in the latest commit, whatever it is. This means that in case someone merged main back into their PR branch, we'd think that the files in the merge commit were modified by the PR, and we'd spuriously trigger a CI run. This should fix this issue.

The downside is that the merge target is hardcoded to main, which might not always be what we want. I still think this is an improvement over the status quo.

[ldionne]

CC @shafik

[jdenny-ornl]

It also means that all changes introduced by the PR are included instead of just the latest commit. That seems right to me. However, I've lost track of how people are thinking about stacked PRs now.

The downside is that the merge target is hardcoded to main, which might not always be what we want. I still think this is an improvement over the status quo.

Is there any way to parameterize that? Sorry, I'm not familiar with these scripts.

[metaflow]

LGTM

[ldionne]

Is there any way to parameterize that? Sorry, I'm not familiar with these scripts.

Right now it's possible to pass MODIFIED_FILES as an environment variable, however we can't customize the merge target. I don't think we have a concrete need for that right now, so let's punt. In the future, IMO the solution will be to move to Github actions instead and stop maintaining our custom triggering logic.

[ldionne]

Reverted in 705d21f because we are seeing tons of spurious CI jobs.