build: avoid conflicts when publishing snapshot builds #19737
Conversation
googlebot
added
the
cla: yes
For the past year, we've had CI failures when multiple PRs have been merged. Since we run CI for all landed commits, the snapshot publish job runs multiple times concurrently and could end up pushing to the snapshot repositories out of order. This can result in conflicts when multiple jobs push artifacts at the same time. e.g. https://circleci.com/gh/angular/components/155407. We intend to fix this by queuing snapshot publishing across CircleCI builds. This is done using the CircleCI API. We constantly check the API to check that there are no previous snapshot jobs running. Instead of implementing this logic our own, we leverage a CircleCI orb that provides this functionality.
devversion
force-pushed
the
build/avoid-snapshot-publish-conflicts
branch
from
d3aec84 to
7121fbe
Compare
|
Note: There is no easy way to test this on my end. The orb does not run on forked builds, so I cannot test this on my fork. @jelbourn @josephperrott If we think this is worth trying, I think we should just merge it, and if it doesn't work, revert it immediately? |
devversion
added
merge safe
target: patch
There was a problem hiding this comment.
LGTM
@devversion for testing, you should be able to push a branch on upstream. Since we build on all upstream branches, you should be able to test there?
josephperrott
removed
the
action: merge
|
Removed |
|
I say just go for it |
|
Yeah, I was thinking about testing this upstream, but snapshot builds do not run for arbitrary upstream branches, and CircleCI auto-cancels redundant builds on branches (except for the default one. i.e. master). I gave it a shot upstream to just see if there is any configuration issue, but looks like we should be good. |
For the past year, we've had CI failures when multiple PRs have been merged. Since we run CI for all landed commits, the snapshot publish job runs multiple times concurrently and could end up pushing to the snapshot repositories out of order. This can result in conflicts when multiple jobs push artifacts at the same time. e.g. https://circleci.com/gh/angular/components/155407. We intend to fix this by queuing snapshot publishing across CircleCI builds. This is done using the CircleCI API. We constantly check the API to check that there are no previous snapshot jobs running. Instead of implementing this logic our own, we leverage a CircleCI orb that provides this functionality. (cherry picked from commit b68ae0d)
|
This issue has been automatically locked due to inactivity. Read more about our automatic conversation locking policy. This action has been performed automatically by a bot. |
For the past year, we've had CI failures when multiple PRs
have been merged. Since we run CI for all landed commits, the
snapshot publish job runs multiple times concurrently and could
end up pushing to the snapshot repositories out of order.
This can result in conflicts when multiple jobs push artifacts
at the same time. e.g. https://circleci.com/gh/angular/components/155407.
We intend to fix this by queuing snapshot publishing across CircleCI
builds. This is done using the CircleCI API. We constantly check the
API to check that there are no previous snapshot jobs running.
Instead of implementing this logic our own, we leverage a CircleCI
orb that provides this functionality.