v3: Upgrades to MarQS

[ericallam]

• It’s “queue choosing” algorithm is now MUCH better (pretty much didn’t work before) and should be performant even when we have a bunch of prod queues
• There are now concurrency limits at the environment and organization level, as well as the task/queue level. So if any of them are at capacity the message won’t be dequeued. This means we can have an org wide concurrency limit, as well as different limits for dev/prod/staging
• I’ve added an admin API that can be used to update the org/env concurrency limits
• Extract the queue priority choosing strategy into an interface
• Implement a much better weighted average strategy
• “Slide the window” of queue candidates if the parent queue sends all at-capacity queues (so we won’t get stuck attempting to choose the same 12 full queues)
• Added some unit tests for the priority stuff
• Added some ideas for expanding the priority choosing strategy with more dynamic features

[changeset-bot]

⚠️ No Changeset found

Latest commit: 422f028

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR