Add a report-only mode to the BalanceCapacity middleware
#2495
Conversation
This adds a single environment variable that can be quickly set or removed to toggle capacity enforcement. If this variable is set to any value (including the empty string) then no requests will be rejected due to load. This will allow us to evolve the heuristics used in this middleware to find the right balance before turning on enforcement. This also increases the default logging percentage to a higher default.
|
This is a quick and dirty implementation of the report-only functionality. I've captured some longer-term thoughts in #2496. My primary goal for this PR is to make the master branch deployable. (We can technically re-deploy master without this but that would require updating multiple environment variables and it isn't even clear that those environment variables are the right knobs long-term.) |
src/middleware/balance_capacity.rs
Outdated
| if self.report_only { | ||
| return handler.call(request); | ||
| } |
There was a problem hiding this comment.
Instead of early exiting, it'd be nice to add some kind of log metadata like "would be dropped" to requests that would be dropped. This way we can analyze the impact with a single query in Papertrail.
|
Also, to unblock deployments to master we could simply set all environment variables to 100 beforehand. |
|
I agree with your suggested changes above, and should be able to update the implementation later today or this weekend.
I just want to point out 1 non-obvious thing regarding this. With Sidebar: If there are more requests than threads then the web server queues the requests. This means that under heavy load the response times we log become inaccurate because a request can be blocked in the web server for some amount of time before recording the start time in the logging middleware. |
Download requests are no longer counted as in-flight and the variable name is updated to reflect this. By default 20% of database connections are still reserved for updating download counts, but other requests will be rejected less often. Additionally this saves 2 atomic updates when serving the most popular request.
jtgeibel
force-pushed
the
web-capacity-report-only
branch
from
f3bcf7d to
6494ac6
Compare
|
Okay, I've added 2 commits. The first should address you comment. In report-only mode, a The second commit moves downloads outside of the in-flight request tracking. By default 20% of database connections are still reserved for updating download counts, but by not counting these requests the limits should be hit less often. |
|
This looks good! @bors r+ |
|
📌 Commit 6494ac6 has been approved by |
|
☀️ Test successful - checks-travis |
This adds a single environment variable that can be quickly set or
removed to toggle capacity enforcement. If this variable is set to any
value (including the empty string) then no requests will be rejected due
to load.
This will allow us to evolve the heuristics used in this middleware to
find the right balance before turning on enforcement.
This also increases the default logging percentage to a higher default.
r? @pietroalbini