move labels equality check out of critical section in summary/histogram #61
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR moves
Labels
equality check outside ofLock
.The rationale behind it is following:
App's metrics list is usually bound and don't grow a lot after some time of the app being operational. The code already has this constraint implicitly by the fact that none of
Prometheus.metrics
,Histogram.subHistograms
,Summary.subSummaries
,Counter.metrics
are bounded, and will eventually cause an OOM if grow indefinitely. From the domain standpoint, unique label names and values are discouraged and don't provide a lot of insight to app's behaviour.With the above constraint we know that after some time of app being operational, all or most of label key-value pairs are already stored in corresponding
Histogram.subHistograms
/Summary.subSummaries
. This makes getting a subSummary/subHistogram from the map a target for optimisations: getOrCreate flow becomes highly skewed towardsget
part. Suggested change improves performance ofget
part ofgetOrCreate
and increases the cost less frequently usedcreate
part. On my local machine this results in more than 2x throughput boost for the following test setups:Checklist
Motivation and Context
This PR optimises metric emission, which results in higher throughput and lower impact on running system.
Description
scope of locking is now reduced to only obtain/modify current value of
PromHistogram.subHistograms
/PromSummary.subSummaries
If subHistogram/subSummaries for given labels is not yet added to the corresponding
PromHistogram
/PromSummary
, the lock will be held twice: to get current value and to re-check + store new value.