Bindings optimisation #1715
Bindings optimisation #1715
Conversation
…in bindings where possible. Route table key contains all the route information, which makes delete equivalent to delete_object. But it's faster. For the same reason match_object with a full object is equivalent to read.
match_object locks entire table, we'd like to avoid that. It's possible to not call delete_for_source if exchange is autodeleted. Checking an autodelete exchange will lock table on scanning for outgoing bindings anyway. But other cases will not lock the table.
|
Exclusive queue cleanup benchmark: Time to cleanup queues on connection:
Since default bindings are a separate PR now, this numbers should be shifted right for the "after" row. Still there is an optimisation. |
|
Time to delete queues/exchanges:
|
|
This can be backported to 3.7 if accepted. |
|
The interesting thing about default bindings and min-master locator is that it's counting queue masters looking at bindings, but all queues were bound. Originally it was counting bindings and selecting a node which min bindings to the queues on the node. Then it was changed in 1d413ea to count multiple bindings for the same queue as one. The assumption was that we should not account for unbound queues. But any queue had at least 1 binding (default). |
|
The test now passes. |
|
So default bindings are shown in the management API. I've missed that somehow. I still don't think we should store them in the database. Maybe we could fake them for the management API |
hairyhum
force-pushed
the
bindings-optimisation
branch
from
8539822 to
a9dc1f8
Compare
|
Moved default bindings removal to a separate PR since most of the failures are about that change. It might be even for 3.8. |
|
@hairyhum we can append them in HTTP API handlers and |
|
…or functions that list bindings, that'd cover more cases with a single change? |
|
@michaelklishin maybe. That's why it's a separate PR. |
|
Merged to 3.7 |
|
#1721, a spin-off from this issue, was merged for 3.8. |
A follow-up to #1589.
Fixes #1690.
A number of optimisations around bindings. Apparently there is not much sense in adding table indexes to binding tables, as index read locks entire table. Indexes could slightly improve performance when there are many bindings and queues, but will require a breaking change in the schema.
The changes in this branch provide performance improvement comparable to introduction of indexes without reworking the schema:
delete_objectwithdeleteas bindings are value-objects and there are no bag tablesmatch_objectis replaced withdirty_match_objectto not lock the entire bindings table. This change was reverted in d0423f9 because it was creating inconsistencies withdirty_delete. Current branch does not contain dirty deletes. Still requires more testing.list_bindings. They are not used, but slow down queue deletion and creation.Benchmark data will follow.