Skip to content

[ETCM-103] Restartable state sync #730

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Oct 16, 2020
Merged

[ETCM-103] Restartable state sync #730

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
99f67a8
[ETCM-103] Make sync state process restartable
KonradStaniec Oct 9, 2020
6fc5a46
[ETCM-103] Add bloom filter
KonradStaniec Oct 9, 2020
a898ebb
Merge remote-tracking branch 'origin/develop' into etcm-103/restartab…
KonradStaniec Oct 9, 2020
0c80251
[ETCM-103] Move hardcoded values to configuration
KonradStaniec Oct 12, 2020
599311b
[ETCM-103] Refactor handling of checking for stale block
KonradStaniec Oct 13, 2020
b30f5e6
[ETCM-103] Refactor SyncControllerSpec
KonradStaniec Oct 14, 2020
6234ff2
[ETCM-103] Add more tests
KonradStaniec Oct 14, 2020
3f9f261
Merge remote-tracking branch 'origin/develop' into etcm-103/restartab…
KonradStaniec Oct 15, 2020
b60038a
[ETCM-103] Minor renaming and cleanup
KonradStaniec Oct 16, 2020
6e3c185
Merge remote-tracking branch 'origin/develop' into etcm-103/restartab…
KonradStaniec Oct 16, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 28 additions & 8 deletions src/main/resources/application.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -261,17 +261,17 @@ mantis {
blockchains {
network = "etc"

etc { include "chains/etc-chain.conf"}
etc {include "chains/etc-chain.conf"}

eth { include "chains/eth-chain.conf"}
eth {include "chains/eth-chain.conf"}

mordor { include "chains/mordor-chain.conf"}
mordor {include "chains/mordor-chain.conf"}

ropsten { include "chains/ropsten-chain.conf"}
ropsten {include "chains/ropsten-chain.conf"}

test { include "chains/test-chain.conf"}
test {include "chains/test-chain.conf"}

testnet-internal { include "chains/testnet-internal-chain.conf"}
testnet-internal {include "chains/testnet-internal-chain.conf"}
}

sync {
Expand All @@ -294,7 +294,7 @@ mantis {
sync-retry-interval = 0.5 seconds

# Response time-out from peer during sync. If a peer fails to respond within this limit, it will be blacklisted
peer-response-timeout = 3.minutes
peer-response-timeout = 30.seconds

# Interval for logging syncing status info
print-status-interval = 30.seconds
Expand Down Expand Up @@ -383,7 +383,7 @@ mantis {

# Maxium difference beetween our target block and best possible target block (current best known block - offset)
# This is to ensure that we start downloading our state as close as possible to top of the chain
max-target-difference = 20
max-target-difference = 10

# Maxium number of failure to update target block, this could happen when target block, or x blocks after target
# fail validation. Or when we keep getting old block from the network.
Expand All @@ -392,6 +392,26 @@ mantis {
# Sets max number of blocks that can be stored in queue to import on fetcher side
# Warning! This setting affects ability to go back in case of branch resolution so it should not be too low
max-fetcher-queue-size = 1000

# Expected size fo state sync bloom filter.
# Current Size of ETC state trie is aroud 150M Nodes, so 200M is set to have some reserve
# If the number of elements inserted into bloom filter would be significally higher that expected, then number
# of false positives would rise which would degrade performance of state sync
state-sync-bloom-filter-size = 200000000

# Max number of mpt nodes held in memory in state sync, before saving them into database
# 100k is around 60mb (each key-value pair has around 600bytes)
state-sync-persist-batch-size = 100000

# If new pivot block received from network will be less than fast sync current pivot block, the re-try to chose new
# pivot will be scheduler after this time. Avarage block time in etc/eth is around 15s so after this time, most of
# network peers should have new best block
pivot-block-reschedule-interval = 15.seconds

# If for most network peers, the following condition will be true:
# (peer.bestKnownBlock - pivot-block-offset) - node.curentPivotBlock > max-pivot-age
# it fast sync pivot block has become stale and it needs update
max-pivot-block-age = 96
}

pruning {
Expand Down
Loading