Skip to content

[ETCM-127] Regular sync integration tests #740

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Oct 22, 2020
Merged

Conversation

biandratti
Copy link
Contributor

Description

  1. Adds integration test suite for regular sync with 3 main scenarios on happy path i.e
  • synchornize with two peers which are at same best block and which have same chain
  • synchronize with two peers which are progressing forward in the same time
  • synchronize with two peers which ultimalty have divergent chains and node will be forced to resolve branches
  1. Refactoring common behavior between regular and fast sync

@lemastero
Copy link
Contributor

Commits should be signed using PGP. Take a look at: https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/about-commit-signature-verification

BTW I have a problem with pushing to repo after switching to GPG, b/c I clone the project with HTTPS. I had to update remote to have:

git remote -v
origin	[email protected]:input-output-hk/mantis.git (fetch)
origin	[email protected]:input-output-hk/mantis.git (push)

@biandratti biandratti force-pushed the ETCM-127-it-regularsync branch 2 times, most recently from ba0ceb2 to ecfd875 Compare October 15, 2020 19:22
discoveryStatus = ServerStatus.NotListening
)

lazy val tempDir = Files.createTempDirectory("temp-regular-sync")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Temporary directories in some places are cleared like so:

  def withDir(testCode: String => Any): Unit = {
    val path = Files.createTempDirectory("testdb").getFileName.toString
    try {
      testCode(path)
    } finally {
      val dir = new File(path)
      assert(!dir.exists() || dir.delete(), "File deletion failed")
    }
  }

i wonder if we should do the same here?

@lemastero
Copy link
Contributor

After margin [ETCM-141] scalafmt . This PR can be updated by git merge ETCM-127-it-regularsync-fmt
Changes


object ValidatorsExecutorAlwaysSucceed extends ValidatorsExecutorAlwaysSucceed

class FakePeer(peerName: String, fakePeerCustomConfig: FakePeerCustomConfig)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be hones regular sync FakePeer looks almost the same as FastSyncItSpecUtils.FakePeer (except some regular sync specific clasess) maybe we could have only one FakePeer in some kinda SyncItSpecUtils and in regular sync tests start it by startRegularSync and in fast sync tests start it by startFastSync ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add new abstract class called “CommonFakePeer”

_ <- peer2.waitForRegularSyncLoadLastBlock(blockNumer + 3)
} yield {
assert(peer1.bl.getBestBlock().number == peer2.bl.getBestBlock().number)
(peer1.bl.getBlockByNumber(blockNumer + 1), peer1.bl.getBlockByNumber(blockNumer + 1)) match {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe it would be more clear to check assert(peer1.bt.getTotalDifficultyByHash(bestBlock.hash) == peer2.bt.getTotalDifficultyByHash(bestBlock.hash)) ? i.e that peers total difficulties are the same after fork resolution

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

_ <- peer2.importBlocksUntil(blockNumer)(IdentityUpdate)
_ <- peer1.connectToPeers(Set(peer2.node))
_ <- peer1.startRegularSync().delayExecution(500.milliseconds)
_ <- peer2.mineNewBlock()(IdentityUpdate).delayExecution(50.milliseconds)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we could mine several blocks (like 10-15) ? (it will check if if all of them are properly queued and handled on syncer par)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the next line, where mine 10 new blocks.
_ <- peer2.mineNewBlocks(100.milliseconds, 10)(IdentityUpdate)

@biandratti biandratti force-pushed the ETCM-127-it-regularsync branch from 841b841 to a1eddfa Compare October 20, 2020 13:23
Copy link
Contributor

@KonradStaniec KonradStaniec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one last minor comment and lgtm


import scala.concurrent.duration._

class RegularSyncItSpec extends FlatSpecBase with Matchers with BeforeAndAfter {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we change BeforeAndAfter to BeforeAndAfterAll and then add :

  override def afterAll(): Unit = {
    testScheduler.shutdown()
    testScheduler.awaitTermination(60.second)
  }

? ( we should probably do this also in FastSyncItSpec)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Contributor

@KonradStaniec KonradStaniec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last comment. Also for the future, all commits titles should start with ticket number like
[ETCM-XXX] commit message

@@ -68,7 +68,8 @@ val root = {
libraryDependencies ++= dep
)
.settings(executableScriptName := name.value)
.settings(inConfig(Integration)(Defaults.testSettings :+ (Test / parallelExecution := false)): _*)
.settings(inConfig(Integration)(Defaults.testSettings
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could add formatting to all subprojects ?

@biandratti biandratti merged commit 61fbcac into develop Oct 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants