Skip to content

Incremental exploration based kprove using rpc endpoints #1443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 204 commits into from

Conversation

ehildenb
Copy link
Member

@ehildenb ehildenb commented Oct 30, 2022

Blocked on: #1447
Blocked on: #1440
Blocked on: runtimeverification/pyk#100
Blocked on: #1458
Blocked on: runtimeverification/pyk#112
Blocked on: #1462
Blocked on: runtimeverification/pyk#118
Blocked on: #1476
Blocked on: runtimeverification/haskell-backend#3386
Blocked on: runtimeverification/k#3059
Blocked on: #1484
Blocked on: #1478
Blocked on: #1480

This brings in a decent amount of functionality needed for making progress incremental on Foundry proofs:

  • Adds KEVM.{is_terminal,extract_branches}, which help with auto-exploration.
  • Adds auto-exploration strategy based prover, which stores branch points for inspection (as opposed to using direct calls to kprove).
    • Use option --max-iterations to limit how many calls to prover to make while exploring.
    • Use option --max-depth to limit how long each exploration step will let the prover run.
    • Use option --no-simplify-init to turn of simplification of the initial state (up to 2 minutes at proof startup, does not need to be repeated on subsequent runs).

Doing something like kevm foundry-prove --test MyContract.MyTest --max-depth 100 will make sure that every 100'th step is saved in the KCFG. That way you can save progress in case the prover crashes, and you can resume exploration from a later state with more lemmas in context.

@ehildenb ehildenb self-assigned this Oct 30, 2022
@ehildenb
Copy link
Member Author

ehildenb commented Oct 30, 2022

As an example of using this functionality, I do:

kevm foundry-kompile tests/foundry/out --verbose

Then I can run the proof I want, with --max-iterations 1 (and default --max-depth 250), and then I display the resulting CFG:

kevm foundry-prove tests/foundry/out --test UintTypeTest.test_uint8 --max-depth 250 --verbose --max-iterations 1
kevm foundry-show-cfg tests/foundry/out UintTypeTest.test_uint8

And I get something like this:

image

I want to inspect the node f6f72d..8384b2, which is 250 steps after the initial node, so I run:

kevm foundry-show-cfg tests/foundry/out UintTypeTest.test_uint8 --node f6f72d..8384b2

And I get this (highly truncated) output:

image

Now I want to continue exploration, without any depth bound, so I simple call foundry-prove again, but without the --max-iterations argument. Any additional --lemma argument I add will be used from this point on, but not to simplify original executions:

kevm foundry-prove tests/foundry/out --test UintTypeTest.test_uint8 --max-depth 250 --verbose

This leads to the following CFG eventually (has explored 2 times, then gotten stuck, cannot make progress, proof not passing):

image

After playing with the CFG a bit, and exploring more of the behaviors, I decide it's time to start the exploration over, so I add eth --reinit command to foundry-prove, which brings us back to the initial proof state (just init and target states present):

kevm foundry-prove tests/foundry/out --test UintTypeTest.test_uint8 --max-depth 250 --verbose --reinit

@ehildenb ehildenb force-pushed the incremental-exploration branch from 6923cbf to d3234e9 Compare November 3, 2022 04:59
@ehildenb ehildenb force-pushed the incremental-exploration branch from d3234e9 to 9a0a41f Compare November 11, 2022 03:52
@ehildenb ehildenb changed the title Incremental exploration and state saving/visualization Incremental exploration based kprove using rpc endpoints Nov 16, 2022
@ehildenb ehildenb force-pushed the incremental-exploration branch 2 times, most recently from 4903fae to b082ed7 Compare November 23, 2022 07:22
@ehildenb ehildenb force-pushed the incremental-exploration branch 2 times, most recently from a966100 to ae8a872 Compare November 23, 2022 08:12
rv-jenkins pushed a commit to runtimeverification/pyk that referenced this pull request Nov 23, 2022
These changes are what I needed to have a first working version of
KProve implemented fully here:
runtimeverification/evm-semantics#1443. In
particular:

- Some KAst <-> Kore problems are addressed:
- If the same variable shows up multiple times in
`KDefinition.sort_vars`, but with different sorts, instead of just
giving up we'll check if one sort is a subsort of the other. If so,
we'll return the smaller sort. Test added to `test_kast_to_kore`.
- The conversions for `Exists` are added. Test added to
`test_bidirectional`.
- `KProve.execute` method now returns `CTerm` instead of `KInner`, as it
shouldn't be able to return `mlTop()` or `mlBottom()` anyway.
- `KCFG.create_cover` now allows you to specify a witness manually
instead of trying to compute it for you. (De)Serialization requires that
we can do `Subst.{from_dict,to_dict}`, so that functionality is added
too.
- `KProve.implies` method is added as a wrapper around the `implies` RPC
endpoint. Tests are added that make sure it's giving back sane results.
By default, we much existentially bind variables in the consequent that
are not present in the antecedent to get anything useful out of the
endpoint. Tests added to `test_implies` in `test_kprove`.

Co-authored-by: devops <[email protected]>
@ehildenb ehildenb force-pushed the incremental-exploration branch 3 times, most recently from 6567463 to 6216a45 Compare November 29, 2022 21:25
@ehildenb ehildenb force-pushed the incremental-exploration branch from ef89e47 to 5600b53 Compare December 19, 2022 19:31
@ehildenb
Copy link
Member Author

Subsumed by: #1538

@ehildenb ehildenb closed this Jan 26, 2023
@ehildenb ehildenb deleted the incremental-exploration branch January 26, 2023 06:08
@ehildenb ehildenb restored the incremental-exploration branch January 26, 2023 06:08
Baltoli pushed a commit to runtimeverification/k that referenced this pull request Apr 9, 2024
These changes are what I needed to have a first working version of
KProve implemented fully here:
runtimeverification/evm-semantics#1443. In
particular:

- Some KAst <-> Kore problems are addressed:
- If the same variable shows up multiple times in
`KDefinition.sort_vars`, but with different sorts, instead of just
giving up we'll check if one sort is a subsort of the other. If so,
we'll return the smaller sort. Test added to `test_kast_to_kore`.
- The conversions for `Exists` are added. Test added to
`test_bidirectional`.
- `KProve.execute` method now returns `CTerm` instead of `KInner`, as it
shouldn't be able to return `mlTop()` or `mlBottom()` anyway.
- `KCFG.create_cover` now allows you to specify a witness manually
instead of trying to compute it for you. (De)Serialization requires that
we can do `Subst.{from_dict,to_dict}`, so that functionality is added
too.
- `KProve.implies` method is added as a wrapper around the `implies` RPC
endpoint. Tests are added that make sure it's giving back sane results.
By default, we much existentially bind variables in the consequent that
are not present in the antecedent to get anything useful out of the
endpoint. Tests added to `test_implies` in `test_kprove`.

Co-authored-by: devops <[email protected]>
Baltoli pushed a commit to runtimeverification/k that referenced this pull request Apr 9, 2024
These changes are what I needed to have a first working version of
KProve implemented fully here:
runtimeverification/evm-semantics#1443. In
particular:

- Some KAst <-> Kore problems are addressed:
- If the same variable shows up multiple times in
`KDefinition.sort_vars`, but with different sorts, instead of just
giving up we'll check if one sort is a subsort of the other. If so,
we'll return the smaller sort. Test added to `test_kast_to_kore`.
- The conversions for `Exists` are added. Test added to
`test_bidirectional`.
- `KProve.execute` method now returns `CTerm` instead of `KInner`, as it
shouldn't be able to return `mlTop()` or `mlBottom()` anyway.
- `KCFG.create_cover` now allows you to specify a witness manually
instead of trying to compute it for you. (De)Serialization requires that
we can do `Subst.{from_dict,to_dict}`, so that functionality is added
too.
- `KProve.implies` method is added as a wrapper around the `implies` RPC
endpoint. Tests are added that make sure it's giving back sane results.
By default, we much existentially bind variables in the consequent that
are not present in the antecedent to get anything useful out of the
endpoint. Tests added to `test_implies` in `test_kprove`.

Co-authored-by: devops <[email protected]>
Baltoli pushed a commit to runtimeverification/k that referenced this pull request Apr 9, 2024
These changes are what I needed to have a first working version of
KProve implemented fully here:
runtimeverification/evm-semantics#1443. In
particular:

- Some KAst <-> Kore problems are addressed:
- If the same variable shows up multiple times in
`KDefinition.sort_vars`, but with different sorts, instead of just
giving up we'll check if one sort is a subsort of the other. If so,
we'll return the smaller sort. Test added to `test_kast_to_kore`.
- The conversions for `Exists` are added. Test added to
`test_bidirectional`.
- `KProve.execute` method now returns `CTerm` instead of `KInner`, as it
shouldn't be able to return `mlTop()` or `mlBottom()` anyway.
- `KCFG.create_cover` now allows you to specify a witness manually
instead of trying to compute it for you. (De)Serialization requires that
we can do `Subst.{from_dict,to_dict}`, so that functionality is added
too.
- `KProve.implies` method is added as a wrapper around the `implies` RPC
endpoint. Tests are added that make sure it's giving back sane results.
By default, we much existentially bind variables in the consequent that
are not present in the antecedent to get anything useful out of the
endpoint. Tests added to `test_implies` in `test_kprove`.

Co-authored-by: devops <[email protected]>
Baltoli pushed a commit to runtimeverification/k that referenced this pull request Apr 10, 2024
These changes are what I needed to have a first working version of
KProve implemented fully here:
runtimeverification/evm-semantics#1443. In
particular:

- Some KAst <-> Kore problems are addressed:
- If the same variable shows up multiple times in
`KDefinition.sort_vars`, but with different sorts, instead of just
giving up we'll check if one sort is a subsort of the other. If so,
we'll return the smaller sort. Test added to `test_kast_to_kore`.
- The conversions for `Exists` are added. Test added to
`test_bidirectional`.
- `KProve.execute` method now returns `CTerm` instead of `KInner`, as it
shouldn't be able to return `mlTop()` or `mlBottom()` anyway.
- `KCFG.create_cover` now allows you to specify a witness manually
instead of trying to compute it for you. (De)Serialization requires that
we can do `Subst.{from_dict,to_dict}`, so that functionality is added
too.
- `KProve.implies` method is added as a wrapper around the `implies` RPC
endpoint. Tests are added that make sure it's giving back sane results.
By default, we much existentially bind variables in the consequent that
are not present in the antecedent to get anything useful out of the
endpoint. Tests added to `test_implies` in `test_kprove`.

Co-authored-by: devops <[email protected]>
Baltoli pushed a commit to runtimeverification/k that referenced this pull request Apr 10, 2024
These changes are what I needed to have a first working version of
KProve implemented fully here:
runtimeverification/evm-semantics#1443. In
particular:

- Some KAst <-> Kore problems are addressed:
- If the same variable shows up multiple times in
`KDefinition.sort_vars`, but with different sorts, instead of just
giving up we'll check if one sort is a subsort of the other. If so,
we'll return the smaller sort. Test added to `test_kast_to_kore`.
- The conversions for `Exists` are added. Test added to
`test_bidirectional`.
- `KProve.execute` method now returns `CTerm` instead of `KInner`, as it
shouldn't be able to return `mlTop()` or `mlBottom()` anyway.
- `KCFG.create_cover` now allows you to specify a witness manually
instead of trying to compute it for you. (De)Serialization requires that
we can do `Subst.{from_dict,to_dict}`, so that functionality is added
too.
- `KProve.implies` method is added as a wrapper around the `implies` RPC
endpoint. Tests are added that make sure it's giving back sane results.
By default, we much existentially bind variables in the consequent that
are not present in the antecedent to get anything useful out of the
endpoint. Tests added to `test_implies` in `test_kprove`.

Co-authored-by: devops <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants