Skip to content

Commit b373660

Browse files
geo2arv-jenkins
andauthored
Only run tests that Kontrol's integration tests job runs (#3879)
- consolidate `pytest` options into a bash variable - select only the test that are run by the integration tests CI [job](https://github.com/runtimeverification/kontrol/blob/921dfe8ddd0774265d59122d0ace19ac5889007a/.github/workflows/test-pr.yml#L119). The deselected tests need to run in isolation, as they fail if ran together. This has something to do with proof artifacts produced by other tests. Ideally, we should run the extra tests too. Co-authored-by: rv-jenkins <[email protected]>
1 parent 6e57788 commit b373660

File tree

1 file changed

+7
-3
lines changed

1 file changed

+7
-3
lines changed

scripts/performance-tests-kontrol.sh

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -114,14 +114,18 @@ cp -r $PYTEST_TEMP_DIR/foundry/* $FOUNDRY_DIR
114114

115115
mkdir -p $SCRIPT_DIR/logs
116116

117-
feature_shell "make test-integration TEST_ARGS='--foundry-root $FOUNDRY_DIR --maxfail=0 --numprocesses=$PYTEST_PARALLEL -vv $BUG_REPORT' | tee $SCRIPT_DIR/logs/kontrol-$KONTROL_VERSION-$FEATURE_BRANCH_NAME.log"
117+
# set test arguments and select which tests to run
118+
QUOTE='"'
119+
TEST_ARGS="--foundry-root $FOUNDRY_DIR --maxfail=0 --numprocesses=$PYTEST_PARALLEL -vv $BUG_REPORT -k 'not (test_kontrol_cse or test_foundry_minimize_proof or test_kontrol_end_to_end)'"
120+
121+
feature_shell "make test-integration TEST_ARGS=$QUOTE$TEST_ARGS$QUOTE | tee $SCRIPT_DIR/logs/kontrol-$KONTROL_VERSION-$FEATURE_BRANCH_NAME.log"
118122
killall kore-rpc-booster || echo "no zombie processes found"
119123

120124
if [ -z "$BUG_REPORT" ]; then
121125
if [ ! -e "$SCRIPT_DIR/logs/kontrol-$KONTROL_VERSION-master-$MASTER_COMMIT_SHORT.log" ]; then
122126
# remove proofs so that they are not reused by the master shell call
123-
rm -r $FOUNDRY_DIR/out/proofs
124-
master_shell "make test-integration TEST_ARGS='--foundry-root $FOUNDRY_DIR --maxfail=0 --numprocesses=$PYTEST_PARALLEL -vv' | tee $SCRIPT_DIR/logs/kontrol-$KONTROL_VERSION-master-$MASTER_COMMIT_SHORT.log"
127+
rm -rf $FOUNDRY_DIR/out/proofs
128+
master_shell "make test-integration TEST_ARGS=$QUOTE$TEST_ARGS$QUOTE | tee $SCRIPT_DIR/logs/kontrol-$KONTROL_VERSION-master-$MASTER_COMMIT_SHORT.log"
125129
killall kore-rpc-booster || echo "no zombie processes found"
126130
fi
127131

0 commit comments

Comments
 (0)