Skip to content

Move stress tester and swiftevolve to new swift_build_support build infrastructure #27580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 21, 2019
Merged

Move stress tester and swiftevolve to new swift_build_support build infrastructure #27580

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
adbd96b
[build-script] Move building skstresstester to swift_build_support
ahoppen Oct 8, 2019
1926da3
[build-script] Move building swiftevolve to swift_build_support
ahoppen Oct 18, 2019
92a0e20
[build-script] Add preset to test the stress tester and swiftevolve
ahoppen Oct 18, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions utils/build-presets.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1529,6 +1529,26 @@ assertions
# Downstream projects that import llbuild+SwiftPM.
sourcekit-lsp

#===------------------------------------------------------------------------===#
# Test Swift Stress Tester
#===------------------------------------------------------------------------===#

[preset: buildbot_skstresstester_macos]
mixin-preset=mixin_swiftpm_package_macos_platform
release
assertions
skstresstester

#===------------------------------------------------------------------------===#
# Test Swift Evolve
#===------------------------------------------------------------------------===#

[preset: buildbot_swiftevolve_macos]
mixin-preset=mixin_swiftpm_package_macos_platform
release
assertions
swiftevolve

#===------------------------------------------------------------------------===#
# Test SourceKit-LSP
#===------------------------------------------------------------------------===#
Expand Down
8 changes: 0 additions & 8 deletions utils/build-script
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -305,8 +305,6 @@ class BuildScriptInvocation(object):
"--xctest-build-type", args.build_variant,
"--swiftpm-build-type", args.build_variant,
"--swiftsyntax-build-type", args.build_variant,
"--skstresstester-build-type", args.build_variant,
"--swiftevolve-build-type", args.build_variant,
"--llbuild-build-type", args.build_variant,
"--swift-enable-assertions", str(args.swift_assertions).lower(),
"--swift-stdlib-enable-assertions", str(
Expand Down Expand Up @@ -437,10 +435,6 @@ class BuildScriptInvocation(object):
impl_args += ["--skip-build-swiftpm"]
if not args.build_swiftsyntax:
impl_args += ["--skip-build-swiftsyntax"]
if not args.build_skstresstester:
impl_args += ["--skip-build-skstresstester"]
if not args.build_swiftevolve:
impl_args += ["--skip-build-swiftevolve"]
if not args.build_playgroundsupport:
impl_args += ["--skip-build-playgroundsupport"]
if args.build_swift_dynamic_stdlib:
Expand Down Expand Up @@ -485,8 +479,6 @@ class BuildScriptInvocation(object):
"--skip-test-llbuild",
"--skip-test-swiftpm",
"--skip-test-swiftsyntax",
"--skip-test-skstresstester",
"--skip-test-swiftevolve",
"--skip-test-xctest",
"--skip-test-foundation",
"--skip-test-libdispatch",
Expand Down
164 changes: 6 additions & 158 deletions utils/build-script-impl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,6 @@ KNOWN_SETTINGS=(
# When this flag is set, the build-script will only build/install the swift-syntax parser
# This is a temporary workaround of having a separate build product for swift-syntax parser
skip-swiftsyntax-swiftside "" "skip building/installing the swift side of swiftsyntax"
skstresstester-build-type "Debug" "the build variant for the SourceKit stress tester"
swiftevolve-build-type "Debug" "the build variant for the swift-evolve tool"
llbuild-enable-assertions "1" "enable assertions in llbuild"
enable-asan "" "enable Address Sanitizer"
enable-ubsan "" "enable Undefined Behavior Sanitizer"
Expand Down Expand Up @@ -129,8 +127,6 @@ KNOWN_SETTINGS=(
skip-build-libcxx "" "set to skip building libcxx"
skip-build-swiftpm "" "set to skip building swiftpm"
skip-build-swiftsyntax "" "set to skip building swiftSyntax"
skip-build-skstresstester "" "set to skip building the SourceKit stress tester"
skip-build-swiftevolve "" "set to skip building the swift-evolve tool"
skip-build-xctest "" "set to skip building xctest"
skip-build-foundation "" "set to skip building foundation"
skip-build-libdispatch "" "set to skip building libdispatch"
Expand All @@ -145,8 +141,6 @@ KNOWN_SETTINGS=(
skip-test-llbuild "" "set to skip testing llbuild"
skip-test-swiftpm "" "set to skip testing swiftpm"
skip-test-swiftsyntax "" "set to skip testing swiftSyntax"
skip-test-skstresstester "" "set to skip testing the SourceKit stress tester"
skip-test-swiftevolve "" "set to skip testing the swift-evolve tool"
skip-test-xctest "" "set to skip testing xctest"
skip-test-foundation "" "set to skip testing foundation"
skip-test-libdispatch "" "set to skip testing libdispatch"
Expand Down Expand Up @@ -206,8 +200,6 @@ KNOWN_SETTINGS=(
install-swiftpm "" "whether to install swiftpm"
install-swiftsyntax "" "whether to install swiftsyntax"
skip-install-swiftsyntax-module "" "set to skip installing swiftsyntax modules"
install-skstresstester "" "whether to install the SourceKit stress tester"
install-swiftevolve "" "whether to install the swift-evolve tool"
install-xctest "" "whether to install xctest"
install-foundation "" "whether to install foundation"
install-libcxx "" "whether to install libc++"
Expand Down Expand Up @@ -1200,8 +1192,6 @@ LLBUILD_SOURCE_DIR="${WORKSPACE}/llbuild"
SWIFTPM_SOURCE_DIR="${WORKSPACE}/swiftpm"
SWIFTSYNTAX_SOURCE_DIR="${WORKSPACE}/swift-syntax"
STRESSTEST_PACKAGE_DIR="${WORKSPACE}/swift-stress-tester"
SKSTRESSTESTER_SOURCE_DIR="${STRESSTEST_PACKAGE_DIR}/SourceKitStressTester"
SWIFTEVOLVE_SOURCE_DIR="${STRESSTEST_PACKAGE_DIR}/SwiftEvolve"
XCTEST_SOURCE_DIR="${WORKSPACE}/swift-corelibs-xctest"
FOUNDATION_SOURCE_DIR="${WORKSPACE}/swift-corelibs-foundation"
FOUNDATION_STATIC_SOURCE_DIR="${WORKSPACE}/swift-corelibs-foundation"
Expand Down Expand Up @@ -1254,9 +1244,8 @@ PRODUCTS=("${PRODUCTS[@]}" swift)
if [[ ! "${SKIP_BUILD_LLDB}" ]] ; then
PRODUCTS=("${PRODUCTS[@]}" lldb)
fi
# LLBuild, SwiftPM, SwiftSyntax, the SourceKit stress tester and XCTest are
# dependent on Foundation, so Foundation must be added to the list of build
# products first.
# LLBuild, SwiftPM, SwiftSyntax and XCTest are dependent on Foundation, so
# Foundation must be added to the list of build products first.
if [[ ! "${SKIP_BUILD_LIBDISPATCH}" ]] ; then
PRODUCTS=("${PRODUCTS[@]}" libdispatch)
if [[ -z "${SKIP_BUILD_SWIFT_STATIC_LIBDISPATCH}" ]] ; then
Expand All @@ -1275,27 +1264,19 @@ fi
if [[ ! "${SKIP_BUILD_PLAYGROUNDSUPPORT}" ]] ; then
PRODUCTS=("${PRODUCTS[@]}" playgroundsupport)
fi
# SwiftPM, SwiftSyntax and the SourceKit stress tester are dependent on XCTest,
# so XCTest must be added to the list of build products first.
# SwiftPM and SwiftSyntax are dependent on XCTest, so XCTest must be added to
# the list of build products first.
if [[ ! "${SKIP_BUILD_XCTEST}" ]] ; then
PRODUCTS=("${PRODUCTS[@]}" xctest)
fi
# SwiftSyntax and the SourceKit stress tester are dependent on SwiftPM, so
# SwiftPM must be added to the list of build products first.
# SwiftSyntax is dependent on SwiftPM, so SwiftPM must be added to the list of
# build products first.
if [[ ! "${SKIP_BUILD_SWIFTPM}" ]] ; then
PRODUCTS=("${PRODUCTS[@]}" swiftpm)
fi
# The SourceKit stress tester and swift-evolve are dependent on SwiftSyntax, so
# it must be added to the list of build products first.
if [[ ! "${SKIP_BUILD_SWIFTSYNTAX}" ]] ; then
PRODUCTS=("${PRODUCTS[@]}" swiftsyntax)
fi
if [[ ! "${SKIP_BUILD_SKSTRESSTESTER}" ]] ; then
PRODUCTS=("${PRODUCTS[@]}" skstresstester)
fi
if [[ ! "${SKIP_BUILD_SWIFTEVOLVE}" ]] ; then
PRODUCTS=("${PRODUCTS[@]}" swiftevolve)
fi

# Checks if a given product is enabled (i.e. part of $PRODUCTS array)
function contains_product() {
Expand Down Expand Up @@ -1604,12 +1585,6 @@ function build_directory_bin() {
swiftsyntax)
echo "${root}/${SWIFTSYNTAX_BUILD_TYPE}/bin"
;;
skstresstester)
echo "${root}/${SKSTRESSTESTER_BUILD_TYPE}/bin"
;;
swiftevolve)
echo "${root}/${SWIFTEVOLVE_BUILD_TYPE}/bin"
;;
xctest)
echo "${root}/${XCTEST_BUILD_TYPE}/bin"
;;
Expand Down Expand Up @@ -1755,12 +1730,6 @@ function cmake_config_opt() {
swiftsyntax)
echo "--config ${SWIFTSYNTAX_BUILD_TYPE}"
;;
skstresstester)
echo "--config ${SKSTRESSTESTER_BUILD_TYPE}"
;;
swiftevolve)
echo "--config ${SWIFTEVOLVE_BUILD_TYPE}"
;;
xctest)
echo "--config ${XCTEST_BUILD_TYPE}"
;;
Expand Down Expand Up @@ -1885,53 +1854,6 @@ function set_swiftsyntax_build_command() {
--filecheck-exec="$(build_directory_bin ${LOCAL_HOST} llvm)/FileCheck")
}

function set_stresstester_build_script_helper_command() {
local package_name="$1"
local package_build_type="$2"
local package_build_dir="$3"

if [ "${SKIP_BUILD_SWIFTSYNTAX}" ]; then
echo "Error: Cannot build $package_name without SwiftSyntax."
exit 1
fi

local swiftsyntax_config="debug"
if [[ $(is_cmake_release_build_type "${SWIFTSYNTAX_BUILD_TYPE}") ]] ; then
swiftsyntax_config="release"
fi
local config="debug"
if [[ $(is_cmake_release_build_type "${package_build_type}") ]] ; then
config="release"
fi

stresstester_build_script_helper_command=("${STRESSTEST_PACKAGE_DIR}/build-script-helper.py")
if [[ "${VERBOSE_BUILD}" ]] ; then
stresstester_build_script_helper_command+=(-v)
fi

stresstester_build_script_helper_command+=(
--package-dir="${package_name}"
--build-dir="${package_build_dir}"
--swiftc-exec="$(build_directory_bin ${LOCAL_HOST} swift)/swiftc"
--swift-build-exec="$(swiftpm_find_tool swift-build)"
--swift-test-exec="$(swiftpm_find_tool swift-test)"
--syntax-parser-header-dir="${SWIFT_SOURCE_DIR}/include/swift-c/SyntaxParser"
--syntax-parser-lib-dir="$(build_directory ${host} swift)/lib"
--sourcekitd-dir="$(build_directory ${host} swift)/lib"
--swiftsyntax-dir="$(build_directory ${host} swiftsyntax)/${swiftsyntax_config}"
--config="${config}")
}

function set_skstresstester_build_command() {
set_stresstester_build_script_helper_command SourceKitStressTester "${SKSTRESSTESTER_BUILD_TYPE}" "$(build_directory ${host} skstresstester)"
skstresstester_build_command=("${stresstester_build_script_helper_command[@]}")
}

function set_swiftevolve_build_command() {
set_stresstester_build_script_helper_command SwiftEvolve "${SWIFTEVOLVE_BUILD_TYPE}" "$(build_directory ${host} swiftevolve)"
swiftevolve_build_command=("${stresstester_build_script_helper_command[@]}")
}

#
# Configure and build each product
#
Expand Down Expand Up @@ -2583,26 +2505,6 @@ for host in "${ALL_HOSTS[@]}"; do
set_swiftsyntax_build_command
call "${swiftsyntax_build_command[@]}"

continue
;;
skstresstester)
if [[ "$(uname -s)" != "Darwin" ]]; then
echo "error: unable to build swift-stress-tester on this platform"
continue
fi
set_skstresstester_build_command
call "${skstresstester_build_command[@]}"

continue
;;
swiftevolve)
if [[ "$(uname -s)" != "Darwin" ]]; then
echo "error: unable to build swift-evolve on this platform"
continue
fi
set_swiftevolve_build_command
call "${swiftevolve_build_command[@]}"

continue
;;
xctest)
Expand Down Expand Up @@ -3117,24 +3019,6 @@ for host in "${ALL_HOSTS[@]}"; do
# As swiftSyntax tests itself, we break early here.
continue
;;
skstresstester)
if [[ "${SKIP_TEST_SKSTRESSTESTER}" ]]; then
continue
fi
echo "--- Running tests for ${product} ---"
set_skstresstester_build_command
call "${skstresstester_build_command[@]}" test
continue
;;
swiftevolve)
if [[ "${SKIP_TEST_SWIFTEVOLVE}" ]]; then
continue
fi
echo "--- Running tests for ${product} ---"
set_swiftevolve_build_command
call "${swiftevolve_build_command[@]}" test
continue
;;
xctest)
if [[ "${SKIP_TEST_XCTEST}" ]]; then
continue
Expand Down Expand Up @@ -3491,42 +3375,6 @@ for host in "${ALL_HOSTS[@]}"; do
call "${swiftsyntax_build_command[@]}" --dylib-dir="${DYLIB_DIR}" --install
fi

continue
;;
skstresstester)
if [[ -z "${INSTALL_SKSTRESSTESTER}" ]] ; then
continue
fi
if [[ -z "${INSTALL_DESTDIR}" ]] ; then
echo "--install-destdir is required to install products."
exit 1
fi
if [[ -z "${INSTALL_SWIFTSYNTAX}" ]] ; then
echo "--install-swiftsyntax is required to install the SourceKit stress tester"
exit 1
fi

echo "--- Installing ${product} ---"
set_skstresstester_build_command
call "${skstresstester_build_command[@]}" --prefix="${host_install_destdir}${host_install_prefix}" install
continue
;;
swiftevolve)
if [[ -z "${INSTALL_SWIFTEVOLVE}" ]] ; then
continue
fi
if [[ -z "${INSTALL_DESTDIR}" ]] ; then
echo "--install-destdir is required to install products."
exit 1
fi
if [[ -z "${INSTALL_SWIFTSYNTAX}" ]] ; then
echo "--install-swiftsyntax is required to install swift-evolve"
exit 1
fi

echo "--- Installing ${product} ---"
set_swiftevolve_build_command
call "${swiftevolve_build_command[@]}" --prefix="${host_install_destdir}${host_install_prefix}" install
continue
;;
xctest)
Expand Down
24 changes: 23 additions & 1 deletion utils/build_swift/driver_arguments.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,8 @@ def _apply_default_arguments(args):
args.test_android = False
args.test_indexstoredb = False
args.test_sourcekitlsp = False
args.test_skstresstester = False
args.test_swiftevolve = False

# --skip-test-ios is merely a shorthand for host and simulator tests.
if not args.test_ios:
Expand Down Expand Up @@ -568,6 +570,10 @@ def create_argument_parser():
help='build SourceKitLSP')
option(['--install-sourcekit-lsp'], toggle_true('install_sourcekitlsp'),
help='install SourceKitLSP')
option(['--install-skstresstester'], toggle_true('install_skstresstester'),
help='install the SourceKit stress tester')
option(['--install-swiftevolve'], toggle_true('install_swiftevolve'),
help='install SwiftEvolve')
option(['--toolchain-benchmarks'],
toggle_true('build_toolchainbenchmarks'),
help='build Swift Benchmarks using swiftpm against the just built '
Expand Down Expand Up @@ -955,6 +961,10 @@ def create_argument_parser():
help='skip testing indexstore-db')
option('--skip-test-sourcekit-lsp', toggle_false('test_sourcekitlsp'),
help='skip testing sourcekit-lsp')
option('--skip-test-skstresstester', toggle_false('test_skstresstester'),
help='skip testing the SourceKit Stress tester')
option('--skip-test-swiftevolve', toggle_false('test_swiftevolve'),
help='skip testing SwiftEvolve')

# -------------------------------------------------------------------------
in_group('Build settings specific for LLVM')
Expand Down Expand Up @@ -1026,6 +1036,17 @@ def create_argument_parser():
option('--skip-test-optimize-none-with-implicit-dynamic', unsupported)
option('--skip-test-optimized', unsupported)

# -------------------------------------------------------------------------
in_group('Build-script-impl arguments (for disambiguation)')
# We need to list --skip-build-swift explicitly because otherwise argparse
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be "skip-test-swift" ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it should be. I’ll fix it in a follow-up PR.

# will auto-expand arguments like --skip-build-swift to the only known
# argument --skip-build-swiftevolve.
# These arguments are forwarded to impl_args in migration.py

option('--install-swift', toggle_true('impl_install_swift'))
option('--skip-test-swift', toggle_true('impl_skip_test_swift'))


# -------------------------------------------------------------------------
return builder.build()

Expand Down Expand Up @@ -1068,7 +1089,8 @@ def create_argument_parser():


Any arguments not listed are forwarded directly to Swift's
'build-script-impl'. See that script's help for details.
'build-script-impl'. See that script's help for details. The listed
build-script-impl arguments are only for disambiguation in the argument parser.

Environment variables
---------------------
Expand Down
Loading