Skip to content

Use 'Incremental Imports' Terminology #523

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -56,11 +56,11 @@ public class IncrementalCompilationState {
self.reporter = nil
}

let enablingOrDisabling = options.contains(.enableCrossModuleIncrementalBuild)
? "Enabling"
: "Disabling"
let enablingOrDisabling = options.contains(.disableIncrementalImports)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if options contains both?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the bitflags, not the parsed options.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops! my bad

? "Disabling"
: "Enabling"
reporter?.report(
"\(enablingOrDisabling) incremental cross-module building")
"\(enablingOrDisabling) incremental imports")


guard let outputFileMap = driver.outputFileMap else {
Expand Down Expand Up @@ -120,8 +120,8 @@ extension IncrementalCompilationState {
/// When this information is combined with the output file map, swiftdeps
/// files can be located and loaded into the graph.
///
/// In a cross-module build, the dependency graph is derived from prior
/// state that is serialized alongside the build record.
/// In a build that is aware of incremental imports, the dependency graph is
/// derived from prior state that is serialized alongside the build record.
let graph: ModuleDependencyGraph
/// The set of compile jobs we can definitely skip given the state of the
/// incremental dependency graph and the status of the input files for this
Expand Down Expand Up @@ -382,18 +382,19 @@ extension IncrementalCompilationState {
/// After integrating each source file dependency graph into the driver's
/// module dependency graph, dump a dot file to the current working
/// directory showing the state of the driver's dependency graph.
///
/// FIXME: This option is not yet implemented.
public static let emitDependencyDotFileAfterEveryImport = Options(rawValue: 1 << 2)
/// After integrating each source file dependency graph, verifies the
/// integrity of the driver's dependency graph and aborts if any errors
/// are detected.
public static let verifyDependencyGraphAfterEveryImport = Options(rawValue: 1 << 3)
/// Enables the cross-module incremental build infrastructure.
/// Disables the cross-module incremental build infrastructure by forcing
/// the incremental build to ignore dependency information embedded in
/// swiftmodule files.
///
/// FIXME: This option is transitory. We intend to make this the
/// default behavior. This option should flip to a "disable" bit after that.
public static let enableCrossModuleIncrementalBuild = Options(rawValue: 1 << 4)
/// Using this flag pessimizes the incremental compilation session by forcing
/// the driver to only consider changes to the modification time of imported
/// swiftmodules, which necessarily requires cascading rebuilds.
public static let disableIncrementalImports = Options(rawValue: 1 << 4)
Comment on lines +390 to +397
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be a similar comment-rewording for the enable flag?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean here. The opposite of this flag is Options(rawValue: ~(1 << 4))

/// Enables an optimized form of start-up for the incremental build state
/// that reads the dependency graph from a serialized format on disk instead
/// of reading O(N) swiftdeps files.
Expand All @@ -405,9 +406,9 @@ extension IncrementalCompilationState {

extension IncrementalCompilationState {
@_spi(Testing) public func writeDependencyGraph() {
// If the cross-module build is not enabled, the status quo dictates we
// If incremental imports not enabled, the status quo dictates we
// not emit this file.
guard moduleDependencyGraph.info.isCrossModuleIncrementalBuildEnabled else {
if moduleDependencyGraph.info.areIncrementalImportsDisabled {
return
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ extension IncrementalCompilationState {
@_spi(Testing) public let diagnosticEngine: DiagnosticsEngine
@_spi(Testing) public let readPriorsFromModuleDependencyGraph: Bool
@_spi(Testing) public let alwaysRebuildDependents: Bool
@_spi(Testing) public let isCrossModuleIncrementalBuildEnabled: Bool
@_spi(Testing) public let areIncrementalImportsDisabled: Bool
@_spi(Testing) public let verifyDependencyGraphAfterEveryImport: Bool
@_spi(Testing) public let emitDependencyDotFileAfterEveryImport: Bool

Expand Down Expand Up @@ -69,7 +69,7 @@ extension IncrementalCompilationState {
self.readPriorsFromModuleDependencyGraph = maybeBuildRecord != nil &&
options.contains(.readPriorsFromModuleDependencyGraph)
self.alwaysRebuildDependents = options.contains(.alwaysRebuildDependents)
self.isCrossModuleIncrementalBuildEnabled = options.contains(.enableCrossModuleIncrementalBuild)
self.areIncrementalImportsDisabled = options.contains(.disableIncrementalImports)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I tend to favor the bit being true when the feature is enabled.

self.verifyDependencyGraphAfterEveryImport = options.contains(.verifyDependencyGraphAfterEveryImport)
self.emitDependencyDotFileAfterEveryImport = options.contains(.emitDependencyDotFileAfterEveryImport)
self.buildTime = maybeBuildRecord?.buildTime ?? .distantPast
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -293,7 +293,7 @@ extension ModuleDependencyGraph {

// If the graph already includes prior externals, then any new externals are changes
// Short-circuit conjunction may avoid the modTime query
let shouldTryToProcess = info.isCrossModuleIncrementalBuildEnabled &&
let shouldTryToProcess = !info.areIncrementalImportsDisabled &&
(isNewToTheGraph || lazyModTimer.hasExternalFileChanged)

// Do this no matter what in order to integrate any incremental external dependencies.
Expand Down
15 changes: 11 additions & 4 deletions Sources/SwiftDriver/Jobs/Planning.swift
Original file line number Diff line number Diff line change
Expand Up @@ -152,12 +152,19 @@ extension Driver {
if self.parsedOptions.contains(veriOpt) {
options.formUnion(.verifyDependencyGraphAfterEveryImport)
}
if self.parsedOptions.hasFlag(positive: .enableIncrementalImports,
negative: .disableIncrementalImports,
default: true) {
options.formUnion(.enableCrossModuleIncrementalBuild)

// Propagate the disable flag for cross-module incremental builds
// if necessary. Note because we're interested in *disabling* this feature,
// we consider the disable form to be the positive and enable to be the
// negative.
if self.parsedOptions.hasFlag(positive: .disableIncrementalImports,
negative: .enableIncrementalImports,
default: false) {
Comment on lines +160 to +162
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be clearer to write positive: .enableIncrementalImports, negative: .disableIncrementalImports, default: true

options.formUnion(.disableIncrementalImports)
} else {
options.formUnion(.readPriorsFromModuleDependencyGraph)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davidungar If you'd like, I can also flip the polarity on readPriorsFromModuleDependencyGraph to be something like readPriorsFromSourceFileDependencyGraphs

}

return options
}

Expand Down
10 changes: 8 additions & 2 deletions Sources/SwiftOptions/Options.swift
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ extension Option {
public static let disableImplicitSwiftModules: Option = Option("-disable-implicit-swift-modules", .flag, attributes: [.frontend, .noDriver], helpText: "Disable building Swift modules implicitly by the compiler")
public static let disableIncrementalImports: Option = Option("-disable-incremental-imports", .flag, attributes: [.frontend], helpText: "Disable cross-module incremental build metadata and driver scheduling for Swift modules")
public static let disableIncrementalLlvmCodegeneration: Option = Option("-disable-incremental-llvm-codegen", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Disable incremental llvm code generation.")
public static let disableInferPublicConcurrentValue: Option = Option("-disable-infer-public-concurrent-value", .flag, attributes: [.frontend, .noDriver], helpText: "Disable inference of ConcurrentValue conformances for public structs and enums")
public static let disableInterfaceLockfile: Option = Option("-disable-interface-lock", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Don't lock interface file when building module")
public static let disableInvalidEphemeralnessAsError: Option = Option("-disable-invalid-ephemeralness-as-error", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Diagnose invalid ephemeral to non-ephemeral conversions as warnings")
public static let disableLegacyTypeInfo: Option = Option("-disable-legacy-type-info", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Completely disable legacy type layout")
Expand Down Expand Up @@ -223,7 +224,6 @@ extension Option {
public static let enableExperimentalAdditiveArithmeticDerivation: Option = Option("-enable-experimental-additive-arithmetic-derivation", .flag, attributes: [.frontend], helpText: "Enable experimental 'AdditiveArithmetic' derived conformances")
public static let enableExperimentalConcisePoundFile: Option = Option("-enable-experimental-concise-pound-file", .flag, attributes: [.frontend, .moduleInterface], helpText: "Enable experimental concise '#file' identifier")
public static let enableExperimentalConcurrency: Option = Option("-enable-experimental-concurrency", .flag, attributes: [.helpHidden, .frontend, .noDriver, .moduleInterface], helpText: "Enable experimental concurrency model")
public static let enableExperimentalCrossModuleIncrementalBuild: Option = Option("-enable-experimental-cross-module-incremental-build", .flag, attributes: [.frontend], helpText: "(experimental) Enable cross-module incremental build metadata and driver scheduling")
public static let enableExperimentalCxxInterop: Option = Option("-enable-experimental-cxx-interop", .flag, helpText: "Allow importing C++ modules into Swift (experimental feature)")
public static let enableExperimentalEnumCodableDerivation: Option = Option("-enable-experimental-enum-codable-derivation", .flag, attributes: [.helpHidden, .frontend, .noDriver, .moduleInterface], helpText: "Enable experimental derivation of Codable for enums")
public static let enableExperimentalFlowSensitiveConcurrentCaptures: Option = Option("-enable-experimental-flow-sensitive-concurrent-captures", .flag, attributes: [.helpHidden, .frontend, .noDriver, .moduleInterface], helpText: "Enable flow-sensitive concurrent captures")
Expand All @@ -233,6 +233,7 @@ extension Option {
public static let enableImplicitDynamic: Option = Option("-enable-implicit-dynamic", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Add 'dynamic' to all declarations")
public static let enableIncrementalImports: Option = Option("-enable-incremental-imports", .flag, attributes: [.frontend], helpText: "Enable cross-module incremental build metadata and driver scheduling for Swift modules")
public static let enableInferImportAsMember: Option = Option("-enable-infer-import-as-member", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Infer when a global could be imported as a member")
public static let enableInferPublicConcurrentValue: Option = Option("-enable-infer-public-concurrent-value", .flag, attributes: [.frontend, .noDriver], helpText: "Enable inference of ConcurrentValue conformances for public structs and enums")
public static let enableInvalidEphemeralnessAsError: Option = Option("-enable-invalid-ephemeralness-as-error", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Diagnose invalid ephemeral to non-ephemeral conversions as errors")
public static let enableLibraryEvolution: Option = Option("-enable-library-evolution", .flag, attributes: [.frontend, .moduleInterface], helpText: "Build the module to allow binary-compatible library evolution")
public static let enableLlvmValueNames: Option = Option("-enable-llvm-value-names", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Add names to local values in LLVM IR")
Expand Down Expand Up @@ -309,6 +310,8 @@ extension Option {
public static let indexIgnoreSystemModules: Option = Option("-index-ignore-system-modules", .flag, attributes: [.noInteractive], helpText: "Avoid indexing system modules")
public static let indexStorePath: Option = Option("-index-store-path", .separate, attributes: [.frontend, .argumentIsPath], metaVar: "<path>", helpText: "Store indexing data to <path>")
public static let indexSystemModules: Option = Option("-index-system-modules", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Emit index data for imported serialized swift system modules")
public static let indexUnitOutputPathFilelist: Option = Option("-index-unit-output-path-filelist", .separate, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Specify index unit output paths in a file rather than on the command line")
public static let indexUnitOutputPath: Option = Option("-index-unit-output-path", .separate, attributes: [.frontend, .argumentIsPath], metaVar: "<path>", helpText: "Use <path> as the output path in the produced index data.")
public static let interpret: Option = Option("-interpret", .flag, attributes: [.helpHidden, .frontend, .noDriver], helpText: "Immediate mode", group: .modes)
public static let I: Option = Option("-I", .joinedOrSeparate, attributes: [.frontend, .argumentIsPath], helpText: "Add directory to the import search path")
public static let i: Option = Option("-i", .flag, group: .modes)
Expand Down Expand Up @@ -593,6 +596,7 @@ extension Option {
Option.disableImplicitSwiftModules,
Option.disableIncrementalImports,
Option.disableIncrementalLlvmCodegeneration,
Option.disableInferPublicConcurrentValue,
Option.disableInterfaceLockfile,
Option.disableInvalidEphemeralnessAsError,
Option.disableLegacyTypeInfo,
Expand Down Expand Up @@ -727,7 +731,6 @@ extension Option {
Option.enableExperimentalAdditiveArithmeticDerivation,
Option.enableExperimentalConcisePoundFile,
Option.enableExperimentalConcurrency,
Option.enableExperimentalCrossModuleIncrementalBuild,
Option.enableExperimentalCxxInterop,
Option.enableExperimentalEnumCodableDerivation,
Option.enableExperimentalFlowSensitiveConcurrentCaptures,
Expand All @@ -737,6 +740,7 @@ extension Option {
Option.enableImplicitDynamic,
Option.enableIncrementalImports,
Option.enableInferImportAsMember,
Option.enableInferPublicConcurrentValue,
Option.enableInvalidEphemeralnessAsError,
Option.enableLibraryEvolution,
Option.enableLlvmValueNames,
Expand Down Expand Up @@ -813,6 +817,8 @@ extension Option {
Option.indexIgnoreSystemModules,
Option.indexStorePath,
Option.indexSystemModules,
Option.indexUnitOutputPathFilelist,
Option.indexUnitOutputPath,
Option.interpret,
Option.I,
Option.i,
Expand Down
16 changes: 8 additions & 8 deletions Tests/SwiftDriverTests/IncrementalCompilationTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -458,8 +458,8 @@ extension IncrementalCompilationTests {
(.driverShowIncremental, {$0.reporter != nil}),
(.driverEmitFineGrainedDependencyDotFileAfterEveryImport, {$0.emitDependencyDotFileAfterEveryImport}),
(.driverVerifyFineGrainedDependencyGraphAfterEveryImport, {$0.verifyDependencyGraphAfterEveryImport}),
(.enableIncrementalImports, {$0.isCrossModuleIncrementalBuildEnabled}),
(.disableIncrementalImports, {!$0.isCrossModuleIncrementalBuildEnabled}),
(.enableIncrementalImports, {!$0.areIncrementalImportsDisabled}),
(.disableIncrementalImports, {$0.areIncrementalImportsDisabled}),
]

for (driverOption, stateOptionFn) in optionPairs {
Expand Down Expand Up @@ -593,7 +593,7 @@ extension IncrementalCompilationTests {
// Leave off the part after the colon because it varies on Linux:
// MacOS: The operation could not be completed. (TSCBasic.FileSystemError error 3.).
// Linux: The operation couldn’t be completed. (TSCBasic.FileSystemError error 3.)
"Enabling incremental cross-module building",
"Enabling incremental imports",
"Incremental compilation: Incremental compilation could not read build record at",
"Incremental compilation: Disabling incremental build: could not read build record",
"Incremental compilation: Created dependency graph from swiftdeps files",
Expand All @@ -617,7 +617,7 @@ extension IncrementalCompilationTests {
checkDiagnostics: checkDiagnostics,
extraArguments: extraArguments,
expectingRemarks: [
"Enabling incremental cross-module building",
"Enabling incremental imports",
"Incremental compilation: Read dependency graph",
"Incremental compilation: May skip current input: {compile: main.o <= main.swift}",
"Incremental compilation: May skip current input: {compile: other.o <= other.swift}",
Expand All @@ -637,7 +637,7 @@ extension IncrementalCompilationTests {
checkDiagnostics: checkDiagnostics,
extraArguments: extraArguments,
expectingRemarks: [
"Enabling incremental cross-module building",
"Enabling incremental imports",
"Incremental compilation: May skip current input: {compile: main.o <= main.swift}",
"Incremental compilation: Scheduing changed input {compile: other.o <= other.swift}",
"Incremental compilation: Queuing (initial): {compile: other.o <= other.swift}",
Expand Down Expand Up @@ -666,7 +666,7 @@ extension IncrementalCompilationTests {
checkDiagnostics: checkDiagnostics,
extraArguments: extraArguments,
expectingRemarks: [
"Enabling incremental cross-module building",
"Enabling incremental imports",
"Incremental compilation: Read dependency graph",
"Incremental compilation: Scheduing changed input {compile: main.o <= main.swift}",
"Incremental compilation: Scheduing changed input {compile: other.o <= other.swift}",
Expand Down Expand Up @@ -696,7 +696,7 @@ extension IncrementalCompilationTests {
checkDiagnostics: checkDiagnostics,
extraArguments: extraArguments,
expectingRemarks: [
"Enabling incremental cross-module building",
"Enabling incremental imports",
"Incremental compilation: Read dependency graph",
"Incremental compilation: Scheduing changed input {compile: main.o <= main.swift}",
"Incremental compilation: May skip current input: {compile: other.o <= other.swift}",
Expand Down Expand Up @@ -736,7 +736,7 @@ extension IncrementalCompilationTests {
checkDiagnostics: checkDiagnostics,
extraArguments: [extraArgument],
expectingRemarks: [
"Enabling incremental cross-module building",
"Enabling incremental imports",
"Incremental compilation: Read dependency graph",
"Incremental compilation: May skip current input: {compile: other.o <= other.swift}",
"Incremental compilation: Queuing (initial): {compile: main.o <= main.swift}",
Expand Down