[CDRIVER-4640] Remove the Release Archive #1333

vector-of-bool · 2023-06-30T19:03:46Z

This changeset removes the release archive.

Background

The release archive was implemented as a source distribution method that contained only the relevant files to build the libraries and tests, as well as containing the built documentation.

Why Remove It?

It is currently unclear if any users are particularly dependent upon this exact release format, and if there is significant benefit over a regular snapshot of the repository at the release tag. Maintaining the source distribution required manually listing the constituent files, which led to a lot of redundancy and trivial CMakeLists.txt that only contained these file lists. Forgetting to add/remove items from the listings would result in CI failures upon push.

Notably, the enumerated file listing restricted the inclusion of any generated build results that were not strictly known ahead-of-time. For documentation, this was a problem as Sphinx would change its output with different theme updates. This led to us vendoring in a Sphinx theme that is still used several years later, and it's age is showing 😦.

It was decided that the additional complexity and build constraints are not worth it going forward, so this changeset removes the release archive tasks.

What's Here?

Most of this changeset is deletion of the file listings that had to be maintained.
Several CI tasks consumed the output source archive as their primary input. These tasks had to be modified to consume the repository directly.
- These changes are found within the link-sample-... scripts, which now copy the source tree into their scratch space rather than expand an archive.
- Because the dependency upon the make-release-archive task is gone, the link-... test tasks now queue much faster and will not bottleneck.
- The **Release Archive Creator variant has been removed and replaced with a Smoke Tests variant, since that is what remains within it.
- A new task make-docs is defined, which builds and uploads the generated documentation pages. (This was previously done as part of make-release-archive)
The release-compile tasks common across most variants no longer uses the release archive. The RELEASE environment variable is now used simply to disable debug assertions. We may be safe to remove this task as low-value?

mongo::detail::c_platform is intended to hold usage requirements common to all libraries in the project. This will replace usage of directory-level compile/link options, which do not propagate on the generated libraries when they are installed. This allows one to export/install libmongoc libraries that expose additional usage requirements, such as linking instrumentation runtimes (coverage, sanitizers, etc.) Also: Sanitizers.cmake and ENABLE_COVERAGE now attach their options to the platform-support library. This allows installed libmongoc with sanitizers enabled to "just work," as downstream users will now receive link options for coverage and sanitizers as usage requirements from the libbson and libmongoc libraries. Also: Enable threading on c_platform rather than linking. Fix the bson and mongoc packages to pull Threads::Threads as a dependency This also affects generated pkg-config files automatically.

This removes code that accumulates lists of sources to include in the source dist, as well as the targets which generate it.

mongo::detail::c_platform is intended to hold usage requirements common to all libraries in the project. This will replace usage of directory-level compile/link options, which do not propagate on the generated libraries when they are installed. This allows one to export/install libmongoc libraries that expose additional usage requirements, such as linking instrumentation runtimes (coverage, sanitizers, etc.) Also: Sanitizers.cmake and ENABLE_COVERAGE now attach their options to the platform-support library. This allows installed libmongoc with sanitizers enabled to "just work," as downstream users will now receive link options for coverage and sanitizers as usage requirements from the libbson and libmongoc libraries. Also: Enable threading on c_platform rather than linking. Fix the bson and mongoc packages to pull Threads::Threads as a dependency This also affects generated pkg-config files automatically.

kevinAlbs

LGTM with additional comments addressed. The build simplification and removal of the CI bottleneck is much appreciated.

If this change is merged, I expect the C driver 1.25.0 will not release create/sign/upload a tarball. I suggest adding a note to the NEWS file for a new "Unreleased" section noting the removal of the tarball:

libmongoc 1.25.0 (Unreleased)
============================

* The release archive tarball is no longer created. Please instead use the snapshot tarball attached in GitHub for this release tag.

Remove the ## Creating and checking a distribution tarball section from CONTRIBUTING.md.

The RELEASE environment variable is now used simply to disable debug assertions. We may be safe to remove this task as low-value?

I incorrectly assumed those tasks were building with a release config (e.g. -DCMAKE_BUILD_TYPE=RelWithDebInfo). Though maybe that does not provide much value either. I support removing the compile-release tasks.

kevinAlbs · 2023-07-03T20:09:52Z

CMakeLists.txt

@@ -548,63 +554,7 @@ if (ENABLE_MONGOC)
   set (PACKAGE_PREFIX "mongo-c-driver-${MONGOC_DIST_VERSION}")
   set (DIST_FILE "${PACKAGE_PREFIX}.tar.gz")


Suggested change

set (DIST_FILE "${PACKAGE_PREFIX}.tar.gz")

I suggest grepping this file for dist. Several comments and lines look no longer needed.

kevinAlbs · 2023-07-03T20:10:53Z

CMakeLists.txt

-      DEPENDS
-         ${ALL_DIST} ${dist_generated_depends}
-   )
-
   if (NOT USE_SYSTEM_LIBBSON AND ENABLE_MAN_PAGES STREQUAL ON AND ENABLE_HTML_DOCS STREQUAL ON)


Remove unneeded if and else.

eramongodb

Minor feedback; otherwise, LGTM.

eramongodb · 2023-07-05T15:13:37Z

.evergreen/config_generator/components/make_docs.py

+                bash tools/poetry.sh run \
+                    bash .evergreen/scripts/build-docs.sh


Suggested change

bash tools/poetry.sh run \

bash .evergreen/scripts/build-docs.sh

bash tools/poetry.sh run bash .evergreen/scripts/build-docs.sh

Avoids messy formatting in the generated config file.

eramongodb · 2023-07-05T15:28:32Z

.evergreen/legacy_config_generator/evergreen_config_lib/tasks.py

@@ -171,7 +171,7 @@ def __init__(

        super().__init__(
            task_name=task_name,
-            depends_on=[OD([("name", "make-release-archive"), ("variant", "releng")])],
+            depends_on=[],


Suggested change

depends_on=[],

Now unused; simply remove?

eramongodb · 2023-07-05T15:30:49Z

.evergreen/legacy_config_generator/evergreen_config_lib/variants.py

@@ -76,6 +76,7 @@ def days(n: int) -> int:
            # Disable the MongoDB legacy shell download, which is not available in 5.0 for u22
            "SKIP_LEGACY_SHELL": "1"
        },
+        tags=["merge-gate"],


Suggested change

tags=["merge-gate"],

tags=["pr-merge-gate"],

To make it clearer what "merge" is referring to.

eramongodb · 2023-07-05T15:35:34Z

.evergreen/scripts/install-uninstall-check.sh

+SCRATCH_DIR=$(pwd)/.scratch
+rm -rf "$SCRATCH_DIR"
+mkdir -p "$SCRATCH_DIR"
+cp -vr -- "$SRCROOT"/* "$SCRATCH_DIR"


Consider using cmake -S . -B "$BUILD_DIR" and cmake --build "$BUILD_DIR" instead of copying the source root directory and running CMake commands with relative paths from inside $BUILD_DIR.

I was originally going to be overhauling many of these scripts, but had to abandon those changes as they are incredibly fragile, especially in assuming $PWD. Doing any slight "engoodening" of these scripts ends up cascading into a Gordian Knot. I decided to do the minimum-possible changes to get them working, but I hope to entirely replace them soon (with something hopefully faster and better as well).

rcsanchez97

The changes look good to me.

I don't think that this has been brought up yet, but remember that this change will also break release.py in the tools repo. That script uses make dist for generating the release artifacts. It will require updating, and that probably also means that the document which describes the release steps will require updating.

vector-of-bool added 12 commits June 30, 2023 10:50

Remove more usage of FindThreads

e03f6a4

Comman naming tweaks

414b568

Merge branch 'master' into platform-target

b62e6a4

Remove CMake code for source distribution

c64c1da

This removes code that accumulates lists of sources to include in the source dist, as well as the targets which generate it.

"RELEASE" mode does not use release archive

f2be43a

link-sample tasks do not use the source archive

7f2b3ee

More uses of release archive gone

5a00f1d

No more make-release-archive

46265fe

Rename make_release_archive -> make_docs

6709f02

Rename the variant

be0222e

vector-of-bool force-pushed the CDRIVER-4640-no-source-dist.3 branch from 309a6f6 to be0222e Compare June 30, 2023 19:09

vector-of-bool added 2 commits June 30, 2023 13:20

Define a variant tag for protecting merges

3f33078

Tweaks from PR comments

ffeb034

vector-of-bool requested review from eramongodb, kevinAlbs and rcsanchez97 July 3, 2023 18:53

vector-of-bool marked this pull request as ready for review July 3, 2023 18:55

Merge branch 'platform-target' into CDRIVER-4640-no-source-dist.3

16058f8

kevinAlbs approved these changes Jul 3, 2023

View reviewed changes

eramongodb approved these changes Jul 5, 2023

View reviewed changes

vector-of-bool added 6 commits July 7, 2023 12:30

Use stable Fedora for RPM build (for now)

d6b34fe

Use git archive to create the source archive

5a0112a

Additional PR cleanups and tweaks

a96c1ad

Bring packaging tasks as PR-protected

40545d1

Merge branch 'master' into CDRIVER-4640-no-source-dist.3

81b46af

Un-break zlib

847da56

rcsanchez97 approved these changes Jul 10, 2023

View reviewed changes

Printing helps?

15e227a

vector-of-bool added 7 commits July 11, 2023 13:00

Tweak pkg version detection

02b93b3

Tweak spec patchfile

133bd13

Better source filename detection

49c34a6

Get source filename from rpmspec

0278889

Tweak version detection v2

d75e48f

Note pending removal of the release archive

4312086

Merge branch 'master' into CDRIVER-4640-no-source-dist.3

0eaf61a

vector-of-bool merged commit 75db6e1 into mongodb:master Jul 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[CDRIVER-4640] Remove the Release Archive #1333

[CDRIVER-4640] Remove the Release Archive #1333

Uh oh!

vector-of-bool commented Jun 30, 2023 •

edited

Loading

Uh oh!

kevinAlbs left a comment

Uh oh!

kevinAlbs Jul 3, 2023

Uh oh!

kevinAlbs Jul 3, 2023

Uh oh!

eramongodb left a comment

Uh oh!

eramongodb Jul 5, 2023

Uh oh!

eramongodb Jul 5, 2023

Uh oh!

eramongodb Jul 5, 2023

Uh oh!

eramongodb Jul 5, 2023

Uh oh!

vector-of-bool Jul 7, 2023

Uh oh!

rcsanchez97 left a comment

Uh oh!

Uh oh!

		@@ -548,63 +554,7 @@ if (ENABLE_MONGOC)
		set (PACKAGE_PREFIX "mongo-c-driver-${MONGOC_DIST_VERSION}")
		set (DIST_FILE "${PACKAGE_PREFIX}.tar.gz")

		bash tools/poetry.sh run \
		bash .evergreen/scripts/build-docs.sh

	bash tools/poetry.sh run \
	bash .evergreen/scripts/build-docs.sh
	bash tools/poetry.sh run bash .evergreen/scripts/build-docs.sh

[CDRIVER-4640] Remove the Release Archive #1333

[CDRIVER-4640] Remove the Release Archive #1333

Uh oh!

Conversation

vector-of-bool commented Jun 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

Why Remove It?

What's Here?

Uh oh!

kevinAlbs left a comment

Choose a reason for hiding this comment

Uh oh!

kevinAlbs Jul 3, 2023

Choose a reason for hiding this comment

Uh oh!

kevinAlbs Jul 3, 2023

Choose a reason for hiding this comment

Uh oh!

eramongodb left a comment

Choose a reason for hiding this comment

Uh oh!

eramongodb Jul 5, 2023

Choose a reason for hiding this comment

Uh oh!

eramongodb Jul 5, 2023

Choose a reason for hiding this comment

Uh oh!

eramongodb Jul 5, 2023

Choose a reason for hiding this comment

Uh oh!

eramongodb Jul 5, 2023

Choose a reason for hiding this comment

Uh oh!

vector-of-bool Jul 7, 2023

Choose a reason for hiding this comment

Uh oh!

rcsanchez97 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vector-of-bool commented Jun 30, 2023 •

edited

Loading