Skip to content

Implement the GitSubmodulePlugin #4623

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Feb 7, 2023
Merged

Implement the GitSubmodulePlugin #4623

merged 7 commits into from
Feb 7, 2023

Conversation

daymxn
Copy link
Member

@daymxn daymxn commented Feb 3, 2023

Per b/267779128,

This adds a new Plugin called GitSubmodulePlugin. It's a light weight plugin that facilitates interaction with Git Submodules from Gradle.

Our current release action is broken (b/267188389), because we have SDKs with submodules- but never actually pull the submodules as apart of our build process. We could get away with just a quick little hacky task to fix this, but I opted for a Plugin instead. Primarily to take advantage of the following benefits:

  • Better build times through native Gradle plugin improvements
  • More modular implementation, should any other SDKs start using Git Submodules
  • Further establishment of plugin design principles in Gradle/Kotlin
  • Easier to maintain and document

Documentation is provided as annotations in KDoc form for everything added, including the tasks themselves. Namely, the plugin adds three tasks:

initializeGitSubmodules
updateGitSubmodules
removeGitSubmodules

Each task is annotated with background information on what it does.

Note that this plugin also binds the updating of submodules to the build process, so this immediately fixes our release pipeline without further action.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2023

Javadoc Changes:
--- /home/runner/diff/original/firebase-kotlindoc/android/com/google/firebase/database/FirebaseDatabase.html	2023-02-06 22:14:01.491894292 +0000
+++ /home/runner/diff/modified/firebase-kotlindoc/android/com/google/firebase/database/FirebaseDatabase.html	2023-02-06 22:06:44.288026205 +0000
@@ -122,7 +122,7 @@
           <tr>
             <td width="40%"><code>synchronized void</code></td>
             <td>
-              <div><code><a href="/docs/reference/android/com/google/firebase/database/FirebaseDatabase.html#setLogLevel(com.google.firebase.database.Logger.Level)">setLogLevel</a>(@<a href="https://developer.android.com/reference/kotlin/androidx/annotation/NonNull.html">NonNull</a> <a href="https://developer.android.com/reference/kotlin/java/lang/Object.html">Object</a>&nbsp;logLevel)</code></div>
+              <div><code><a href="/docs/reference/android/com/google/firebase/database/FirebaseDatabase.html#setLogLevel(com.google.firebase.database.Logger.Level)">setLogLevel</a>(@<a href="https://developer.android.com/reference/kotlin/androidx/annotation/NonNull.html">NonNull</a> <a href="/docs/reference/android/com/google/firebase/database/Logger.Level.html">Logger.Level</a>&nbsp;logLevel)</code></div>
               <p>By default, this is set to <code><a href="/docs/reference/android/com/google/firebase/database/Logger.Level.html#INFO">INFO</a></code>.</p>
             </td>
           </tr>
@@ -462,7 +462,7 @@
     </div>
     <div><a name="setLogLevel-com.google.firebase.database.Logger.Level-"></a><a name="setloglevel"></a>
       <h3 class="api-name" id="setLogLevel(com.google.firebase.database.Logger.Level)">setLogLevel</h3>
-      <pre class="api-signature no-pretty-print">synchronized&nbsp;public&nbsp;void&nbsp;<a href="/docs/reference/android/com/google/firebase/database/FirebaseDatabase.html#setLogLevel(com.google.firebase.database.Logger.Level)">setLogLevel</a>(@<a href="https://developer.android.com/reference/kotlin/androidx/annotation/NonNull.html">NonNull</a> <a href="https://developer.android.com/reference/kotlin/java/lang/Object.html">Object</a>&nbsp;logLevel)</pre>
+      <pre class="api-signature no-pretty-print">synchronized&nbsp;public&nbsp;void&nbsp;<a href="/docs/reference/android/com/google/firebase/database/FirebaseDatabase.html#setLogLevel(com.google.firebase.database.Logger.Level)">setLogLevel</a>(@<a href="https://developer.android.com/reference/kotlin/androidx/annotation/NonNull.html">NonNull</a> <a href="/docs/reference/android/com/google/firebase/database/Logger.Level.html">Logger.Level</a>&nbsp;logLevel)</pre>
       <p>By default, this is set to <code><a href="/docs/reference/android/com/google/firebase/database/Logger.Level.html#INFO">INFO</a></code>. This includes any internal errors (<code><a href="/docs/reference/android/com/google/firebase/database/Logger.Level.html#ERROR">ERROR</a></code>) and any security debug messages (<code><a href="/docs/reference/android/com/google/firebase/database/Logger.Level.html#INFO">INFO</a></code>) that the client receives. Set to <code><a href="/docs/reference/android/com/google/firebase/database/Logger.Level.html#DEBUG">DEBUG</a></code> to turn on the diagnostic logging, and <code><a href="/docs/reference/android/com/google/firebase/database/Logger.Level.html#NONE">NONE</a></code> to disable all logging.</p>
       <div class="devsite-table-wrapper">
         <table class="responsive">
@@ -473,7 +473,7 @@
           </thead>
           <tbody class="list">
             <tr>
-              <td width="40%"><code>@<a href="https://developer.android.com/reference/kotlin/androidx/annotation/NonNull.html">NonNull</a> <a href="https://developer.android.com/reference/kotlin/java/lang/Object.html">Object</a>&nbsp;logLevel</code></td>
+              <td width="40%"><code>@<a href="https://developer.android.com/reference/kotlin/androidx/annotation/NonNull.html">NonNull</a> <a href="/docs/reference/android/com/google/firebase/database/Logger.Level.html">Logger.Level</a>&nbsp;logLevel</code></td>
               <td>
                 <p>The desired minimum log level</p>
               </td>
--- /home/runner/diff/original/firebase-kotlindoc/kotlin/com/google/firebase/database/FirebaseDatabase.html	2023-02-06 22:14:01.551896533 +0000
+++ /home/runner/diff/modified/firebase-kotlindoc/kotlin/com/google/firebase/database/FirebaseDatabase.html	2023-02-06 22:06:44.320027220 +0000
@@ -122,7 +122,7 @@
           <tr>
             <td width="40%"><code>synchronized <a href="https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-unit/index.html">Unit</a></code></td>
             <td>
-              <div><code><a href="/docs/reference/kotlin/com/google/firebase/database/FirebaseDatabase.html#setLogLevel(com.google.firebase.database.Logger.Level)">setLogLevel</a>(logLevel:&nbsp;<a href="https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-any/index.html">Any</a>)</code></div>
+              <div><code><a href="/docs/reference/kotlin/com/google/firebase/database/FirebaseDatabase.html#setLogLevel(com.google.firebase.database.Logger.Level)">setLogLevel</a>(logLevel:&nbsp;<a href="/docs/reference/kotlin/com/google/firebase/database/Logger.Level.html">Logger.Level</a>)</code></div>
               <p>By default, this is set to <code><a href="/docs/reference/kotlin/com/google/firebase/database/Logger.Level.html#INFO">INFO</a></code>.</p>
             </td>
           </tr>
@@ -462,7 +462,7 @@
     </div>
     <div><a name="setLogLevel-com.google.firebase.database.Logger.Level-"></a><a name="setloglevel"></a>
       <h3 class="api-name" id="setLogLevel(com.google.firebase.database.Logger.Level)">setLogLevel</h3>
-      <pre class="api-signature no-pretty-print">synchronized&nbsp;fun&nbsp;<a href="/docs/reference/kotlin/com/google/firebase/database/FirebaseDatabase.html#setLogLevel(com.google.firebase.database.Logger.Level)">setLogLevel</a>(logLevel:&nbsp;<a href="https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-any/index.html">Any</a>):&nbsp;<a href="https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-unit/index.html">Unit</a></pre>
+      <pre class="api-signature no-pretty-print">synchronized&nbsp;fun&nbsp;<a href="/docs/reference/kotlin/com/google/firebase/database/FirebaseDatabase.html#setLogLevel(com.google.firebase.database.Logger.Level)">setLogLevel</a>(logLevel:&nbsp;<a href="/docs/reference/kotlin/com/google/firebase/database/Logger.Level.html">Logger.Level</a>):&nbsp;<a href="https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-unit/index.html">Unit</a></pre>
       <p>By default, this is set to <code><a href="/docs/reference/kotlin/com/google/firebase/database/Logger.Level.html#INFO">INFO</a></code>. This includes any internal errors (<code><a href="/docs/reference/kotlin/com/google/firebase/database/Logger.Level.html#ERROR">ERROR</a></code>) and any security debug messages (<code><a href="/docs/reference/kotlin/com/google/firebase/database/Logger.Level.html#INFO">INFO</a></code>) that the client receives. Set to <code><a href="/docs/reference/kotlin/com/google/firebase/database/Logger.Level.html#DEBUG">DEBUG</a></code> to turn on the diagnostic logging, and <code><a href="/docs/reference/kotlin/com/google/firebase/database/Logger.Level.html#NONE">NONE</a></code> to disable all logging.</p>
       <div class="devsite-table-wrapper">
         <table class="responsive">
@@ -473,7 +473,7 @@
           </thead>
           <tbody class="list">
             <tr>
-              <td width="40%"><code>logLevel:&nbsp;<a href="https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-any/index.html">Any</a></code></td>
+              <td width="40%"><code>logLevel:&nbsp;<a href="/docs/reference/kotlin/com/google/firebase/database/Logger.Level.html">Logger.Level</a></code></td>
               <td>
                 <p>The desired minimum log level</p>
               </td>

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2023

📝 PRs merging into main branch

Our main branch should always be in a releasable state. If you are working on a larger change, or if you don't want this change to see the light of the day just yet, consider using a feature branch first, and only merge into the main branch when the code complete and ready to be released.

Add the 'main-merge-ack' label to your PR to confirm merging into the main branch is intended.

@vkryachko
Copy link
Member

For the release job, isn't it easier to just add the following to our workflow config?

with:
  submodules: true

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2023

Unit Test Results

   399 files  ±0     399 suites  ±0   18m 14s ⏱️ +14s
4 755 tests ±0  4 733 ✔️ +2  21 💤 ±0  1  - 2 
4 771 runs  ±0  4 749 ✔️ +2  21 💤 ±0  1  - 2 

For more details on these failures, see this check.

Results for commit 83a86c2. ± Comparison against base commit fea70f7.

♻️ This comment has been updated with latest results.

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Feb 3, 2023

Coverage Report 1

This report is too large (197,377 characters) to be displayed here in a GitHub comment. Please use the below link to see the full report on Google Cloud Storage.

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/jQqijANkFJ.html

@daymxn
Copy link
Member Author

daymxn commented Feb 3, 2023

For the release job, isn't it easier to just add the following to our workflow config?

with:
  submodules: true

Good question. Yes and no. While it would quickly and easily fix the release job, that's only in the context of the workflow. The underlying task itself would still be broken. So if you were to be debugging this locally, it wouldn't align with what you would get from the workflow. Additionally, this may come up in several places in the future regarding submodules not being initialized for a build. I believe it'd be much easier long-term, and more robust- to instead fix it at the build level versus the workflow level. WDYT?

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2023

buildSrc Test Results

18 tests   18 ✔️  1m 31s ⏱️
  4 suites    0 💤
  4 files      0

Results for commit 83a86c2.

♻️ This comment has been updated with latest results.

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Feb 3, 2023

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Feb 3, 2023

Startup Time Report 1

Note: Layout is sometimes suboptimal due to limited formatting support on GitHub. Please check this report on GCS.

Startup time comparison between the CI merge commit (7c166a3) and the base commit (fea70f7) are not available.

No macrobenchmark data found for the base commit (fea70f7). Analysis for the CI merge commit (7c166a3) can be found at:

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/1dJHOl0u6P/index.html

@daymxn daymxn enabled auto-merge (squash) February 5, 2023 21:10
Copy link
Contributor

@yifanyang yifanyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we vision that going forward product teams shall manipulate submodules only through gradle?

  • If so, do we need to support other submodule commands? Not sure if all existing submodule use scenarios are covered.
  • If not, can product teams run standalone git submodule commands? Will they be interfered with implicit invocations from gradle?

@daymxn
Copy link
Member Author

daymxn commented Feb 6, 2023

Do we vision that going forward product teams shall manipulate submodules only through gradle?

Only in the context of it being a requirement for any given gradle task. So if they need certain submodule actions to occur before a build, or before an artifact is generated- then ideally this would be bound via gradle. Aligns with keeping tasks aligned with one another, is more easily conveyed, and in the context of gradle specifically- allows us to take advantage of gradle cache & speed optimizations.

  • If so, do we need to support other submodule commands? Not sure if all existing submodule use scenarios are covered.

Right now, we only have one place using submodules at all. Should more teams start to use them in more elaborate ways (that have implications for other gradle tasks) then we can expand this. I intentionally wrote this in a way to which adding additional functionality in the future will be as painless as possible. But seeing as there is only one usage of submodules right now (fetching them)- this avoids unnecessary implementation / maintenance overhead. I'd like to cover our use cases rn, as it has implications for our current issues, and re-visit in the future as needed- as we have more pressing tasks on our plate.

  • If not, can product teams run standalone git submodule commands? Will they be interfered with implicit invocations from gradle?

Absolutely. These tasks track state via the actual state of the module, and will skip when the task is unnecessary.

@daymxn daymxn force-pushed the daymon-fix-release-job branch from 793a7b1 to 83a86c2 Compare February 6, 2023 21:57
@yifanyang yifanyang disabled auto-merge February 7, 2023 18:50
@yifanyang yifanyang merged commit 7b7a6b1 into master Feb 7, 2023
@yifanyang yifanyang deleted the daymon-fix-release-job branch February 7, 2023 18:50
@firebase firebase locked and limited conversation to collaborators Mar 10, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants