Fix: Retry CLI launch if needed #8221

engelke · 2022-07-28T20:54:58Z

Description

snippet-bot · 2022-07-28T20:55:07Z

No region tags are edited in this PR.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

Refresh this comment

ace-n

I reviewed the first file; I assume the changes are copy-pasted into the others?

(Either way, I'll take a closer look tomorrow.)

appengine/standard_python3/bundled-services/blobstore/django/main_test.py

leahecole

Unless there is there a reason we have to use time please use backoff, which is our preferred library for retries. This example in the authoring guide is based on time, but you can also pass max_tries instead of max_time to do something closer to what I think you're trying to implement

ace-n

There seems to be quite a bit of code duplication here. 🙁

@engelke is there any chance we can move the duplicative code into some "shared" file, and either
a) symlink to it, or
b) import it using Python's path-ing mechanisms?

(Also - I believe we [historically] don't worry about making tests easy-to-understand unless they're explicitly included in the cloud.google.com docs.)

engelke · 2022-08-03T17:31:16Z

@ace-n I've always had each sample self-contained. I prefer it the way it currently is.

ace-n

First file LGTM, other than concerns about result being logged on failure.

(I'm assuming the other files' changes are similar; @engelke LMK if you'd like me to review them one-by-one. 👍 )

ace-n · 2022-08-03T22:13:40Z

appengine/standard_python3/bundled-services/blobstore/django/main_test.py

+    """
+
+    result = gcloud_cli(f"app deploy --no-promote --version={uuid.uuid4().hex}")
+    version_id = result["versions"][0]["id"]


If result is defined but result["versions"] is not, will this print out a meaningful error message? (Ideally, one that includes result itself?)

It has been addressed, as requested.

* chenged the dag to load ghcn dataset * data preprocessing done * modified preprocessing * dataproc file added * code runs great * modifyed code based on Brad, still buggy * finished modifying, haven't sync wit hDAG * finished modifying DAG codes * ready for draft PR * pass lint * addressed Brad and Leah's comments * pass nox lint * pass nox lint * Fix: Retry CLI launch if needed (#8221) * Fix: add region tags * Fix: region tag typos * Fix: urlpatterns moved to end * Fix: typo * Fix: cli retries to fix flakiness * Fix: remove duplicate tags * Fix: use backoff for retries * Fix: lint import order error * address Leah's comments about typo and comments Co-authored-by: Charles Engelke <[email protected]>

* Kaiyang expansion project 2022 (#8224) * chenged the dag to load ghcn dataset * data preprocessing done * modified preprocessing * dataproc file added * code runs great * modifyed code based on Brad, still buggy * finished modifying, haven't sync wit hDAG * finished modifying DAG codes * ready for draft PR * pass lint * addressed Brad and Leah's comments * pass nox lint * pass nox lint * Fix: Retry CLI launch if needed (#8221) * Fix: add region tags * Fix: region tag typos * Fix: urlpatterns moved to end * Fix: typo * Fix: cli retries to fix flakiness * Fix: remove duplicate tags * Fix: use backoff for retries * Fix: lint import order error * address Leah's comments about typo and comments Co-authored-by: Charles Engelke <[email protected]> * run blacken on dag and dataproc code * WIP: not working test for process job * working test for expansion dataproc script * move dataproc expansion files to separate directory * add readme * update readme * run black * ignore data file * fix import order * ignore one line of lint because it's being silly * add check for Notfound for test * add requirements files * add noxfile config * update try/except * experiment - fully qualify path * update filepath * update path * try different path * remove the directory that was causing test problems * fix typo in header checker * tell folks to skip cleanup of prereq * clean up hyperlinks for distance weighting and arithmetic mean * fix math links again * remove debug statements * remove commented out variables * Update composer/2022_airflow_summit/data_analytics_dag_expansion_test.py Co-authored-by: Dan Lee <[email protected]> * Apply suggestions from code review Co-authored-by: Dan Lee <[email protected]> * Apply suggestions from code review * update apache-beam version (#8302) Bumping the `apache-beam[gcp]` version to (indirectly) bump the `google-cloud-pubsub` version to accept the keyword argument `request` on `create_topic()` * dataflow: replace job name underscores with hyphens (#8303) * dataflow: replace job name underscores with hyphens It looks like Dataflow no longer accepts underscores in the job names. Replacing them with hyphens should work. * fix test checks * improve error reporting * fix test name for exception handling * chore(deps): update dependency datalab to v1.2.1 (#8309) * fix: unsanitized output (#8316) * fix: unsanitized output * fix: add license to template * chore(deps): update dependency cryptography to v38 (#8317) * chore(deps): update dependency cryptography to v38 * lint Co-authored-by: Anthonios Partheniou <[email protected]> * Remove region tags to be consistent with other languages (#8322) * fix lint in conftest (#8324) * Pin perl version to 5.34.0 as latest doesn't work with the example. (#8319) Co-authored-by: Leah E. Cole <[email protected]> * refactor fixtures * revert last change * revert last change * chore(deps): update dependency tensorflow to v2.7.2 [security] (#8329) * remove backoff, add manual retry (#8328) * remove backoff, add manual retry * fix lint * remove unused import Co-authored-by: Anthonios Partheniou <[email protected]> * refactor test to match #8328 * update most write methods, fix test issue with comparing to exception * Bmiro kaiyang edit (#8350) * modified code to more closely adhere to Spark best practices * remove unnecessary import * improved explanation of Inverse Distance Weighting * Apply suggestions from code review Co-authored-by: Leah E. Cole <[email protected]> Co-authored-by: Leah E. Cole <[email protected]> * run black on process files * fix relative import issue * fixed jvm error (#8360) * Add UDF type hinting (#8361) * fixed jvm error * add type hinting to UDF * Update composer/2022_airflow_summit/data_analytics_process_expansion.py * fix comment alignment * change dataproc region to northamerica-northeast1 * refactor import * switch other test to also use northamerica-northeast1 Co-authored-by: kaiyang-code <[email protected]> Co-authored-by: Charles Engelke <[email protected]> Co-authored-by: Maciej Strzelczyk <[email protected]> Co-authored-by: Dan Lee <[email protected]> Co-authored-by: David Cavazos <[email protected]> Co-authored-by: WhiteSource Renovate <[email protected]> Co-authored-by: Anthonios Partheniou <[email protected]> Co-authored-by: Averi Kitsch <[email protected]> Co-authored-by: mhenc <[email protected]> Co-authored-by: Brad Miro <[email protected]>

engelke added 5 commits July 26, 2022 16:24

Fix: add region tags

42e5a88

Fix: region tag typos

a2a81a0

Fix: urlpatterns moved to end

646a476

Fix: typo

fb9805a

Fix: cli retries to fix flakiness

2242160

engelke requested review from a team as code owners July 28, 2022 20:54

product-auto-label bot added the samples Issues that are directly related to samples. label Jul 28, 2022

blunderbuss-gcf bot assigned leahecole Jul 28, 2022

Merge branch 'main' into engelke-tags

4378d77

engelke added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 28, 2022

kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 28, 2022

Fix: remove duplicate tags

e545f66

engelke requested review from ace-n and rogerthatdev July 28, 2022 23:53

ace-n reviewed Jul 29, 2022

View reviewed changes

appengine/standard_python3/bundled-services/blobstore/django/main_test.py Outdated Show resolved Hide resolved

leahecole previously requested changes Jul 29, 2022

View reviewed changes

engelke added 2 commits August 2, 2022 13:32

Fix: use backoff for retries

b22144c

Fix: lint import order error

2f22bf1

engelke requested a review from leahecole August 2, 2022 21:10

Merge branch 'main' into engelke-tags

ee086ec

ace-n reviewed Aug 3, 2022

View reviewed changes

ace-n approved these changes Aug 3, 2022

View reviewed changes

engelke mentioned this pull request Aug 4, 2022

appengine.standard_python3.bundled-services.blobstore.django.main_test: test_upload_and_view failed #8229

Closed

Merge branch 'main' into engelke-tags

c522391

engelke merged commit 0c85f7b into main Aug 4, 2022

engelke deleted the engelke-tags branch August 4, 2022 23:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix: Retry CLI launch if needed #8221

Fix: Retry CLI launch if needed #8221

Uh oh!

engelke commented Jul 28, 2022

Uh oh!

snippet-bot bot commented Jul 28, 2022 •

edited

Loading

Uh oh!

ace-n left a comment

Uh oh!

Uh oh!

leahecole left a comment

Uh oh!

ace-n left a comment •

edited

Loading

Uh oh!

engelke commented Aug 3, 2022

Uh oh!

ace-n left a comment

Uh oh!

ace-n Aug 3, 2022

Uh oh!

Uh oh!

Fix: Retry CLI launch if needed #8221

Fix: Retry CLI launch if needed #8221

Uh oh!

Conversation

engelke commented Jul 28, 2022

Description

Uh oh!

snippet-bot bot commented Jul 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

No region tags are edited in this PR.

Uh oh!

ace-n left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

leahecole left a comment

Choose a reason for hiding this comment

Uh oh!

ace-n left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

engelke commented Aug 3, 2022

Uh oh!

ace-n left a comment

Choose a reason for hiding this comment

Uh oh!

ace-n Aug 3, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

snippet-bot bot commented Jul 28, 2022 •

edited

Loading

ace-n left a comment •

edited

Loading