Skip to content
This repository was archived by the owner on May 17, 2024. It is now read-only.

Datafold database credentials from cli #466

Merged
merged 27 commits into from
Apr 11, 2023
Merged

Datafold database credentials from cli #466

merged 27 commits into from
Apr 11, 2023

Conversation

pik94
Copy link
Contributor

@pik94 pik94 commented Mar 30, 2023 •
edited
Loading

Add functions to create a new data source via the data diff CLI

  1. Create a new data source if it does not exist neither in the datafold cloud nor in the dbt_project.yml
    Screencast from 30.03.2023 17:58:04.webm

  2. Use an existing data source in the cloud if it is not set in dbt_project.yml
    Screencast from 30.03.2023 18:11:30.webm

@pik94 pik94 added enhancement New feature or request cloud Issues related to data-diff <> Datafold Cloud interaction --dbt Issues/features related to the dbt integration labels Mar 30, 2023
@pik94 pik94 self-assigned this Mar 30, 2023
@pik94 pik94 force-pushed the datafold-database-credentials-from-cli branch 2 times, most recently from 503f576 to af8105f Compare March 30, 2023 11:38
@pik94 pik94 marked this pull request as ready for review March 30, 2023 12:00
@williebsweet
Copy link
Contributor

@pik94 Can you address the conflicts that were created and re-request review?

@pik94 pik94 force-pushed the datafold-database-credentials-from-cli branch from 616d92f to c855e25 Compare April 5, 2023 10:24
@pik94
Copy link
Contributor Author

pik94 commented Apr 5, 2023

@pik94 Can you address the conflicts that were created and re-request review?

@williebsweet sure, i have updated my PR

@williebsweet
Copy link
Contributor

@dlawin Can you review?

dlawin
dlawin reviewed Apr 5, 2023
View reviewed changes
dlawin
dlawin reviewed Apr 5, 2023
View reviewed changes
dlawin
dlawin reviewed Apr 5, 2023
View reviewed changes
@dlawin
Copy link
Contributor

dlawin commented Apr 5, 2023

It's unclear at first that the values in blue parentheses are defaults:
Screenshot 2023-04-05 at 4 11 33 PM

Adding a line at the top with an instruction like:
Press enter to accept the (Default value) would help

@dlawin
Copy link
Contributor

dlawin commented Apr 5, 2023

It's unclear at first that the values in blue parentheses are defaults: Screenshot 2023-04-05 at 4 11 33 PM

Adding a line at the top with an instruction like: Press enter to accept the (Default value) would help

also note I was able to leave Host blank, which passed a None to the post

@pik94
Copy link
Contributor Author

pik94 commented Apr 6, 2023

It's unclear at first that the values in blue parentheses are defaults: Screenshot 2023-04-05 at 4 11 33 PM
Adding a line at the top with an instruction like: Press enter to accept the (Default value) would help

also note I was able to leave Host blank, which passed a None to the post

@dlawin good catch, thank you very much! I have added validation for fields without non-default values.

nolar
nolar suggested changes Apr 6, 2023
View reviewed changes
@pik94 pik94 force-pushed the datafold-database-credentials-from-cli branch from d7d29a7 to ab11c95 Compare April 6, 2023 18:33
@pik94 pik94 requested review from dlawin and nolar April 6, 2023 18:33
dlawin
dlawin reviewed Apr 6, 2023
View reviewed changes
pik94 added 22 commits April 7, 2023 16:16
@pik94
fix unit tests
7212021
@pik94
add some unit tests for creating of data source configs and data sour…
fc718b1 
…ce existing
@pik94
return test results instead of a flag for _test_data_source function
641b1a5
@pik94
add unit tests for data source tests
520f5a4
@pik94
fix an incorrect float_tolerance value
c219b1e
@pik94
add a function to render available data sources
3998f94
@pik94
refactor
356ddaf
@pik94
improve logging for cloud CLI
e0a7616
@pik94
disable profiling and lineage for cloud data sources
78324e6
@pik94
fix unit tests
d9f85c4
@pik94
update a link
abc4b73
@pik94
update after rebasing to the latest master
c5a6cd4
@pik94
update unit tests
27b2472
@pik94
reformat using black
4b551d3
@pik94
remove unused function
64d62ae
@pik94
fix typos and validate fields without default values
4183459
@pik94
add a unit test for validating a value of a required field
679014b
@pik94
fix a typo
f07d1c8
@pik94
set required parameters for data sources in pydantic
e3aa793
@pik94
fix unit tests
fd94a7b
@pik94
use monotonic time and improve error messages
03220e8
@pik94
replace internal endpoints to external
67b8b70
@pik94 pik94 force-pushed the datafold-database-credentials-from-cli branch from ab11c95 to 67b8b70 Compare April 7, 2023 10:23
@pik94 pik94 requested a review from dlawin April 10, 2023 08:48
@pik94 pik94 merged commit 8d1fb40 into datafold:master Apr 11, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@nolar nolar nolar approved these changes

@williebsweet williebsweet Awaiting requested review from williebsweet

@wvanbergen wvanbergen Awaiting requested review from wvanbergen

@vvkh vvkh Awaiting requested review from vvkh

@dlawin dlawin Awaiting requested review from dlawin

Assignees

@pik94 pik94

Labels
--dbt Issues/features related to the dbt integration cloud Issues related to data-diff <> Datafold Cloud interaction enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@pik94 @williebsweet @dlawin @nolar