Skip to content
This repository was archived by the owner on May 17, 2024. It is now read-only.

Add redshift external tables #341

Conversation

danieldiamond
Copy link
Contributor

@danieldiamond danieldiamond commented Dec 9, 2022

Description

Existing data-diff redshift capabilities are unable to read external tables. These are found in the svv_external_columns table.

Original thought was to add a database flag -e --external-table but I don't believe flags can be specific to one database connection and not the other. A simpler approach is to simply query the external tables in redshift if none was found in the redshift one.

  • Adding select_external_table_schema in redshift sqeleton
  • Add condition to query redshift external tables if no results were found in the redshift database.

erezsh added a commit that referenced this pull request Dec 9, 2022
@erezsh
Copy link
Contributor

erezsh commented Dec 9, 2022

@danieldiamond Made a small change so the code is more modular.

#345

Please check that it works for you, and I'll merge it in.

If you can write a test for it, that would be even better! (maybe here)

@danieldiamond
Copy link
Contributor Author

@erezsh was running into issues with the query_table_schema function, updated here

@erezsh
Copy link
Contributor

erezsh commented Dec 13, 2022

(don't worry about the CI error, it's a problem on our end)

erezsh added a commit that referenced this pull request Dec 13, 2022
@erezsh erezsh merged commit 7ada9ec into datafold:master Dec 13, 2022
@erezsh
Copy link
Contributor

erezsh commented Dec 13, 2022

Thank you for contributing!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants