Updated gcs_to_bq method (#4105)

Dylan Peck · leahecole · gcf-merge-on-green[bot] · web-flow · commit 9a9ce78e9d3e · 2020-06-16T13:47:27.000-07:00
Copying from gcs_to_bq method was throwing errors in airflow due to having no defined schema values, object, or autodetect set.

Co-authored-by: Leah E. Cole &lt;6719667+leahecole@users.noreply.github.com&gt;
Co-authored-by: gcf-merge-on-green[bot] &lt;60162190+gcf-merge-on-green[bot]@users.noreply.github.com&gt;
diff --git a/composer/workflows/bq_copy_across_locations.py b/composer/workflows/bq_copy_across_locations.py
@@ -174,7 +174,8 @@ def read_table_list(table_list_file):
             source_objects=['{}-*.avro'.format(table_source)],
             destination_project_dataset_table=table_dest,
             source_format='AVRO',
-            write_disposition='WRITE_TRUNCATE'
+            write_disposition='WRITE_TRUNCATE',
+            autodetect=True
         )
 
         start >> BQ_to_GCS >> GCS_to_GCS >> GCS_to_BQ >> end

Original file line number	Diff line number	Diff line change
`@@ -174,7 +174,8 @@ def read_table_list(table_list_file):`
`174`	`174`	`source_objects=['{}-*.avro'.format(table_source)],`
`175`	`175`	`destination_project_dataset_table=table_dest,`
`176`	`176`	`source_format='AVRO',`
`177`		`- write_disposition='WRITE_TRUNCATE'`
	`177`	`+ write_disposition='WRITE_TRUNCATE',`
	`178`	`+ autodetect=True`
`178`	`179`	`)`
`179`	`180`
`180`	`181`	`start >> BQ_to_GCS >> GCS_to_GCS >> GCS_to_BQ >> end`