Fix: Presto Basic authentication error

[pppsunil]

Problem

I tried running data-diff against a presto table using command like this

data-diff "presto://username:password@<presto-url>:8443/hive/public?auth=basic&http_scheme=https" -k test_table_a -l 5 --json test_table_a_1

And it failed with following error

Entering query_cursor SELECT column_name, data_type, 3 as datetime_precision, 3 as numeric_precision, NULL as numeric_scale FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name = 'test_table_a' AND table_schema = 'public'
Entering query_cursor SELECT column_name, data_type, 3 as datetime_precision, 3 as numeric_precision, NULL as numeric_scale FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name = 'test_table_a' AND table_schema = 'public'
14:38:07 ERROR    error 400: b'User must be set'                                                                          __main__.py:332
Traceback (most recent call last):
  File "/Users/spatil/study/datadiff/presto/venv/bin/data-diff", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/data_diff/__main__.py", line 328, in main
    return _data_diff(
           ^^^^^^^^^^^
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/data_diff/__main__.py", line 464, in _data_diff
    schemas = list(differ._thread_map(_get_schema, safezip(dbs, table_paths)))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 619, in result_iterator
    yield _result_or_cancel(fs.pop())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 317, in _result_or_cancel
    return fut.result(timeout)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/data_diff/__main__.py", line 77, in _get_schema
    return db.query_table_schema(table_path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/data_diff/databases/base.py", line 945, in query_table_schema
    rows = self.query(self.select_table_schema(path), list)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/data_diff/databases/base.py", line 893, in query
    res = self._query(sql_code)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/data_diff/databases/presto.py", line 196, in _query
    return query_cursor(c, sql_code)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/data_diff/databases/presto.py", line 42, in query_cursor
    c.execute(sql_code)
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/prestodb/dbapi.py", line 267, in execute
    self._iterator = iter(self._query.execute())
                          ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/prestodb/client.py", line 539, in execute
    status = self._request.process(response)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/prestodb/client.py", line 404, in process
    self.raise_response_error(http_response)
  File "/Users/spatil/study/datadiff/presto/venv/lib/python3.11/site-packages/prestodb/client.py", line 394, in raise_response_error
    raise exceptions.HttpError(
prestodb.exceptions.HttpError: error 400: b'User must be set'

The problem is in the following line of code in the current presto.py

kw["auth"] = prestodb.auth.BasicAuthentication(kw.pop("user"), kw.pop("password"))

This code reads the value of user attribute that is supplied and pops it out from the kw args.

Solution

We can change the code so that it uses the value of user attribute in basic authentication but does not pops it out of kwargs

prestodb.auth.BasicAuthentication(kw["user"], kw.pop("password"))

I tried this in my local and now I am able to run data-diff in my local

[vvkh]

@pppsunil thanks for contributing!