Skip to content

v0.11.5

Compare
Choose a tag to compare
@wjsi wjsi released this 05 Jan 10:59
· 38 commits to master since this release
4c12aa6

Features

  • Add support for arrow table preview reader
  • Enhance support for Apache Superset
  • Add support for storage tier on tables and partitions
  • (Experimental) Add support for tunnel upsert
  • (Experimental) Add image argument for DataFrame

Bugfixes

  • Fill partition value for tunnel records
  • Use PERCENTILE_APPROX for doubles under ODPS 2.0
  • Convert all requirement files to UNIX format for pyodps-pack
  • Fix error when reloading volume tunnel session
  • Fix logview setting not working in options
  • Dump SQL statement when encountered ParseError
  • Remove misplaced warnings when pickling user functions
  • Fix errors of to_pandas for InSessionInstance readers
  • Fix position of tablesample clause for sample
  • Fix compatibility for SQLAlchemy 2.0
  • Fix results of value_counts when values are None
  • Remove empty equal mark for url actions
  • Stop copying and caching for DataFrame(pd).persist if possible to reduce memory usage
  • Fix missing quotaName in full lifecycle of tunnel requests
  • Fill partition value for tunnel records
  • Fix starting of Mars notebook and Mars import in some case
  • Delete deflate Content-Encoding header for halo in storage api

Enhancements

  • Supports scanning dependencies for pkg_resources
  • Add PEP517 args for pyodps-pack
  • Persist pandas dataframes in batches
  • Use date in response headers to replace fields in Schemas
  • Add detailed logs for sign server on errors
  • Make option context as thread locals
  • Adapt to extended types for ODPS arrow format
  • Supports schema API along with SQL implementations
  • Add support for MaxFieldSize passed by server end
  • Add options to allow keeping resources for DataFrame
  • Add support for timestamp_ntz type
  • Refine error message for malfunctioning create instance response
  • Allow adding custom log handlers to support displaying logs in notebook kernels
  • Allow using run_sql to execute merge smallfiles or compact commands
  • Allow specifying transactional table property
  • Unify verbose_log into standard Python logging and dump progress when waiting for instances
  • Return struct values as namedtuples by default and fix DataFrame customized functions on complex types
  • Add retry for BufferedRecordWriter when writing blocks
  • Reuse task utilities to simplify MCQA submission

Documentations

  • Fix pyodps-pack doc on docker requirements
  • Add doc for timezone setting
  • Make bare tunnel docs more explicit
  • Refine documents of instance tunnel limit

Compatibility issues

  • PyODPS now returns struct values as namedtuples for tunnels to keep consistency with UDFs. For most of the cases your code might still work. If it doesn't, try configuring options.struct_as_dict = True.
  • From v0.11.5 nullable property of columns is added for transactional tables, and default value for partition columns is False. If you use these column instances in some scenario, for instance, using them as common columns to create tables, non-nullable columns could be created and insertion of null values will result in errors. To ignore nullable flags in columns, try configuring sql.ignore_fields_not_null = True.