v0.50.4-public

Changelog

[0.50.4-public] - 2024-10-29

Features

• PartitionSpec Added a new constructor (String, boolean) that uses a boolean parameter to specify whether to trim partition values. This caters to scenarios (such as using char type as a partition field) where users may not want to trim partition values.

Changes

• Instance The OdpsException thrown when calling the stop method will no longer be wrapped a second time.

Fixes

• SQLExecutor
  • Fixed an issue in MCQA 1.0 mode where the user-specified fallbackPolicy.isFallback4AttachError did not take effect correctly.
  • Fixed an issue in MCQA 2.0 mode where the cancel method threw an exception when the job failed.
  • Fixed an issue in MCQA 2.0 mode where using instanceTunnel to fetch results resulted in an error when the isSelect check was incorrect.
• Table Fixed an issue with the getPartitionSpecs method that trimmed partition values, causing the retrieval of non-existing partitions.

---

更新日志

[0.50.4-public] - 2024-10-29

功能

• PartitionSpec 新增(String, boolean)构造方法，通过布尔参数指定是否对分区值进行trim操作，以满足某些场景（如使用char类型作为分区字段）用户不希望trim的需求。

变更

• Instance 在调用stop方法时，抛出的OdpsException将不再被二次包装。

修复

• SQLExecutor
  • 修复了在MCQA 1.0模式下，用户指定fallbackPolicy.isFallback4AttachError时未正确生效的问题。
  • 修复了在MCQA 2.0模式下，作业失败时cancel方法抛出异常的问题。
  • 修复了在MCQA 2.0模式下，当isSelect判断错误时，通过instanceTunnel取结果报错的问题。
• Table 修复了getPartitionSpecs方法会trim分区值，导致无法获取存在的分区的问题。

v0.50.3-public

Changelog

[0.50.3-public] - 2024-10-23

Features

• SQLExecutor In MCQA 1.0 mode, it is allowed to add custom fallback policies, add subclass FallbackPolicy.UserDefinedFallbackPolicy.

---

更新日志

[0.50.3-public] - 2024-10-23

功能

• SQLExecutor 在 MCQA 1.0 模式下，允许增加自定义回退策略，新增类FallbackPolicy.UserDefinedFallbackPolicy。

v0.50.2-public

Changelog

[0.50.2-public] - 2024-10-23

Features

• SQLExecutor Enhanced MCQA 2.0 functionality:
  • isActive will return false, indicating that there are no active Sessions in MCQA 2.0 mode.
  • Added a cancel method to terminate ongoing jobs.
  • getExecutionLog now returns a deep copy of the current log and clears the current log,
    preventing duplicates.
  • New quota method in SQLExecutorBuilder allows reusing already loaded Quota, reducing
    load times.
  • New regionId method in SQLExecutorBuilder allows specifying the region where the quota is
    located.
• Quotas Added getWlmQuota method with regionId parameter to fetch quota for a specified regionId.
• Quota Introduced setMcqaConnHeader method to allow users to override quota using a custom
  McqaConnHeader, supporting MCQA 2.0.
• Instances Added get method applicable for MCQA 2.0 jobs, requiring additional parameters for QuotaName
  and RegionId.
• Instance Further adapted for MCQA 2.0 jobs.
• TableSchema basicallyEquals method will no longer strictly check for identical Class types.

Optimization

• SQLExecutor The run method's hints will now be deep-copied, preserving the user-provided Map and
  supporting immutable types (e.g., ImmutableMap).

Fixes

• Stream Fixed potential SQL syntax errors in the create method.

---

更新日志

[0.50.2-public] - 2024-10-23

功能

• SQLExecutor 增强 MCQA 2.0 功能：
  • isActive 将返回 false，指示在 MCQA 2.0 模式下没有活跃的 Session。
  • 新增 cancel 方法，用于中止正在执行的作业。
  • getExecutionLog 现在返回当前日志的深拷贝并清空当前日志，避免重复获取。
  • 在 SQLExecutorBuilder 新增 quota 方法，支持复用已加载的 Quota，减少加载时间。
  • 在 SQLExecutorBuilder 新增 regionId 方法，允许指定 quota 所在的 region。
• Quotas 新增带 regionId 参数的 getWlmQuota 方法，用于获取指定 regionId 的 quota。
• Quota 新增 setMcqaConnHeader 方法，支持用户通过自定义的 McqaConnHeader 重载 quota，以适配 MCQA 2.0。
• Instances 新增适用于 MCQA 2.0 的 get 方法，需额外传入 MCQA 2.0 的 QuotaName 和 RegionId。
• Instance 进一步适配 MCQA 2.0 作业。
• TableSchema basicallyEquals 方法将不再严格检查两个类的 Class 类型一致性。

优化

• SQLExecutor run 方法中的 hints 现在会进行深拷贝，保护用户传入的 Map，支持不可变类型（如 ImmutableMap）。

修复

• Stream 修复 create 方法中的潜在 SQL 语法错误。

v0.50.1-public

Changelog

[0.50.1-public] - 2024-10-11

Fixes

• TableAPI Fixed an issue where ArrayRecord could not correctly invoke toString when using SplitRecordReaderImpl to retrieve results.
• TableAPI Fixed an issue where a get operation would throw an array index out of bounds exception when the number of Records corresponding to a Split is 0 while using SplitRecordReaderImpl to retrieve results.
• TableAPI Fixed an issue with composite predicates CompositePredicate that could lead to an additional operator being added when encountering an empty predicate.

---

更新日志

[0.50.1-public] - 2024-10-11

修复

• TableAPI 修复了使用SplitRecordReaderImpl获取结果时，拿到了ArrayRecord无法正确toString的问题。
• TableAPI 修复了使用SplitRecordReaderImpl获取结果时，如果Split对应的Record数量为0，在get
  操作时会抛出数组越界异常的问题。
• TableAPI 修复了复合谓词CompositePredicate在遇到空谓词时，可能额外增加一次操作符的问题。

v0.48.9-public

Changelog

[0.48.9-public] - 2024-10-11

Fixes

• TableAPI Fixed an issue where ArrayRecord could not correctly invoke toString when using SplitRecordReaderImpl to retrieve results.
• TableAPI Fixed an issue where a get operation would throw an array index out of bounds exception when the number of Records corresponding to a Split is 0 while using SplitRecordReaderImpl to retrieve results.
• TableAPI Fixed an issue with composite predicates CompositePredicate that could lead to an additional operator being added when encountering an empty predicate.

---

更新日志

[0.48.9-public] - 2024-10-11

修复

• TableAPI 修复了使用SplitRecordReaderImpl获取结果时，拿到了ArrayRecord无法正确toString的问题。
• TableAPI 修复了使用SplitRecordReaderImpl获取结果时，如果Split对应的Record数量为0，在get
  操作时会抛出数组越界异常的问题。
• TableAPI 修复了复合谓词CompositePredicate在遇到空谓词时，可能额外增加一次操作符的问题。

v0.50.0-public

[0.50.0-public] - 2024-10-09

Features

• Added SchemaMismatchException: This exception will be thrown when using StreamUploadSession if the Record structure uploaded by the user does not match the table structure. This exception will additionally carry the latest schema version to assist users in rebuilding the Session and performing retry operations.
• Added allowSchemaMismatch method in StreamUploadSession.Builder: This method specifies whether to tolerate mismatches between the user's uploaded Record structure and the table structure without throwing an exception. The default value is true.

Fixes

• Fixed an issue where specifying tunnelEndpoint in Odps was ineffective when using StreamUploadSession.
• Fixed a potential NPE issue in TunnelRetryHandler.

---

[0.50.0-public] - 2024-10-09

功能

• 新增 SchemaMismatchException 异常：当使用 StreamUploadSession 时，如果用户上传的 Record 结构与表结构不匹配，将抛出该异常。此异常将额外携带最新的 schema version，方便用户重建 Session 并进行重试操作。
• 在 StreamUploadSession.Builder 中新增 allowSchemaMismatch 方法，用于指定是否容忍用户上传的 Record 结构与表结构不匹配时是否抛出异常。默认值为 true。

修复

• 修复了在 Odps 中指定 tunnelEndpoint 时，使用 StreamUploadSession 无法生效的问题。
• 修复了 TunnelRetryHandler 潜在的 NPE 问题。

v0.50.0-rc1

Changelog

[0.50.0-rc1] - 2024-09-19

Features

• SQLExecutor added isUseInstanceTunnel method:
  • Used to determine whether to use instanceTunnel to obtain results

Fix

• Fixed an issue where when using SQLExecutor to execute MCQA 2.0 jobs, executing the CommandApi task would affect the next job, causing NPE to be thrown when retrieving results.

---

更新日志

[0.50.0-rc1] - 2024-09-19

功能

• SQLExecutor 新增 isUseInstanceTunnel 方法：
  • 用来判断是否使用 instanceTunnel 取结果

修复

• 修复了使用 SQLExecutor 执行 MCQA 2.0 作业时，执行 CommandApi 任务会影响下一次作业，导致取结果时抛出NPE的问题。

v0.50.0-rc0

Changelog

[0.50.0-rc0] - 2024-09-18

Features

• SQLExecutor supports submitting MCQA 2.0 jobs
  • SQLExecutorBuilder adds method enableMcqaV2
  • SQLExecutorBuilder adds getter methods for fields
• SQLExecutor adds getQueryId method:
  • For offline jobs and MCQA 2.0 jobs, it returns the currently executing job's InstanceId
  • For MCQA 1.0 jobs, it returns the InstanceId and SubQueryId
• TableAPI adds SharingQuotaToken parameter in EnvironmentSettings to support sharing quota resources during job submission
• Quotas introduces getWlmQuota method:
  • Allows retrieval of detailed quota information based on projectName and quotaNickName, including whether it belongs to interactive quotas
• Quota class adds isInteractiveQuota method to determine if a quota belongs to interactive quotas (suitable for MCQA 2.0)
• Adds getResultByInstanceTunnel(Instance instance, String taskName, Long limit, boolean limitEnabled) method:
  • Allows unlimited retrieval of results via instanceTunnel (lifting restrictions requires higher permissions)
• UpsertSession.Builder adds setLifecycle method to configure the session lifecycle

Fixes

• Fixed the issue where using SQLExecutor to execute offline jobs with limitEnabled specified resulted in no effect
• Modified the SQLExecutor so that getQueryId method returns the job's instanceID instead of null when executing offline jobs
• Fixed the issue where using instanceTunnel to retrieve results on encountering non-select statements no longer throws exceptions, instead falling back to non-tunnel logic
• Fixed the problem of missing one data entry when using DownloadSession to download data and an error occurred while the read count equaled the number of records to be read minus one
• The clone method of the Odps class now correctly clones other fields, including tunnelEndpoint
• The Instance's getRawTaskResults method now does not make multiple requests when processing synchronous jobs

---

更新日志

[0.50.0-rc0] - 2024-09-18

功能

• SQLExecutor 支持提交 MCQA 2.0 作业
  • SQLExecutorBuilder 新增方法 enableMcqaV2
  • SQLExecutorBuilder 新增对字段的 getter 方法
• SQLExecutor 新增 getQueryId 方法：
  • 对于离线作业和 MCQA 2.0 作业，会返回当前执行的作业 InstanceId
  • 对于 MCQA 1.0 作业，会返回 InstanceId 和 SubQueryId
• TableAPI EnvironmentSettings 新增 SharingQuotaToken 参数，以支持提交作业时携带Quota资源共享临时凭证
• Quotas 新增 getWlmQuota 方法：
  • 能够根据 projectName 和 quotaNickName 获取到 quota 的详细信息，比如是否属于交互式 quota
• Quota 类新增 isInteractiveQuota 方法，用来判断 quota 是否属于交互式 quota（适用于 MCQA 2.0）
• 新增 getResultByInstanceTunnel(Instance instance, String taskName, Long limit, boolean limitEnabled) 方法：
  • 用来无限制地通过 instanceTunnel 获取结果（解除限制需要更高的权限）
• UpsertSession.Builder 新增 setLifecycle 方法，用来配置 Session 生命周期

修复

• 修复了使用 SQLExecutor 执行离线作业时，指定 limitEnabled 取结果但不生效的问题
• 修改了 SQLExecutor 执行离线作业时，getQueryId 方法会返回作业的 instanceID 而非 null
• 修复了 SQLExecutor 执行离线作业时，当遇到非 select 语句时，使用 instanceTunnel 取结果不再抛出异常，而是回退到非 tunnel 逻辑
• 修复了使用 DownloadSession 下载数据时，发生错误且读取数量刚好等于要读取记录的数量 - 1 时重建漏掉一条数据的问题
• Odps 类的 clone 方法现在能正确克隆包括 tunnelEndpoint 等其他字段
• Instance 的 getRawTaskResults 方法现在在处理同步作业时不会多次发起请求

v0.49.0-public

Changelog

[0.49.0-public] - 2024-09-12

Features

• OdpsRecordConverter Enhancement: Now supports converting data to SQL-compatible formats. For
  example, for the LocalDate type, data can be converted to "DATE 'yyyy-mm-dd'" format.
  Additionally, for the Binary type, hex representation format is now supported.

• Enhanced Predicate Pushdown for Storage Constants: Improved the behavior of the Constant
  class and added the Constant.of(Object, TypeInfo) method. Now, when setting or identifying types
  as time types, the conversion to SQL-compatible format can be done correctly (enabling correct
  pushdown of time types). Other type conversion issues have been fixed;
  an IllegalArgumentException will be thrown during session creation when conversion to
  SQL-compatible mode is not possible.

• UpsertSession Implements Closable Interface: Notifies users to properly release local
  resources of the UpsertSession.

• SQLExecutorBuilder New Method offlineJobPriority: Allows setting the priority of offline
  jobs when a job rolls back.

• New Method in Table Class getLastMajorCompactTime: Used to retrieve the last time the table
  underwent major compaction.

• New Method in Instance Class create(Job job, boolean tryWait): When the tryWait parameter
  is true, the job will attempt to wait on the server for a period of time to obtain results more
  quickly.

• Resource Class Enhancement: Now able to determine if the corresponding resource is a temporary
  resource.

• CreateProjectParma class enhancement Added defaultCtrlService parameter to specify the default control cluster of the project.

Fixes

• UpsertStream NPE Fix: Fixed an issue where an NPE was thrown during flush when a local error
  occurred, preventing a proper retry.

• Varchar/Char type fix: Fixed the problem that when the Varchar/Char type obtains its length
  and encounters special characters such as Chinese symbols or emoticons, it will be incorrectly
  calculated twice.

---

更新日志

[0.49.0-public] - 2024-09-12

功能

• OdpsRecordConverter 功能增强：现在支持将数据转换为 SQL 兼容格式，比如对于 LocalDate
  类型，数据可以转换为 "DATE 'yyyy-mm-dd'" 格式。同时对于 Binary 类型，现在支持了 hex 表示格式。

• 开放存储谓词下推常量增强：改进了 Constant 类行为，新增了 Constant.of(Object, TypeInfo)
  方法。现在当设定或识别出类型为时间类型时，可以正确转变为 SQL
  兼容格式（也就是可以正确下推时间类型了）。同时修复了一些其他类型的问题，当无法转换成 SQL
  兼容模式时，会在创建 Session 的时候抛出 IllegalArgumentException。

• UpsertSession 实现 Closable 方法：提醒用户应当正确释放 UpsertSession 的本地资源。

• SQLExecutorBuilder 新增方法 offlineJobPriority：用来设置当作业发生回退时，离线作业的优先级。

• Table 类新增方法 getLastMajorCompactTime：用来获取表最后一次 major compact 的时间。

• Instance 类新增方法 create(Job job, boolean tryWait)：当用户执行 tryWait 为 true
  时，作业会尝试在服务端等待一段时间，以更快获取结果。

• Resource 类增强：现在能够判断对应的资源是否属于临时资源。

• CreateProjectParma 类增强 新增defaultCtrlService参数，用来指定项目的默认控制集群。

修复

• UpsertStream NPE 修复：修复了在 flush 时，当发生本地错误时抛出 NPE 而无法正确重试的问题。

• Varchar/Char 类型修复：修复了 Varchar/Char 类型获取其长度时，当遇到中文符号或表情等特殊字符，会错误的计算两次的问题。

v0.48.8-public

Changelog

[0.48.8-public] - 2024-08-12

Enhancement

• Introduced internal validation of compound predicate expressions, fixed logic when handling
  invalid or always true/false predicates, enhanced test coverage, and ensured stability and
  accuracy in complex query optimization.

---

更新日志

[0.48.8-public] - 2024-08-12

增强

• 引入了对复合谓词表达式的内部验证，修复了处理无效或总是真/假谓词时的逻辑，增强了测试覆盖，确保了在复杂查询优化中的稳定性和准确性。