Skip to content

Add ml.get_transform_stats issues in response #4065

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 26, 2025

Conversation

pquentin
Copy link
Member

This is a field that we have missed until now even though it was documented, except for type which was added in 8.8. It appeared in this YAML test today.

Regarding server code:

Copy link
Contributor

Following you can find the validation results for the API you have changed.

API Status Request Response
transform.get_transform_stats 🟢 34/34 34/34

You can validate this API yourself by using the make validate target.

@pquentin pquentin requested review from flobernd and l-trotta March 24, 2025 21:07
/** Number of times this issue has occurred since it started */
count: integer
/** The timestamp this issue occurred for for the first time */
first_occurrence?: EpochTime<UnitMillis>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

around the spec I saw various instance where we also map the human readable field, so both first_occurrence and first_occurrence_string

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. We can indeed get this when passing ?human, here are samples outputs for get_transform

{
  "count": 1,
  "transforms": [
    {
      "id": "test",
      "authorization": {
        "roles": [
          "superuser"
        ]
      },
      "version": "10.0.0",
      "create_time_string": "2025-03-26T11:25:45.642Z",
      "create_time": 1742988345642,
      "source": {
        "index": [
          "kibana_sample_data_ecommerce"
        ],
        "query": {
          "match_all": {}
        }
      },
      "dest": {
        "index": "test"
      },
      "pivot": {
        "group_by": {
          "category": {
            "terms": {
              "field": "category.keyword"
            }
          }
        },
        "aggregations": {
          "customer_id.cardinality": {
            "cardinality": {
              "field": "customer_id"
            }
          }
        }
      },
      "settings": {}
    }
  ]
}

and get_transform_stats:

{
  "count": 1,
  "transforms": [
    {
      "id": "test",
      "state": "stopped",
      "stats": {
        "pages_processed": 2,
        "documents_processed": 7409,
        "documents_indexed": 6,
        "documents_deleted": 0,
        "trigger_count": 1,
        "index_time_in_ms": 2,
        "index_total": 1,
        "index_failures": 0,
        "search_time_in_ms": 3,
        "search_total": 2,
        "search_failures": 0,
        "processing_time_in_ms": 0,
        "processing_total": 2,
        "delete_time_in_ms": 0,
        "exponential_avg_checkpoint_duration_ms": 19,
        "exponential_avg_documents_indexed": 6,
        "exponential_avg_documents_processed": 7409
      },
      "checkpointing": {
        "last": {
          "checkpoint": 1,
          "timestamp": "2025-03-26T11:25:46.849Z",
          "timestamp_millis": 1742988346849
        },
        "changes_last_detected_at_string": "2025-03-26T11:25:46.849Z",
        "changes_last_detected_at": 1742988346849
      },
      "health": {
        "status": "green"
      }
    }
  ]
}

I fixed the ones I saw here.

They are used when `?human` is passed.
Copy link
Contributor

Following you can find the validation results for the APIs you have changed.

API Status Request Response
transform.get_transform_stats 🟢 34/34 34/34
transform.get_transform 🟢 26/26 26/26

You can validate these APIs yourself by using the make validate target.

Comment on lines 102 to 110
export class Checkpointing {
changes_last_detected_at?: long
changes_last_detected_at_date_time?: DateTime
changes_last_detected_at_string?: DateTime
last: CheckpointStats
next?: CheckpointStats
operations_behind?: long
last_search_time?: long
last_search_time_string?: DateTime
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pquentin pquentin merged commit 573e699 into main Mar 26, 2025
8 checks passed
@pquentin pquentin deleted the fix-ml-get-transform-stats branch March 26, 2025 12:11
github-actions bot pushed a commit that referenced this pull request Mar 26, 2025
* Add ml.get_transform_stats issues in response

* Add missing DateTime fields

They are used when `?human` is passed.

(cherry picked from commit 573e699)
github-actions bot pushed a commit that referenced this pull request Mar 26, 2025
* Add ml.get_transform_stats issues in response

* Add missing DateTime fields

They are used when `?human` is passed.

(cherry picked from commit 573e699)
github-actions bot pushed a commit that referenced this pull request Mar 26, 2025
* Add ml.get_transform_stats issues in response

* Add missing DateTime fields

They are used when `?human` is passed.

(cherry picked from commit 573e699)
pquentin added a commit that referenced this pull request Mar 26, 2025
* Add ml.get_transform_stats issues in response

* Add missing DateTime fields

They are used when `?human` is passed.

(cherry picked from commit 573e699)

Co-authored-by: Quentin Pradet <[email protected]>
pquentin added a commit that referenced this pull request Mar 26, 2025
* Add ml.get_transform_stats issues in response

* Add missing DateTime fields

They are used when `?human` is passed.

(cherry picked from commit 573e699)

Co-authored-by: Quentin Pradet <[email protected]>
pquentin added a commit that referenced this pull request Mar 26, 2025
* Add ml.get_transform_stats issues in response

* Add missing DateTime fields

They are used when `?human` is passed.

(cherry picked from commit 573e699)

Co-authored-by: Quentin Pradet <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants