Skip to content

Fix infer_trained_model API #2786

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 21, 2024
Merged

Conversation

pquentin
Copy link
Member

It can return list of lists of integers and floats.

It can return list of lists of integers and floats.
@pquentin pquentin requested review from a team as code owners August 14, 2024 09:28

This comment was marked as outdated.

@davidkyle
Copy link
Member

I can't find any places in the Elasticsearch code where an array of doubles or ints is returned. Do you have an example pls?

Why isn't the list case covered by this line in the spec? Is the result a lists of lists of numbers?

predicted_value?: PredictedValue | PredictedValue[]

@pquentin
Copy link
Member Author

pquentin commented Aug 21, 2024

@davidkyle Sorry that I missed your comment. It happens in four "3rd party deployment" ML YAML tests, for example https://github.com/elastic/elasticsearch/blob/0d38528e0e6f01010ba4a6d11d84ff7a5ddf32f4/x-pack/plugin/src/yamlRestTest/resources/rest-api-spec/test/ml/3rd_party_deployment.yml#L600-L622. In all cases, the following JSON is returned:

{
  "inference_results": [
    {
      "predicted_value": [
        [
          1,
          1
        ]
      ]
    }
  ]
}

Agreed that if predicted_value can be a list already, that should be covered somehow.

@pquentin pquentin closed this Aug 21, 2024
@pquentin
Copy link
Member Author

Actually yes it is a list of lists, so there's a missing list somewhere, PredictedValue[] isn't enough. Reopening.

@pquentin pquentin reopened this Aug 21, 2024
Copy link
Contributor

Following you can find the validation results for the APIs you have changed.

API Status Request Response
ml.clear_trained_model_deployment_cache 🟢 1/1 1/1
ml.close_job 🟢 64/64 63/63
ml.delete_calendar_event 🟢 4/4 4/4
ml.delete_calendar_job 🟢 3/3 3/3
ml.delete_calendar 🟢 5/5 5/5
ml.delete_data_frame_analytics 🟢 2/2 2/2
ml.delete_datafeed 🟢 3/3 3/3
ml.delete_expired_data 🟢 5/5 5/5
ml.delete_filter 🟢 27/27 27/27
ml.delete_forecast 🟢 3/3 3/3
ml.delete_job 🟢 47/47 47/47
ml.delete_model_snapshot 🟢 2/2 2/2
ml.delete_trained_model_alias 🟢 3/3 3/3
ml.delete_trained_model 🟢 5/5 5/5
ml.estimate_model_memory 🟢 16/16 16/16
ml.evaluate_data_frame 🟢 15/15 15/15
ml.explain_data_frame_analytics 🟢 7/7 7/7
ml.flush_job 🟢 16/16 16/16
ml.forecast 🟢 2/2 2/2
ml.get_buckets 🟢 14/14 14/14
ml.get_calendar_events 🟢 17/17 17/17
ml.get_calendars 🟢 17/17 17/17
ml.get_categories 🟢 12/12 12/12
ml.get_data_frame_analytics_stats 🟢 12/12 12/12
ml.get_data_frame_analytics 🔴 17/17 14/17
ml.get_datafeed_stats 🟢 27/27 27/27
ml.get_datafeeds 🔴 20/20 9/20
ml.get_filters 🟢 13/13 13/13
ml.get_influencers 🟢 11/11 11/11
ml.get_job_stats 🔴 32/32 30/32
ml.get_jobs 🔴 31/31 22/31
ml.get_memory_stats 🟢 6/6 6/6
ml.get_model_snapshot_upgrade_stats 🟢 3/3 3/3
ml.get_model_snapshots 🟢 18/18 18/18
ml.get_overall_buckets 🟢 16/16 15/15
ml.get_records 🟢 8/8 8/8
ml.get_trained_models_stats 🔴 17/17 10/17
ml.get_trained_models 🔴 36/37 32/37
ml.infer_trained_model 🟢 10/10 10/10
ml.info 🔴 10/10 2/10
ml.open_job 🟢 83/83 83/83
ml.post_calendar_events 🟢 21/21 21/21
ml.post_data 🔴 10/12 15/19
ml.preview_data_frame_analytics 🟢 3/3 3/3
ml.preview_datafeed 🟢 17/17 17/17
ml.put_calendar_job 🟢 12/12 12/12
ml.put_calendar 🟢 135/135 135/135
ml.put_data_frame_analytics 🔴 32/33 32/33
ml.put_datafeed 🔴 70/71 53/71
ml.put_filter 🟢 27/27 27/27
ml.put_job 🔴 219/227 223/225
ml.put_trained_model_alias 🟢 13/13 13/13
ml.put_trained_model_definition_part 🟢 1/1 1/1
ml.put_trained_model_vocabulary 🟢 1/1 1/1
ml.put_trained_model 🔴 15/16 6/16
ml.reset_job 🟢 2/2 2/2
ml.revert_model_snapshot 🟢 2/2 2/2
ml.set_upgrade_mode 🟢 6/6 6/6
ml.start_data_frame_analytics 🟢 1/1 1/1
ml.start_datafeed 🟢 24/24 24/24
ml.start_trained_model_deployment 🔴 14/14 4/14
ml.stop_data_frame_analytics 🟢 5/5 5/5
ml.stop_datafeed 🟢 17/17 17/17
ml.stop_trained_model_deployment 🟢 10/10 10/10
ml.update_data_frame_analytics 🟢 2/2 2/2
ml.update_datafeed 🔴 7/7 2/7
ml.update_filter 🟢 3/3 3/3
ml.update_job 🔴 4/5 5/5
ml.update_model_snapshot 🟢 3/3 3/3
ml.update_trained_model_deployment 🔴 4/4 2/4
ml.upgrade_job_snapshot 🟢 3/3 3/3
ml.validate_detector 🟢 2/2 2/2
ml.validate 🟢 3/3 3/3

You can validate these APIs yourself by using the make validate target.

Copy link
Member

@davidkyle davidkyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thanks @pquentin, the yml test you linked to uses the special pass_through type that returns the raw response. I'd forgotten about that one sorry

@pquentin
Copy link
Member Author

pquentin commented Aug 21, 2024

Merging, assuming that only ints and floats can be lists of lists. Happy to open another PR if that's not the case.

Thank you!

@pquentin pquentin merged commit 0fc0098 into main Aug 21, 2024
12 checks passed
@pquentin pquentin deleted the fix-infer-trained-model-responses branch August 21, 2024 12:21
@davidkyle
Copy link
Member

Yes only ints and floats

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants