Add Redact processor

[pquentin]

When coupled with #2854, this makes all ingest APIs pass validation (specifically, a redact processor is used in ingest.put_pipeline and ingest.simulate)

[github-actions]

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
ingest.delete_geoip_database	🟢	1/1	1/1
ingest.delete_pipeline	🟢	15/15	15/15
ingest.geo_ip_stats	🟢	1/1	1/1
ingest.get_geoip_database	🟢	4/4	4/4
ingest.get_pipeline	🔴	22/22	21/22
ingest.processor_grok	🟢	1/1	1/1
ingest.put_geoip_database	🟢	3/3	3/3
ingest.put_pipeline	🔴	58/59	59/59
ingest.simulate	🟢	9/9	9/9

You can validate these APIs yourself by using the make validate target.

[flobernd]

LGTM! Just wondering if we should introduce a GrokPattern alias to allow the clients to implement custom functionality for this type.

[pquentin]

Just wondering if we should introduce a GrokPattern alias to allow the clients to implement custom functionality for this type.

We could, but we would need to do that for the GrokProcessor too. Would it be breaking?

[flobernd]

It would not be breaking for .NET and Java as we simplify to the aliased type by default. Not 100% sure if that's the case for Go as well. cc @Anaethelion

[github-actions]

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
ingest.delete_geoip_database	🟢	1/1	1/1
ingest.delete_pipeline	🟢	15/15	15/15
ingest.geo_ip_stats	🟢	1/1	1/1
ingest.get_geoip_database	🟢	4/4	4/4
ingest.get_pipeline	🟢	22/22	22/22
ingest.processor_grok	🟢	1/1	1/1
ingest.put_geoip_database	🟢	3/3	3/3
ingest.put_pipeline	🟢	59/59	59/59
ingest.simulate	🟢	9/9	9/9

You can validate these APIs yourself by using the make validate target.

[github-actions]

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
ingest.delete_geoip_database	🟢	1/1	1/1
ingest.delete_pipeline	🟢	15/15	15/15
ingest.geo_ip_stats	🟢	1/1	1/1
ingest.get_geoip_database	🟢	4/4	4/4
ingest.get_pipeline	🟢	22/22	22/22
ingest.processor_grok	🟢	1/1	1/1
ingest.put_geoip_database	🟢	3/3	3/3
ingest.put_pipeline	🟢	59/59	59/59
ingest.simulate	🟢	9/9	9/9
ml.clear_trained_model_deployment_cache	🟢	1/1	1/1
ml.close_job	🟢	64/64	63/63
ml.delete_calendar_event	🟢	4/4	4/4
ml.delete_calendar_job	🟢	3/3	3/3
ml.delete_calendar	🟢	5/5	5/5
ml.delete_data_frame_analytics	🟢	2/2	2/2
ml.delete_datafeed	🟢	3/3	3/3
ml.delete_expired_data	🟢	5/5	5/5
ml.delete_filter	🟢	27/27	27/27
ml.delete_forecast	🟢	3/3	3/3
ml.delete_job	🟢	47/47	47/47
ml.delete_model_snapshot	🟢	2/2	2/2
ml.delete_trained_model_alias	🟢	3/3	3/3
ml.delete_trained_model	🟢	5/5	5/5
ml.estimate_model_memory	🟢	16/16	16/16
ml.evaluate_data_frame	🟢	15/15	15/15
ml.explain_data_frame_analytics	🟢	7/7	7/7
ml.flush_job	🟢	15/15	15/15
ml.forecast	🟢	1/1	1/1
ml.get_buckets	🟢	14/14	14/14
ml.get_calendar_events	🟢	17/17	17/17
ml.get_calendars	🟢	17/17	17/17
ml.get_categories	🟢	12/12	12/12
ml.get_data_frame_analytics_stats	🟢	12/12	12/12
ml.get_data_frame_analytics	🔴	17/17	14/17
ml.get_datafeed_stats	🟢	27/27	27/27
ml.get_datafeeds	🔴	20/20	9/20
ml.get_filters	🟢	13/13	13/13
ml.get_influencers	🟢	11/11	11/11
ml.get_job_stats	🔴	32/32	30/32
ml.get_jobs	🔴	31/31	22/31
ml.get_memory_stats	🟢	6/6	6/6
ml.get_model_snapshot_upgrade_stats	🟢	3/3	3/3
ml.get_model_snapshots	🟢	18/18	18/18
ml.get_overall_buckets	🟢	16/16	15/15
ml.get_records	🟢	8/8	8/8
ml.get_trained_models_stats	🔴	17/17	10/17
ml.get_trained_models	🔴	36/37	32/37
ml.infer_trained_model	🟢	10/10	10/10
ml.info	🔴	10/10	2/10
ml.open_job	🟢	83/83	83/83
ml.post_calendar_events	🟢	21/21	21/21
ml.post_data	🔴	9/11	14/18
ml.preview_data_frame_analytics	🟢	3/3	3/3
ml.preview_datafeed	🟢	17/17	17/17
ml.put_calendar_job	🟢	12/12	12/12
ml.put_calendar	🟢	135/135	135/135
ml.put_data_frame_analytics	🔴	32/33	32/33
ml.put_datafeed	🔴	70/71	53/71
ml.put_filter	🟢	27/27	27/27
ml.put_job	🔴	219/227	223/225
ml.put_trained_model_alias	🟢	13/13	13/13
ml.put_trained_model_definition_part	🟢	1/1	1/1
ml.put_trained_model_vocabulary	🟢	1/1	1/1
ml.put_trained_model	🔴	15/16	6/16
ml.reset_job	🟢	2/2	2/2
ml.revert_model_snapshot	🟢	2/2	2/2
ml.set_upgrade_mode	🟢	6/6	6/6
ml.start_data_frame_analytics	🟢	1/1	1/1
ml.start_datafeed	🟢	24/24	24/24
ml.start_trained_model_deployment	🔴	14/14	4/14
ml.stop_data_frame_analytics	🟢	5/5	5/5
ml.stop_datafeed	🟢	17/17	17/17
ml.stop_trained_model_deployment	🟢	10/10	10/10
ml.update_data_frame_analytics	🟢	2/2	2/2
ml.update_datafeed	🔴	7/7	2/7
ml.update_filter	🟢	3/3	3/3
ml.update_job	🔴	4/5	5/5
ml.update_model_snapshot	🟢	3/3	3/3
ml.update_trained_model_deployment	🔴	4/4	2/4
ml.upgrade_job_snapshot	🟢	3/3	3/3
ml.validate_detector	🟢	2/2	2/2
ml.validate	🟢	3/3	3/3
text_structure.find_structure	🟢	2/2	2/2
text_structure.test_grok_pattern	🟢	2/2	2/2

You can validate these APIs yourself by using the make validate target.

[github-actions]

The backport to 8.15 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-8.15 8.15
# Navigate to the new working tree
cd .worktrees/backport-8.15
# Create a new branch
git switch --create backport-2858-to-8.15
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 39ba8e8550e75ed515c0fd76803738df3b8aafcb
# Push it to GitHub
git push --set-upstream origin backport-2858-to-8.15
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-8.15

Then, create a pull request where the base branch is 8.15 and the compare/head branch is backport-2858-to-8.15.