Skip to content

Audit logs moved to security #743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 12, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion deploy-manage/deploy/deployment-comparison.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ For more details about feature availability in Serverless, check [](elastic-clou
| [Security configurations](/deploy-manage/security.md) | Full control | Limited control | Limited control |
| [Authentication realms](/deploy-manage/users-roles.md) | Available | Available | Available, through Elastic Cloud only |
| [Custom roles](/deploy-manage/users-roles.md) | Available | Available | Available |
| [Audit logging](/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md) | Available | Available | No |
| [Audit logging](/deploy-manage/security/logging-configuration/security-event-audit-logging.md) | Available | Available | No |

## Infrastructure and cluster management

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ If you’re using {{agent}}, do not deploy {{filebeat}} for log collection. Inst

2. Identify which logs you want to monitor.

The {{filebeat}} {{es}} module can handle [audit logs](../logging-configuration/logfile-audit-output.md), [deprecation logs](../logging-configuration/elasticsearch-log4j-configuration-self-managed.md#deprecation-logging), [gc logs](elasticsearch://reference/elasticsearch/jvm-settings.md#gc-logging), [server logs](../logging-configuration/elasticsearch-log4j-configuration-self-managed.md), and [slow logs](elasticsearch://reference/elasticsearch/index-settings/slow-log.md). For more information about the location of your {{es}} logs, see the [path.logs](../../deploy/self-managed/important-settings-configuration.md#path-settings) setting.
The {{filebeat}} {{es}} module can handle [audit logs](../../security/logging-configuration/logfile-audit-output.md), [deprecation logs](../logging-configuration/elasticsearch-log4j-configuration-self-managed.md#deprecation-logging), [gc logs](elasticsearch://reference/elasticsearch/jvm-settings.md#gc-logging), [server logs](../logging-configuration/elasticsearch-log4j-configuration-self-managed.md), and [slow logs](elasticsearch://reference/elasticsearch/index-settings/slow-log.md). For more information about the location of your {{es}} logs, see the [path.logs](../../deploy/self-managed/important-settings-configuration.md#path-settings) setting.

::::{important}
If there are both structured (`*.json`) and unstructured (plain text) versions of the logs, you must use the structured logs. Otherwise, they might not appear in the appropriate context in {{kib}}.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ When shipping logs to a monitoring deployment there are more logging features av

#### For {{es}}: [ece-extra-logging-features-elasticsearch]

* [Audit logging](../logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment
* [Audit logging](../../security/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment
* [Slow query and index logging](elasticsearch://reference/elasticsearch/index-settings/slow-log.md) - helps find and debug slow queries and indexing
* Verbose logging - helps debug stack issues by increasing component logs

Expand All @@ -190,7 +190,7 @@ After you’ve enabled log delivery on your deployment, you can [add the Elastic

#### For Kibana: [ece-extra-logging-features-kibana]

* [Audit logging](../logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment
* [Audit logging](../../security/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment

After you’ve enabled log delivery on your deployment, you can [add the Kibana user settings](../../deploy/cloud-enterprise/edit-stack-settings.md) to enable this feature.

Expand Down
1 change: 1 addition & 0 deletions deploy-manage/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ There is no orchestration layer for self-managed deployments because you directl
- [**Traffic filtering**](security/traffic-filtering.md): IP filtering, private links, and static IPs
- [**Secure communications**](security/secure-cluster-communications.md): TLS configuration, certificates management
- [**Data protection**](security/data-security.md): Encryption at rest, secure settings, saved objects
- [**Security event audit logging**](security/logging-configuration/security-event-audit-logging.md): {{es}} and {{kib}} audit logs
- [**Session management**](security/kibana-session-management.md): Kibana session controls
- [**FIPS 140-2 compliance**](security/fips-140-2.md): Federal security standards

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ When auditing security events, a single client request might generate multiple a

For a complete description of event details and format, refer to the following resources:
* [{{es}} audit events details and schema](elasticsearch://reference/elasticsearch/elasticsearch-audit-events.md)
* [{{es}} log entry output format](/deploy-manage/monitor/logging-configuration/logfile-audit-output.md#audit-log-entry-format)
* [{{es}} log entry output format](./logfile-audit-output.md#audit-log-entry-format)

### Kibana auditing configuration

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,10 @@ You can log security-related events such as authentication failures and refused
This section describes how to enable and configure audit logging in both {{es}} and {{kib}} for all supported deployment types, including self-managed clusters, {{ech}}, {{ece}} (ECE), and {{eck}} (ECK).

::::{important}
In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md).
In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](/deploy-manage/monitor/logging-configuration.md).
::::

When audit logging is enabled, security events are persisted to a dedicated `<clustername>_audit.json` file on the host’s file system, on every cluster node. For more information, refer to [{{es}} logfile audit output](logfile-audit-output.md).
When audit logging is enabled, security events are persisted to a dedicated `<clustername>_audit.json` file on the host’s file system, on every cluster node. For more information, refer to [{{es}} logfile audit output](./logfile-audit-output.md).

## Enable audit logging [enable-audit-logging-procedure]

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,6 @@ The audit events are formatted as JSON documents, and each event is printed on a

There are however a few attributes that are exceptions to the above format. The `put`, `delete`, `change`, `create` and `invalidate` attributes, which are only present for events with the `event.type: "security_config_change"` attribute, contain the **nested JSON** representation of the security change taking effect. The contents of the security config change are hence not displayed as top-level dot-named fields in the audit event document. That’s because the fields are specific to the particular kind of security change and do not show up in any other audit events. The benefits of a columnar format are therefore much more limited; the space-saving benefits of the nested structure is the favoured trade-off in this case.

When the `request.body` attribute is present (see [Auditing search queries](auditing-search-queries.md)), it contains a string value containing the full HTTP request body, escaped as per the JSON RFC 4677.
When the `request.body` attribute is present (see [Auditing search queries](./auditing-search-queries.md)), it contains a string value containing the full HTTP request body, escaped as per the JSON RFC 4677.

Refer to [audit event types](elasticsearch://reference/elasticsearch/elasticsearch-audit-events.md) for a complete list of fields, as well as examples, for each entry type.
18 changes: 9 additions & 9 deletions deploy-manage/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -522,6 +522,15 @@ toc:
- file: security/enabling-cipher-suites-for-stronger-encryption.md
- file: security/secure-settings.md
- file: security/secure-saved-objects.md
- file: security/logging-configuration/security-event-audit-logging.md
children:
- file: security/logging-configuration/enabling-audit-logs.md
- file: security/logging-configuration/configuring-audit-logs.md
children:
- file: security/logging-configuration/logfile-audit-events-ignore-policies.md
- file: security/logging-configuration/logfile-audit-output.md
- file: security/logging-configuration/auditing-search-queries.md
- file: security/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md
- file: security/kibana-session-management.md
- file: security/fips-140-2.md
- file: security/secure-clients-integrations.md
Expand Down Expand Up @@ -742,15 +751,6 @@ toc:
children:
- file: monitor/logging-configuration/kibana-log-settings-examples.md
- file: monitor/logging-configuration/kibana-logging-cli-configuration.md
- file: monitor/logging-configuration/security-event-audit-logging.md
children:
- file: monitor/logging-configuration/enabling-audit-logs.md
- file: monitor/logging-configuration/configuring-audit-logs.md
children:
- file: monitor/logging-configuration/logfile-audit-events-ignore-policies.md
- file: monitor/logging-configuration/logfile-audit-output.md
- file: monitor/logging-configuration/auditing-search-queries.md
- file: monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md
- file: cloud-organization.md
children:
- file: cloud-organization/billing.md
Expand Down
2 changes: 1 addition & 1 deletion deploy-manage/upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ It is very important to map all the components that are being used on the {{stac
* External services (Kafka, etc.)

:::{tip}
When you do your inventory, you can [enable audit logging](/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) to evaluate resources accessing your deployment.
When you do your inventory, you can [enable audit logging](/deploy-manage/security/logging-configuration/enabling-audit-logs.md) to evaluate resources accessing your deployment.
:::

**Test your development environment**
Expand Down
2 changes: 1 addition & 1 deletion deploy-manage/users-roles.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Preventing unauthorized access is only one element of a complete security strate

* Restrict the nodes and clients that can connect to the cluster using [traffic filters](/deploy-manage/security/traffic-filtering.md).
* Take steps to maintain your data integrity and confidentiality by [encrypting HTTP and inter-node communications](/deploy-manage/security/secure-cluster-communications.md), as well as [encrypting your data at rest](/deploy-manage/security/encrypt-deployment.md).
* Maintain an [audit trail](/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md) for security-related events.
* Maintain an [audit trail](/deploy-manage/security/logging-configuration/security-event-audit-logging.md) for security-related events.
* Control access to dashboards and other saved objects in your UI using [{{kib}} spaces](/deploy-manage/manage-spaces.md).
* Connect your cluster to a [remote cluster](/deploy-manage/remote-clusters.md) to enable cross-cluster replication and search.
* Manage [API keys](/deploy-manage/api-keys.md) used for programmatic access to Elastic.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,5 @@ The {{stack-security-features}} use eight *internal* users (`_system`, `_xpack`,

These users are only used by requests that originate from within the cluster. For this reason, they cannot be used to authenticate against the API and there is no password to manage or reset.

From time-to-time you may find a reference to one of these users inside your logs, including [audit logs](../../monitor/logging-configuration/enabling-audit-logs.md).
From time-to-time you may find a reference to one of these users inside your logs, including [audit logs](../../security/logging-configuration/enabling-audit-logs.md).

Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,7 @@ const client = new Client({
})
```

Check [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on {{ech}} or {{ece}}, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics).
Check [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/security/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on {{ech}} or {{ece}}, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics).


### Best practices [ec_best_practices]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,7 @@ es = Elasticsearch(
)
```

Check [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on {{ech}} or {{ece}}, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics).
Check [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/security/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on {{ech}} or {{ece}}, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics).

For more information on refreshing an index, searching, updating, and deleting, check the [elasticsearch-py examples](elasticsearch-py://reference/examples.md).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ When shipping logs to a monitoring deployment there are more logging features av

#### For {{es}}: [ech-extra-logging-features-elasticsearch]

* [Audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment
* [Audit logging](../../../deploy-manage/security/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment
* [Slow query and index logging](elasticsearch://reference/elasticsearch/index-settings/slow-log.md) - helps find and debug slow queries and indexing
* Verbose logging - helps debug stack issues by increasing component logs

Expand All @@ -181,7 +181,7 @@ After you’ve enabled log delivery on your deployment, you can [add the Elastic

#### For Kibana: [ech-extra-logging-features-kibana]

* [Audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment
* [Audit logging](../../../deploy-manage/security/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment

After you’ve enabled log delivery on your deployment, you can [add the Kibana user settings](../../../deploy-manage/deploy/elastic-cloud/edit-stack-settings.md) to enable this feature.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ When shipping logs to a monitoring deployment there are more logging features av

#### For {{es}}: [ec-extra-logging-features-elasticsearch]

* [Audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment
* [Audit logging](../../../deploy-manage/security/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment
* [Slow query and index logging](elasticsearch://reference/elasticsearch/index-settings/slow-log.md) - helps find and debug slow queries and indexing
* Verbose logging - helps debug stack issues by increasing component logs

Expand All @@ -181,7 +181,7 @@ After you’ve enabled log delivery on your deployment, you can [add the Elastic

#### For Kibana: [ec-extra-logging-features-kibana]

* [Audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment
* [Audit logging](../../../deploy-manage/security/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment

After you’ve enabled log delivery on your deployment, you can [add the Kibana user settings](../../../deploy-manage/deploy/elastic-cloud/edit-stack-settings.md) to enable this feature.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,7 @@ const client = new Client({
})
```

Check [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on {{ech}}, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics).
Check [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/security/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on {{ech}}, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics).


### Best practices [ec_best_practices]
Expand Down
Loading