[logs-ui][docs] Logs analysis docs (#50832) (#51314)

This commit is contained in:
Brandon Morelli 2019-11-21 09:44:33 -08:00 committed by GitHub
parent 99ce102193
commit a50eac24c9
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
7 changed files with 109 additions and 2 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 235 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 287 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 190 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 53 KiB

View file

@ -27,3 +27,5 @@ include::getting-started.asciidoc[]
include::using.asciidoc[]
include::configuring.asciidoc[]
include::log-rate.asciidoc[]

View file

@ -0,0 +1,94 @@
[role="xpack"]
[[xpack-logs-analysis]]
== Detecting and inspecting log anomalies
beta::[]
When the {ml} {anomaly-detect} features are enabled,
you can use the **Log rate** page in the Logs app.
**Log rate** helps you to detect and inspect log anomalies and the log partitions where the log anomalies occur.
This means you can easily spot anomalous behavior without significant human intervention --
no more manually sampling log data, calculating rates, and determining if rates are normal.
*Log rate* automatically highlights periods of time where the log rate is outside expected bounds,
and therefore may be anomalous.
You can use this information as a basis for further investigations.
For example:
* A significant drop in the log rate might suggest that a piece of infrastructure stopped responding,
and thus we're serving less requests.
* A spike in the log rate could denote a DDoS attack.
This may lead to an investigation of IP addresses from incoming requests.
You can also view log anomalies directly in the <<xpack-ml-anomalies,Machine Learning app>>.
[float]
[[logs-analysis-create-ml-job]]
=== Enable log rate analysis and anomaly detection
Create a machine learning job to enable log rate analysis and anomaly detection.
[role="screenshot"]
image::logs/images/analysis-tab-create-ml-job.png[Create machine learning job]
1. To enable log rate analysis and anomaly detection,
you must first create your own {kibana-ref}/xpack-spaces.html[space].
2. Within a space, navigate to the Logs app and select *Log rate*.
Here, you'll be prompted to create a machine learning job which will carry out the log rate analysis.
3. Choose a time range for the machine learning analysis.
4. Add the Indices that contain the logs you want to analyze.
5. Click *Create ML job*.
6. You're now ready to analyze your log partitions.
Even though the machine learning job's time range is fixed,
you can still use the time filter to adjust the results that are shown in your analysis.
[role="screenshot"]
image::logs/images/log-time-filter.png[Log rate time filter]
[float]
[[logs-analysis-entries-chart]]
=== Log entries chart
The log entries chart shows an overall, color-coded visualization of the log entry rate,
partitioned according to the value of the Elastic Common Schema (ECS)
{ecs-ref}/ecs-event.html[`event.dataset`] field.
This chart helps you quickly spot increases or decreases in each partition's log rate.
[role="screenshot"]
image::logs/images/log-rate-entries.png[Log rate entries chart]
If you have a lot of log partitions, use the following to filter your data:
* Hover over a time range to see the log rate for each partition.
* Click or hover on a partition name to show, hide, or highlight the partition values.
[float]
[[logs-analysis-anomalies-chart]]
=== Anomalies charts
The Anomalies chart shows the time range where anomalies were detected.
The typical rate values are shown in grey, while the anomalous regions are color-coded and superimposed on top.
[role="screenshot"]
image::logs/images/log-rate-anomalies.png[Log rate entries chart]
When a time range is flagged as anomalous,
the machine learning algorithms have detected unusual log rate activity.
This might be because:
* The log rate is significantly higher than usual.
* The log rate is significantly lower than usual.
* Other anomalous behavior has been detected.
For example, the log rate is within bounds, but not fluctuating when it is expected to.
The level of anomaly detected in a time period is color-coded, from red, orange, yellow, to blue.
Red indicates a critical anomaly level, while blue is a warning level.
To help you further drill down into a potential anomaly,
you can view an anomaly chart for each individual partition:
Anomaly scores range from 0 (no anomalies) to 100 (critical).
To analyze the anomalies in more detail, click *Analyze in ML*, which opens the
{kibana-ref}/xpack-ml.html[Anomaly Explorer in Machine Learning].

View file

@ -78,8 +78,19 @@ This opens the *Log event document details* fly-out that shows the fields associ
To quickly filter the logs stream by one of the field values, in the log event details, click the *View event with filter* icon image:logs/images/logs-view-event-with-filter.png[View event icon] beside the field.
This automatically adds a search filter to the logs stream to filter the entries by this field and value.
To see other actions related to the event, in the log event details, click *Actions*.
Depending on the event and the features you have installed and configured, you may also be able to:
[float]
[[view-log-anomalies]]
=== View log anomalies
When the machine learning anomaly detection features are enabled, click *Log rate*, which allows you to
<<xpack-logs-analysis,use machine learning to detect and inspect anomalies>> in your log data.
[float]
[[logs-integrations]]
=== Logs app integrations
To see other actions related to the event, click *Actions* in the log event details.
Depending on the event and the features you have configured, you may also be able to:
* Select *View status in Uptime* to <<uptime-overview, view related uptime information>> in the *Uptime* app.
* Select *View in APM* to <<traces, view related APM traces>> in the *APM* app.