[DOCS] Adds docs for managing, creating, and visualizing rollups (#25450)

* [DOCS] Adds docs for managing, creating, and visualizing rollups

* [DOCS] Fixed broken link]

* [DOCS] Updates docs based on review feedback
This commit is contained in:
gchaps 2018-11-09 12:57:59 -08:00 committed by gchaps
parent 9fd4e26f5d
commit 72b50919a8
9 changed files with 132 additions and 0 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 147 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 177 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 251 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 146 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 301 KiB

View file

@ -17,6 +17,10 @@ include::management/managing-licenses.asciidoc[]
include::management/index-patterns.asciidoc[]
include::management/rollups/create_and_manage_rollups.asciidoc[]
include::management/rollups/visualize_rollup_data.asciidoc[]
include::management/managing-fields.asciidoc[]
include::management/managing-indices.asciidoc[]

View file

@ -0,0 +1,80 @@
[[data-rollups]]
== Working with rollup indices
The {ref}/xpack-rollup.html[rollup feature in {es}]
enables you to summarize historical data and store it compactly for future analysis,
so you can query, aggregate, and visualize the data using a fraction of the storage.
This is a good way to keep costs down when you need to store months or years of
historical data for use in visualizations and reports.
{kib} supports rolled up data in two ways:
* You can create and manage a rollup job in Management
* You can create a visualization using rolled up data in
Visualize and view it in a dashboard
[[create-and-manage-rollup-job]]
=== Create and manage rollup jobs
In Management, you'll find a UI for viewing, creating, starting, stopping, and
deleting rollup jobs. A rollup job is a periodic task that summarizes data from
indices specified by an index pattern and rolls it into a new index. To navigate
to the UI, go to *Management*, and under *Elasticsearch*, click *Rollup Jobs*.
[role="screenshot"]
image::images/management_rollup_list.png[][List of currently active rollup jobs]
[float]
[[create-rollup-job]]
==== Creating a rollup job
{kib} makes it easy for you to create a rollup job by walking you through the
process step by step. The first step is to define the job logistics. These include
the name of the rollup job, the index or indices to summarize, and the output rollup index.
The index pattern cannot match the name of the output rollup index. For example,
if your index pattern is `metricbeat-*`, you cannot name your rollup index
`metricbeat-rollup`. Otherwise, the job will attempt to capture the data in the
rollup index.
[role="screenshot"]
image::images/management_create_rollup_job.png[][Wizard that walks you through creation of a rollup job]
You must set a schedule for the rollup job: how often to collect the data,
the number of documents to roll up at a time, and the duration of its latency.
The latency buffer field is provided to protect against the late arrival of data
from Beats or other sources. By delaying the rollup for the specified amount of
time from when the job starts, you allow for the inclusion of late-arriving data
in the rollup.
In the subsequent phases, you define the Date Histogram aggregation for the job
and optionally the Terms and Histogram aggregations.
* The Date Histogram aggregation defines the time intervals for summarizing the data.
This value is important because you cannot search the data with a smaller value
than this interval. However, you can aggregate buckets in a larger time interval.
* The Terms histogram enables you to split the time buckets into sub buckets for
term field values.
* The Histogram aggregation enables you to split the time buckets into sub buckets
for numeric field values.
The final step is to specify the fields for calculating metrics. For each selected
field, you can collect any or all of the following: value count, average, sum, min, and max.
Before you save the rollup job, {kib} displays a summary of the rollup job for
validation.
[float]
[[manage-rollup-job]]
==== Managing rollup jobs
Selecting a job on the *Rollup jobs* page shows its details. The Manage menu in
the lower right enables you to start, stop, and delete the rollup job.
You must first stop a rollup job before deleting it.
[role="screenshot"]
image::images/management_rollup_job_details.png[][Rollup job details]

View file

@ -0,0 +1,48 @@
[[visualize-rollup-data]]
=== Create a visualization using rolled up data
beta[]
You can visualize your rolled up data in a variety of charts, tables, maps, and
more. Most visualizations support rolled up data, with the exception of
Timelion, Visual Builder, and Vega visualizations.
You create an index pattern for rolled up data the same way you do for any data,
in *Management > Kibana > Index patterns*. Clicking *Create index pattern* includes
an item for creating a rollup index pattern, if a rollup index is detected in the cluster.
[role="screenshot"]
image::images/management_create_rollup_menu.png[Create index pattern menu]
You can match an index pattern to only rolled up data, or mix both rolled up
and raw data to visualize all data together. An index
pattern can match only one rolled up index, not multiple. There is no restriction
on the number of standard indices that an index pattern can match. To match multiple indices, use a comma
to separate the names, with no space after the comma.
When creating an index pattern, youre asked to set a time field for filtering.
With a rollup index, the time filter field is the same field used for
the rolled up date histogram aggregation.
Keep the following in mind when creating a visualization from rolled up data:
* The data in a rollup index only has summarized metrics for specific fields.
You cant search any other field from the original raw data.
* Data is summarized into time buckets that might be split into sub buckets for
numeric field values or terms. You can ask for a time aggregation that takes
several time buckets and combines them to lower granularity. For example,
if the rollup job was aggregated by hours, you can ask for buckets of days.
The data represented in this visualization comes from a rollup index and
standard indices.
[role="screenshot"]
image::images/management_rollups_visualization.png[][Rollups in visualizations]
You can mix rollup visualizations and regular visualizations in a dashboard.
The following dashboard shows this mix, along with a field filter. Note
that not all queries and filters are supported by rollups.
[role="screenshot"]
image::images/management_rolled_dashboard.png[][Rollups in dashboards]