436 lines
16 KiB
Plaintext
436 lines
16 KiB
Plaintext
[[timelion]]
|
|
== Timelion
|
|
|
|
Timelion is a time series data visualizer that enables you to combine totally
|
|
independent data sources within a single visualization. It's driven by a simple
|
|
expression language you use to retrieve time series data, perform calculations
|
|
to tease out the answers to complex questions, and visualize the results.
|
|
|
|
For example, Timelion enables you to easily get the answers to questions like:
|
|
|
|
* <<time-series-intro, What is the real-time percentage of CPU time spent in user space to the results offset by one hour?>>
|
|
* <<mathematical-functions-intro, What does my inbound and outbound network traffic look like?>>
|
|
* <<timelion-conditional-intro, How much memory is my system actually using?>>
|
|
|
|
[float]
|
|
[[time-series-before-you-begin]]
|
|
=== Before you begin
|
|
|
|
In this tutorial, you'll use the time series data from https://www.elastic.co/guide/en/beats/metricbeat/current/index.html[Metricbeat]. To ingest the data locally, link:https://www.elastic.co/downloads/beats/metricbeat[download Metricbeat].
|
|
|
|
[float]
|
|
[[time-series-intro]]
|
|
=== Create time series visualizations
|
|
|
|
To compare the real-time percentage of CPU time spent in user space to the results offset by one hour, create a time series visualization.
|
|
|
|
[float]
|
|
[[time-series-define-functions]]
|
|
==== Define the functions
|
|
|
|
To start tracking the real-time percentage of CPU, enter the following in the *Timelion Expression* field:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(index=metricbeat-*, timefield='@timestamp', metric='avg:system.cpu.user.pct')
|
|
----------------------------------
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-create01.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[time-series-compare-data]]
|
|
==== Compare the data
|
|
|
|
To compare the two data sets, add another series with data from the previous hour, separated by a comma:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='avg:system.cpu.user.pct'),
|
|
.es(offset=-1h, <1>
|
|
index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='avg:system.cpu.user.pct')
|
|
----------------------------------
|
|
|
|
<1> `offset` offsets the data retrieval by a date expression. In this example, `-1h` offsets the data back by one hour.
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-create02.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[time-series-add-labels]]
|
|
==== Add label names
|
|
|
|
To easily distinguish between the two data sets, add the label names:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(offset=-1h,index=metricbeat-*, timefield='@timestamp', metric='avg:system.cpu.user.pct').label('last hour'), .es(index=metricbeat-*, timefield='@timestamp', metric='avg:system.cpu.user.pct').label('current hour') <1>
|
|
----------------------------------
|
|
|
|
<1> `.label()` adds custom labels to the visualization.
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-create03.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[time-series-title]]
|
|
==== Add a title
|
|
|
|
Add a meaningful title:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(offset=-1h,
|
|
index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='avg:system.cpu.user.pct')
|
|
.label('last hour'),
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='avg:system.cpu.user.pct')
|
|
.label('current hour')
|
|
.title('CPU usage over time') <1>
|
|
----------------------------------
|
|
|
|
<1> `.title()` adds a title with a meaningful name. Titles make is easier for unfamiliar users to understand the purpose of the visualization.
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-customize01.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[time-series-change-chart-type]]
|
|
==== Change the chart type
|
|
|
|
To differentiate between the current hour data and the last hour data, change the chart type:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(offset=-1h,
|
|
index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='avg:system.cpu.user.pct')
|
|
.label('last hour')
|
|
.lines(fill=1,width=0.5), <1>
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='avg:system.cpu.user.pct')
|
|
.label('current hour')
|
|
.title('CPU usage over time')
|
|
----------------------------------
|
|
|
|
<1> `.lines()` changes the appearance of the chart lines. In this example, `.lines(fill=1,width=0.5)` sets the fill level to `1`, and the border width to `0.5`.
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-customize02.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[time-series-change-color]]
|
|
==== Change the line colors
|
|
|
|
To make the current hour data stand out, change the line colors:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(offset=-1h,
|
|
index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='avg:system.cpu.user.pct')
|
|
.label('last hour')
|
|
.lines(fill=1,width=0.5)
|
|
.color(gray), <1>
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='avg:system.cpu.user.pct')
|
|
.label('current hour')
|
|
.title('CPU usage over time')
|
|
.color(#1E90FF)
|
|
----------------------------------
|
|
|
|
<1> `.color()` changes the color of the data. Supported color types include standard color names, hexadecimal values, or a color schema for grouped data. In this example, `.color(gray)` represents the last hour, and `.color(#1E90FF)` represents the current hour.
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-customize03.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[time-series-adjust-legend]]
|
|
==== Make adjustments to the legend
|
|
|
|
Change the position and style of the legend:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(offset=-1h,index=metricbeat-*, timefield='@timestamp', metric='avg:system.cpu.user.pct').label('last hour').lines(fill=1,width=0.5).color(gray), .es(index=metricbeat-*, timefield='@timestamp', metric='avg:system.cpu.user.pct').label('current hour').title('CPU usage over time').color(#1E90FF).legend(columns=2, position=nw) <1>
|
|
----------------------------------
|
|
|
|
<1> `.legend()` sets the position and style of the legend. In this example, `.legend(columns=2, position=nw)` places the legend in the north west position of the visualization with two columns.
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-customize04.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[mathematical-functions-intro]]
|
|
=== Create visualizations with mathematical functions
|
|
|
|
To create a visualization for inbound and outbound network traffic, use mathematical functions.
|
|
|
|
[float]
|
|
[[mathematical-functions-define-functions]]
|
|
==== Define the functions
|
|
|
|
To start tracking the inbound and outbound network traffic, enter the following in the *Timelion Expression* field:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(index=metricbeat*, timefield=@timestamp, metric=max:system.network.in.bytes)
|
|
----------------------------------
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-math01.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[mathematical-functions-plot-change]]
|
|
==== Plot the rate of change
|
|
|
|
Change how the data is displayed so that you can easily monitor the inbound traffic:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(index=metricbeat*, timefield=@timestamp, metric=max:system.network.in.bytes).derivative() <1>
|
|
----------------------------------
|
|
|
|
<1> `.derivative` plots the change in values over time.
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-math02.png[]
|
|
{nbsp}
|
|
|
|
Add a similar calculation for outbound traffic:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(index=metricbeat*, timefield=@timestamp, metric=max:system.network.in.bytes).derivative(), .es(index=metricbeat*, timefield=@timestamp, metric=max:system.network.out.bytes).derivative().multiply(-1) <1>
|
|
----------------------------------
|
|
|
|
<1> `.multiply()` multiplies the data series by a number, the result of a data series, or a list of data series. For this example, `.multiply(-1)` converts the outbound network traffic to a negative value since the outbound network traffic is leaving your machine.
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-math03.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[mathematical-functions-convert-data]]
|
|
==== Change the data metric
|
|
|
|
To make the visualization easier to analyze, change the data metric from bytes to megabytes:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(index=metricbeat*, timefield=@timestamp, metric=max:system.network.in.bytes).derivative().divide(1048576), .es(index=metricbeat*, timefield=@timestamp, metric=max:system.network.out.bytes).derivative().multiply(-1).divide(1048576) <1>
|
|
----------------------------------
|
|
|
|
<1> `.divide()` accepts the same input as `.multiply()`, then divides the data series by the defined divisor.
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-math04.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[mathematical-functions-add-labels]]
|
|
==== Customize and format the visualization
|
|
|
|
Customize and format the visualization using functions:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(index=metricbeat*,
|
|
timefield=@timestamp,
|
|
metric=max:system.network.in.bytes)
|
|
.derivative()
|
|
.divide(1048576)
|
|
.lines(fill=2, width=1)
|
|
.color(green)
|
|
.label("Inbound traffic") <1>
|
|
.title("Network traffic (MB/s)"), <2>
|
|
.es(index=metricbeat*,
|
|
timefield=@timestamp,
|
|
metric=max:system.network.out.bytes)
|
|
.derivative()
|
|
.multiply(-1)
|
|
.divide(1048576)
|
|
.lines(fill=2, width=1) <3>
|
|
.color(blue) <4>
|
|
.label("Outbound traffic")
|
|
.legend(columns=2, position=nw) <5>
|
|
----------------------------------
|
|
|
|
<1> `.label()` adds custom labels to the visualization.
|
|
<2> `.title()` adds a title with a meaningful name.
|
|
<3> `.lines()` changes the appearance of the chart lines. In this example, `.lines(fill=2, width=1)` sets the fill level to `2`, and the border width to `1`.
|
|
<4> `.color()` changes the color of the data. Supported color types include standard color names, hexadecimal values, or a color schema for grouped data. In this example, `.color(green)` represents the inbound network traffic, and `.color(blue)` represents the outbound network traffic.
|
|
<5> `.legend()` sets the position and style of the legend. For this example, `legend(columns=2, position=nw)` places the legend in the north west position of the visualization with two columns.
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-math05.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[timelion-conditional-intro]]
|
|
=== Create visualizations with conditional logic and tracking trends
|
|
|
|
To easily detect outliers and discover patterns over time, modify time series data with conditional logic and create a trend with a moving average.
|
|
|
|
With Timelion conditional logic, you can use the following operator values to compare your data:
|
|
|
|
[horizontal]
|
|
`eq`:: equal
|
|
`ne`:: not equal
|
|
`lt`:: less than
|
|
`lte`:: less than or equal to
|
|
`gt`:: greater than
|
|
`gte`:: greater than or equal to
|
|
|
|
[float]
|
|
[[conditional-define-functions]]
|
|
==== Define the functions
|
|
|
|
To chart the maximum value of `system.memory.actual.used.bytes`, enter the following in the *Timelion Expression* field:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(index=metricbeat-*, timefield='@timestamp', metric='max:system.memory.actual.used.bytes')
|
|
----------------------------------
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-conditional01.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[conditional-track-memory]]
|
|
==== Track used memory
|
|
|
|
To track the amount of memory used, create two thresholds:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='max:system.memory.actual.used.bytes'),
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='max:system.memory.actual.used.bytes')
|
|
.if(gt, <1>
|
|
11300000000, <2>
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='max:system.memory.actual.used.bytes'),
|
|
null)
|
|
.label('warning')
|
|
.color('#FFCC11'),
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='max:system.memory.actual.used.bytes')
|
|
.if(gt,
|
|
11375000000,
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='max:system.memory.actual.used.bytes'),
|
|
null)
|
|
.label('severe')
|
|
.color('red')
|
|
----------------------------------
|
|
|
|
<1> Timelion conditional logic for the _greater than_ operator. In this example, the warning threshold is 11.3GB (`11300000000`), and the severe threshold is 11.375GB (`11375000000`). If the threshold values are too high or low for your machine, adjust the values accordingly.
|
|
<2> `if()` compares each point to a number. If the condition evaluates to `true`, adjust the styling. If the condition evaluates to `false`, use the default styling.
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-conditional02.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[conditional-determine-trend]]
|
|
==== Determine the trend
|
|
|
|
To determine the trend, create a new data series:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(index=metricbeat-*, timefield='@timestamp', metric='max:system.memory.actual.used.bytes'), .es(index=metricbeat-*, timefield='@timestamp', metric='max:system.memory.actual.used.bytes').if(gt,11300000000,.es(index=metricbeat-*, timefield='@timestamp', metric='max:system.memory.actual.used.bytes'),null).label('warning').color('#FFCC11'), .es(index=metricbeat-*, timefield='@timestamp', metric='max:system.memory.actual.used.bytes').if(gt,11375000000,.es(index=metricbeat-*, timefield='@timestamp', metric='max:system.memory.actual.used.bytes'),null).label('severe').color('red'), .es(index=metricbeat-*, timefield='@timestamp', metric='max:system.memory.actual.used.bytes').mvavg(10) <1>
|
|
----------------------------------
|
|
|
|
<1> `mvavg()` calculates the moving average over a specified period of time. In this example, `.mvavg(10)` creates a moving average with a window of 10 data points.
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-conditional03.png[]
|
|
{nbsp}
|
|
|
|
[float]
|
|
[[conditional-format-visualization]]
|
|
==== Customize and format the visualization
|
|
|
|
Customize and format the visualization using functions:
|
|
|
|
[source,text]
|
|
----------------------------------
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='max:system.memory.actual.used.bytes')
|
|
.label('max memory') <1>
|
|
.title('Memory consumption over time'), <2>
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='max:system.memory.actual.used.bytes')
|
|
.if(gt,
|
|
11300000000,
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='max:system.memory.actual.used.bytes'),
|
|
null)
|
|
.label('warning')
|
|
.color('#FFCC11') <3>
|
|
.lines(width=5), <4>
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='max:system.memory.actual.used.bytes')
|
|
.if(gt,
|
|
11375000000,
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='max:system.memory.actual.used.bytes'),
|
|
null)
|
|
.label('severe')
|
|
.color('red')
|
|
.lines(width=5),
|
|
.es(index=metricbeat-*,
|
|
timefield='@timestamp',
|
|
metric='max:system.memory.actual.used.bytes')
|
|
.mvavg(10)
|
|
.label('mvavg')
|
|
.lines(width=2)
|
|
.color(#5E5E5E)
|
|
.legend(columns=4, position=nw) <5>
|
|
----------------------------------
|
|
|
|
<1> `.label()` adds custom labels to the visualization.
|
|
<2> `.title()` adds a title with a meaningful name.
|
|
<3> `.color()` changes the color of the data. Supported color types include standard color names, hexadecimal values, or a color schema for grouped data.
|
|
<4> `.lines()` changes the appearance of the chart lines. In this example, .lines(width=5) sets border width to `5`.
|
|
<5> `.legend()` sets the position and style of the legend. For this example, `(columns=4, position=nw)` places the legend in the north west position of the visualization with four columns.
|
|
|
|
[role="screenshot"]
|
|
image::images/timelion-conditional04.png[]
|
|
{nbsp}
|
|
|
|
For additional information on Timelion conditional capabilities, go to https://www.elastic.co/blog/timeseries-if-then-else-with-timelion[I have but one .condition()].
|