2018-01-13 21:14:04 +01:00
[[vega-graph]]
2020-04-20 22:56:41 +02:00
== Vega
2018-01-13 21:14:04 +01:00
2020-04-20 22:56:41 +02:00
experimental[]
2018-01-13 21:14:04 +01:00
2020-04-20 22:56:41 +02:00
Build custom visualizations from multiple data sources using Vega
and Vega-Lite.
2019-02-14 11:35:05 +01:00
2020-04-20 22:56:41 +02:00
* *Vega* — A declarative format to create visualizations using JSON.
Generate interactive displays using D3.
2019-02-14 11:35:05 +01:00
2020-04-20 22:56:41 +02:00
* *Vega-Lite* — An easier format to use than Vega that enables more rapid
data analysis. Compiles into Vega.
2018-01-13 21:14:04 +01:00
2020-04-20 22:56:41 +02:00
For more information about Vega and Vega-Lite, refer to
<<vega-useful-links, Resources and examples>>.
2018-04-13 17:32:49 +02:00
2020-04-20 22:56:41 +02:00
[float]
[[create-vega-viz]]
=== Create Vega visualizations
2019-02-19 16:31:37 +01:00
2020-04-20 22:56:41 +02:00
You create Vega visualizations by using the text editor, which is
preconfigured with the options you need.
2018-01-13 21:14:04 +01:00
2020-04-20 22:56:41 +02:00
[role="screenshot"]
image::images/vega_lite_default.png[]
2019-02-19 16:31:37 +01:00
2020-04-20 22:56:41 +02:00
[float]
[[vega-schema]]
==== Change the Vega version
2020-03-09 16:29:25 +01:00
2020-04-20 22:56:41 +02:00
The default visualization uses Vega-Lite version 2. To use Vega version 4, edit
the `schema`.
2019-02-19 16:31:37 +01:00
2020-04-20 22:56:41 +02:00
Go to `$schema`, enter `https://vega.github.io/schema/vega/v4.json`, then click
*Update*.
2020-03-09 16:29:25 +01:00
2020-04-20 22:56:41 +02:00
[float]
[[vega-type]]
==== Change the visualization type
2019-02-19 16:31:37 +01:00
2020-04-20 22:56:41 +02:00
The default visualization is a line chart. To change the visualization type,
change the `mark` value. The supported visualization types are listed in the
text editor.
2018-01-13 21:14:04 +01:00
2020-04-20 22:56:41 +02:00
Go to `mark`, change the value to a different visualization type, then click
*Update*.
2019-02-19 16:31:37 +01:00
2020-04-20 22:56:41 +02:00
[float]
[[vega-sizing-and-positioning]]
==== Change the layout
2018-01-13 21:14:04 +01:00
2020-04-20 22:56:41 +02:00
By default, Vega visualizations use the `autosize = { type: 'fit', contains: 'padding' }` layout.
`fit` uses all available space, ignores `width` and `height` values,
and respects the padding values. To override this behavior, change the
`autosize` value.
2018-04-13 17:32:49 +02:00
2018-01-13 21:14:04 +01:00
[[vega-querying-elasticsearch]]
2020-04-20 22:56:41 +02:00
=== Query {es}
2018-01-13 21:14:04 +01:00
2020-04-20 22:56:41 +02:00
experimental[] Vega https://vega.github.io/vega/docs/data/[data] elements
use embedded and external data with a `"url"` parameter. {kib} adds support for
direct {es} queries by overloading
2018-01-13 21:14:04 +01:00
the `"url"` value.
2020-04-20 22:56:41 +02:00
NOTE: With Vega, you dynamically load your data by setting signals as data URLs.
Since {kib} is unable to support dynamically loaded data, all data is fetched
before it's passed to the Vega renderer.
For example, count the number of documents in all indices:
2018-01-13 21:14:04 +01:00
[source,yaml]
----
2020-04-20 22:56:41 +02:00
// An object instead of a string for the URL value
2018-01-13 21:14:04 +01:00
// is treated as a context-aware Elasticsearch query.
url: {
2020-04-20 22:56:41 +02:00
// Specify the time filter.
2018-01-13 21:14:04 +01:00
%timefield%: @timestamp
// Apply dashboard context filters when set
%context%: true
// Which indexes to search
index: _all
// The body element may contain "aggs" and "query" subfields
body: {
aggs: {
time_buckets: {
date_histogram: {
// Use date histogram aggregation on @timestamp field
2020-04-20 22:56:41 +02:00
field: @timestamp <1>
// interval value will depend on the time filter
2018-01-13 21:14:04 +01:00
// Use an integer to set approximate bucket count
interval: { %autointerval%: true }
// Make sure we get an entire range, even if it has no data
extended_bounds: {
min: { %timefilter%: "min" }
max: { %timefilter%: "max" }
}
// Use this for linear (e.g. line, area) graphs
// Without it, empty buckets will not show up
min_doc_count: 0
}
}
}
// Speed up the response by only including aggregation results
size: 0
}
}
----
2020-04-20 22:56:41 +02:00
<1> `@timestamp` — Filters the time range and breaks it into histogram
buckets.
The full result includes the following structure:
2018-01-13 21:14:04 +01:00
[source,yaml]
----
{
"aggregations": {
"time_buckets": {
"buckets": [{
"key_as_string": "2015-11-30T22:00:00.000Z",
2020-04-20 22:56:41 +02:00
"key": 1448920800000,<1>
2018-01-13 21:14:04 +01:00
"doc_count": 28
}, {
"key_as_string": "2015-11-30T23:00:00.000Z",
2020-04-20 22:56:41 +02:00
"key": 1448924400000, <1>
2018-01-13 21:14:04 +01:00
"doc_count": 330
}, ...
----
2020-04-20 22:56:41 +02:00
<1> `"key"` — The unix timestamp you can use without conversions by the
2018-01-13 21:14:04 +01:00
Vega date expressions.
2020-04-20 22:56:41 +02:00
For most visualizations, you only need the list of bucket values. To focus on
only the data you need, use `format: {property: "aggregations.time_buckets.buckets"}`.
2018-01-13 21:14:04 +01:00
2020-04-20 22:56:41 +02:00
Specify a query with individual range and dashboard context. The query is
equivalent to `"%context%": true, "%timefield%": "@timestamp"`,
except that the time range is shifted back by 10 minutes:
2018-01-13 21:14:04 +01:00
[source,yaml]
----
{
body: {
query: {
bool: {
must: [
// This string will be replaced
// with the auto-generated "MUST" clause
"%dashboard_context-must_clause%"
{
range: {
// apply timefilter (upper right corner)
// to the @timestamp variable
@timestamp: {
// "%timefilter%" will be replaced with
// the current values of the time filter
// (from the upper right corner)
"%timefilter%": true
// Only work with %timefilter%
// Shift current timefilter by 10 units back
shift: 10
// week, day (default), hour, minute, second
unit: minute
}
}
}
]
must_not: [
// This string will be replaced with
// the auto-generated "MUST-NOT" clause
"%dashboard_context-must_not_clause%"
]
2020-06-11 08:08:55 +02:00
filter: [
// This string will be replaced
// with the auto-generated "FILTER" clause
"%dashboard_context-filter_clause%"
]
2018-01-13 21:14:04 +01:00
}
}
}
}
----
2020-06-11 08:08:55 +02:00
NOTE: When using `"%context%": true` or defining a value for "%timefield%"` the body cannot contain a query. To customize the query within the VEGA specification (e.g. add an additional filter, or shift the timefilter), define your query and use the placeholders as in the example above. The placeholders will be replaced by the actual context of the dashboard or visualization once parsed.
2018-01-13 21:14:04 +01:00
The `"%timefilter%"` can also be used to specify a single min or max
2020-03-09 16:29:25 +01:00
value. The date_histogram's `extended_bounds` can be set
2018-01-13 21:14:04 +01:00
with two values - min and max. Instead of hardcoding a value, you may
use `"min": {"%timefilter%": "min"}`, which will be replaced with the
beginning of the current time range. The `shift` and `unit` values are
also supported. The `"interval"` can also be set dynamically, depending
on the currently picked range: `"interval": {"%autointerval%": 10}` will
try to get about 10-15 data points (buckets).
[[vega-esmfiles]]
2020-04-20 22:56:41 +02:00
=== Access Elastic Map Service files
2018-01-13 21:14:04 +01:00
2020-04-20 22:56:41 +02:00
experimental[] Access the Elastic Map Service files via the same mechanism:
2018-01-13 21:14:04 +01:00
[source,yaml]
----
url: {
// "type" defaults to "elasticsearch" otherwise
type: emsfile
// Name of the file, exactly as in the Region map visualization
name: World Countries
}
// The result is a geojson file, get its features to use
// this data source with the "shape" marks
// https://vega.github.io/vega/docs/marks/shape/
format: {property: "features"}
----
2020-04-20 22:56:41 +02:00
To enable Elastic Maps, the graph must specify `type=map` in the host
configuration:
2018-04-06 22:09:41 +02:00
[source,yaml]
----
{
"config": {
"kibana": {
"type": "map",
// Initial map position
"latitude": 40.7, // default 0
"longitude": -74, // default 0
"zoom": 7, // default 2
// defaults to "default". Use false to disable base layer.
"mapStyle": false,
// default 0
"minZoom": 5,
// defaults to the maximum for the given style,
// or 25 when base is disabled
"maxZoom": 13,
// defaults to true, shows +/- buttons to zoom in/out
"zoomControl": false,
2020-03-09 16:29:25 +01:00
// Defaults to 'false', disables mouse wheel zoom. If set to
2018-10-29 22:31:16 +01:00
// 'true', map may zoom unexpectedly while scrolling dashboard
2018-04-06 22:09:41 +02:00
"scrollWheelZoom": false,
// When false, repaints on each move frame.
// Makes the graph slower when moving the map
"delayRepaint": true, // default true
}
},
/* the rest of Vega JSON */
}
----
2020-04-20 22:56:41 +02:00
The visualization automatically injects a `"projection"`, which you can use to
calculate the position of all geo-aware marks.
Additionally, you can use `latitude`, `longitude`, and `zoom` signals.
2018-04-06 22:09:41 +02:00
These signals can be used in the graph, or can be updated to modify the
2020-04-20 22:56:41 +02:00
position of the map.
Vega visualization ignore the `autosize`, `width`, `height`, and `padding`
values, using `fit` model with zero padding.
2018-04-06 22:09:41 +02:00
2018-01-13 21:14:04 +01:00
[[vega-debugging]]
2020-04-20 22:56:41 +02:00
=== Debugging Vega
2018-01-13 21:14:04 +01:00
[[vega-browser-debugging-console]]
2020-04-20 22:56:41 +02:00
==== Browser debugging console
2018-01-13 21:14:04 +01:00
2020-04-20 22:56:41 +02:00
experimental[] Use browser debugging tools (for example, F12 or Ctrl+Shift+J in Chrome) to
2018-01-13 21:14:04 +01:00
inspect the `VEGA_DEBUG` variable:
2020-04-20 22:56:41 +02:00
+
* `view` — Access to the Vega View object. See https://vega.github.io/vega/docs/api/debugging/[Vega Debugging Guide]
on how to inspect data and signals at runtime. For Vega-Lite, `VEGA_DEBUG.view.data('source_0')` gets the main data set.
For Vega, it uses the data name as defined in your Vega spec.
* `vega_spec` — Vega JSON graph specification after some modifications by {kib}. In case
2019-02-19 16:31:37 +01:00
of Vega-Lite, this is the output of the Vega-Lite compiler.
2020-04-20 22:56:41 +02:00
* `vegalite_spec` — If this is a Vega-Lite graph, JSON specification of the graph before
2019-02-19 16:31:37 +01:00
Vega-Lite compilation.
2018-01-13 21:14:04 +01:00
[[vega-data]]
2018-04-13 17:32:49 +02:00
==== Data
2018-01-13 21:14:04 +01:00
2020-04-20 22:56:41 +02:00
experimental[] If you are using an {es} query, make sure your resulting data is
what you expected. The easiest way to view it is by using the "networking"
tab in the browser debugging tools (for example, F12). Modify the graph slightly
2018-01-13 21:14:04 +01:00
so that it makes a search request, and view the response from the
server. Another approach is to use
2020-04-20 22:56:41 +02:00
https://www.elastic.co/guide/en/kibana/current/console-kibana.html[Dev Tools]. Place the index name into the first line:
`GET <INDEX_NAME>/_search`, then add your query as the following lines
(just the value of the `"query"` field).
2018-01-13 21:14:04 +01:00
2020-04-20 22:56:41 +02:00
If you need to share your graph with someone, copy the
2018-01-13 21:14:04 +01:00
raw data response to https://gist.github.com/[gist.github.com], possibly
with a `.json` extension, use the `[raw]` button, and use that url
directly in your graph.
To restrict Vega from using non-ES data sources, add `vega.enableExternalUrls: false`
to your kibana.yml file.
[[vega-notes]]
[[vega-useful-links]]
2020-04-20 22:56:41 +02:00
=== Resources and examples
experimental[] To learn more about Vega and Vega-List, refer to the resources and examples.
2018-01-13 21:14:04 +01:00
2020-04-20 22:56:41 +02:00
==== Vega editor
2020-03-09 16:29:25 +01:00
The https://vega.github.io/editor/[Vega Editor] includes examples for Vega & Vega-Lite, but does not support any
2019-02-19 16:31:37 +01:00
{kib}-specific features like {es} requests and interactive base maps.
==== Vega-Lite resources
* https://vega.github.io/vega-lite/tutorials/getting_started.html[Tutorials]
* https://vega.github.io/vega-lite/docs/[Docs]
* https://vega.github.io/vega-lite/examples/[Examples]
2020-03-09 16:29:25 +01:00
==== Vega resources
2019-02-19 16:31:37 +01:00
* https://vega.github.io/vega/tutorials/[Tutorials]
* https://vega.github.io/vega/docs/[Docs]
* https://vega.github.io/vega/examples/[Examples]
2020-04-20 22:56:41 +02:00
TIP: When you use the examples, you may
2018-01-13 21:14:04 +01:00
need to modify the "data" section to use absolute URL. For example,
replace `"url": "data/world-110m.json"` with
2020-04-20 22:56:41 +02:00
`"url": "https://vega.github.io/editor/data/world-110m.json"`.
2018-01-13 21:14:04 +01:00
[[vega-additional-configuration-options]]
==== Additional configuration options
2020-04-20 22:56:41 +02:00
These options are specific to the {kib}. link:#vega-with-a-map[Map support] has
2018-04-06 22:09:41 +02:00
additional configuration options.
2018-01-13 21:14:04 +01:00
[source,yaml]
----
{
config: {
kibana: {
// Placement of the Vega-defined signal bindings.
// Can be `left`, `right`, `top`, or `bottom` (default).
controlsLocation: top
// Can be `vertical` or `horizontal` (default).
controlsDirection: vertical
// If true, hides most of Vega and VegaLite warnings
hideWarnings: true
// Vega renderer to use: `svg` or `canvas` (default)
renderer: canvas
}
}
/* the rest of Vega code */
}
----