264 lines
8.6 KiB
Plaintext
264 lines
8.6 KiB
Plaintext
[[managing-index-patterns]]
|
|
== Manage index pattern data fields
|
|
|
|
To customize the data fields in your index pattern, you can add runtime fields to the existing documents, add scrited fields to compute data on the fly, and change how {kib} displays the data fields.
|
|
|
|
[float]
|
|
[[runtime-fields]]
|
|
=== Explore your data with runtime fields
|
|
|
|
Runtime fields are fields that you add to documents after you've ingested, and are evaluated at query time. With runtime fields, you allow for a smaller index and faster ingest time so that you can use less resources and reduce your operating costs. You can use runtime fields anywhere index patterns are used.
|
|
|
|
When you use runtime fields, you can:
|
|
|
|
* Define fields for a specific use without modifying the underlying schema.
|
|
|
|
* Override the returned values from index fields.
|
|
|
|
* Start working on your data without first understanding the structure.
|
|
|
|
* Add fields to existing documents without reindexing your data.
|
|
|
|
* Explore runtime field data in *Discover*.
|
|
|
|
* Create visualizations with runtime field data using *Lens*, *Maps*, and *TSVB*.
|
|
|
|
WARNING: Runtime fields can impact {kib} performance. When you run a query, {es} uses the fields you index first to shorten the response time.
|
|
Index the fields that you commonly search for and filter on, such as `timestamp`, then use runtime fields to limit the number of fields {es} uses to calculate values.
|
|
|
|
For more information, refer to {ref}/runtime.html[Runtime fields].
|
|
|
|
[float]
|
|
[[create-runtime-fields]]
|
|
==== Create runtime fields
|
|
|
|
Create runtime fields in your index patterns, or create runtime fields in *Discover* and *Lens*.
|
|
|
|
. Open the main menu, then click *Stack Management > Index Patterns*.
|
|
|
|
. Select the index pattern you want to add the runtime field to, then click *Add field*.
|
|
|
|
. Enter a *Name* for the runtime field, then select the field *Type*.
|
|
|
|
. Select *Set value*, then define the field value by emitting a single value using the {ref}/modules-scripting-painless.html[Painless scripting language].
|
|
+
|
|
The script must match the field *Type*, or the script fails.
|
|
|
|
. Click *Create field*.
|
|
+
|
|
For information on how to create runtime fields in *Discover*, refer to <<add-field-in-discover,Add a field>>.
|
|
+
|
|
For information on how to create runtime fields in *Lens*, refer to <<add-fields-in-lens,Add fields>>.
|
|
|
|
[float]
|
|
[[runtime-field-examples]]
|
|
==== Runtime field examples
|
|
|
|
Try the runtime field examples on your own using the *Sample web logs* data index pattern.
|
|
|
|
[float]
|
|
[[simple-hello-world-example]]
|
|
==== Return a keyword value
|
|
|
|
To return `Hello World!` value:
|
|
|
|
[source,text]
|
|
----
|
|
emit("Hello World!");
|
|
----
|
|
|
|
[float]
|
|
[[perform-a-calculation-on-a-single-field]]
|
|
===== Perform a calculation on a single field
|
|
|
|
Calculate kilobytes from bytes:
|
|
|
|
[source,text]
|
|
----
|
|
emit(doc['bytes'].value / 1024)
|
|
----
|
|
|
|
[float]
|
|
[[return-substring]]
|
|
===== Return a substring
|
|
|
|
Return the string that appears after the last slash in the URL:
|
|
|
|
[source,text]
|
|
----
|
|
def path = doc["url.keyword"].value;
|
|
if (path != null) {
|
|
int lastSlashIndex = path.lastIndexOf('/');
|
|
if (lastSlashIndex > 0) {
|
|
emit(path.substring(lastSlashIndex+1));
|
|
return;
|
|
}
|
|
}
|
|
emit("");
|
|
----
|
|
|
|
[float]
|
|
[[replace-nulls-with-blanks]]
|
|
===== Replace nulls with blanks
|
|
|
|
Replace null values with none values:
|
|
|
|
[source,text]
|
|
----
|
|
def source = doc['referer'].value;
|
|
if (source != null) {
|
|
emit(source);
|
|
return;
|
|
}
|
|
else {
|
|
emit("None");
|
|
}
|
|
----
|
|
|
|
Specify operating system condition:
|
|
|
|
[source,text]
|
|
----
|
|
def source = doc['machine.os.keyword'].value;
|
|
if (source != "") {
|
|
emit(source);
|
|
}
|
|
else {
|
|
emit("None");
|
|
}
|
|
----
|
|
|
|
[float]
|
|
[[manage-runtime-fields]]
|
|
==== Manage runtime fields
|
|
|
|
Edit the settings for runtime fields, or remove runtime fields from index patterns.
|
|
|
|
. Open the main menu, then click *Stack Management > Index Patterns*.
|
|
|
|
. Select the index pattern that contains the runtime field you want to manage, then open the runtime field edit options or delete the runtime field.
|
|
|
|
[float]
|
|
[[scripted-fields]]
|
|
=== Add scripted fields to index patterns
|
|
|
|
deprecated::[7.13,Use {ref}/runtime.html[runtime fields] instead of scripted fields. Runtime fields support Painless scripts and provide greater flexibility.]
|
|
|
|
Scripted fields compute data on the fly from the data in your {es} indices. The data is shown on
|
|
the Discover tab as part of the document data, and you can use scripted fields in your visualizations. You query scripted fields with the <<kuery-query, {kib} query language>>, and can filter them using the filter bar. The scripted field values are computed at query time, so they aren't indexed and cannot be searched using the {kib} default
|
|
query language.
|
|
|
|
WARNING: Computing data on the fly with scripted fields can be very resource intensive and can have a direct impact on
|
|
{kib} performance. Keep in mind that there's no built-in validation of a scripted field. If your scripts are
|
|
buggy, you'll get exceptions whenever you try to view the dynamically generated data.
|
|
|
|
When you define a scripted field in {kib}, you have a choice of the {ref}/modules-scripting-expression.html[Lucene expressions] or the
|
|
{ref}/modules-scripting-painless.html[Painless] scripting language.
|
|
|
|
You can reference any single value numeric field in your expressions, for example:
|
|
|
|
----
|
|
doc['field_name'].value
|
|
----
|
|
|
|
For more information on scripted fields and additional examples, refer to
|
|
https://www.elastic.co/blog/using-painless-kibana-scripted-fields[Using Painless in {kib} scripted fields]
|
|
|
|
[float]
|
|
[[create-scripted-field]]
|
|
==== Create scripted fields
|
|
|
|
Create and add scripted fields to your index patterns.
|
|
|
|
. Open the main menu, then click *Stack Management > Index Patterns*.
|
|
|
|
. Select the index pattern you want to add a scripted field to.
|
|
|
|
. Select the *Scripted fields* tab, then click *Add scripted field*.
|
|
|
|
. Enter a *Name* for the scripted field, then enter the *Script* you want to use to compute a value on the fly from your index data.
|
|
|
|
. Click *Create field*.
|
|
|
|
For more information about scripted fields in {es}, refer to {ref}/modules-scripting.html[Scripting].
|
|
|
|
[float]
|
|
[[update-scripted-field]]
|
|
==== Manage scripted fields
|
|
|
|
. Open the main menu, then click *Stack Management > Index Patterns*.
|
|
|
|
. Select the index pattern that contains the scripted field you want to manage.
|
|
|
|
. Select the *Scripted fields* tab, then open the scripted field edit options or delete the scripted field.
|
|
|
|
WARNING: Built-in validation is unsupported for scripted fields. When your scripts contain errors, you receive
|
|
exceptions when you view the dynamically generated data.
|
|
|
|
[float]
|
|
[[managing-fields]]
|
|
=== Format data fields
|
|
|
|
{kib} uses the same field types as {es}, however, some {es} field types are unsupported in {kib}.
|
|
To customize how {kib} displays data fields, use the formatting options.
|
|
|
|
. Open the main menu, then click *Stack Management > Index Patterns*.
|
|
|
|
. Click the index pattern that contains the field you want to change.
|
|
|
|
. Find the field, then open the edit options (image:management/index-patterns/images/edit_icon.png[Data field edit icon]).
|
|
|
|
. Select *Set custom label*, then enter a *Custom label* for the field.
|
|
|
|
. Select *Set format*, then enter the *Format* for the field.
|
|
|
|
[float]
|
|
[[string-field-formatters]]
|
|
==== String field formatters
|
|
|
|
String fields support *String* and *Url* formatters.
|
|
|
|
include::field-formatters/string-formatter.asciidoc[]
|
|
|
|
include::field-formatters/url-formatter.asciidoc[]
|
|
|
|
[float]
|
|
[[field-formatters-date]]
|
|
==== Date field formatters
|
|
|
|
Date fields support *Date*, *String*, and *Url* formatters.
|
|
|
|
The *Date* formatter enables you to choose the display format of date stamps using the https://momentjs.com/[moment.js]
|
|
standard format definitions.
|
|
|
|
include::field-formatters/string-formatter.asciidoc[]
|
|
|
|
include::field-formatters/url-formatter.asciidoc[]
|
|
|
|
[float]
|
|
[[field-formatters-geopoint]]
|
|
==== Geographic point field formatters
|
|
|
|
Geographic point fields support the *String* formatter.
|
|
|
|
include::field-formatters/string-formatter.asciidoc[]
|
|
|
|
[float]
|
|
[[field-formatters-numeric]]
|
|
==== Number field formatters
|
|
|
|
Numeric fields support *Bytes*, *Color*, *Duration*, *Histogram*, *Number*, *Percentage*, *String*, and *Url* formatters.
|
|
|
|
The *Bytes*, *Number*, and *Percentage* formatters enable you to choose the display formats of numbers in the field using
|
|
the <<numeral, Elastic numeral pattern>> syntax that {kib} maintains.
|
|
|
|
The *Histogram* formatter is used only for the {ref}/histogram.html[histogram field type]. When you use the *Histogram* formatter,
|
|
you can apply the *Bytes*, *Number*, or *Percentage* format to aggregated data.
|
|
|
|
include::field-formatters/url-formatter.asciidoc[]
|
|
|
|
include::field-formatters/string-formatter.asciidoc[]
|
|
|
|
include::field-formatters/duration-formatter.asciidoc[]
|
|
|
|
include::field-formatters/color-formatter.asciidoc[] |