264 lines
9 KiB
Plaintext
264 lines
9 KiB
Plaintext
[[managing-index-patterns]]
|
|
== Manage index pattern data fields
|
|
|
|
To customize the data fields in your index pattern, you can add runtime fields to the existing documents, add scrited fields to compute data on the fly, and change how {kib} displays the data fields.
|
|
|
|
[float]
|
|
[[runtime-fields]]
|
|
=== Explore your data with runtime fields
|
|
|
|
Runtime fields are fields that you add to documents after you've ingested your data, and are evaluated at query time. With runtime fields, you allow for a smaller index and faster ingest time so that you can use less resources and reduce your operating costs. You can use runtime fields anywhere index patterns are used, for example, you can explore runtime fields in *Discover* and create visualizations with runtime fields for your dashboard.
|
|
|
|
With runtime fields, you can:
|
|
|
|
* Define fields for a specific use case without modifying the underlying schema.
|
|
|
|
* Override the returned values from index fields.
|
|
|
|
* Start working on your data without understanding the structure.
|
|
|
|
* Add fields to existing documents without reindexing your data.
|
|
|
|
WARNING: Runtime fields can impact {kib} performance. When you run a query, {es} uses the fields you index first to shorten the response time.
|
|
Index the fields that you commonly search for and filter on, such as `timestamp`, then use runtime fields to limit the number of fields {es} uses to calculate values.
|
|
|
|
For detailed information on how to use runtime fields with {es}, refer to {ref}/runtime.html[Runtime fields].
|
|
|
|
[float]
|
|
[[create-runtime-fields]]
|
|
==== Add runtime fields
|
|
|
|
To add runtime fields to your index patterns, open the index pattern you want to change, then define the field values by emitting a single value using the {ref}/modules-scripting-painless.html[Painless scripting language]. You can also add runtime fields in <<add-field-in-discover,*Discover*>> and <<change-the-fields,*Lens*>>.
|
|
|
|
. Open the main menu, then click *Stack Management > Index Patterns*.
|
|
|
|
. Select the index pattern you want to add the runtime field to, then click *Add field*.
|
|
|
|
. Enter the field *Name*, then select the *Type*.
|
|
|
|
. Select *Set custom label*, then enter the label you want to display where the index pattern is used, such as *Discover*.
|
|
|
|
. Select *Set value*, then define the script. The script must match the *Type*, or the index pattern fails anywhere it is used.
|
|
|
|
. To help you define the script, use the *Preview*:
|
|
|
|
* To view the other available fields, use the *Document ID* arrows.
|
|
|
|
* To filter the fields list, enter the keyword in *Filter fields*.
|
|
|
|
* To pin frequently used fields to the top of the list, hover over the field, then click image:images/stackManagement-indexPatterns-pinRuntimeField-7.15.png[Icon to pin field to the top of the list].
|
|
|
|
. Click *Create field*.
|
|
|
|
[float]
|
|
[[runtime-field-examples]]
|
|
==== Runtime field examples
|
|
|
|
Try the runtime field examples on your own using the <<gs-get-data-into-kibana,*Sample web logs*>> data index pattern.
|
|
|
|
[float]
|
|
[[simple-hello-world-example]]
|
|
==== Return a keyword value
|
|
|
|
Return `Hello World!`:
|
|
|
|
[source,text]
|
|
----
|
|
emit("Hello World!");
|
|
----
|
|
|
|
[float]
|
|
[[perform-a-calculation-on-a-single-field]]
|
|
===== Perform a calculation on a single field
|
|
|
|
Calculate kilobytes from bytes:
|
|
|
|
[source,text]
|
|
----
|
|
emit(doc['bytes'].value / 1024)
|
|
----
|
|
|
|
[float]
|
|
[[return-substring]]
|
|
===== Return a substring
|
|
|
|
Return the string that appears after the last slash in the URL:
|
|
|
|
[source,text]
|
|
----
|
|
def path = doc["url.keyword"].value;
|
|
if (path != null) {
|
|
int lastSlashIndex = path.lastIndexOf('/');
|
|
if (lastSlashIndex > 0) {
|
|
emit(path.substring(lastSlashIndex+1));
|
|
return;
|
|
}
|
|
}
|
|
emit("");
|
|
----
|
|
|
|
[float]
|
|
[[replace-nulls-with-blanks]]
|
|
===== Replace nulls with blanks
|
|
|
|
Replace `null` values with `None`:
|
|
|
|
[source,text]
|
|
----
|
|
def source = doc['referer'].value;
|
|
if (source != null) {
|
|
emit(source);
|
|
return;
|
|
}
|
|
else {
|
|
emit("None");
|
|
}
|
|
----
|
|
|
|
Specify the operating system condition:
|
|
|
|
[source,text]
|
|
----
|
|
def source = doc['machine.os.keyword'].value;
|
|
if (source != "") {
|
|
emit(source);
|
|
}
|
|
else {
|
|
emit("None");
|
|
}
|
|
----
|
|
|
|
[float]
|
|
[[manage-runtime-fields]]
|
|
==== Manage runtime fields
|
|
|
|
Edit the settings for runtime fields, or remove runtime fields from index patterns.
|
|
|
|
. Open the main menu, then click *Stack Management > Index Patterns*.
|
|
|
|
. Select the index pattern that contains the runtime field you want to manage, then open the runtime field edit options or delete the runtime field.
|
|
|
|
[float]
|
|
[[scripted-fields]]
|
|
=== Add scripted fields to index patterns
|
|
|
|
deprecated::[7.13,Use {ref}/runtime.html[runtime fields] instead of scripted fields. Runtime fields support Painless scripts and provide greater flexibility.]
|
|
|
|
Scripted fields compute data on the fly from the data in your {es} indices. The data is shown on
|
|
the Discover tab as part of the document data, and you can use scripted fields in your visualizations. You query scripted fields with the <<kuery-query, {kib} query language>>, and can filter them using the filter bar. The scripted field values are computed at query time, so they aren't indexed and cannot be searched using the {kib} default
|
|
query language.
|
|
|
|
WARNING: Computing data on the fly with scripted fields can be very resource intensive and can have a direct impact on
|
|
{kib} performance. Keep in mind that there's no built-in validation of a scripted field. If your scripts are
|
|
buggy, you'll get exceptions whenever you try to view the dynamically generated data.
|
|
|
|
When you define a scripted field in {kib}, you have a choice of the {ref}/modules-scripting-expression.html[Lucene expressions] or the
|
|
{ref}/modules-scripting-painless.html[Painless] scripting language.
|
|
|
|
You can reference any single value numeric field in your expressions, for example:
|
|
|
|
----
|
|
doc['field_name'].value
|
|
----
|
|
|
|
For more information on scripted fields and additional examples, refer to
|
|
https://www.elastic.co/blog/using-painless-kibana-scripted-fields[Using Painless in {kib} scripted fields]
|
|
|
|
[float]
|
|
[[create-scripted-field]]
|
|
==== Create scripted fields
|
|
|
|
Create and add scripted fields to your index patterns.
|
|
|
|
. Open the main menu, then click *Stack Management > Index Patterns*.
|
|
|
|
. Select the index pattern you want to add a scripted field to.
|
|
|
|
. Select the *Scripted fields* tab, then click *Add scripted field*.
|
|
|
|
. Enter a *Name* for the scripted field, then enter the *Script* you want to use to compute a value on the fly from your index data.
|
|
|
|
. Click *Create field*.
|
|
|
|
For more information about scripted fields in {es}, refer to {ref}/modules-scripting.html[Scripting].
|
|
|
|
[float]
|
|
[[update-scripted-field]]
|
|
==== Manage scripted fields
|
|
|
|
. Open the main menu, then click *Stack Management > Index Patterns*.
|
|
|
|
. Select the index pattern that contains the scripted field you want to manage.
|
|
|
|
. Select the *Scripted fields* tab, then open the scripted field edit options or delete the scripted field.
|
|
|
|
WARNING: Built-in validation is unsupported for scripted fields. When your scripts contain errors, you receive
|
|
exceptions when you view the dynamically generated data.
|
|
|
|
[float]
|
|
[[managing-fields]]
|
|
=== Format data fields
|
|
|
|
{kib} uses the same field types as {es}, however, some {es} field types are unsupported in {kib}.
|
|
To customize how {kib} displays data fields, use the formatting options.
|
|
|
|
. Open the main menu, then click *Stack Management > Index Patterns*.
|
|
|
|
. Click the index pattern that contains the field you want to change.
|
|
|
|
. Find the field, then open the edit options (image:management/index-patterns/images/edit_icon.png[Data field edit icon]).
|
|
|
|
. Select *Set custom label*, then enter a *Custom label* for the field.
|
|
|
|
. Select *Set format*, then enter the *Format* for the field.
|
|
|
|
[float]
|
|
[[string-field-formatters]]
|
|
==== String field formatters
|
|
|
|
String fields support *String* and *Url* formatters.
|
|
|
|
include::field-formatters/string-formatter.asciidoc[]
|
|
|
|
include::field-formatters/url-formatter.asciidoc[]
|
|
|
|
[float]
|
|
[[field-formatters-date]]
|
|
==== Date field formatters
|
|
|
|
Date fields support *Date*, *String*, and *Url* formatters.
|
|
|
|
The *Date* formatter enables you to choose the display format of date stamps using the https://momentjs.com/[moment.js]
|
|
standard format definitions.
|
|
|
|
include::field-formatters/string-formatter.asciidoc[]
|
|
|
|
include::field-formatters/url-formatter.asciidoc[]
|
|
|
|
[float]
|
|
[[field-formatters-geopoint]]
|
|
==== Geographic point field formatters
|
|
|
|
Geographic point fields support the *String* formatter.
|
|
|
|
include::field-formatters/string-formatter.asciidoc[]
|
|
|
|
[float]
|
|
[[field-formatters-numeric]]
|
|
==== Number field formatters
|
|
|
|
Numeric fields support *Bytes*, *Color*, *Duration*, *Histogram*, *Number*, *Percentage*, *String*, and *Url* formatters.
|
|
|
|
The *Bytes*, *Number*, and *Percentage* formatters enable you to choose the display formats of numbers in the field using
|
|
the <<numeral, Elastic numeral pattern>> syntax that {kib} maintains.
|
|
|
|
The *Histogram* formatter is used only for the {ref}/histogram.html[histogram field type]. When you use the *Histogram* formatter,
|
|
you can apply the *Bytes*, *Number*, or *Percentage* format to aggregated data.
|
|
|
|
include::field-formatters/url-formatter.asciidoc[]
|
|
|
|
include::field-formatters/string-formatter.asciidoc[]
|
|
|
|
include::field-formatters/duration-formatter.asciidoc[]
|
|
|
|
include::field-formatters/color-formatter.asciidoc[] |