2019-04-01 18:56:46 +02:00
|
|
|
|
[role="xpack"]
|
2018-05-24 20:59:34 +02:00
|
|
|
|
[[managing-indices]]
|
2020-02-06 18:16:32 +01:00
|
|
|
|
== Index Management
|
2018-05-24 20:59:34 +02:00
|
|
|
|
|
2019-02-13 22:51:36 +01:00
|
|
|
|
*Index Management* enables you to view index settings,
|
|
|
|
|
mappings, and statistics and perform index-level operations.
|
|
|
|
|
These include refreshing, flushing, clearing the cache, force merging segments,
|
2020-01-22 22:45:39 +01:00
|
|
|
|
freezing indices, and more. Practicing good index management helps ensure
|
2019-09-16 17:45:58 +02:00
|
|
|
|
that your data is stored in the most cost-effective way possible.
|
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
*Index Management* also helps you create index templates. A template reduces
|
|
|
|
|
the amount of bookkeeping when working with indices. Instead of manually
|
|
|
|
|
setting up your indices, you can create them automatically from a template,
|
2019-09-16 17:45:58 +02:00
|
|
|
|
ensuring that your settings, mappings, and aliases are consistently defined.
|
|
|
|
|
|
2020-06-12 16:39:36 +02:00
|
|
|
|
To manage your indices, open the menu, then go to *Stack Management > {es} > Index Management*.
|
2019-09-16 17:45:58 +02:00
|
|
|
|
|
|
|
|
|
[role="screenshot"]
|
|
|
|
|
image::images/management_index_labels.png[Index Management UI]
|
2018-05-24 20:59:34 +02:00
|
|
|
|
|
2019-02-13 22:51:36 +01:00
|
|
|
|
If security is enabled,
|
2020-01-22 22:45:39 +01:00
|
|
|
|
you must have the `monitor` cluster privilege and the `view_index_metadata`
|
|
|
|
|
and `manage` index privileges to view the data.
|
2019-09-16 17:45:58 +02:00
|
|
|
|
For index templates, you must have the `manage_index_templates` cluster privilege.
|
2019-11-26 21:36:35 +01:00
|
|
|
|
See {ref}/security-privileges.html[Security privileges] for more
|
2018-07-13 17:31:14 +02:00
|
|
|
|
information.
|
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
Before using this feature, you should be familiar with index management
|
2019-09-16 17:45:58 +02:00
|
|
|
|
operations. Refer to the {ref}/indices.html[index management APIs]
|
|
|
|
|
and the {ref}/indices-templates.html[index template APIs].
|
|
|
|
|
|
|
|
|
|
[float]
|
|
|
|
|
=== View and edit indices
|
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
When you open *Index Management*, you’re presented an overview of your configured indices.
|
|
|
|
|
Badges indicate if an index is {ref}/frozen-indices.html[frozen],
|
|
|
|
|
a {ref}/ccr-put-follow.html[follower index],
|
|
|
|
|
or a {ref}/rollup-get-rollup-index-caps.html[rollup index].
|
2019-09-16 17:45:58 +02:00
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
Clicking a badge narrows the list to only indices of that type.
|
2019-09-16 17:45:58 +02:00
|
|
|
|
You can also filter your indices using the search bar.
|
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
You can drill down into each index to investigate the index
|
|
|
|
|
{ref}/index-modules.html#index-modules-settings[settings], {ref}/mapping.html[mapping], and statistics.
|
2019-09-16 17:45:58 +02:00
|
|
|
|
From this view, you can also edit the index settings.
|
2019-02-13 22:51:36 +01:00
|
|
|
|
|
2018-05-24 20:59:34 +02:00
|
|
|
|
[role="screenshot"]
|
2019-09-16 17:45:58 +02:00
|
|
|
|
image::images/management_index_details.png[Index Management UI]
|
2019-02-13 22:51:36 +01:00
|
|
|
|
|
2019-09-16 17:45:58 +02:00
|
|
|
|
[float]
|
|
|
|
|
=== Perform index-level operations
|
2019-02-13 22:51:36 +01:00
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
Use the *Manage* menu to perform index-level operations. This menu
|
|
|
|
|
is available in the index details view, or when you select the checkbox of one or more
|
|
|
|
|
indices on the overview page. The menu includes the following actions:
|
2019-02-13 22:51:36 +01:00
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
* *Close index*. Blocks the index from read/write operations.
|
|
|
|
|
A closed index exists in the cluster, but doesn't consume resources
|
|
|
|
|
other than disk space. If you reopen a closed index, it goes through the
|
|
|
|
|
normal recovery process.
|
2019-02-13 22:51:36 +01:00
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
* *Force merge index*. Reduces the number of segments in your shard by
|
2019-02-13 22:51:36 +01:00
|
|
|
|
merging smaller files and clearing deleted ones. Only force merge a read-only index.
|
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
* *Refresh index*. Writes the operations in the indexing buffer to the
|
|
|
|
|
filesystem cache. This action is automatically performed once per second. Forcing a manual
|
|
|
|
|
refresh is useful during testing, but should not be routinely done in
|
2019-02-13 22:51:36 +01:00
|
|
|
|
production because it has a performance impact.
|
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
* *Clear index cache*. Clears all caches associated with the index.
|
2019-02-13 22:51:36 +01:00
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
* *Flush index*. Frees memory by syncing the filesystem cache to disk and
|
2019-02-13 22:51:36 +01:00
|
|
|
|
clearing the cache. Once the sync is complete, the internal transaction log is reset.
|
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
* *Freeze index*. Makes the index read-only and reduces its memory footprint
|
|
|
|
|
by moving shards to disk. Frozen indices remain
|
2019-02-13 22:51:36 +01:00
|
|
|
|
searchable, but queries take longer.
|
2018-05-24 20:59:34 +02:00
|
|
|
|
|
2019-09-16 17:45:58 +02:00
|
|
|
|
* *Delete index*. Permanently removes the index and all of its documents.
|
2018-05-24 20:59:34 +02:00
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
* *Add lifecycle policy*. Specifies a policy for managing the lifecycle of the
|
2019-02-13 22:51:36 +01:00
|
|
|
|
index.
|
2018-05-24 20:59:34 +02:00
|
|
|
|
|
2019-09-16 17:45:58 +02:00
|
|
|
|
[float]
|
|
|
|
|
[[manage-index-templates]]
|
|
|
|
|
=== Manage index templates
|
|
|
|
|
|
|
|
|
|
An index template defines {ref}/index-modules.html#index-modules-settings[settings],
|
2020-01-22 22:45:39 +01:00
|
|
|
|
{ref}/mapping.html[mappings], and {ref}/indices-add-alias.html[aliases]
|
|
|
|
|
that you can automatically apply when creating a new index. {es} applies a
|
2019-09-16 17:45:58 +02:00
|
|
|
|
template to a new index based on an index pattern that matches the index name.
|
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
The *Index Templates* view lists your templates and enables you to examine, edit, clone, and
|
|
|
|
|
delete them. Changes you make to an index template
|
2019-09-16 17:45:58 +02:00
|
|
|
|
do not affect existing indices.
|
|
|
|
|
|
|
|
|
|
[role="screenshot"]
|
|
|
|
|
image::images/management-index-templates.png[Index templates]
|
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
If you don't have any templates, you can create one using the *Create template* wizard.
|
|
|
|
|
Index templates are applied during index creation,
|
|
|
|
|
so you must create the
|
2019-09-16 17:45:58 +02:00
|
|
|
|
template before you create the indices.
|
|
|
|
|
|
|
|
|
|
[float]
|
|
|
|
|
==== Example: Create an index template
|
|
|
|
|
|
|
|
|
|
In this example, you’ll create an index template for randomly generated log files.
|
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
Open the *Create template* wizard, and enter `logs_template` in the *Name*
|
2019-09-16 17:45:58 +02:00
|
|
|
|
field. Set *Index pattern* to `logstash*` so the template matches any index
|
2020-01-22 22:45:39 +01:00
|
|
|
|
with that index pattern. The merge order and version are both optional,
|
2019-09-16 17:45:58 +02:00
|
|
|
|
and you'll leave them blank in this example.
|
2020-01-22 22:45:39 +01:00
|
|
|
|
|
2019-09-16 17:45:58 +02:00
|
|
|
|
[role="screenshot"]
|
|
|
|
|
image::images/management_index_create_wizard.png[Create wizard]
|
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
The second step in the *Create template* wizard allows you to define index settings.
|
|
|
|
|
These settings are optional, and this example skips this step.
|
2019-09-16 17:45:58 +02:00
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
The logs data set requires a
|
|
|
|
|
mapping to label the latitude and longitude pairs as geographic locations
|
|
|
|
|
by applying the geo_point type. In the third step of the wizard, define this mapping
|
|
|
|
|
under the *Mapped fields* tab as follows:
|
2019-09-16 17:45:58 +02:00
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
[role="screenshot"]
|
|
|
|
|
image::images/management-index-templates-mappings.png[Mapped fields page]
|
|
|
|
|
|
2020-04-06 11:05:29 +02:00
|
|
|
|
Alternatively, you can click the *Load JSON* link and define the mapping as JSON:
|
|
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
|
----------------------------------
|
|
|
|
|
{
|
|
|
|
|
"properties": {
|
|
|
|
|
"geo": {
|
|
|
|
|
"properties": {
|
|
|
|
|
"coordinates": {
|
|
|
|
|
"type": "geo_point"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
----------------------------------
|
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
You can create additional mapping configurations in the *Dynamic templates* and
|
2020-02-06 18:16:32 +01:00
|
|
|
|
*Advanced options* tabs. No additional mappings are required for this example.
|
2019-09-16 17:45:58 +02:00
|
|
|
|
|
|
|
|
|
In the fourth step, define an alias named `logstash`.
|
2020-01-22 22:45:39 +01:00
|
|
|
|
|
2019-09-16 17:45:58 +02:00
|
|
|
|
[source,js]
|
|
|
|
|
----------------------------------
|
|
|
|
|
{
|
|
|
|
|
"logstash": {}
|
|
|
|
|
}
|
|
|
|
|
----------------------------------
|
2020-01-22 22:45:39 +01:00
|
|
|
|
|
|
|
|
|
A summary of the template is in step 5. If everything looks right, click *Create template*.
|
2019-09-16 17:45:58 +02:00
|
|
|
|
|
|
|
|
|
At this point, you’re ready to use the {es} index API to load the logs data.
|
|
|
|
|
In the {kib} *Console*, index two documents:
|
|
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
|
----------------------------------
|
|
|
|
|
POST /logstash-2019.05.18/_doc
|
|
|
|
|
{
|
|
|
|
|
"@timestamp": "2019-05-18T15:57:27.541Z",
|
|
|
|
|
"ip": "225.44.217.191",
|
|
|
|
|
"extension": "jpg",
|
|
|
|
|
"response": "200",
|
|
|
|
|
"geo": {
|
|
|
|
|
"coordinates": {
|
|
|
|
|
"lat": 38.53146222,
|
|
|
|
|
"lon": -121.7864906
|
|
|
|
|
}
|
|
|
|
|
},
|
|
|
|
|
"url": "https://media-for-the-masses.theacademyofperformingartsandscience.org/uploads/charles-fullerton.jpg"
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
POST /logstash-2019.05.20/_doc
|
|
|
|
|
{
|
|
|
|
|
"@timestamp": "2019-05-20T03:44:20.844Z",
|
|
|
|
|
"ip": "198.247.165.49",
|
|
|
|
|
"extension": "php",
|
|
|
|
|
"response": "200",
|
|
|
|
|
"geo": {
|
|
|
|
|
"coordinates": {
|
|
|
|
|
"lat": 37.13189556,
|
|
|
|
|
"lon": -76.4929875
|
|
|
|
|
}
|
|
|
|
|
},
|
|
|
|
|
"memory": 241720,
|
|
|
|
|
"url": "https://theacademyofperformingartsandscience.org/people/type:astronauts/name:laurel-b-clark/profile"
|
|
|
|
|
}
|
|
|
|
|
----------------------------------
|
|
|
|
|
|
2020-01-22 22:45:39 +01:00
|
|
|
|
The mappings and alias are configured automatically based on the template. To verify, you
|
2019-09-16 17:45:58 +02:00
|
|
|
|
can view one of the newly created indices using the {ref}/indices-get-index.html#indices-get-index[index API].
|