213 lines
7.1 KiB
Text
213 lines
7.1 KiB
Text
[role="xpack"]
|
|
[[monitoring-metricbeat]]
|
|
=== Collecting {kib} monitoring data with {metricbeat}
|
|
[subs="attributes"]
|
|
++++
|
|
<titleabbrev>Collecting monitoring data with {metricbeat}</titleabbrev>
|
|
++++
|
|
|
|
In 6.4 and later, you can use {metricbeat} to collect data about {kib}
|
|
and ship it to the monitoring cluster, rather than routing it through the
|
|
production cluster as described in <<monitoring-kibana>>.
|
|
|
|
image::monitoring/images/metricbeat.png[Example monitoring architecture]
|
|
|
|
To learn about monitoring in general, see
|
|
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
|
|
|
|
//NOTE: The tagged regions are re-used in the Stack Overview.
|
|
|
|
. Disable the default collection of {kib} monitoring metrics. +
|
|
+
|
|
--
|
|
// tag::disable-kibana-collection[]
|
|
Add the following setting in the {kib} configuration file (`kibana.yml`):
|
|
|
|
[source,yaml]
|
|
----------------------------------
|
|
xpack.monitoring.kibana.collection.enabled: false
|
|
----------------------------------
|
|
|
|
Leave the `xpack.monitoring.enabled` set to its default value (`true`).
|
|
// end::disable-kibana-collection[]
|
|
For more information, see
|
|
<<monitoring-settings-kb,Monitoring settings in {kib}>>.
|
|
--
|
|
|
|
. <<start-stop,Start {kib}>>.
|
|
|
|
. Set the `xpack.monitoring.collection.enabled` setting to `true` on
|
|
each node in the production cluster. By default, it is disabled (`false`).
|
|
+
|
|
--
|
|
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
|
|
node or across the cluster as a dynamic cluster setting. If {es}
|
|
{security-features} are enabled, you must have `monitor` cluster privileges to
|
|
view the cluster settings and `manage` cluster privileges to change them.
|
|
|
|
--
|
|
|
|
** In {kib}:
|
|
|
|
... Open {kib} in your web browser.
|
|
+
|
|
--
|
|
If you are running {kib} locally, go to `http://localhost:5601/`.
|
|
|
|
If the Elastic {security-features} are enabled, log in.
|
|
--
|
|
|
|
... In the side navigation, click *Stack Monitoring*. If data collection is
|
|
disabled, you are prompted to turn it on.
|
|
|
|
** From the Console or command line, set `xpack.monitoring.collection.enabled`
|
|
to `true` on the production cluster. +
|
|
+
|
|
--
|
|
For example, you can use the following APIs to review and change this setting:
|
|
|
|
[source,js]
|
|
----------------------------------
|
|
GET _cluster/settings
|
|
|
|
PUT _cluster/settings
|
|
{
|
|
"persistent": {
|
|
"xpack.monitoring.collection.enabled": true
|
|
}
|
|
}
|
|
----------------------------------
|
|
|
|
For more information, see {ref}/monitoring-settings.html[Monitoring settings in {es}]
|
|
and {ref}/cluster-update-settings.html[Cluster update settings].
|
|
--
|
|
|
|
. {metricbeat-ref}/metricbeat-installation.html[Install {metricbeat}] on the
|
|
same server as {kib}.
|
|
|
|
. Enable the {kib} {xpack} module in {metricbeat}. +
|
|
+
|
|
--
|
|
// tag::enable-kibana-module[]
|
|
For example, to enable the default configuration in the `modules.d` directory,
|
|
run the following command:
|
|
|
|
["source","sh",subs="attributes,callouts"]
|
|
----------------------------------------------------------------------
|
|
metricbeat modules enable kibana-xpack
|
|
----------------------------------------------------------------------
|
|
|
|
For more information, see
|
|
{metricbeat-ref}/configuration-metricbeat.html[Specify which modules to run] and
|
|
{metricbeat-ref}/metricbeat-module-kibana.html[{kib} module].
|
|
// end::enable-kibana-module[]
|
|
--
|
|
|
|
. Configure the {kib} {xpack} module in {metricbeat}. +
|
|
+
|
|
--
|
|
// tag::configure-kibana-module[]
|
|
The `modules.d/kibana-xpack.yml` file contains the following settings:
|
|
|
|
[source,yaml]
|
|
----------------------------------
|
|
- module: kibana
|
|
metricsets:
|
|
- stats
|
|
period: 10s
|
|
hosts: ["localhost:5601"]
|
|
#basepath: ""
|
|
#username: "user"
|
|
#password: "secret"
|
|
xpack.enabled: true
|
|
----------------------------------
|
|
|
|
By default, the module collects {kib} monitoring metrics from `localhost:5601`.
|
|
If that host and port number are not correct, you must update the `hosts`
|
|
setting. If you configured {kib} to use encrypted communications, you must
|
|
access it via HTTPS. For example, use a `hosts` setting like
|
|
`https://localhost:5601`.
|
|
// end::configure-kibana-module[]
|
|
|
|
// tag::remote-monitoring-user[]
|
|
If the Elastic {security-features} are enabled, you must also provide a user
|
|
ID and password so that {metricbeat} can collect metrics successfully:
|
|
|
|
.. Create a user on the production cluster that has the
|
|
`remote_monitoring_collector` {stack-ov}/built-in-roles.html[built-in role].
|
|
Alternatively, use the `remote_monitoring_user`
|
|
{stack-ov}/built-in-users.html[built-in user].
|
|
|
|
.. Add the `username` and `password` settings to the {kib} module configuration
|
|
file.
|
|
// end::remote-monitoring-user[]
|
|
--
|
|
|
|
. Optional: Disable the system module in {metricbeat}.
|
|
+
|
|
--
|
|
// tag::disable-system-module[]
|
|
By default, the {metricbeat-ref}/metricbeat-module-system.html[system module] is
|
|
enabled. The information it collects, however, is not shown on the *Monitoring*
|
|
page in {kib}. Unless you want to use that information for other purposes, run
|
|
the following command:
|
|
|
|
["source","sh",subs="attributes,callouts"]
|
|
----------------------------------------------------------------------
|
|
metricbeat modules disable system
|
|
----------------------------------------------------------------------
|
|
// end::disable-system-module[]
|
|
--
|
|
|
|
. Identify where to send the monitoring data. +
|
|
+
|
|
--
|
|
TIP: In production environments, we strongly recommend using a separate cluster
|
|
(referred to as the _monitoring cluster_) to store the data. Using a separate
|
|
monitoring cluster prevents production cluster outages from impacting your
|
|
ability to access your monitoring data. It also prevents monitoring activities
|
|
from impacting the performance of your production cluster.
|
|
|
|
For example, specify the {es} output information in the {metricbeat}
|
|
configuration file (`metricbeat.yml`):
|
|
|
|
[source,yaml]
|
|
----------------------------------
|
|
output.elasticsearch:
|
|
# Array of hosts to connect to.
|
|
hosts: ["http://es-mon-1:9200", "http://es-mon2:9200"] <1>
|
|
|
|
# Optional protocol and basic auth credentials.
|
|
#protocol: "https"
|
|
#username: "elastic"
|
|
#password: "changeme"
|
|
----------------------------------
|
|
<1> In this example, the data is stored on a monitoring cluster with nodes
|
|
`es-mon-1` and `es-mon-2`.
|
|
|
|
If you configured the monitoring cluster to use encrypted communications, you
|
|
must access it via HTTPS. For example, use a `hosts` setting like
|
|
`https://es-mon-1:9200`.
|
|
|
|
IMPORTANT: The {es} {monitor-features} use ingest pipelines, therefore the
|
|
cluster that stores the monitoring data must have at least one ingest node.
|
|
|
|
If the {es} {security-features} are enabled on the monitoring cluster, you
|
|
must provide a valid user ID and password so that {metricbeat} can send metrics
|
|
successfully:
|
|
|
|
.. Create a user on the monitoring cluster that has the
|
|
`remote_monitoring_agent` {stack-ov}/built-in-roles.html[built-in role].
|
|
Alternatively, use the `remote_monitoring_user`
|
|
{stack-ov}/built-in-users.html[built-in user].
|
|
|
|
.. Add the `username` and `password` settings to the {es} output information in
|
|
the {metricbeat} configuration file.
|
|
|
|
For more information about these configuration options, see
|
|
{metricbeat-ref}/elasticsearch-output.html[Configure the {es} output].
|
|
--
|
|
|
|
. {metricbeat-ref}/metricbeat-starting.html[Start {metricbeat}].
|
|
|
|
. <<monitoring-data,View the monitoring data in {kib}>>.
|