kibana/x-pack/plugins/monitoring
Tim Sullivan d0595fdc4b
Separate bulk upload behavior from CollectorSet (#19691)
* Separate bulk upload behavior from CollectorSet

 - takes out a lot of behavior from CollectorSet and moves it to a class called BulkUploader
 - simplifies kibana monitoring init by taking out the indirection that startCollectorSet / createCollectorSet had
 - removes start() method from CollectorSet and calls the collector objects' init() functions from CollectorSet.register()
 - removes cleanup method from collectorSet since that was doing work for bulk uploading

* remove cleanup and fetchAfterInit methods

* test for bulk_uploader class

* improve test for collector_set

* fix reporting

* expose collectorSet if there actually is a collectorSet

* comment for enclosed function

* make collectorSet creation/expose unconditional, bulkUploader more conditional

* fix collector_set tests

* lifecycle events

* stab at collectorSet error logging from the API call

* clean up comments

* clean up comments

* fix BulkUploader mocha test

* check kibanaCollectionEnabled config before registering bulk upload and the plugin status listeners

* no singleton timer object

* just log a warning if bulk uploader start called twice

* normal quotes

* check if bulk is enabled inside of the _fetchAndUpload method

* log for stopping bulk stats

* call bulkUploader.start with the collectorSet object

* call bulkUploader.start with the collectorSet object

* roll back change for module scoped variable

* oops I broke init

* init and logging: if / elseif / elseif

* remove unnecessary check/log

* help log

* remove redundant, use data.filter.map

* use xpackInfo.onLicenseInfoChange not xpackMainPlugin.status.on('green')

* help logging

* fix unit test

* remove handler that stops upload when connection is lost
2018-06-21 13:08:32 -07:00
..
__tests__ Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
common [Monitoring/Reporting Stats] Move hardcoded Reporting stats type collection out of monitoring plugin (#18894) 2018-05-24 11:14:27 -07:00
public Fix capitalization typo for className attribute. (#20104) 2018-06-21 10:04:38 -04:00
server Separate bulk upload behavior from CollectorSet (#19691) 2018-06-21 13:08:32 -07:00
.agignore Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
config.js Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
deprecations.js Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
index.js Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
init.js Separate bulk upload behavior from CollectorSet (#19691) 2018-06-21 13:08:32 -07:00
README.md Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
ui_exports.js Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00

Using Monitoring

The easiest way to get to know the new Monitoring is probably by reading the docs.

Install the distribution the way a customer would is pending the first release of Unified X-Pack plugins.

Developing

You will need to get Elasticsearch and X-Pack plugins for ES that match the version of the UI. The best way to do this is to run gradle run from a clone of the x-plugins repository.

To set up Monitoring and automatic file syncing code changes into Kibana's plugin directory, clone the kibana and x-plugins repos in the same directory and from x-plugins/kibana/monitoring, run yarn start.

Once the syncing process has run at least once, start the Kibana server in development mode. It will handle restarting the server and re-optimizing the bundles as-needed. Go to https://localhost:5601 and click Monitoring from the App Drawer.

Running tests

  • Run the command:

    yarn test
    
  • Debug tests Add a debugger line to create a breakpoint, and then:

    gulp sync && mocha debug --compilers js:babel-register /pathto/kibana/plugins/monitoring/pathto/__tests__/testfile.js
    

Deploying

Monitoring is part of XPack, and only a single XPack artifact needs to be deployed. Previously, the instructions to deploy were:

The release task creates archives and uploads them to download.elasticsearch.org/elasticsearch/monitoring/VERSION. You will need S3 credentials in $HOME/.aws-config.json. Format as so:

{
  "key":"MY_KEY_HERE",
  "secret":"your/long/secret/string"
}

To upload the current archive as the "latest" release, use:

gulp release

Multicluster Setup for Development

To run the UI with multiple clusters, the easiest way is to run 2 nodes out of the same Elasticsearch directory, but use different start up commands for each one. One node will be assigned to the "monitoring" cluster and the other will be for the "production" cluster.

  1. Add the Security users:
% ./bin/x-pack/users useradd -r remote_monitoring_agent -p notsecure remote
% ./bin/x-pack/users useradd -r monitoring_user -p notsecure monitoring_user
  1. Start up the Monitoring cluster:
% ./bin/elasticsearch \
-Ehttp.port=9210 \
-Ecluster.name=monitoring \
-Epath.data=monitoring-data \
-Enode.name=monitor1node1
  1. Start up the Production cluster:
% ./bin/elasticsearch \
-Expack.monitoring.exporters.id2.type=http \
-Expack.monitoring.exporters.id2.host=http://127.0.0.1:9210 \
-Expack.monitoring.exporters.id2.auth.username=remote \
-Expack.monitoring.exporters.id2.auth.password=notsecure \
-Ecluster.name=production \
-Enode.name=prod1node1 \
-Epath.data=production-data
  1. Set the Kibana config:
% cat config/kibana.dev.yml
xpack.monitoring.elasticsearch:
  url: "http://localhost:9210"
  username: "kibana"
  password: "changeme"
  1. Start another Kibana instance:
% yarn start
  1. Start a Kibana instance connected to the Monitoring cluster (for running queries in Sense on Monitoring data):
% ./bin/kibana --config config/kibana.dev.yml --elasticsearch.url http://localhost:9210 --server.name monitoring-kibana --server.port 5611