kibana/x-pack/plugins/monitoring
Tim Sullivan 274617de59
Stats API: implement the "kibana status" spec from the Monitoring data model for stats (#20577)
* [Stats API] Set API field names per spec

* fix jest tests

* fix api integration test

* trash the original metrics collector

- constantly accumulating stats over time does not align with the existing behavior, which is to reset the stats to 0 whenever they are pulled

* move some logic out of the collector types combiner into inline

- change the signature of sourceKibana

* Make a new stats collector for the API

- to not clear the data when pulling via the api
- fetching is a read-only thing

* isolate data transforms for api data and upload data

* no static methods

* remove external in bytes

* remove the _stats prefix for kibana and reporting

* update jest test snapshot

* fix collector_types_combiner test

* fix usage api

* add test suite todo comment

* reduce some loc change

* roll back mysterious change

* reduce some more loc change

* comment correction

* reduce more loc change

* whitespace

* comment question

* fix cluster_uuid

* fix stats integration test

* fix bulk uploader test, combineTypes is no longer external

* very important comments about the current nature of stats represented and long-term goals

* add stats api tests with/without authentication

* fix more fields to match data model

* fix more tests

* fix jest test

* remove TODO

* remove sockets

* use snake_case for api field names

* restore accidental removal + copy/paste error

* sourceKibana -> getKibanaInfoForStats

* skip usage test on legacy endpoint

* fix api tests

* more comment

* stop putting a field in that used to be omitted

* fix the internal type to ID the usage data for bulk uploader

* correct the kibana usage type value, which is shown as-is in the API

* more fixes for the constants identifying collector types + test against duplicates

* add a comment on a hack, and a whitespace fix
2018-07-18 16:37:04 -07:00
..
__tests__ Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
common Stats API: implement the "kibana status" spec from the Monitoring data model for stats (#20577) 2018-07-18 16:37:04 -07:00
public Bump eui (#20774) 2018-07-17 19:28:07 -04:00
server Stats API: implement the "kibana status" spec from the Monitoring data model for stats (#20577) 2018-07-18 16:37:04 -07:00
.agignore Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
config.js Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
deprecations.js Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
index.js Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
init.js Stats API: implement the "kibana status" spec from the Monitoring data model for stats (#20577) 2018-07-18 16:37:04 -07:00
README.md Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
ui_exports.js Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00

Using Monitoring

The easiest way to get to know the new Monitoring is probably by reading the docs.

Install the distribution the way a customer would is pending the first release of Unified X-Pack plugins.

Developing

You will need to get Elasticsearch and X-Pack plugins for ES that match the version of the UI. The best way to do this is to run gradle run from a clone of the x-plugins repository.

To set up Monitoring and automatic file syncing code changes into Kibana's plugin directory, clone the kibana and x-plugins repos in the same directory and from x-plugins/kibana/monitoring, run yarn start.

Once the syncing process has run at least once, start the Kibana server in development mode. It will handle restarting the server and re-optimizing the bundles as-needed. Go to https://localhost:5601 and click Monitoring from the App Drawer.

Running tests

  • Run the command:

    yarn test
    
  • Debug tests Add a debugger line to create a breakpoint, and then:

    gulp sync && mocha debug --compilers js:babel-register /pathto/kibana/plugins/monitoring/pathto/__tests__/testfile.js
    

Deploying

Monitoring is part of XPack, and only a single XPack artifact needs to be deployed. Previously, the instructions to deploy were:

The release task creates archives and uploads them to download.elasticsearch.org/elasticsearch/monitoring/VERSION. You will need S3 credentials in $HOME/.aws-config.json. Format as so:

{
  "key":"MY_KEY_HERE",
  "secret":"your/long/secret/string"
}

To upload the current archive as the "latest" release, use:

gulp release

Multicluster Setup for Development

To run the UI with multiple clusters, the easiest way is to run 2 nodes out of the same Elasticsearch directory, but use different start up commands for each one. One node will be assigned to the "monitoring" cluster and the other will be for the "production" cluster.

  1. Add the Security users:
% ./bin/x-pack/users useradd -r remote_monitoring_agent -p notsecure remote
% ./bin/x-pack/users useradd -r monitoring_user -p notsecure monitoring_user
  1. Start up the Monitoring cluster:
% ./bin/elasticsearch \
-Ehttp.port=9210 \
-Ecluster.name=monitoring \
-Epath.data=monitoring-data \
-Enode.name=monitor1node1
  1. Start up the Production cluster:
% ./bin/elasticsearch \
-Expack.monitoring.exporters.id2.type=http \
-Expack.monitoring.exporters.id2.host=http://127.0.0.1:9210 \
-Expack.monitoring.exporters.id2.auth.username=remote \
-Expack.monitoring.exporters.id2.auth.password=notsecure \
-Ecluster.name=production \
-Enode.name=prod1node1 \
-Epath.data=production-data
  1. Set the Kibana config:
% cat config/kibana.dev.yml
xpack.monitoring.elasticsearch:
  url: "http://localhost:9210"
  username: "kibana"
  password: "changeme"
  1. Start another Kibana instance:
% yarn start
  1. Start a Kibana instance connected to the Monitoring cluster (for running queries in Sense on Monitoring data):
% ./bin/kibana --config config/kibana.dev.yml --elasticsearch.url http://localhost:9210 --server.name monitoring-kibana --server.port 5611