kibana/x-pack/plugins/monitoring
Tiago Costa 20bdf7d4a9
Migration to Babel7 and @babel/preset-typescript (#33093)
* chore(NA): first changes on every package.json order to support new babel 7. chore(NA): build for kbn-pm with babel 7.

* chore(NA): patch babel register to load typescrit

* chore(NA): first working version with babel 7 replacing typescript compiler.

* fix(NA): common preset declaration in order to make it work with babel-loader.

* chore(na): organizing babel preset env package json.

* chore(NA): mocha tests enabled.

* fix(NA): typo on importing

* test(NA): majority of x-pack tests ported to use babel-jest

* fix(NA): report info button test with babel-jest.

* fix(NA): polling service tests.

* test(na): fix server plugins plugin tests.

* test(NA): batch of test fixs for jest tests under babel-jest hoisting.

* chore(NA): add babel plugin to hoist mock prefixed vars on jest tests.

* chore(NA): update yarn.lock file.

* chore(NA): tests passing.

* chore(NA): remove wrong dep

* chore(NA): fix tsconfig

* chore(NA): skip babel for ts-jest.

* chore(NA): selectively apply the plugin to strip off namespace from ts files.

* chore(NA): remove not needed changes from ts tests

* chore(NA): removed ts-jest dependency. chore(NA): migrate ts tests on x-pack to use babel-jest with the new pattern.

* chore(NA): migrate kibana default distribution typescript tests to run with babel-jest and the new test mock pattern.

* chore(NA): merge and solve conflicts with master.

* chore(NA): fix problems reported by eslint

* chore(NA): fix license ovveride for babel-plugin-mock-imports

* chore(NA): update jest integration tests for kbn pm

* chore(NA): update babel jest integration tests for kbn pm.

* test(NA): update jest integration snapshot for kbn pm.

* chore(NA): apply changes according to the pull request reviews.

* chore(NA): apply changes according to the pull request reviews.

* refact(NA): migrate jest tests to the new pattern.

* fix(NA): babel 7 polyfill in the tests bundle.

* chore(NA): restore needed step in order to compile x-pack with typescript.

* chore(NA): change build to compile typescript with babel for the oss code. chore(NA): change transpile typescript task to only transpile types for x-pack. refact(NA): common preset for babel 7

* Revert "chore(NA): change build to compile typescript with babel for the oss code. chore(NA): change transpile typescript task to only transpile types for x-pack. refact(NA): common preset for babel 7"

This reverts commit 2707d538f5.

* fix(NA): import paths for tabConfigConst

* chore(NA): fix transpiling error on browser tests

* chore(NA): simplify kbn babel preset package.

* chore(NA): migrate build to use babel transpiler for typescript excluding xpack.

* fix(NA): introduced error on test quick task.

* fix(NA): fix preset for client side code on build.

* fix(NA): build with babel

* fix(NA): negated patterns in the end.

* fix(NA): kbn_tp_sample_panel_action creation.

* fix(NA): babel typescript transform plugin workaround when exporting interface name.

* refact(NA): remove not needed type cast to any on jest test.

* docs(NA): add developement documentation about jest mocks test pattern.

* chore(NA): missing unmerged path.

* chore(NA): fix jest tests for template.

* [CCR] Client integration tests (table lists) (#33525)

* Force user to re-authenticate if token refresh fails with `400` status code. (#33774)

* Improve performance of the Logstash Pipeline Viewer (#33793)

Resolves #27513.

_This PR is a combination of #31293 (the code changes) + #33570 (test updates). These two PRs were individually reviewed and merged into a feature branch. This combo PR here simply sets up the merge from the feature branch to `master`._

Summary of changes, taken from #31293:

## Before this PR
The Logstash Pipeline Viewer UI would make a single Kibana API call to fetch all the information necessary to render the Logstash pipeline. This included information necessary to render the detail drawer that opens up when a user clicks on an individual vertex in the pipeline.

Naturally, this single API call fetched _a lot_ of data, not just from the Kibana server but also, in turn, from Elasticsearch as well. The "pro" of this approach was that the user would see instantaneous results if they clicked on a vertex in a pipeline and opened the detail drawer for that vertex. The "cons" were the amount of computation Elasticsearch had to perform and the amount of data being transferred over the wire between Elasticsearch and the Kibana server as well as between the Kibana server and the browser.

## With this PR
This PR makes the Kibana API call to fetch data necessary for **initially** rendering the pipeline — that is, with the detail drawer closed — much lighter. When the user clicks on a vertex in a pipeline, a second API call is then made to fetch data necessary for the detail drawer.

## Gains, by the numbers

Based on a simple, 1-input, 1-filter, and 1-output pipeline.

* Before this PR, the Elasticsearch `logstash_stats` API responses (multiple calls were made using the `composite` aggregation over the `date_histogram` aggregation) generated a total of 1228 aggregation buckets (before any `filter_path`s were applied but across all `composite` "pages"). With this PR, the single `logstash_stats` API response (note that this is just for the initial rendering of the pipeline, with the detail drawer closed) generated 12 buckets (also before any `filter_path`s were applied). That's a **99.02% reduction** in number of buckets.

* Before this PR, the Elasticsearch `logstash_stats` API responses added up to 70319 bytes. With this PR, the single `logstash_stats` API response for the same pipeline is 746 bytes. That's a **98.93% reduction** in size.

* Before this PR, the Elasticsearch `logstash_state` API response was 7718 bytes. With this PR, the API response for the same pipeline is 2328 bytes. That's a **69.83% reduction** in size.

* Before this PR the Kibana API response was 51777 bytes. With this PR, the API response for the same pipeline is 2567 bytes (again, note that this is just for the initial rendering of the pipeline, with the detail drawer closed). That's a **95.04% reduction** in size.

* [Maps] split settings into layer and source panels (#33788)

* [Maps] split settings into layer and source panels

* fix SCSS import

* [env] exit if starting as root (#21563)

* [env] exit if starting as root

* fix windows

* s/--allow-root

* Typescript sample panel action (#33602)

* Typescript sample panel action

* Update EUI version to match main cabana version

* update yarn.lock

* add back typings include

* use correct relative path

* Home page "recent links" should communicate saved object type #21896 (#33694)

* adds object type for screen order
* adds object type for pointer hovering
* Update src/legacy/ui/public/chrome/directives/header_global_nav/components/header.tsx

Co-Authored-By: rockfield <philipp.b@ya.ru>
2019-03-26 20:44:03 +00:00
..
__tests__ [@kbn/expect] "fork" expect.js into repo (#33761) 2019-03-25 09:56:48 -07:00
common [@kbn/expect] "fork" expect.js into repo (#33761) 2019-03-25 09:56:48 -07:00
public Migration to Babel7 and @babel/preset-typescript (#33093) 2019-03-26 20:44:03 +00:00
server Improve performance of the Logstash Pipeline Viewer (#33793) 2019-03-26 06:27:20 -07:00
.agignore Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
.kibana-plugin-helpers.json Convert all Less files to Sass in monitoring, use EUI variable scope (#20995) 2018-08-13 08:18:08 -07:00
config.js Removing deprecated xpack.monitoring.report_stats setting (#30017) 2019-02-04 19:37:32 -08:00
deprecations.js Removing deprecated xpack.monitoring.report_stats setting (#30017) 2019-02-04 19:37:32 -08:00
index.js [I18n] Translate monitoring - uiExports (#27195) 2018-12-17 22:50:15 +03:00
init.js Introduce Elasticsearch service. (#28344) 2019-02-28 17:22:07 +02:00
README.md Migration to Babel7 and @babel/preset-typescript (#33093) 2019-03-26 20:44:03 +00:00
ui_exports.js Update the title in the left nav (#28102) 2019-01-07 12:28:07 -05:00

Using Monitoring

The easiest way to get to know the new Monitoring is probably by reading the docs.

Install the distribution the way a customer would is pending the first release of Unified X-Pack plugins.

Developing

You will need to get Elasticsearch and X-Pack plugins for ES that match the version of the UI. The best way to do this is to run gradle run from a clone of the x-plugins repository.

To set up Monitoring and automatic file syncing code changes into Kibana's plugin directory, clone the kibana and x-plugins repos in the same directory and from x-plugins/kibana/monitoring, run yarn start.

Once the syncing process has run at least once, start the Kibana server in development mode. It will handle restarting the server and re-optimizing the bundles as-needed. Go to https://localhost:5601 and click Monitoring from the App Drawer.

Running tests

  • Run the command:

    yarn test
    
  • Debug tests Add a debugger line to create a breakpoint, and then:

    gulp sync && mocha debug --compilers js:@babel/register /pathto/kibana/plugins/monitoring/pathto/__tests__/testfile.js
    

Multicluster Setup for Development

To run the UI with multiple clusters, the easiest way is to run 2 nodes out of the same Elasticsearch directory, but use different start up commands for each one. One node will be assigned to the "monitoring" cluster and the other will be for the "production" cluster.

  1. Add the Security users:
% ./bin/x-pack/users useradd -r remote_monitoring_agent -p notsecure remote
% ./bin/x-pack/users useradd -r monitoring_user -p notsecure monitoring_user
  1. Start up the Monitoring cluster:
% ./bin/elasticsearch \
-Ehttp.port=9210 \
-Ecluster.name=monitoring \
-Epath.data=monitoring-data \
-Enode.name=monitor1node1
  1. Start up the Production cluster:
% ./bin/elasticsearch \
-Expack.monitoring.exporters.id2.type=http \
-Expack.monitoring.exporters.id2.host=http://127.0.0.1:9210 \
-Expack.monitoring.exporters.id2.auth.username=remote \
-Expack.monitoring.exporters.id2.auth.password=notsecure \
-Ecluster.name=production \
-Enode.name=prod1node1 \
-Epath.data=production-data
  1. Set the Kibana config:
% cat config/kibana.dev.yml
xpack.monitoring.elasticsearch:
  url: "http://localhost:9210"
  username: "kibana"
  password: "changeme"
  1. Start another Kibana instance:
% yarn start
  1. Start a Kibana instance connected to the Monitoring cluster (for running queries in Sense on Monitoring data):
% ./bin/kibana --config config/kibana.dev.yml --elasticsearch.hosts http://localhost:9210 --server.name monitoring-kibana --server.port 5611