Commit graph

30 commits

Author SHA1 Message Date
Jiří Programagor
e2b304a344 [Monitoring] Properly Deduplicate Replica shards (#23183)
Issue #23179 was caused by ignoring multiple replicas of the same shard, even if they are on different nodes. This commit fixes that.
2018-09-14 12:40:11 -04:00
Larry Gregory
d021f71f6f
Only log deprecation warning if xpack:defaultAdminEmail is actually set (#22774)
This PR fixes the logic for logging the deprecation warning introduced with https://github.com/elastic/kibana/pull/22765. Previously Kibana would log the warning if the new `xpack.monitoring.cluster_alerts.email_notifications.email_address` setting was not defined, regardless of `xpack:defaultAdminEmail`'s setting.

Now, we will only log the deprecation warning if all of the following are true:
1) `xpack.monitoring.cluster_alerts.email_notifications.email_address` is not set.
2) `xpack:defaultAdminEmail` is set. (**<-- this is the new part**)
3) We haven't already logged the deprecation warning
2018-09-07 10:06:35 -04:00
Larry Gregory
5f96c903f3
Deprecate xpack:defaultAdminEmail for monitoring alerts (#22195) 2018-09-06 10:26:05 -04:00
Tim Sullivan
10e88ab6b3
[Monitoring] Fix logging when Monitoring reinitializes with HUP signal (#22464) 2018-08-29 11:33:27 -07:00
Chris Roberson
db089ea5ce
[Monitoring] Ensure the settings api always return the xpack/default_admin_email (#22220)
* If the settings collector returns nothing, ensure the settings api still returns a null value for default_admin_email

* Update test

* Feedback from PR
2018-08-23 09:04:16 -04:00
Tim Sullivan
8718d1ee4d
Retrofit the Bulk Uploader types combiner [ch2198] (#22030)
* Retrofit the Bulk Uploader types combiner [ch2198]

fix usage collector, add comments to formatForBulk

remove unnecessary customizations

* override default format for bulk upload for usage type collectors

* rename to ignoreForInternalUploader

* collectors -> collectorSet

* use constant for kibana_stats type

* example of data formatting for bulk in function comment
2018-08-21 12:29:04 -07:00
Tim Sullivan
108d59cb91
Allow kibana_settings collector to return nothing (#22091)
* Fix kibana_settings collector to return nothing when no settings data is found

* make code more clear
2018-08-17 11:41:02 -07:00
Chris Earle
2f082208f6
[Monitoring] Ignore Duplicate Shards (#21057)
[Monitoring] Ignore Duplicate Shards

This eliminates duplicate shards from the shard table by actively ignoring
them from the response.
2018-07-24 16:04:11 -04:00
Chris Roberson
92dd341f5c
[Monitoring] Use 0 as the default for shard count if the node is not found (#21000)
* Use 0 as the default for shard count if the node is not found

* Remove debug

* Updating snapshot tests

* Update api integration test
2018-07-24 12:56:10 -04:00
Chris Roberson
ade24f2104
[Monitoring] Remove kibana_stats.requests.status_codes from bulk uploader (#20855)
* Remove `status_codes` from bulk uploader

* Remove more references to `status_codes`
2018-07-24 10:15:26 -04:00
Tim Sullivan
274617de59
Stats API: implement the "kibana status" spec from the Monitoring data model for stats (#20577)
* [Stats API] Set API field names per spec

* fix jest tests

* fix api integration test

* trash the original metrics collector

- constantly accumulating stats over time does not align with the existing behavior, which is to reset the stats to 0 whenever they are pulled

* move some logic out of the collector types combiner into inline

- change the signature of sourceKibana

* Make a new stats collector for the API

- to not clear the data when pulling via the api
- fetching is a read-only thing

* isolate data transforms for api data and upload data

* no static methods

* remove external in bytes

* remove the _stats prefix for kibana and reporting

* update jest test snapshot

* fix collector_types_combiner test

* fix usage api

* add test suite todo comment

* reduce some loc change

* roll back mysterious change

* reduce some more loc change

* comment correction

* reduce more loc change

* whitespace

* comment question

* fix cluster_uuid

* fix stats integration test

* fix bulk uploader test, combineTypes is no longer external

* very important comments about the current nature of stats represented and long-term goals

* add stats api tests with/without authentication

* fix more fields to match data model

* fix more tests

* fix jest test

* remove TODO

* remove sockets

* use snake_case for api field names

* restore accidental removal + copy/paste error

* sourceKibana -> getKibanaInfoForStats

* skip usage test on legacy endpoint

* fix api tests

* more comment

* stop putting a field in that used to be omitted

* fix the internal type to ID the usage data for bulk uploader

* correct the kibana usage type value, which is shown as-is in the API

* more fixes for the constants identifying collector types + test against duplicates

* add a comment on a hack, and a whitespace fix
2018-07-18 16:37:04 -07:00
Hendrik Muhs
a0a22cfc12
[Monitoring/ML] add number of forecasts to monitoring overview (#20758)
Add forecasts (Number of forecasts for an ml jobs) as column to the ML Jobs monitoring page. Related to: elastic/elasticsearch#31647
2018-07-17 16:41:19 +02:00
Tim Sullivan
c043788c97
[Telemetry] move x-pack/monitoring collector classes to src/server/usage (#20248)
* [Monitoring/Telemetry] Move Usage service from Monitoring to Kibana core

* fix tests

* fix reporting integration

* roll back more diffs

* roll logger into bulk uploader to remove file duplication

* fix xpack usage api

* subclass constructor is not needed

* collectorSet has factory methods for collector object creation

* fix reporting usage jest test
2018-07-09 09:32:02 -07:00
Josh Soref
2b27fb1c5f Fix misspellings (#19981) 2018-06-26 20:17:41 -07:00
Tim Sullivan
d0595fdc4b
Separate bulk upload behavior from CollectorSet (#19691)
* Separate bulk upload behavior from CollectorSet

 - takes out a lot of behavior from CollectorSet and moves it to a class called BulkUploader
 - simplifies kibana monitoring init by taking out the indirection that startCollectorSet / createCollectorSet had
 - removes start() method from CollectorSet and calls the collector objects' init() functions from CollectorSet.register()
 - removes cleanup method from collectorSet since that was doing work for bulk uploading

* remove cleanup and fetchAfterInit methods

* test for bulk_uploader class

* improve test for collector_set

* fix reporting

* expose collectorSet if there actually is a collectorSet

* comment for enclosed function

* make collectorSet creation/expose unconditional, bulkUploader more conditional

* fix collector_set tests

* lifecycle events

* stab at collectorSet error logging from the API call

* clean up comments

* clean up comments

* fix BulkUploader mocha test

* check kibanaCollectionEnabled config before registering bulk upload and the plugin status listeners

* no singleton timer object

* just log a warning if bulk uploader start called twice

* normal quotes

* check if bulk is enabled inside of the _fetchAndUpload method

* log for stopping bulk stats

* call bulkUploader.start with the collectorSet object

* call bulkUploader.start with the collectorSet object

* roll back change for module scoped variable

* oops I broke init

* init and logging: if / elseif / elseif

* remove unnecessary check/log

* help log

* remove redundant, use data.filter.map

* use xpackInfo.onLicenseInfoChange not xpackMainPlugin.status.on('green')

* help logging

* fix unit test

* remove handler that stops upload when connection is lost
2018-06-21 13:08:32 -07:00
Tim Sullivan
cbd319f92c
[Monitoring] Fix Node Advanced page (#19740)
* [Monitoring] Fix Node Advanced page

* fix metrics snapshot
2018-06-07 17:05:43 -07:00
Tim Sullivan
1617da93dd
[X-Pack Usage API] use authentication from request headers (#19613)
* [X-Pack Usage API] use authentication from request headers

* add test for usage api no-auth

* whitespace / syntax nits

* reduce loc changed

* remove a weird looking comment
2018-06-05 10:38:02 -07:00
Justin Kambic
aec1904685
Provide events in & out per millisecond for Logstash pipelines (#19446)
* Port changes to dedicated branch to isolate PR.

* Rename variable in test to match function name.
2018-05-25 15:26:02 -04:00
Tim Sullivan
ce61ae41a3
[Monitoring/Reporting Stats] Move hardcoded Reporting stats type collection out of monitoring plugin (#18894)
* [Monitoring/Telemetry collection] use `typeCollector.registerType` in Reporting

* a few cleanup changes
2018-05-24 11:14:27 -07:00
Tim Sullivan
d649f02c07
Monitoring/directives use react view elasticsearch (#19362)
* [Monitoring/React] Make Elasticsearch directives use React component internally

* fix more functional tests

* remove TODO

* lessen loc change
2018-05-23 16:57:37 -07:00
Tim Sullivan
744fc0b316
[XpackMain] Add _xpack/usage API (#19232)
* [XpackMain] Add _xpack/usage API

* add xpack usage http api integration test

* comment

* misc test describe fixes

* fix integration test

* fix reply called twice

* enable api test

* enable kibana collection for integration test to work

* throw error comment
2018-05-21 16:55:26 -07:00
Tim Sullivan
a7c3255665
Add some collector classes for objects that get registered in a CollectorSet (#19098)
* Add some collector classes for objects that get registered in a CollectorSet

* comment cleanup

* don't pass an inline-defined logger to collectorSet

* add a helper logger function so collector has access to logger at construction
2018-05-18 13:02:54 -07:00
Tim Sullivan
0940f5755d
[Monitoring] Rename TypeCollector to CollectorSet for semantics (#18987)
* [Monitoring] Rename TypeCollector to CollectorSet for semantics

* boring test changes

* usage collector renames

* rename initKibanaMonitoring => createCollectorSet

* fix lint
2018-05-15 13:46:20 -07:00
Tim Sullivan
9e0a6e2a31
[Monitoring] Add a yellow status phase in plugin init (#18939)
* [Monitoring] Add a yellow status phase in the startup lifecycle

* comments

* more comments

* more comment

* undo register => registerType function name change
2018-05-15 09:32:14 -07:00
Tim Sullivan
782d4d9804
[Monitoring/React] Render ES Indices Listing with Base Controller / React (#18595)
* [Monitoring/React] Render ES Indices Listing with Base Controller

Refactors the ES Indices Listing to use Base Controller and a React component instead of an Angular directive

* fix functional tests

* fix another test

* better default for summary status component

* nicer code for whitespace after label

* apiFn => apiUrlFn

* Deleted / Closed
2018-05-09 21:37:24 -07:00
Tim Sullivan
011e8b2c28
[Monitoring] reorganize metrics per stack product (#18590)
* [Monitoring/Metrics] clean up metric classes and getMetricFields static helpers

* more pull out

* more to break out

* [Monitoring] reorganize metrics per stack product

This organizes the metrics classes per product as a starting point for adding more functions to the metrics so external logic in the `get_metrics` and `get_series` modules can be simplified.

New structure:
```
plugins/monitoring/server/lib/metrics
├── __test__
│   ├── __snapshots__
│   │   └── metrics.test.js.snap
│   └── metrics.test.js
├── beats
│   ├── __test__
│   │   └── cpu_utilization_calculation.test.js
│   ├── classes.js
│   └── metrics.js
├── classes
│   ├── __test__
│   │   ├── latency_metric_calculation.test.js
│   │   └── quota_metric_calculation.test.js
│   ├── cluster_metric.js
│   ├── index.js
│   ├── metric.js
│   └── quota_metric.js
├── elasticsearch
│   ├── __test__
│   │   └── latency_calculation.test.js
│   ├── classes.js
│   └── metrics.js
├── index.js
├── kibana
│   ├── classes.js
│   └── metrics.js
├── logstash
│   ├── classes.js
│   └── metrics.js
└── metrics.js
```

* fix strange test failure

* reduce loc change
2018-05-09 18:23:27 -07:00
Tim Sullivan
482953c9b8
[Monitoring/Metrics] clean up metric classes and getMetricFields stat… (#18741)
* [Monitoring/Metrics] clean up metric classes and getMetricFields static helpers

* more pull out

* more to break out

* snapshot update
2018-05-02 17:29:25 -07:00
Tim Sullivan
9e4d4952b6
[Monitoring] add get_clusters_summary to shrink cluster api response (#18596)
This reduces the output from the clusters API in the data processing stage.

Mainly, it throws away a lot of unused fields in `elasticsearch.cluster_stats` and `elasticsearch.cluster_state`

cc @rasroh
2018-04-30 09:11:01 -07:00
Chris Earle
015e94b5fe
[Monitoring] Recognize Write Threadpool (#18147)
This changes the Advanced Node page for Elasticsearch to recognize the
"write" threadpool, while also incorporating the deprecated "index"
threadpool and renamed "bulk" usage (renamed to "write").

This sums the three writing threadpools together and displays it as a
unified view.
2018-04-24 19:26:50 -04:00
Jenkins CI
fe4609647d Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00