Commit graph

1093 commits

Author SHA1 Message Date
Tyler Smalley
504c8739de
test:jest improvements to better support our monorepo (#84848)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2020-12-14 14:07:50 -08:00
Pete Harverson
bf869422e1
[ML] Only show _doc_count for anomaly detection job summary count field (#85444)
* [ML] Only show _doc_count for anomaly detection job summary count field

* [ML] Use array filter for removing _DOC_COUNT field
2020-12-10 22:30:39 +00:00
Devon Thomson
ac189b8c25
Deangularize Dashboard (#82909)
Deangularized dashboard application
2020-12-10 16:46:50 -05:00
Craig Chamberlain
313d85e985
[ML] Adds security_linux and security_windows Modules (#85065)
* initial commit

refactored multi-index, multi-pipeline jobs for 7.11. These are new modules that will live alongside the existing jobs.

* Update ml_modules.tsx

added new module names to the list

* Update get_module.ts

added new module names

* Linter fixes

* Order matters

* manifest fixes

added colon char to the module name and shortened the description

* additon to description

after talking with the security team today, adding this suggested text to the beginning of the description so it will tend to be visible to the user:
"This is a new refactored job which works on ECS compatible events across multiple indices."

* Adjust module recognizer test for auditbeat dataset

* influencers

changes to the metadata jobs to make influencers identical to the originals

* change for security app

changes to two datafeeds needed for logic in the Security app - added the suffix "_ecs" to two ids.

Co-authored-by: Garrett Spong <spong@users.noreply.github.com>
Co-authored-by: Robert Oskamp <robert.oskamp@elastic.co>
2020-12-10 14:02:41 -05:00
Quynh Nguyen
008a420f81
[ML] Improve messaging and support for datafeed using aggregated and scripted fields (#84594) 2020-12-10 11:35:51 -06:00
Melissa Alvarez
93670ec81f
[ML] Data frame analytics: Adds job deletion and creation to map view (#84299)
* wip: delete job node and update map

* adds ability to delete job and update map

* create job from index node

* can clone job from map

* reset map button

* remove trained model node when deleting job

* remove related model node. remove map tab when root node deleted

* ensure model with no job shows up correctly

* update types and naming

* use urlGenerator

* fix inner scrollbar

* Adjust cytoscapeOptions after EUI update

Co-authored-by: Robert Oskamp <robert.oskamp@elastic.co>
2020-12-09 10:11:08 -05:00
Dima Arnautov
b6fe6d0fea
[ML] Fix watcher URL to the Anomaly Explorer page (#85123)
* [ML] fix watcher url to the anomaly explorer page

* [ML] remove refreshInterval from the watcher url to the anomaly explorer page
2020-12-08 11:30:54 +01:00
Chandler Prall
2f72c8ad10
Upgrade EUI to v30.5.1 (#84677)
* Updated to eui@30.4.1, fixed types and unit tests

* Cleanup some imports

* Fix a text color swap, now back to danger text

* Bump EUI to v30.4.2

* Revert snapshot changes from ownFocus modification

* Clean up alert flyout test actions to better represent user actions

* Upgrade EUI to 30.5.1

* More accurate test interaction

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-04 09:39:03 -07:00
Tyler Smalley
b593781009
Jest multi-project configuration (#77894)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2020-12-02 11:42:23 -08:00
Quynh Nguyen
0e43beed4f
[ML] Fix prediction probability text for classification (#84593)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-02 13:35:38 -06:00
Walter Rafelsberger
5889e366da
[ML] Fix unnecessary trigger of wildcard field type search for ML plugin routes. (#84605)
Passing in an empty string '' to useResolver() would trigger a wild card search across all indices and fields, potentially causing a timeout and the page would fail to load. The following pages were affected: Single Metric Viewer, Data frame analytics models list, Data frame analytics jobs list, Data frame analytics exploration page, File Data Visualizer (Data visualizer - Import data from a log file). This PR fixes it by passing undefined instead of '' to useResolver to avoid calling _fields_for_wildcard with an empty pattern. Jest tests were added to cover the two parameter scenarios empty string/undefined.
2020-12-01 18:05:57 +01:00
Dima Arnautov
f39da3e680
[ML] Persisted URL state for Data Frame Analytics Exploration page (#84499)
* [ML] store query string in the URL state

* [ML] query state for the config step

* [ML] pagination in the URL state

* [ML] persisted URL state for outlier results exploration

* [ML] update URL generator

* [ML] do not update the url state when query string hasn't been changed

* [ML] store expandable panels state in the URL

* [ML] fix TS issue

* [ML] fix TS issue
2020-11-30 18:54:15 +01:00
James Gowdy
090359ca5a
[ML] Renaming saved object repair to sync (#84311)
* [ML] Remaiming saved object repair to sync

* removing 0 from sync lists

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-30 12:24:03 +00:00
James Gowdy
767286e2da
[ML] Fix spaces job ID check (#84404)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-30 09:34:33 +00:00
Dima Arnautov
280ce7e5fa
[ML] Persisted URL state for Anomalies table (#84314)
* [ML] Persisted URL state for Anomalies table

* [ML] adjust cell selection according to the time range
2020-11-25 16:49:45 +01:00
Pete Harverson
c1b807eda3
[ML] Fix Anomaly Explorer population charts when multiple causes in anomaly (#84254) 2020-11-25 11:36:31 +00:00
Dima Arnautov
dc15aa8ea2
[ML] Fix swim lane for top influencers (#84258)
* [ML] fix swim lane with page size for top influencers, fix swim lane sorting

* [ML] fix typo
2020-11-24 23:26:08 +01:00
James Gowdy
24f262b9ca
[ML] Space permision checks for job deletion (#83871)
* [ML] Space permision checks for job deletion

* updating spaces dependency

* updating endpoint comments

* adding delete job capabilities check

* small change based on review

* improving permissions checks

* renaming function and endpoint

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-24 16:29:32 +00:00
Dima Arnautov
bfbb43e59b
[ML] Improve browser history navigation (#83792)
* [ML] replace history support

* [ML] explorer url state

* [ML] timeseriesexplorer url state

* [ML] fix state keys for mlSelectSeverity and mlSelectInterval

* [ML] fix useSelectedCells

* [ML] update urls and tests in security app

* [ML] fix TS

* [ML] fix apm unit tests

* [ML] fix typo

* [ML] remove state sync

* [ML] fix initial zoom set

* [ML] fix initial zoom set

* [ML]: update with useMlHref

* [ML] fix TS issue
2020-11-24 15:29:32 +01:00
Pete Harverson
921a119804
[ML] Remove all usage of font awesome icons in ML plugin (#84066) 2020-11-23 17:39:14 +00:00
Mikhail Shustov
95861a0fb0
[DX] Prettier v2.2 (#83899)
* update prettier with ts version support

* mute type-error

* run prettier on codebase

* fix examples

* fix errors after master merged
2020-11-23 13:17:05 +01:00
Melissa Alvarez
00e59512fa
[ML] Data frame analytics: Adds functionality to map view (#83710)
* get all jobs from index node

* create map from modelId and enable url share

* highlight source node

* add map endpoint to api doc

* use variables in css.fix types.ensure map tab is shown

* fix translations
2020-11-20 10:39:30 -05:00
Quynh Nguyen
2074b2991c
[ML] Fix Single Metric Viewer not loading if job is metric with no partition (#83880) 2020-11-20 08:50:10 -06:00
Larry Gregory
7f962e5839
Removing circular dependency between spaces and security (#81891)
* Removing circular dependency between spaces and security

* Apply suggestions from code review

Co-authored-by: Constance <constancecchen@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>

* Tests refactor

- Reorganize top level describes into 3 space-based blocks into based on spaces:
  - space disabled
  - spaces plugin unavailable
  - space enabled (most previous tests go under this new block) with new beforeEach

- wrote new tests for uncovered lines 58, 66-69

* Review1: address PR feedback

* changing fake requests for alerts/actions

* Fixing tests

* fixing more tests

* Additional testing and refactoring

* Apply suggestions from code review

Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>

* Review 2: Address feedback

* Make ESLint happy again

Co-authored-by: Constance <constancecchen@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Constance Chen <constance.chen.3@gmail.com>
2020-11-19 13:41:13 -05:00
James Gowdy
78d7bfdf97
[ML] Space management UI (#83320)
* [ML] Space management UI

* fixing types

* small react refactor

* adding repair toasts

* text and style changes

* handling spaces being disabled

* correcting initalizing endpoint response

* text updates

* text updates

* fixing spaces manager use when spaces is disabled

* more text updates

* switching to delete saved object first rather than overwrite

* filtering non ml spaces

* renaming file

* fixing types

* updating list style

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-19 15:26:01 +00:00
Quynh Nguyen
77da781144
[ML] Persist URL state for Anomaly detection jobs using metric function (#83507)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-18 10:57:22 -06:00
Quynh Nguyen
7a7057eba7
[ML] Performance improvements to annotations editing in Single Metric Viewer & buttons placement (#83216) 2020-11-18 10:49:26 -06:00
Dima Arnautov
bc22b67ba7
[ML] Persisted URL state for the Data frame analytics jobs and models pages (#83439)
* [ML] support table settings from the URL state

* [ML] fix management page

* [ML] models page support

* [ML] update URL generator

* [ML] rename id column

* [ML] update types and tests

* [ML] fix id column name and field

* [ML] remove legacy functions

* [ML] set id key for the job query text

* [ML] fix id column rendering

* [ML] ad jobs with usePageUrlState

* [ML] update unit tests for solutions
2020-11-17 19:29:47 +01:00
Quynh Nguyen
55119c2152
[ML] Improve support for script and aggregation fields in anomaly detection jobs (#81923)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-17 11:34:15 -06:00
Quynh Nguyen
9c0164a2d8
[ML] Add UI test for feature importance features (#82677)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-17 10:07:46 -06:00
Melissa Alvarez
389e7a1d6a
Replace experimental badge with Beta (#83468) 2020-11-17 10:57:52 -05:00
James Gowdy
6a1cd73082
[ML] Additional job spaces initialization (#83127)
* [ML] Additional job spaces initialization

* adding logs test

* updating integrations

* updating test text

* fixing logs jobs error

* fix bug with duplicate ids

* updating initialization log text

* fixing initialization text

* adding metrics overrides

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-16 21:56:41 +00:00
Nathan L Smith
3f51bf5e9f
Fix advanced settings category sorting (#83394)
In the advanced settings categories, "Observability" and "Machine Learning" were using uppercase letters in their keys while everything else was using lowercase

This caused them both to show up before the rest of the options in the dropdown and in the sorting in the advanced settings UI.

Add keys for them to the get_category_name module in the advanced settings plugin and use those keys in the plugins that apply these categories.

This also makes it so i18n keys are available for these items.

Fixes #81974.
2020-11-16 09:23:28 -06:00
Robert Oskamp
406a568d0c
[ML] Add basic license test run details to ML+Transform READMEs (#83259)
This PR updates the READMEs of the ml and transform plugins to include instructions how to run basic license tests.
2020-11-16 10:25:42 +01:00
John Schulz
380fa5b3f6
[bundle optimization] Update to semver 7.x to get tree-shaking (#83020)
## What's changed in this PR
### Update to latest available `semver`: `7.3.2`
 * `semver` 5.x pulls in the entire library in one large file (~38k uncompressed / ~9k gz), when we might only use 1-2K.
 * `semver` 7.0+ supports tree-shaking: https://github.com/npm/node-semver/blob/master/CHANGELOG.md#700

### Update paths to only import individual function(s) used instead of the entire library
  * Getting the smaller bundle requires a different import style [as shown in the docs](https://github.com/npm/node-semver#usage)
  * Only changed code in `public` & `common` folders; not `server`. We could also update `server` as well for consistency, but I skipped because the new import style is more verbose and the filesize didn't seem as important on the server

### Results
The build stats show a 10K+ improvement for initial page bundles https://github.com/elastic/kibana/pull/83020#issuecomment-724724432

| id | [before](c6afc47f32) | [after](213bb52a8c) | diff |
| --- | --- | --- | --- |
| `ingestManager` | 386.2KB | 373.9KB | -12.3KB |
| `telemetry` | 63.5KB | 50.1KB | -13.5KB |
| `upgradeAssistant` | 74.5KB | 60.5KB | -14.0KB |
| total |  |  | -39.7KB |

### The import paths look odd. Are they required?
I agree and, no, they're not strictly required. If you'd like me to revert to the prior style just drop a comment and I'll undo them.

The caveat is that the current style (in `master` & this PR) pulls in the entire `semver` library. In 7.x that added ~15K to the initial size. Some more details in the comments: https://github.com/elastic/kibana/pull/83020#issuecomment-724859130

### Possible issues
Moving 2 major versions. We're currently on 5.7 and the latest available is 7.3. 
  * changelog says 5.x (our current) to 6.0 should be safe: https://github.com/npm/node-semver/blob/master/CHANGELOG.md#60
  * There 6.x & 7.x changes all appear to be new features or bugfixes around the `includePrerelease` flag added in 5.6, but I'm not sure if those "fixes" will break existing code
    * https://github.com/npm/node-semver/blob/master/CHANGELOG.md#613
    * https://github.com/npm/node-semver/blob/master/CHANGELOG.md#722

### Stats / screenshots
generated with `node scripts/build_kibana_platform_plugins.js --profile --focus=ingestManager`
<details><summary><b>Ingest Manager in `master`</b>: imports entire `semver` lib, totals 40k+, only 1 large file (orange arc below)</summary>

<img width="972" alt="Screen Shot 2020-11-09 at 6 50 23 PM" src="https://user-images.githubusercontent.com/57655/98666188-a50ac380-231a-11eb-9b8a-6ca784752714.png">
</details>

<details><summary><b>Ingest Manager in PR after upgrade to 7</b>: still imports entire lib. file size *increased* to ~60k, but now individual files are imported (orange arcs below)</summary>
<img width="825" alt="Screen Shot 2020-11-09 at 5 46 30 PM" src="https://user-images.githubusercontent.com/57655/98666355-e602d800-231a-11eb-803f-bc04beb4eaf1.png">
<img width="963" alt="Screen Shot 2020-11-09 at 5 47 06 PM" src="https://user-images.githubusercontent.com/57655/98666357-e69b6e80-231a-11eb-92d3-c66904f92c30.png">
</details>

<details><summary><b>Ingest Manager in PR after changing `import`s:</b> total imported size down to ~20k. Can see individual imported files</summary>
<img width="926" alt="Screen Shot 2020-11-10 at 6 10 23 AM" src="https://user-images.githubusercontent.com/57655/98667058-e64fa300-231b-11eb-9690-5e36ed6475e0.png">
<img width="895" alt="Screen Shot 2020-11-10 at 6 10 53 AM" src="https://user-images.githubusercontent.com/57655/98667059-e780d000-231b-11eb-8abf-98d8bdbcf061.png">
</details>

### Checklist

- [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2020-11-14 07:32:02 -05:00
Pete Harverson
dde2d112d0
[ML] Adds functional tests for the index data visualizer card contents (#83174)
* [ML] Adds functional tests for the index data visualizer card contents

* [ML] Fix translations

* [ML] Fix type errors in permissions tests

* [ML] Address comments from review
2020-11-13 09:39:33 +00:00
Tyler Smalley
aba2068291
Consolidates Jest configuration files and scripts (#82671)
Jest tests are currently organized into main configuration files (src/dev/jest/config.js and x-pack/dev-tools/jest/create_jest_config.js). Both of these are similar, but very slightly due to  previously being in separate repositories. This change consolidates the scripts referenced in those configs and moves them to the `@kbn/test` project.

OSS contained an alias for `test_utils`. Those aliases have been removed in favor of importing these utilities from `@kbn/test/jest`

Blocker to #72569

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2020-11-12 16:19:56 -08:00
Melissa Alvarez
6519b83e48
[ML] Data frame analytics: Adds map view (#81666)
* add analytics map endpoint and server model

* add map action to job and models list

* wip:fetch models for jobs. Use url generator

* get models when extending node. deduplicate elements

* add job type icons. disable map action if job not finished.

* move shared const to common dir

* persist map tab. handle indexPattern from visualizer

* use url generator in models list

* temporarily disable delete action in flyout

* update legend style. make map horizontal

* update dfa model to use spaces changes

* format creation time

* update from indexPattern to index.remove refresh button

* handle index patterns with wildcard
2020-11-12 12:40:08 -05:00
Dima Arnautov
6e9177caac
[ML] Update apidoc config with the Trained models endpoints (#83274)
* [ML] fix apidoc annotations

* [ML] add trained models

* [ML] use full path to the apidoc-markdown package
2020-11-12 17:51:42 +01:00
Dima Arnautov
169dcef2bf
[ML] Persisted URL state for the "Anomaly detection jobs" page (#83149)
* [ML] table config in the URL state

* [ML] fix job list on the management page

* [ML] store query filter in the URL

* [ML] fix context for the management page

* [ML] update module_list_card.tsx in Logs UI

* [ML] fix unit tests

* [ML] fix unit tests

* [ML] fix unit tests

* [ML] move utils functions

* [ML] url generator to support both job and group ids
2020-11-12 15:16:23 +01:00
Quynh Nguyen
af51394986
[ML] Replace EuiBasicTable with EuiInMemoryTable (#83057) 2020-11-11 11:30:30 -06:00
Dima Arnautov
c32215d7ff
[ML] Fix apiDocs extractor script (#82582)
* [ML] fix serializer script

* [ML] fix extractor

* [ML] bump apiDoc version

* [ML] update yarn.lock
2020-11-11 15:42:53 +01:00
James Gowdy
096acb4da8
[ML] job saved objects initialization (#82639)
* [ML] job saved objects initialization

* fixing job count logic

* adding missing files

* attempting to fix build crash

* fixing kibana.json

* changes based on review

* removing accidentally added export

* adding intialization promise

* use finally so errors dont stop initialization

* function rename

* removing duplicate header

* adding job initialization count to log message

* adding error to log message

* moving initialization file

* moving intialization file back again to fix git stash issues

* removing .kibana index search

* creating internal saved object client

* code clean up

* removing commented code

* adding check for spaces enabled

* adding ids to saved objects

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-10 21:02:09 +00:00
James Gowdy
dc287eaadc
[ML] Adding non-space aware checks for existing jobs (#82814)
* [ML] Adding non-space aware checks for existing jobs

* merge with existing jobs exist endpoint

* updating comments

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-10 20:54:32 +00:00
Quynh Nguyen
9c984f4723
[ML] Add option for anomaly charts for metric detector should plot min, mean or max as appropriate (#81662) 2020-11-09 10:41:04 -06:00
Melissa Alvarez
ae20a3a2a9
[ML] Add unsigned_long support to data frame analytics and anomaly detection (#82636)
* add support for unsigned_long field in dfa

* add support for unsigned_long for anomaly detection
2020-11-06 16:51:14 -05:00
James Gowdy
f4386fc5b0
[ML] Updating analysis config schema (#82703)
* [ML] Updating analyss config schema

* better schema order
2020-11-05 13:41:16 +00:00
Pete Harverson
4b9570f209
[ML] Fixes formatting of fields in index data visualizer (#82593) 2020-11-04 18:01:59 +00:00
Quynh Nguyen
89887984d8
[ML] Add annotation markers to time series brush area to indicate annotations exist outside of selected range (#81490)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-04 11:01:58 -06:00
Quynh Nguyen
b8307b498c
[ML] Add probability values in decision path visualization for classification data frame analytics (#80229)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-03 18:47:21 -06:00