Commit graph

142 commits

Author SHA1 Message Date
James Gowdy
fd16c46128
[ML] Re-enabling file upload telemetry (#60418)
* [ML] Re-enabling file upload telemetry

* small refactor

* removing exported function

* removing commented out code

* removing commented out include

* cleaning up types
2020-03-18 08:33:53 +00:00
James Gowdy
35302ed273
[ML] Client side cut over (#60100)
* [ML] Client side cut over

* updating paths and commented code

* changes based on review

* disabling telemetry tests

* fixing start job stylesheets

* fixing everything that is broken

* fixing types and ml icon order

* using icon constant
2020-03-13 19:16:41 +00:00
spalger
410019ab5a skip flaky suite (#59541) 2020-03-12 13:17:23 -07:00
James Gowdy
d5c092811b
[ML] Adding shared services to ml setup contract (#59730)
* [ML] Adding shared services to ml setup contract

* adding data recognizer

* typescripting js client

* adding results service

* code clean up

* adding generic ml index search

* making cloud optional
2020-03-12 10:04:40 +00:00
James Gowdy
8a4bb61f05
[ML] Fixes bucket span estimators loading of max_buckets setting (#59639)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-03-09 14:43:23 +00:00
James Gowdy
92cc04c7df
[ML] Refactoring anomaly detector job types (#59556)
* [ML] Refactoring anomaly detector job types

* removing calendar type

* update validateCardinality

* fixing test

* updating  types in functional tests

* using state constants
2020-03-09 13:51:07 +00:00
Melissa Alvarez
d55524c338
[ML] NP Server: make security and spaces plugins optional (#59156)
* make security and spaces plugins optional

* update spacesPlugin name. update current user check
2020-03-05 10:06:36 -05:00
Pete Harverson
b104980b88
[ML] Add support for date_nanos time field in anomaly job wizard (#59017)
* [ML] Add support for date_nanos time field in anomaly job wizard

* [ML] Edits following review

* [ML] Add functional test for creating job off date_nanos data
2020-03-05 09:28:31 +00:00
James Gowdy
a6d2b57777
[ML] Fixing records and buckets results endpoints (#59313) 2020-03-04 17:04:47 +00:00
James Gowdy
eb40046b93
[ML] Fixing licensing after server NP cutover (#59275) 2020-03-04 10:14:11 +00:00
Jonathan Budzenski
ff23fa3849
Revert "[ML] Fixing licensing after server NP cutover (#58853)" (#59244)
This reverts commit fd5c6c16cd.
2020-03-03 17:09:29 -06:00
James Gowdy
fd5c6c16cd
[ML] Fixing licensing after server NP cutover (#58853)
* [ML] Fixing licensing after server NP cutover

* client side refactor

* management refactor

* moving management license check

* adding server side license class

* added common license class

* client sides using license class

* updating server api guards

* fix privileges tests

* updating translations

* fixing disabled tabs on basic license

* refactoring client side license checks

* setting license initialized flag

* fixing license check on index data viz

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-03-03 18:57:40 +00:00
James Gowdy
4ef594c208
[ML] Adding indices_options to datafeed (#59119)
* [ML] Adding indices_options to datafeed

* adding extra checks to the schema

* updating expand_wildcards
2020-03-03 11:59:35 +00:00
James Gowdy
38067da7ac
[ML] Fixing annotations alias checks (#58722) 2020-02-28 14:30:04 +00:00
Melissa Alvarez
8c3d71b370
[ML] NP: migrate server (#58680)
* remove obsolete legacy server deps

* licensePreRoutingFactory uses licensing plugin rather than legacy xpack

* move schemas to dir in routes

* use NP license check method for license check

* store license data in plugin for passing to check

* create server plugin files in NP plugin dir

* remove dependency on legacy xpack plugin

* add sample data links first step

* move all server dirs from legacy to np dir

* fix requiredPlugin spaces name and update import routes

* delete unnecessary files and add sample data links

* update license and privilege check tests

* add routeInit types
2020-02-27 21:15:08 -05:00
Court Ewing
7ac8e4d9cc Move x-pack plugins/server/common to x-pack/legacy 2019-06-20 13:34:48 -04:00
James Gowdy
89892ba097
[ML] Fixing server side logger after new platform change (#39154) 2019-06-18 15:18:44 +01:00
Melissa Alvarez
c47a0d5cdf
[ML] New Platform Migration - server (#38360)
* wip: pull out all server dependencies

* create new platform plugin

* shim newPlatform plugin into legacy init

* update server tests for migration refactor

* cleanup interfaces

* Only add ML links for sample data sets if full license from - 38120

* update test and fix typescript errors

* Remove unused types
2019-06-17 16:08:13 -04:00
Pete Harverson
1a03c5135b
[ML] Increase max data frames in list from 100 to 1000 (#38973) 2019-06-14 14:32:55 +01:00
Pete Harverson
41d74498e8
[ML] Edit job_service jobsExist call to query ML Get Jobs endpoint (#38649) 2019-06-11 15:02:35 +01:00
Pete Harverson
4a3034de49
[ML] Fixes loading of Single Metric Viewer if partition field is text (#37975)
* [ML] Fixes loading of Single Metric Viewer if partition field is text

* [ML] Fix failing test and edit translations for cardinality check

* [ML] Edits to fields_service.js following review
2019-06-05 12:02:03 +01:00
Matthew Kime
964f9e2d41
remove unneeded eslint-disable directives (#37922)
* remove unneeded eslint-disable directives
2019-06-04 07:53:22 -05:00
Melissa Alvarez
ca2a66dd26
return false for each jobId if it does not exist (#37116) 2019-05-28 11:58:43 -04:00
Matthew Kime
e390be2f11
No unused vars cleanup set two (#37051)
* no unused vars cleanup
2019-05-24 14:35:21 -05:00
stevedodson
9463fd8778
[ML] Improve anomalies in Kibana sample data (#36982)
* [ML] Sample ML jobs: Updates to make anomalies clearer

Minor updates to web logs data and ML job
definitions to make anomalous activity more
distinct.

* [ML] Sample ML jobs: Further updates to jobs and data

Adding missing files from previous commit and
changing job naming (removing weblog_* prefix etc.)

* [ML] Resolving PR review comments.

* [ML] Improve anomalies in Kibana sample data

Minor fixes:
- Rename job group to `kibana_sample_data` throughout
- Fix index pattern link in `sample_data_sets.ts`
- Fix log.json.gz count test
2019-05-24 15:32:06 +02:00
James Gowdy
0babbf150d
[ML] Changes interval to fixed_interval in aggregations in created jobs (#36759)
* [ML] Changes interval to fixed_interval in aggregations in created jobs

* updates following review
2019-05-22 09:57:41 +01:00
Walter Rafelsberger
1e25d1d5e5
[ML] Data Frames: Improve start/stop behavior (#36757)
- Adds a confirm modal to the Start button of the data frame jobs list. image
- Adds a snippet with a warning about possibly cluster load to wizard. image
- Uses _stop?force=true to stop jobs.
- Adds a check to disable the start button in the jobs list if the job is a completed batch job.
2019-05-21 20:55:36 +02:00
Pete Harverson
cc7a6ec2c1
[ML] Add links to ML jobs to the Kibana sample dataset view data menu (#36702)
* [ML] Add links to ML jobs to the Kibana sample dataset view data menu

* [ML] Remove redundant TODO comment from check_module.js

* [ML] Edits to sample data ML links following review

* [ML] Edits to comments about use of reject in check_module.js
2019-05-21 15:34:41 +01:00
Camilla
e997ffe0e9
[ML] Kibana Sample Dataset data recognizer modules (#35138)
* Initial job configuration files for Kibana Sample Ecommerce data recognizer

* Initial configuration files for Kibana Sample Data for web logs

* Initial configuration for Kibana Sample Data recognizer for Flights data

* Correcting ID, title and description of Flights sample data recognizer

* Adding Custom URLs to link to Discover from Kibana Sample Dataset ML jobs

* Adding custom URLs to Ecommerce Data Recognizer results

* Adding new custom URL to Kibana Sample Ecommerce data recognizer

* Adding custom URL for Sample Flights dataset dashboard

* Adding custom URLs to sample Web Logs data recogniser config

* Fixing single quotes and url names in custom URLs for flights and ecommerce datasets

* Adding Kibana Sample Data modules to list of modules that need testing

* Renaming Kibana Sample Data recognizer modules, fixing IDs and adding modules to test framework

* Fixing data recognizer titles to sentence case

* Fixing sentence case issues in Kibana eCommerce sample data

* Fixing sentence case in Kibana sample flights data

* Fixing sentence case in Kibana sample weblogs data

* Changing job configuration for Kibana sample flights data

* Making changes to job descriptions

* Changing job configurations and renaming files for Kibana eCommerce sample data recognizer

* Removing mean bytes by file extension from Web Logs sample jobs and adding three new configurations

* Fixing minor issues with Kibana Sample Data recognizer configurations

* Adding jobs to job groups

* Changing job group names to use underscores instead of hyphens

* Adding filtering to Kibana Sample Data dashboards

* Fixing linting issues in JSON configs

* Finetuning data recognizer module descriptions

* Renaming Kibana Sample Data flights jobs

* Removing plural from Kibana Sample Flights dataset description

* Modifying custom URLs for Kibana Flights jobs

* Fixing custom URLs for Kibana Sample eCommerce dataset jobs

* Adding new influencers and fixing Discover custom URLs for Kibana Weblogs sample dataset job

* Fixing space before colon issue in job configurations

* Converting tabs to spaces

* Fixing description of detector in sample dataset eCommerce job
2019-05-21 16:32:57 +02:00
Chris Roberson
c87e8811cb
[Monitoring/Telemetry] Force collectors to indicate when they are ready (#36153)
* Initial code to force collectors to indicate when they are ready

* Add and fix tests

* Remove debug

* Add ready check in api call

* Fix prettier complaints

* Return 503 if not all collectors are ready

* PR feedback

* Add retry logic for  usage collection in the reporting tests

* Fix incorrect boomify usage

* Fix more issues with the tests

* Just add debug I guess

* More debug

* Try and handle this exception

* Try and make the tests more defensive and remove console logs

* Retry logic here too

* Debug for the reporting tests failure

* I don't like this, but lets see if it works

* Move the retry logic into the collector set directly

* Add support for this new collector

* Localize this

* This shouldn't be static on the class, but rather static for the entire runtime
2019-05-20 13:03:24 -04:00
Walter Rafelsberger
e06569c437
[ML] Data Frame: Wizard Create Step Improvements (#36483)
Revamps the job creation step of the data frame wizard.

- Create & start buttons now have additional descriptions
- Adds a "copy to clipboard" button to copy a Kibana Dev Console statement of the job creation command to the clipboard
- Adds a progress bar once the job is started
- Adds a card which links to Kibana Discover if an index pattern was created for the job
- Hides the wizard's Previous button once the job is successfully created.
2019-05-20 17:30:25 +02:00
Melissa Alvarez
1cc0fa37f6
[ML] Job Selector conversion to React (#35638)
* wip: create react jobSelector wrapper + main component

* Load jobs and select first if none selected via url

* wip: create flyout content

* Add endpoint for fetching jobs with timerange for table

* display selected ids in flyout

* Add custom table allowing external selection

* add groups table in groups tab

* Get groups and jobs in initial api call

* add ability to select groups

* Hook jobSelector into SingleMetricView

* Show selected group badges with count

* Organize jobSelector component directories

* Move timerange logic to server

* Move group color selection to utils

* hide/show badges and add localization

* fetch jobs in route to enable selector jobid validation

* upate globalState on setting jobId in SingleMetricView

* Add pager options.Retain search query on tab change

* Ensure gantBar timeRanges correct

* cleanup old commented code. tweak flyout header/footer style

* running gantt bar and remove unnecessary api call

* GanttBar running style. Pass timezone to server.

* Running gantt bar limited to timerange. Clean up comments.

* Refactor jobSelector endpoint to use fullJobs

* Retain group selection in globalState

* Recalculate ganttbars on resize

* add test for JobSelectorTable
2019-05-07 11:42:34 -04:00
Peter
bc90140f8e
enable security plugin in basic (#35891)
enable security on file dataviz and import (ML plugin)

update unit tests

add api test coverage for security in basic

move audit logging to standard+ license level
2019-05-03 11:21:32 -07:00
Pete Harverson
3577c43d90
[ML] Edit queries in Metricbeat module to use event.dataset field (#35653) 2019-04-29 11:09:16 +01:00
Pete Harverson
43b9c7ad89
[ML] Adds Metricbeat System data recognizer module (#35580)
* [ML] Adds Metricbeat System data recognizer module

* [ML] Edit description of Metricbeat System module
2019-04-25 16:24:39 +01:00
Walter Rafelsberger
d316dca98a
[ML] Data Frame UI (#33427) 2019-04-23 16:46:52 +02:00
James Gowdy
c0edcbe9ab
[ML] Round value when using max model memory limit (#35223)
* [ML] Round value when using max model memory limit

* changing ceil to floor to avoid accidentally exceeding max mml
2019-04-17 16:00:35 +01:00
James Gowdy
3930c36898
ML] Fixes bug in module setup endpoint caused by PR 30935 (#34414)
* [ML] Fixes module setup endpoint

* tweaks endpoint to allow trailing slash
2019-04-03 12:42:59 +01:00
James Gowdy
2315ac1173
[ML] List data recognizer modules (#30935)
* [ML] [WIP] List data recognizer modules

* making module id optional

* sort modules

* removing list_module endpoint

* removing list_modules call

* listed modules include saveObjects

* adding tests
2019-04-02 17:23:56 +01:00
Walter Rafelsberger
2882d2853f
[ML] Fixes check to determine whether telemetry is enabled. (#33975) 2019-03-27 19:43:01 +01:00
Pete Harverson
0603f7a8db
[ML] Adds created_by field to auditbeat and APM data recognizer jobs (#33863) 2019-03-27 11:05:12 +00:00
Spencer
2e232c2e31
[@kbn/expect] "fork" expect.js into repo (#33761)
* [@kbn/expect] "fork" expect.js into repo

* [eslint] autofix references to expect.js

* [tslint] autofix all expect.js imports

* now that expect.js is in strict mode, avoid reassigning fn.length
2019-03-25 09:56:48 -07:00
Pete Harverson
5ff422d353
[ML] Skip invalid modules when data recognizer lists matches (#33703)
* [ML] Skip invalid modules when data recognizer lists matches

* [ML] Append error to data recognizer module load console logs
2019-03-22 16:45:01 +00:00
Pete Harverson
3581f1d54b
[ML] Fixes URL to get job stats when getting stats for all jobs (#33237) 2019-03-14 14:25:39 +00:00
Pete Harverson
a1a9511446
[ML] Edits data recognizer modules to use KQL in saved objects (#32766) 2019-03-08 16:35:20 +00:00
Melissa Alvarez
576c90ae8d
[ML] Add query bar to Anomaly Explorer for filtering of anomaly results for one or more influencers (#31763)
* wip: add autocomplete kueryFilterBar.update endpoints with query.

* Create indexPattern on job change. Value is autocompleted

* Mask unrelated swimlanes on filter apply

* Only show relevant viewBy swimlane

* reload viewBy swimlane when view By change is relevant to filter

* handle various query types when building filteredFields list

* remove deprecated code from old filter bar

* Show error callout when invalid filter syntax

* remove dependency on deprecated import var

* remove all left over apm dependencies

* persist filter on refresh

* Create initial filter bar placeholder dynamically

* add description text when filter active

* switch to relevant viewBy fieldname on filter

* recalculate placeholder on job change

* Create tests for all components

* View by dropdown only contains relevant fields for filter

* fix localization message noInfluencers component

* Move indexPattern to state and viewBy option filtering to utils

* remove unnecessary setState when setting viewBy for filter

* Use preloaded jobs from mlJobService to get influencers

* Only filter viewByOptions if valid field

* use kql and move to components dir

* move filter bar to public/components/
2019-03-05 15:07:30 -06:00
Walter Rafelsberger
46f7a350fe
[ML] Adds missing error handling to annotations request. (#32384)
- This adds more thorough error handling to retrieving annotations from the server to avoid breaking Anomaly Explorer when there's a problem retrieving annotations. The UI will silently fail and hide the annotations table if for whatever reasons annotations cannot be retrieved for the selected jobs (this could be permissions or setup related if e.g. the annotations feature index/aliases are not present). This is now the same behavior like in Single Metric Viewer. The browser console will still output an error message. This fixes an error scenario where Anomaly Explorer would fail to load if a user upgraded to 6.6 because the backend task wasn't run yet to create the annotation index/aliases.
- Disables annotations feature in Single Metric Viewer if annotations fail to load.
- Use componentDidUpdate and an update check instead of deprecated componentWillUpdate for updating the annotations table in jobs management.
2019-03-05 17:04:47 +01:00
Aleh Zasypkin
0835cd30ca
Introduce Elasticsearch service. (#28344) 2019-02-28 17:22:07 +02:00
James Gowdy
f89faec278
[ML] Changing job wizard query to use match_all (#32008)
* [ML] Using job wizard query to use match_all

* adding query back in for data visualizer

* fixing typo
2019-02-26 16:51:17 +00:00
James Gowdy
a28855a183
[ML] Fixing error when checking privileges for user with no ml_user role (#31429) 2019-02-18 18:37:14 +00:00