* [ML] File datavisualizer initial commit (#22828)
* [ML] File datavisualizer initial commit
* removing mocked data and adding initial stats
* adding card styling to fields
* Revert "". accidentally added with no commit message
This reverts commit d762d20b706e6a770e631f863b9e7d8879bb7ee6.
* adding date type to timestamp field
* renaming FileStats to FieldsStats
* code clean up
* changes based on review
* changes to error handling
* [ML] Adding file datavisualizer overrides (#23194)
* [ML] Adding file datavisualizer overrides
* improvements to overrides
* removing comment
* small refactor
* removing accidentally added file
* updates based on review
* fixing broken test
* adding missing grok pattern override
* fixing test
* [ML] Refactoring override option lists (#23424)
* [ML] Refactoring override option lists
* moving lists out of class
* updating test snapshot
* [ML] Fixing field editing (#23500)
* [ML] Changes to timestamp formats (#23498)
* [ML] Changes to timestamp formats
* updating test snapshot
* [ML] Allow Datavisualizer use on basic license (#23748)
* [ML] Allow ML use on basic license
* removing timeout change
* adding permission checks
* updating tests
* removing unnecessary checks
* [ML] Adds new page for choosing file or index based data visualizer (#23763)
* [ML] Adding license check to datavisualizer landing page (#23809)
* [ML] Adding license check to datavisualizer landing page
* removing comments
* updating redirect to landing page
* [ML] Adding ability to upload data to elasticsearch from datavisualizer (#24042)
* [ML] Initial work for delimited file upload
* adding results links cards
* adding nav menu
* removing accidental debugger
* initial work for importing semi structured text
* using ingest pipeline for import
* adding json importer and better error reporting
* better progress steps
* time range added to results links
* first import only creates index and pipeline
* adding status constants
* using status constants
* adding explanation comment
* updating yarn.lock
* changes based on review
* fixing space
* fixing space again, stort it out git
* removing oversized background container causing constant scrollbar
* [ML] Adding basic license check when loading privileges (#24173)
* [ML] Adding basic license check
* missing import
* [ML] Adds an About panel to the file data visualizer landing page (#24260)
* [ML] Adds an About panel to the file data visualizer landing page
* [ML] Remove unnecessary style from file data visualizer scss
* [ML] Adding better error reporting for reading and importing data (#24269)
* [ML] Adding better error reporting for reading and importing data
* changes to endpoint errors
* displaying errors
* step logic refactor
* removing log statements
* [ML] Switch file data visualizer to use Papa Parse for CSV parsing (#24329)
* [ML] Fixes layout of Data Visualizer selector page for IE (#24387)
* [ML] Adding ability to override various settings when importing data (#24346)
* [ML] Adding ability to override various settings when importing data
* second commit with most of the outstanding code
* improving index pattern name validation
* better index pattern matching
* adding comments
* adding empty index pattern check
* changes based on review
* fixing test
- Fixes the job validation for the lower bound of the model memory limit. Previously the check was against zero, now it's again less than 1MB, which is the same as the backend expects.
- If the user entered model memory limit is less than half the value of the estimated model memory limit, a warning type message gets triggered. If the user entered model memory limit is more than half the value but less then the actual value of the estimated model memory limit, then the already existing info type message is shown. The unit tests have been updated to reflect that behavior.
* [ML] Fixing issue with incorrect timezones in jobs list
* refactoring min and max calculation
* changes based on review
* changing TimeStamp to Timestamp
This fixes bucket span estimation for the machine learning admin user when security is enabled. The original issue was that the machine learning admin by default doesn't have the necessary permissions to call cluster.getSettings to get the settings search.max_buckets. This is now fixed by using callWithInternalUser() and additional permissions checks.
* [ML] [WIP] Adding group selector to jobs management
* adding group name validation
* removing comment
* adding keyboard events
* moving new group input to its own component
* changes based on review
* adding tooltip
* adding better error reporting
- Fixes the bucket span estimator when median is selected as a detector function. agg.type.name is median and therefor not usable for an Elasticsearch aggregation. agg.type.dslName is percentile and is the correct mapping. .dslName is also used for the aggregations used for the preview charts.
- 7.0 will introduce a search.max_buckets setting which defaults to 10000. This could lead to failing bucket estimations because the values used for creating the required aggregations could result in more buckets. This PR fixes it by taking search.max_buckets into account when calculating the time range used for the bucket estimation. (Since 6.2 that setting is available so backporting this to current unreleased minor releases 6.4 and 6.5)
Improvements to job validation messages:
- Changed cardinality_over_field_low "might be less suitable" to "might not be suitable".
- Fixed documentation URLs.
- Extended categorization_filters_invalid message + documentation URL.
- If a job configuration's datafeed contains script_fields, they are now available in the field dropdowns of the advanced wizard's detector modal.
- Additionally, job validation now considers script_fields and doesn't report them as non aggregatable anymore.
This provides more helpful texts for job validation success messages. Previously only a list of checks was shown to the user without any further explanation. This PR addresses this issue in the following way:
- At the bottom of the modal an introductory brief text about job validation including a link to documentation is inserted.
- The success messages in the list now provide a more helpful text including (where applicable) a deep link to documentation
- The messages now support a richer Callout layout including a header and additional text.
While the output in the UI was fine, certain job configurations containing non-aggregatable fields could trigger errors on the Elasticsearch side.
This PR fixes it by adding an additional query for fieldCaps to check first which fields are actually aggregatable.
For categorization jobs, job validation would report that mlcategory isn't an aggregatable field. This fix checks the job configuration and only reports the error if the job config isn't using categorization_field_name and the detector field isn't set to mlcategory.
* [ML] Adding close jobs menu item
* changing icon
* updates based on review
* adding extra check for close failure
* adding extra guard against missing response
* add _bulk_create rest endpoint
* document bulk_create api
* provide 409 status code when conflicts exist
* add overwrite and version to documenation
* clean up assert statements and 2 places where bulkCreate is getting used with new output
* properly stub bulkCreate return
* remove version of documenation example
* [ML] Adding jobs stats bar to top of jobs list
* unsetting update function
* changing node variable name
* small refactor
* using props copy of `updateJobStats`
* adding proptypes
* adding fixed height
* [ML] Adding filter bar to jobs list
* fixing page index when filtering
* refreshing job selection after actions have happened
* adding job counts to groups
* catching multi-select start datafeed errors
* style tweaks
* more style tweaks
* changes based on review
* refactoring search logic
* [ML] Removing angular from services
* removing commented out code
* fix include path for tests
* adding flag to initPromise
* moving job service for jest tests to pass
* moving initPromise to its own file
This adds an additional check and corresponding error message to the validation of the model memory limit. If the parsing of the memory model limit results in 0 bytes the message will be triggered. This behaviour catches both correctly formatted but invalid strings (like 0mb) and invalid strings (like asdf).
Updates the behaviour of the job validation's model memory limit checks: The success messages success_mml gets only returned if the checks were actually run. This omits the success message for example in the single metric and population wizard.
