- Introduces dedicated Kibana API endpoints as part of ML and transform plugin API endpoints and moves the logic to query and transform the required data from client to server.
- Adds support for sampling to retrieve the data for the field histograms. For now this is not configurable by the end user and is hard coded to 5000. This is to have a first iteration of this functionality in for 7.9 and protect users when querying large clusters. The button to enable the histogram charts now includes a tooltip that mentions the sampler.
* Move resolver query param logic into shared hook
* Store document location in state
* Rename documentLocation to resolverComponentInstanceID
* Use undefined for initial resolverComponentID value
* Update type for initial state of component id
* fixes bug where the anomaly detection setup link was showing alert incorrectly, adds unit tests
* Fixes typo in getMlBucketSize query, uses terminate_after
* Improve readbility of helper function to show alerts and unit tests
* progress on bulk close
* works but could be slow
* clean up, add tests
* fix reduce types
* address 'event.' fields
* remove duplicate import
* don't replace nested fields
* my best friend typescript
* Add Frontend components for Value Lists Management Modal
Imports and uses the hooks provided by the lists plugin. Tests coming
next.
* Update value list components to use newest Lists API
* uses useEffect on a task's state instead of promise chaining
* handles the fact that API calls can be rejected with strings
* uses exportList function instead of hook
* Close modal on outside click
* Add hook for using a cursor with paged API calls.
For e.g. findLists, we can send along a cursor to optimize our query. On
the backend, this cursor is used as part of a search_after query.
* Better implementation of useCursor
* Does not require args for setCursor as they're already passed to the
hook
* Finds nearest cursor for the same page size
Eventually this logic will also include sortField as part of the
hash/lookup, but we do not currently use that on the frontend.
* Fixes useCursor hook functionality
We were previously storing the cursor on the _current_ page, when it's
only truly valid for the _next_ page (and beyond).
This was causing a few issues, but now that it's fixed everything works
great.
* Add cursor to lists query
This allows us to search_after a previous page's search, if available.
* Do not validate response of export
This is just a blob, so we have nothing to validate.
* Fix double callback post-import
After uploading a list, the modal was being shown twice. Declaring the
constituent state dependencies separately fixed the issue.
* Update ValueListsForm to manually abort import request
These hooks no longer care about/expose an abort function. In this one
case where we need that functionality, we can do it ourselves relatively
simply.
* Default modal table to five rows
* Update translation keys following plugin rename
* Try to fit table contents on a single row
Dates were wrapping (and raw), and so were wrapped in a FormattedDate
component. However, since this component didn't wrap, we needed to
shrink/truncate the uploaded_by field as well as allow the fileName to
truncate.
* Add helper function to prevent tests from logging errors
https://github.com/enzymejs/enzyme/issues/2073 seems to be an ongoing
issue, and causes components with useEffect to update after the test is
completed.
waitForUpdates ensures that updates have completed within an act()
before continuing on.
* Add jest tests for our form, table, and modal components
* Fix translation conflict
* Add more waitForUpdates to new overview page tests
Each of these logs a console.error without them.
* Fix bad merge resolution
That resulted in duplicate exports.
* Make cursor an optional parameter to findLists
This param is an optimization and not required for basic functionality.
* Tweaking Table column sizes
Makes actions column smaller, leaving more room for everything else.
* Fix bug where onSuccess is called upon pagination change
Because fetchLists changes when pagination does, and handleUploadSuccess
changes with fetchLists, our useEffect in Form was being fired on every
pagination change due to its onSuccess changing.
The solution in this instance is to remove fetchLists from
handleUploadSuccess's dependencies, as we merely want to invoke
fetchLists from it, not change our reference.
* Fix failing test
It looks like this broke because EuiTable's pagination changed from a
button to an anchor tag.
* Hide page size options on ValueLists modal table
These have style issues, and anything above 5 rows causes the modal to
scroll, so we're going to disable it for now.
* Update error callbacks now that we have Errors
We don't display the nice errors in the case of an ApiError right now,
but this is better than it was.
* Synchronize delete with the subsequent fetch
Our start() no longer resolves in a meaningful way, so we instead need
to perform the refetch in an effect watching the result of our delete.
* Cast our unknown error to an Error
useAsync generally does not know how what its tasks are going to be
rejected with, hence the unknown.
For these API calls we know that it will be an Error, but I don't
currently have a way to type that generally. For now, we'll cast it
where we use it.
* Import lists code from our new, standardized modules
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
* Adds the ability for exception lists to be multi-list queried
* Fixes a bunch of script issues where I did not update everywhere I needed to use `ip_list` and deletes an old list that now lives within the new/lists folder
* Fixes a few io-ts issues with Encode Decode while I was in there.
* Adds two more types and their tests for supporting converting between comma separated strings and arrays for GET calls.
* Fixes one weird circular dep issue while adding more types.
You now send into the find an optional comma separated list of exception lists their namespace type and any filters like so:
```ts
GET /api/exception_lists/items/_find?list_id=simple_list,endpoint_list&namespace_type=single,agnostic&filtering=filter1,filter2"
```
And this will return the results of both together with each filter applied to each list. If you use a sort field and ordering it will order across the lists together as if they are one list. Filter is optional like before. If you provide less filters than there are lists, the lists will only apply the filters to each list until it runs out of filters and then not filter the other lists.
If at least one list is found this will _not_ return a 404 but it will _only_ query the list(s) it did find. If none of the lists are found, then this will return a 404 not found exception.
**Script testing**
See these files for more information:
* find_exception_list_items.sh
* find_exception_list_items_by_filter.sh
But basically you can create two lists and an item for each of the lists:
```ts
./post_exception_list.sh ./exception_lists/new/exception_list.json
./post_exception_list_item.sh ./exception_lists/new/exception_list_item.json
./post_exception_list.sh ./exception_lists/new/exception_list_agnostic.json
./post_exception_list_item.sh ./exception_lists/new/exception_list_item_agnostic.json
```
And then you can query these two lists together:
```ts
./find_exception_list_items.sh simple_list,endpoint_list single,agnostic
```
Or for filtering you can query both and add a filter for each one:
```ts
./find_exception_list_items_by_filter.sh simple_list,endpoint_list "exception-list.attributes.name:%20Sample%20Endpoint%20Exception%20List,exception-list-agnostic.attributes.name:%20Sample%20Endpoint%20Exception%20List" single,agnostic
```
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
* Fix dimension popover layout and color picker “Auto”
* Created ToolbarButton
* Move disabled help text to tooltip for missing values
* Darker side panel backgrounds
* Adding to .asciidoc about where to put the SASS import
* Moving `SASS` guidelines to STYLEGUIDE.md
* Fix keyboard focus of XY settings popover
* Fix dark mode
* Add API functions and hooks for reading and creating the lists index
* Ensure KibanaApiError extends the Error interface
It has a name, so we should type it as such. This way, we can use it
anywhere that an Error is accepted.
* Return an Error from validationEither and thus from our useAsync hooks
Because an io-ts pipeline needs a consistent type across its left value,
and validateEither was returning a string, we were forcing all our
errors to strings. In the case of an API error, however, this meant a
loss of data, since the original error's extra fields were lost.
By returning an Error from validateEither, we can now pass through Api
errors from useAsync and thus use them directly in kibana utilities like
toasts.addError.
* WIP: implements checking for and consequent creation of lists index
This adds most of the machinery that I think we're going to need. Not
featured here:
* lists privileges (stubbed out currently)
* handling when lists is disabled
* tests
* Add frontend plugin for lists
We need this to deteremine in security_solution whether lists is enabled
or not. There's no other functionality here, just boilerplate.
* Fix cross-plugin imports/exports
Now that lists has a client plugin, the optimizer cares about code
coming into and out of it.
By default, you cannot import another plugin's common/ folder into your
own common/ nor public/ folders. This is fixed by adding 'common' to
extraPublicDirs, however: extraPublicDirs need to resolve to modules.
Rather than adding each folder from which we export modules to
extraPublicDirs, I've added common/index.ts and exporting everything
through there.
By convention, I'm adding shared_exports.ts as an index of these exported modules,
and shared_imports.ts is used to import on the other end.
For now, I've left the ad hoc _deps files so as to limit the changes
here, but we should come back through and remove them at some point. NB
that I did remove lists_common_deps as it was only used in one or two
spots.
* Fix test failing due to lack of context
This component now uses useKibana indirectly through useListsConfig.
* Lists and securitySolution require each other's bundles
Without lists being a requiredBundle of securitySolution, we cannot
import its code when the plugin is disabled. The opposite is also true,
but there's no lists "app" to break.
* Fix logic in useListsConfig
Lists needs configuration if the index explicitly does not exist. If it
is true (already exists) or null (lists is disabled or we could not read
the index), we're good.
* useList* behavior when lists plugin is disabled
When the lists plugin is disabled, our calls in useListsIndex become no-ops so that:
* useListsIndex state does not change
* useListsConfig.needsConfiguration remains false as indexExists is
never non-null
This also removes use of our `useIsMounted` hook. Since the effects
we're consuming come from useAsync hooks, state will (already) not be
updated if the component is unmounted.
* Fix warning due to dynamic creation of a styled component
* Revert "Fix warning due to dynamic creation of a styled component"
This reverts commit 7124a8fbd9.
(This was already fixed on master)
* Check user's lists index privileges when determining configuration status
If there is no lists index and the user cannot create it, we will
display a configuration message in lieu of Detections
* Adds a lists hook to read privileges (missing schemae)
* Adds security hook useListsPrivileges to perform and parse the
privileges request
* Updates useListsConfig to use useListsPrivileges hook
* Move lists hooks to their own subfolder
* Redirect to main detections page if lists needs configuration
If:
* lists are enabled, and
* lists indexes DNE, and
* user cannot manage the lists indexes
Then they will be redirected to the main detections page where they'll
be instructed to configure detections. If any of the above is false,
things work as normal.
* Lock out of detections when user cannot write to value lists
Rather than add conditional logic to all our UI components dealing with
lists, we're going the heavy-handed route for now.
* Mock lists config hook in relevant Detections page tests
* Disable Detections when Lists is enabled
This refactors useListsConfig.needsConfiguration to mean:
* lists plugin is disabled, OR
* lists indexes DNE and can't be created, OR,
* user can't write to the lists index
In any of these situations, we want to disable detections, and so we
export that as a single boolean, needsConfiguration.
* Remove unneeded complexity exception
We refactored this to work 👍
* Remove outdated TODO
We link to our documentation, which will describe the lists aspects of
configuration.
* [Data Plugin] Allow server-side date formatters to accept custom timezone
When Advanced Settings shows the date format timezone to be "Browser,"
this means nothing to field formatters in the server-side context. The
field formatters need a way to accept custom format parameters. This
allows a server-side module that creates a FieldFormatMap to set a
timezone as a custom parameter. When custom formatting parameters exist,
they get combined with the defaults.
* add more to tests - need help though
* simplify changes
* api doc changes
* fix src/plugins/data/public/field_formats/constants.ts
* rerun api changes
* re-use public code in server, add test
* fix path for tests
* weird api change needed but no real diff
* 3td time api doc chagens
* move shared code to common
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This modifies the ML job setup flyout of the anomalies tab to offer a list of the two available modules. Via the list each of the modules' jobs can be created or re-created.
* Add simple variables to Canvas workpads
* Fix type for workpad variable action and clarify comment
* Fix types in fixtures and templates
* Fixing type check errors on actions
* Addressing pr feedback and refactoring canvas sidebar accordions
* Render true/false instead of Yes/no on variables
* add warning callout when editing a variable
* Address review feedback
* More feedback
* updating storyshot with new edit mode callout
* Some animation tweaks for the panel
* one more panel tweak
* Removing the slide transition for now
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* rm index: false from binary mappings
* test against unverified snapshot
* two more
* Mapping adjustments
* Revert "Mapping adjustments"
This reverts commit 52d68dcd6d.
* Revert "test against unverified snapshot"
This reverts commit 4284ac37f1.
Co-authored-by: Madison Caldwell <madison.caldwell@elastic.co>
