* Remove legacydetection rule stat summaries
* Remove ML usage summary and consolidate with ML metric telemetry.
* Remove ML usage summary and consolidate with ML metric telemetry.
* Move legacy helper constructs into index.
* Separate rule logic from ml logic. Add ml unit tests.
* Abstract types away into their own file.
* Update telemetry schema.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR starts the migration of the Security Solution rules to use the rule-registry introduced in https://github.com/elastic/kibana/pull/95903. This is a pathfinding effort in porting over the existing Security Solution rules, and may include some temporary reference rules for testing out different paradigms as we move the rules over. See https://github.com/elastic/kibana/issues/95735 for details
Enable via the following feature flags in your `kibana.dev.yml`:
```
# Security Solution Rules on Rule Registry
xpack.ruleRegistry.index: '.kibana-[USERNAME]-alerts' # Only necessary to scope from other devs testing, if not specified defaults to `.alerts-security-solution`
xpack.securitySolution.enableExperimental: ['ruleRegistryEnabled']
```
> Note: if setting a custom `xpack.ruleRegistry.index`, for the time being you must also update the [DEFAULT_ALERTS_INDEX](9e213fb7a5/x-pack/plugins/security_solution/common/constants.ts (L28)) in order for the UI to display alerts within the alerts table.
---
Three reference rule types have been added (`query`, `eql`, `threshold`), along with scripts for creating them located in:
```
x-pack/plugins/security_solution/server/lib/detection_engine/reference_rules/scripts/
```
Main Detection page TGrid queries have been short-circuited to query `.alerts-security-solution*` for displaying alerts from the new alerts as data indices.
To test, checkout, enable the above feature flag(s), and run one of the scripts from the above directory, e.g. `./create_reference_rule_query.sh` (ensure your ENV vars as set! :)
Alerts as data within the main Detection Page 🎉
<p align="center">
<img width="500" src="https://user-images.githubusercontent.com/2946766/119911768-39cfba00-bf17-11eb-8996-63c0b813fdcc.png" />
</p>
cc @madirey @dgieselaar @pmuellr @yctercero @dhurley14 @marshallmain
# Conflicts:
# x-pack/plugins/security_solution/server/plugin.ts
* Shows event filters card on fleet page
* Uses aggs instead of while loop to retrieve summary data
* Add request and response types in the lists package
* Fixes old import
* Removes old i18n keys
* Removes more old i18n keys
* Use consts for exception lists url and endpoint event filter list id
* Uses event filters service to retrieve summary data
* Fixes addressed pr comments such as changing the route without underscore, adding aggs type, validating response, and more
* Uses useMemo instead of useState to memoize object
* Add new e2e test for summart endpoint
* Handle api errors on event filters and trusted apps summary api calls
* Add api error message to the toast
* Fix wrong i18n key
* Change span tag by react fragment
* Uses styled components instead of modify compontent style directly and small improvements on test -> ts
* Adds curls script for summary route
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: David Sánchez <davidsansol92@gmail.com>
Call `setHeaderActionMenu(undefined)` when the HeaderMenuPortal is unmounted.
Found this line in the docs:
> Calling the handler with `undefined` will unmount the current mount point.
Which we weren't doing before.
Previous behavior:
* Go to /app/observability/alerts
* Click the "View in app" button for an APM alert
* Click back
* Click the "View in app" button for an APM alert
* Get a weird toast error message and the header menu is gone forever
Now:
* Go to /app/observability/alerts
* Click the "View in app" button for an APM alert
* Click back
* Click the "View in app" button for an APM alert
* Get a working header menu
Fixes#97140
Co-authored-by: Nathan L Smith <nathan.smith@elastic.co>
* Taking space id into account when creating email footer link
* Handling undefined space when spaces is disabled
* Handling undefined space when spaces is disabled
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: ymao1 <ying.mao@elastic.co>
* Fetch rule statuses using single aggregation instead of N separate requests
* Optimize _find API and _find_statuses
* Merge alerting framework errors into rule statuses
* Add sortSchema for top hits agg, update terms.order schema
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>
* Remove recommendation of coordinating only node
A documentation should be easy and straight forward and giving the user the easiest possible way to get to a fully functioning cluster.
Running a coordinating only node is a bad idea because:
1. Introduces a single point of failure, especially when it is running on the same host as Kibana.
2. Introduces complexity, because you need to run an additional node.
The easiest way to solve the issue of load balancing is to add multiple hosts in the `Elasticsearch.hosts` array in the `Kibana.yml`. This should be far easier than deploying a coordinating node.
* fixed CI errors, there were references to the deleted `load-balancing-es` https://github.com/elastic/kibana/pull/100632
Co-authored-by: Philipp Kahr <philipp.kahr@elastic.co>
* Changing variable name of cases_count_daily to cases_count_total.
* Taking comments out of tests and reverting tests to previosu state.
* Changing meta description to be more descriptive.
* Changing meta description to be more descriptive.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Claire Burn <80253545+clburn-elastic@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Re-introduces the changes from #100727 which was backed out due to a bug. Changes included:
* Generate random isolation values for endpoint metadata
* Generator for Fleet Actions
* Added creation of actions to the index test data loader
Plus:
* Fix generator `randomBoolean()` to ensure it works with seeded random numbers
* Update resolver snapshots due to additional call to randomizer
Co-authored-by: Paul Tavares <56442535+paul-tavares@users.noreply.github.com>
* Update datafeed_windows_rare_user_type10_remote_login.json
refactor df query to work with newer field values
* Update datafeed_windows_rare_user_type10_remote_login.json
remove event.code test - was failing a test on the build server using the original data b/c this field was not there when the query was first developed.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Craig Chamberlain <randomuserid@users.noreply.github.com>
* [Fleet] Link to docs for Fleet Server and ES hosts
* Fix CN/JP i18n
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Zacqary Adam Xeper <Zacqary@users.noreply.github.com>
* WIP - creating alerting authorization client factory and exposing authorization client on plugin start contract
* Updating alerting feature privilege builder to handle different alerting types
* Passing in alerting authorization type to AlertingActions class string builder
* Passing in authorization type in each function call
* Passing in exempt consumer ids. Adding authorization type to audit logger
* Changing alertType to ruleType
* Changing alertType to ruleType
* Updating unit tests
* Updating unit tests
* Passing field names into authorization query builder. Adding kql/es dsl option
* Converting to es query if requested
* Fixing functional tests
* Removing ability to specify feature privilege name in constructor
* Fixing some types and tests
* Consolidating alerting authorization kuery filter options
* Cleanup and tests
* Cleanup and tests
* Initial commit with changes needed for subfeature privilege
* Throwing error when AlertingAuthorizationClientFactory is not defined
* Renaming authorizationType to entity
* Renaming AlertsAuthorization to AlertingAuthorization
* Fixing unit tests
* Changing schema of alerting feature privilege
* Changing schema of alerting feature privilege
* Updating feature privilege iterator
* Updating feature privilege builder
* Fixing types check
* Updating privilege string terminology
* Updating privilege string terminology
* Wip
* Fixing unit tests
* Unit tests
* Updating README and removing stack subfeature privilege changes
* Fixing README
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: ymao1 <ying.mao@elastic.co>
