**Ticket:** https://github.com/elastic/kibana/issues/106469, https://github.com/elastic/kibana/issues/101013
## Summary
TL;DR: New internal endpoint for reading data from Event Log (raw version), legacy status SO under the hood.
With this PR we now read the Failure History (last 5 failures) on the Rule Details page from Event Log. We continue getting the Current Status from the legacy `siem-detection-engine-rule-status` saved objects. Rule Management page also gets data from the legacy saved objects.
- [x] Deprecate existing methods for reading data in `IRuleExecutionLogClient`: `.find()` and `.findBulk()`
- [x] Introduce new methods for reading data in IRuleExecutionLogClient:
- for reading last N execution events for 1 rule from event log
- for reading current status and metrics for 1 rule from legacy status SOs
- for reading current statuses and metrics for N rules from legacy status SOs
- [x] New methods should return data in the legacy status SO format.
- [x] Update all the existing endpoints that depend on `IRuleExecutionLogClient` to use the new methods.
- [x] Implement a new internal endpoint for fetching current status of the rule execution and execution events from Event Log for a given rule.
- [x] The API of the new endpoint should be the same as `rules/_find_statuses` to minimise changes in the app.
- [x] Use the new endpoint on the Rule Details page.
## Near-term plan for technical implementation of the Rule Execution Log (https://github.com/elastic/kibana/issues/101013)
**Stage 1. Reading last 5 failures from Event Log v1 - raw implementation** - ✔️ done in this PR
TL;DR: New internal endpoint for reading data from Event Log (raw version), legacy status SO under the hood.
- Deprecate existing methods for reading data in `IRuleExecutionLogClient`: `.find()` and `.findBulk()`
- Introduce new methods for reading data in IRuleExecutionLogClient:
- for reading last N execution events for 1 rule from event log
- for reading current status and metrics for 1 rule from legacy status SOs
- for reading current statuses and metrics for N rules from legacy status SOs
- New methods should return data in the legacy status SO format.
- Update all the existing endpoints that depend on `IRuleExecutionLogClient` to use the new methods.
- Implement a new internal endpoint for fetching current status of the rule execution and execution events from Event Log for a given rule.
- The API of the new endpoint should be the same as `rules/_find_statuses` to minimise changes in the app.
- Use the new endpoint on the Rule Details page.
**Stage 2: Reading last 5 failures from Event Log v2 - clean implementation**
TL;DR: Clean HTTP API, legacy Rule Status SO under the hood.
🚨🚨🚨 Possible breaking changes in Detections API 🚨🚨🚨
- Design a new data model for the Current Rule Execution Info (the TO-BE new SO type and later the TO-BE data in the rule object itself).
- Design a new data model for the Rule Execution Event (read model to be used on the Rule Details page)
- Think over changes in `IRuleExecutionLogClient` to support the new data model.
- Think over changes in all the endpoints that return any data related to rule monitoring (statuses, metrics, etc). Make sure to check our docs to identify what's documented there regarding rule monitoring.
- Update `IRuleExecutionLogClient` to return data in the new format.
- Update all the endpoints (including the raw new one) to return data in the new format.
- Update Rule Details page to consume data in the new format.
- Update Rule Management page to consume data in the new format.
**Stage 3: Reading last 5 failures from Event Log v3 - new SO**
TL;DR: Clean HTTP API, new Rule Execution Info SO under the hood.
- Implement a new SO type for storing the current rule execution info. Relation type: 1 rule - 1 current execution info.
- Swap the legacy SO with the new SO in the implementation of `IRuleExecutionLogClient`.
**Stage 4: Cleanup and misc**
- Revisit the problem of deterministic ordering ([comment](https://github.com/elastic/kibana/pull/115574#discussion_r735803087))
- Remove rule execution log's glue code: adapters, feature switch.
- Remove the legacy rule status SO.
- Mark the legacy rule status SO as deleted in Kibana Core.
- Encapsulate the current space id in the instance of IRuleExecutionLogClient. Remove it from parameters of its methods.
- Introduce a Rule Execution Logger scoped to a rule instance. For use in rule executors.
- Add test coverage.
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* add tests for pending status api changes
related to elastic/kibana/pull/115441
refs elastic/security-team/issues/1705
* update mock
refs elastic/kibana/pull/116214
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Fix error appearing if user leaves Sources page very quickly
The issue was that the response from /sources endpoint could came after the user has left the page.
Since the user has already left the page, the Sources logic is unmounted,
and any code that was using the response couldn't update the value in that logic file and caused an error.
Fortunately Kea provides a `breakpoint` API exactly for such cases:
https://kea.js.org/docs/guide/additional#breakpoints
This commit uses that API to fix the issue.
* Fix error appearing after leaving Sources page less quickly
This commit solves the same problem, but for the /status endpoint:
1) for the first status call that saves server values to the sources_logic
2) for the subsequent status calls that poll the server for the status updates
* Increase test coverage
The new test duplicates the test below it, but it doesn't set up
initial source statuses. For some reason this case was considered to be covered before,
but after seemingly unrelated changes in this PR, the coverage started to show that this
code branch was missed.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* WIP - need to figure out how to delete old siem-detection action SO's after each test
* WIP - adds some fixes for the update rules utility that differ from patch rules utility
* fix type checks
* cleanup
* remove commented out code
* rename const to use capital snake case
* naming integration tests, adds expect for disabled rules that get migrated, adds expect for pre-migrated rules
## Summary
Removes isUuid and tests as they're not used anymore
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* Use useEndpointPrivileges instead of checking the license directly
* Use the correct privilege key
* rename variable
* Skips flaky test
* Remove skip
* Remove extra dependency
* Add back entries check
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Adds e2e tests for the legacy notification system for:
* Exporting rules
* Reading rules
* Finding rules
Also adds missing e2e tests for the non-legacy actions where they previously did not have e2e tests.
These tests ensure that the legacy notifications system will run for a while.
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* Send Endpoint Alert _id field up as insights docs track that on status changes
* Added test to make sure top-level underscore-prefixed fields are allowed
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* clean up IField API
* disable switch when using MVTs for es docs
* clean up interface comment style
* implement supportsFieldMetaFromEs and supportsFieldMetaFromLocalData in all Field classes
* fix dynamic_color_property test
* fix jest tests
* mock getRangeFieldMeta instead of passing in VectorLayerMock with MockStyle
* review feedback
* clean up supportsFieldMetaFromLocalData test
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* ensure included fields not overwritten + reduce unnecessary renders.
* ensure editor validation works
* ensure depVar always in includes
* ensure selected runtimeField depVar option is shown
