7fd6539dca
**Needed for:** rule execution log for Security https://github.com/elastic/kibana/pull/94143 **Related to:** - alerts-as-data: https://github.com/elastic/kibana/issues/93728, https://github.com/elastic/kibana/issues/93729, https://github.com/elastic/kibana/issues/93730 - RFC for index naming https://github.com/elastic/kibana/issues/98912 ## Summary This PR adds a mechanism for writing to / reading from / bootstrapping indices for RAC project into the `rule_registry` plugin. Particularly, indices for alerts-as-data and rule execution events. This implementation is similar to existing implementations like `event_log` plugin (see https://github.com/elastic/kibana/pull/98353#issuecomment-833045980 for historical perspective), but we're going to converge all of them into 1 or 2 implementations. At least we should have a single one in `rule_registry` itself. In this PR I tried to incorporate most of the feedback received in the RFC (https://github.com/elastic/kibana/issues/98912), but if you notice I missed/forgot something, please let me know in the comments. Done in this PR: - [x] Schema-agnostic APIs for working with Elasticsearch. - [x] Schema-aware log definition and bootstrapping API (creating hierarchical logs). - [x] Schema-aware write API (logging events). - [x] Schema-aware read API (searching logs, filtering, sorting, pagination, aggregation). - [x] Support for Kibana spaces, space-aware index bootstrapping (either at rule creation or rule execution time). As for reviewing this PR, perhaps it might be easier to start with: - checking description of https://github.com/elastic/kibana/issues/98912 - checking usage examples https://github.com/elastic/kibana/pull/98353/files#diff-c049ff2198cc69bd50a69e92d29e88da7e10b9a152bdaceaf3d41826e712c12b - checking public api https://github.com/elastic/kibana/pull/98353/files#diff-8e9ef0dbcbc60b1861d492a03865b2ae76a56ec38ada61898c991d3a74bd6268 ## Next steps Next steps towards rule execution log in Security (https://github.com/elastic/kibana/pull/94143): - define actual schema for rule execution events - inject instance of rule execution log into Security rule executors and route handlers - implement actual execution logging in rule executors - update route handlers to start fetching execution events and metrics from the log instead of custom saved objects Next steps in the context of RAC and unified implementation: - converge this implementation with `RuleDataService` implementation - implement robust index bootstrapping - reconsider using FieldMap as a generic type parameter - implement validation for documents being indexed - cover the final implementation with tests - write comprehensive docs: update plugin README, add JSDoc comments to all public interfaces |
||
|---|---|---|
| .. | ||
| .storybook | ||
| common | ||
| public | ||
| scripts | ||
| server | ||
| typings | ||
| jest.config.js | ||
| kibana.json | ||
| README.md | ||
| tsconfig.json | ||
Observability plugin
This plugin provides shared components and services for use across observability solutions, as well as the observability landing page UI.
Rules, Alerts, and Cases
The Observability plugin contains experimental support for improved alerting and case management.
If you have:
xpack.observability.unsafe.cases.enabled: true
In your Kibana configuration, the Cases page will be available.
If you have:
xpack.observability.unsafe.alertingExperience.enabled: true
In your Kibana configuration, the Alerts page will be available.
This will only enable the UI for this page when. In order to have alert data indexed you'll need to enable writing in the Rule Registry plugin:
xpack.ruleRegistry.write.enabled: true
When both of the these are set to true, your alerts should show on the alerts page.
Shared navigation
The Observability plugin maintains a navigation registry for Observability solutions, and exposes a shared page template component. Please refer to the docs in the component directory for more information on registering your solution's navigation structure, and rendering the navigation via the shared component.
Unit testing
Note: Run the following commands from kibana/x-pack/plugins/observability.
Run unit tests
npx jest --watch
Update snapshots
npx jest --updateSnapshot
Coverage
HTML coverage report can be found in target/coverage/jest after tests have run.
open target/coverage/jest/index.html
API integration testing
API tests are separated in two suites:
- a basic license test suite
- a trial license test suite (the equivalent of gold+)
This requires separate test servers and test runners.
Basic
# Start server
node scripts/functional_tests_server --config x-pack/test/observability_api_integration/basic/config.ts
# Run tests
node scripts/functional_test_runner --config x-pack/test/observability_api_integration/basic/config.ts
The API tests for "basic" are located in x-pack/test/observability_api_integration/basic/tests.
Trial
# Start server
node scripts/functional_tests_server --config x-pack/test/observability_api_integration/trial/config.ts
# Run tests
node scripts/functional_test_runner --config x-pack/test/observability_api_integration/trial/config.ts
The API tests for "trial" are located in x-pack/test/observability_api_integration/trial/tests.
API test tips
- For debugging access Elasticsearch on http://localhost:9220` (elastic/changeme)
- To update snapshots append
--updateSnapshotsto the functional_test_runner command