* [Event Log] Extended README.md with the documentation for a REST API and Start plugin contract.
* Apply suggestions from code review
Co-authored-by: Mike Côté <mikecote@users.noreply.github.com>
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
* fixed due to comments
Co-authored-by: Mike Côté <mikecote@users.noreply.github.com>
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
* first pass
* migrate more plugins
* migrate yet more plugins
* more oss plugins
* fix test file
* change Plugin signature on the client-side too
* fix test types
* migrate OSS client-side plugins
* migrate OSS client-side test plugins
* migrate xpack client-side plugins
* revert fix attempt on fleet plugin
* fix presentation start signature
* fix yet another signature
* add warnings for server-side async plugins in dev mode
* remove unused import
* fix isPromise
* Add client-side deprecations
* update migration examples
* update generated doc
* fix xpack unit tests
* nit
* (will be reverted) explicitly await for license to be ready in the auth hook
* Revert "(will be reverted) explicitly await for license to be ready in the auth hook"
This reverts commit fdf73feb
* restore await on on promise contracts
* Revert "(will be reverted) explicitly await for license to be ready in the auth hook"
This reverts commit fdf73feb
* Revert "restore await on on promise contracts"
This reverts commit c5f2fe51
* add delay before starting tests in FTR
* update deprecation ts doc
* add explicit contract for monitoring setup
* migrate monitoring plugin to sync
* change plugin timeout to 10sec
* use delay instead of silence
* [Event Log] Added KQL queries support for Event Log API.
* refactored to use core.elasticsearch.client
* Fixed tests
* removed get index pattern for event log
* Fixed tests
* Fixed due to comments.
* Added alerting API to get all active instances
* modofied event log findEventsBySavedObject to support bulk ids, renamed to findEventsBySavedObjectIds
* fixed faling typechecks
* fixed crash on zpd/api/event_log/alert/84c00970-5130-11eb-9fa7/_find for non existing id
* fixed faling typechecks
* fixed faling typechecks
* fixed due to comments
* fixed due to comments
* fixed failing test
* fixed due to comments
This PR introduces a new concept of an _Action Subgroup_ (naming is open for discussion) which can be used by an Alert Type when scheduling actions.
An Action Subgroup can be dynamically specified, unlike Action Groups which have to be specified on the AlertType definition.
When scheduling actions, and AlertType can specify an _Action Subgroup_ along side the scheduled _Action Group_, which denotes that the alert instance falls into some kind of narrower grouping in the action group.
resolves#55634resolves#65746
Buffers event docs being written for a fixed interval / buffer size,
and indexes those docs via a bulk ES call.
Also now flushing those buffers at plugin stop() time, which
we couldn't do before with the single index calls, which were
run via `setImmediate()`.
This is a redo of PR https://github.com/elastic/kibana/pull/80941 which
had to be reverted.
resolves https://github.com/elastic/kibana/issues/55634
resolves https://github.com/elastic/kibana/issues/65746
Buffers event docs being written for a fixed interval / buffer size,
and indexes those docs via a bulk ES call.
Also now flushing those buffers at plugin stop() time, which
we couldn't do before with the single index calls, which were
run via `setImmediate()`.
* Removing circular dependency between spaces and security
* Apply suggestions from code review
Co-authored-by: Constance <constancecchen@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
* Tests refactor
- Reorganize top level describes into 3 space-based blocks into based on spaces:
- space disabled
- spaces plugin unavailable
- space enabled (most previous tests go under this new block) with new beforeEach
- wrote new tests for uncovered lines 58, 66-69
* Review1: address PR feedback
* changing fake requests for alerts/actions
* Fixing tests
* fixing more tests
* Additional testing and refactoring
* Apply suggestions from code review
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
* Review 2: Address feedback
* Make ESLint happy again
Co-authored-by: Constance <constancecchen@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Constance Chen <constance.chen.3@gmail.com>
resolves https://github.com/elastic/kibana/issues/79785
Until now, the execution status was available in the the event
log document for the execute action. In this PR we add it.
The event log is extended to add the following fields:
- `kibana.alerting.status` - from executionStatus.status
- `event.reason` - from executionStatus.error.reason
The date from the executionStatus and start date in the event
log will be set to the same value.
Previously, errors encountered while trying to execute an
alert executor, eg decrypting the alert, would not end up
with an event doc generated. Now they will.
In addition, there were a few places where events that could
have had the action group in them did not, and one where the
instance id was undefined - those were fixed up.
* Adding action group id to event log. Showing action group as part of status in alert details view
* Simplifying getting action group id
* Cleanup
* Adding unit tests
* Updating functional tests
* Updating test
* Fix types check
* Updating test
* PR fixes
* PR fixes
* rename uuid service to environment service
* adapt resolve_uuid to directly use the configurations
* move data folder creation to core
* update generated doc
* fix types
* fix monitoring tests
* move instanceUuid to plugin initializer context
* update generated doc
Introduces a pluggable API to Event Log which allows custom Providers for Saved Objects which is used to ensure a user is authorised to get the Saved Object referenced in the Event Log whenever the find api is called.
resolves https://github.com/elastic/kibana/issues/70086
Configures the saved object client for the event log to access the recently
hidden action and alert saved objects.
We didn't have tests for action/alert event log activity, so added some now.
Also found a buglet that was preventing access to event log data from actions
and alerts in non-default spaces.
resolves https://github.com/elastic/kibana/issues/68309
Previously, if the initialization of the elasticsearch resources failed
during initialization, the event logger would still try to write events.
Which is somewhat of a catastrophic failure, as typically the logger would
try writing to the alias name, but no alias exists, so a new index would
be created with the name of the alias. Making it impossible to initialize
successfully later until that index was deleted.
The core initialization calls already returned success indicators, so this
PR just responds to those and prevents the logger from writing to the index
if initialization failed.
* chore(NA): upgrade oss to lodash4
chore(NA): migrate cli, cli_plugin, cli_keystore, dev, test_utils and apm src script to lodash4
chore(NA): missing file for cli plugin
chore(NA): add src core
chore(NA): es archiver and fixtures
chore(NA): try to fix functional test failure
chore(NA): migrate src/legacy entirely to lodash4 except src/legacy/core_plugins
chore(NA): move legacy core plugins to lodash4
chore(NA): upgrade optimize to lodash4
chore(NA): upgrade to lodash4 on advanced_settings, charts, console and dashboard
chore(NA): migrate to lodash4 on dev_tools, discover, embeddable, es_ui)shared, expressions, home plugins
chore(NA): upgrade data plugin to lodash4
chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4
chore(NA): missing data upgrades to lodash4
Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4"
This reverts commit 137055c5fed2fc52bb26547e0bc1ad2e3d4fe309.
Revert "Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4""
This reverts commit f7e73688782998513d9fb6d7e8f0765e9beb28d1.
Revert "chore(NA): missing data upgrades to lodash4"
This reverts commit 92b85bf947a89bfc70cc4052738a6b2128ffb076.
Revert "chore(NA): upgrade data plugin to lodash4"
This reverts commit 88fdb075ee1e26c4ac979b6681d8a2b002df74c6.
chore(NA): upgrade idx_pattern_mgt, input_control_vis, inspector, kbn_legacy, kbn_react, kbn_usage_collections, kbn_utils, management and maps_legacy to lodash4
chore(NA): map src plugin data to lodash3
chore(NA): missing lodash.clonedeep dep
chore(NA): change packages kbn-config-schema deps
chore(NA): update renovate config
chore(NA): upgrade vis_type plugins to lodash4
chore(NA): move vis_type_vislib to lodash3
chore(NA): update visualizations and visualize to lodash4
chore(NA): remove lodash 3 types from src and move test to lodash4
chore(NA): move home, usage_collection and management to lodash 3
Revert "chore(NA): move home, usage_collection and management to lodash 3"
This reverts commit f86e8585f02d21550746569af54215b076a79a3d.
chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3
chore(NA): update x-pack test to mock lodash4
Revert "chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3"
This reverts commit 2d10fe450533e1b36db21d99cfae3ce996a244e0.
* chore(NA): move x-pack and packages to lodash 4
* chore(NA): remove mention to lodash from main package.json
* chore(NA): remove helper alias for lodash4 and make it the default lodash
* chore(NA): fix last failing types in the repo
* chore(NA): fix public api
* chore(NA): fix types for agg_row.tsx
* chore(NA): fix increment of optimizer modules in the rollup plugin
* chore(NA): migrate `src/core/public/http/fetch.ts` (#5)
* omit undefined query props
* just remove merge usage
* fix types
* chore(NA): fixes for feedback from apm team
* chore(NA): recover old behaviour on apm LoadingIndeicatorContext.tsx
* chore(NA): fixes for feedback from watson
* Platform lodash4 tweaks (#6)
* chore(NA): fix types and behaviour on src/core/server/elasticsearch/errors.ts
* Canvas fixes for lodash upgrade
* [APM] Adds unit test for APM service maps transform (#7)
* Adds a snapshot unit test for getConnections and rearranges some code to make testing easier
* reverts `ArrayList` back to `String[]` in the painless script within `fetch_service_paths_from_trace_ids.ts`
* chore(NA): update yarn.lock
* chore(NA): remove any and use a real type for alerts task runner
Co-authored-by: Gidi Meir Morris <github@gidi.io>
* chore(NA): used named import for triggers_actions_ui file
* chore(NA): fix eslint
* chore(NA): fix types
* Delete most uptime lodash references.
* Simplify. Clean up types.
* [Uptime] Delete most uptime lodash references (#8)
* Delete most uptime lodash references.
* Simplify. Clean up types.
* chore(NA): add eslint rule to avoid using lodash3
* chore(NA): apply changes on feedback from es-ui team
* fix some types (#9)
* Clean up some expressions types.
* chore(NA): missing ts-expect-error statements
* Upgrade lodash 4 vislib (#11)
* replace lodash 3 with lodash 4 on vislib plugin
* Further changes
* further replacement of lodash3 to 4
* further work on upgrading to lodash 4
* final changes to update lodash
* chore(NA): upgrade data plugin to lodash4
chore(NA): upgrade data plugin public to lodash4
chore(NA): fix typecheck task
chore(NA): fix agg_config with hasIn
chore(NA): assign to assignIn and has to hasIn
chore(NA): upgrade data plugin server to lodash4
chore(NA): new signature for core api
fix(NA): match behaviour between lodash3 and lodash4 for set in search_source
* chore(NA): remove lodash3 completely from the repo
* chore(NA): fix x-pack/test/api_integration/apis/metrics_ui/snapshot.ts missing content
* chore(NA): fix lodash usage on apm
* chore(NA): fix typecheck for maps
* Patch lodash template (#12)
* Applying changes from https://github.com/elastic/kibana/pull/64985
* Using isIterateeCall, because it seems less brittle
* Also patching `lodash/template` and `lodash/fp/template`
* Reorganizing some files...
* Revising comment
* Ends up `_` is a function also... I hate JavaScript
Co-authored-by: Pierre Gayvallet <pierre.gayvallet@gmail.com>
Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: Clint Andrew Hall <clint.hall@elastic.co>
Co-authored-by: Oliver Gupte <ogupte@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
Co-authored-by: Justin Kambic <justin.kambic@elastic.co>
Co-authored-by: Stratoula Kalafateli <stratoula1@gmail.com>
Co-authored-by: Luke Elmers <luke.elmers@elastic.co>
Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com>
Co-authored-by: kobelb <brandon.kobel@elastic.co>
* move last snapshot to inline
* move legacy files to legacy subfolder
* move request types out of legacy
* export Headers from http instead of elasticsearch
* renaming - first pass
* renaming - second pass
* fix core mocks
* adapt new calls
* update generated doc
* fix IT test mocks
* fix new usages
resolves https://github.com/elastic/kibana/issues/68265
This changes the ILM requests made by the eventLog from relative to absolute
URLs. These requests test the existence of and create ILM policies, and are
made with a cluster client using `transport.request`. Relative URLs work fine
locally and in CI, however do not work on the cloud.
resolves https://github.com/elastic/kibana/issues/62668
Adds a property named `rel` to the nested saved objects in the event
documents, whose value should not be set, or set to `primary`.
The query by saved object function changes to only match event documents
with that saved objects if it has the `rel: primary` value.
This is used to limit searching alerting's executeAction event document
with only the alert saved object, and not the action saved object (this
document has an alert and action saved object). The alert saved object
has the `rel: primary` field set, and the action does not. Previously,
those documents were returned with a query of the action saved object.
resolves https://github.com/elastic/kibana/issues/64275
Changes the fields used to query the event log by time range to use the
`@timestamp` field.
Also allow `@timestamp` as a sort option, and make it the default sort option.
resolves https://github.com/elastic/kibana/issues/61891
Adds a relatively new ECS field `event.outcome`. Value of `success`, `failure`,
or `unknown`. This is nice, as the only way we have currently of determining an
error for an alert or action execution in the log is the existence of an
`error.message` field. It is added to to the documents for those events.
see: https://www.elastic.co/guide/en/ecs/current/ecs-event.html
This removes unneeded use of `any` throughout:
1. alerting
2. alerting_builtin
3. actions
4. task manager
5. event log
It also adds a linting rule that will prevent us from adding more `any` in the future unless an explicit exemption is made.
Major cleanup of the no_restricted_paths rule for imports of core.
For relative imports, we use eslint-module-utils/resolve which resolves
to the full filesystem path. So, to support relative and absolute
imports from the src alias we need to define both the directory and the
index including file extension.
This rule was handling both core imports, as well as imports from other
plugins. Imports from other plugins are being used much more liberally
allowed through the exceptions in tests. I choose to break these up,
removing this exception for tests for core imports.
Fixes:
Absolute imports of src/core/server/mocks were not allowed in src. This
was not an issue in x-pack due to the target excluding
!x-pack/**/*.test.* and !x-pack/test/**/*.
Non-top-level public and server imports were allowed from X-Pack tests
to the previously mentioned exclusion.
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
* Added server api tests for event log service
* fixed tests
* fixed type check issue
* Fixed failing tests
* fixed jest tests
* Fixed due to comments
* Removed flackiness tests
* fixed type check error
* Fixed func test
* x-pack/watcher: use Elasticsearch from CoreStart
* x-pack/upgrade_assistant: use Elasticsearch from CoreStart
* x-pack/actions: use Elasticsearch from CoreStart
* x-pack/alerting: use Elasticsearch from CoreStart
* x-pack/lens: use Elasticsearch from CoreStart
* expressions: use Elasticsearch from CoreStart
* x-pack/remote_clusters: remove unused Elasticsearch dependency on CoreSetup
* x-pack/oss_telemetry: use Elasticsearch from CoreStart
* Cleanup after #59886
* x-pack/watcher: create custom client only once
* Revert "x-pack/watcher: create custom client only once"
This reverts commit 78fc4d2e93.
* Revert "x-pack/watcher: use Elasticsearch from CoreStart"
This reverts commit b621af9388.
* x-pack/task_manager: use Elasticsearch from CoreStart
* x-pack/event_log: use Elasticsearch from CoreStart
* x-pack/alerting: use Elasticsearch from CoreStart
* x-pack/apm: use Elasticsearch from CoreStart
* x-pack/actions: use Elasticsearch from CoreStart
* PR Feedback
* APM review nits
* Remove unused variable
* Remove unused variable
* x-pack/apm: better typesafety
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Adds a namespace attribute to the saved object object within the Event Log so that each Saved Object can have its own. This change also removes the existing kibana.namespace field.
As Event Log is not yet in use, this does not include a migration.
resolves https://github.com/elastic/kibana/issues/58518
Changes the object properties in the eventLog mappings to use `dynamic: false`
instead of `dynamic: strict`. This provides a bit of a safety net for cases
where the mappings change during development, or potentially in production
cases. Rather than completely lose entire events and see errors in logs, we'll
silently drop properties from events.
While researching the `dynamic` property, I remember there's also a `meta`
property available, which is a convenient place to drop our meta-data about
a field being an array value, for internal processing. Bonus, it can live in
the generated mappings as well.
references:
- https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-field-meta.html
- https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic.html
