* added ability to run ephemeral tasks
* fixed typing
* added typing on plugin
* WIP
* Fix type issues
* Hook up the ephemeral task into the task runner for actions
* Tasks can now run independently of one another
* Use deferred language
* Refactor taskParams slightly
* Use Promise.all
* Remove deferred logic
* Add config options to limit the amount of tasks executing at once
* Add ephemeral task monitoring
* WIP
* Add single test so far
* Ensure we log after actions have executed
* Remove confusing * 1
* Add logic to ensure we fallback to default enqueueing if the total actions is above the config
* Add additional test
* Fix tests a bit, ensure we log the alerting:actions-execute right away and the tests should listen for alerts:execute
* Better tests
* If the queue is at capacity, attempt to execute the ephemeral task as a regular action
* Ensure we run ephemeral tasks before to avoid them getting stuck in the queue
* Do not handle the promise anymore
* Remove unnecessary code
* Properly handle errors from ephemeral task lifecycle
* moved acitons domain out of alerting and into actions plugin
* Remove some tests
* Fix TS and test issues
* Fix type issues
* Fix more type issues
* Fix more type issues
* Fix jest tests
* Fix more jest tests
* Off by default
* Fix jest tests
* Update config for this suite too
* Start of telemetry code
* Fix types and add missing files
* Fix telemetry schema
* Fix types
* Fix more types
* moved load event emission to pollingcycle and added health stats on Ephemeral tasks
* Add more telemetry data based on new health metrics for the ephemeral queue
* Fix tests and types
* Add separate request capacity for ephemeral queue
* Fix telemetry schema and add tests for usage collection
* track polled tasks by persistence and use in capacity estimation instead of executions
* fixed typing
* Bump default capacity
* added delay metric to ephemeral stats
* Fix bad merge
* Fix tests
* Fix tests
* Fix types
* Skip failing tests
* Exclude ephemeral stats from capacity estimation tests
* PR feedback
* More PR feedback
* PR feedback
* Fix merge conflict
* Try fixing CI
* Fix broken lock file from merge
* Match master
* Add this back
* PR feedback
* Change to queue and add test
* Disable ephemeral queue in tests
* Updated desc
* Comment out ephemeral-specific tests tha require the entire test suite to support ephemeral tasks
* Add clarifying comment
Co-authored-by: Gidi Meir Morris <github@gidi.io>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
resolves https://github.com/elastic/kibana/issues/99225
Prior to this PR, when an alerting connection action was executed, the event
log document generated did not contain a reference to the originating rule.
This makes it difficult to diagnose problems with connector errors, since
the error is often in the parameters specified in the actions in the alert.
In this PR, a reference to the alerting rule is added to the saved_objects
field in the event document for these events.
* bump @elastic/elasticsearch to canary.7
* address errors in core
* address errors in data plugin
* address errors in Alerting team plugins
* remove outdated messages in Lens
* remove unnecessary comments in ML
* address errors in Observability plugin
* address errors in reporting plugin
* address errors in Rule registry plugin
* fix errors in Security plugins
* fix errors in ES-UI plugin
* remove unnecessary union.
* update core tests
* fix kbn-es-archiver
* update to canary 8
* bump to v9
* use new typings
* fix new errors in core
* fix errors in core typeings
* fix type errors in data plugin
* fix type errors in telemetray plugin
* fix data plugin tests
* fix search examples type error
* fix errors in discover plugin
* fix errors in index_pattern_management
* fix type errors in vis_type_*
* fix errors in typings/elasticsearch
* fix type errors in actions plugin
* fix type errors in alerting and apm plugins
* fix type errors in canvas and cases
* fix errors in event_log
* fix type errors in ILM and ingest_pipelines
* fix errors in lens plugin
* fix errors in lists plugin
* fix errors in logstash
* fix errors in metrics_entities
* fix errors in o11y
* fix errors in watcher
* fix errors in uptime
* fix errors in upgrade_assistant
* fix errors in task_manager
* fix errors in stack_alerts
* fix errors in security_solution
* fix errors in rule_registry
* fix errors in snapshot_restore
* fix remaining errors
* fix search intergration tests
* adjust assetion
* bump version to canary.10
* adapt code to new naming schema
* use mapping types provided by the client library
* Revert "adjust assetion"
This reverts commit 19b8fe0464.
* fix so intergration tests
* fix http integration tests
* bump version to canary 11
* fix login test
* fix http integration test
* fix apm test
* update docs
* fixing some ml types
* fix new errors in data plugin
* fix new errors in alerting plugin
* fix new errors in lists plugin
* fix new errors in reporting
* fix or mute errors in rule_registry plugin
* more ML type fixes
* bump to canary 12
* fix errors after merge conflict
* additional ML fixes
* bump to canary 13
* fix errors in apm plugin
* fix errors in fleet plugin
* fix errors in infra plugin
* fix errors in monitoring plugin
* fix errors in osquery plugin
* fix errors in security solution plugins
* fix errors in transform plugin
* Update type imports for ES
* fix errors in x-pack plugins
* fix errors in tests
* update docs
* fix errors in x-pack/test
* update error description
* fix errors after master merge
* update comment in infra plugin
* fix new errors on xpack tests/
Co-authored-by: James Gowdy <jgowdy@elastic.co>
Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co>
* [Connectors][API] Updated connectors with enabledAfterImport flag
* fixed functional tests
* added new field to connectors API docs
* added update unit test
* fixed test
* renamed enableAfterImport to isMissingSecrets
* removed onExport
* revert the logic of true/false for isMissingSecrets
* fixed test
* fixed tests
* added unit test
* fixed docs
* fixed import text and button labels
* fixed import text
* fixed text
* Use client from branch
* Get type checking working in core
* Fix types in other plugins
* Update client types + remove type errors from core
* migrate Task Manager Elasticsearch typing from legacy library to client library
* use SortOrder instead o string in alerts
* Update client types + fix core type issues
* fix maps ts errors
* Update Lens types
* Convert Search Profiler body from a string to an object to conform to SearchRequest type.
* Fix SOT types
* Fix/mute Security/Spaces plugins type errors.
* Fix bootstrap types
* Fix painless_lab
* corrected es typing in Event Log
* Use new types from client for inferred search responses
* Latest type defs
* Integrate latest type defs for APM/UX
* fix core errors
* fix telemetry errors
* fix canvas errors
* fix data_enhanced errors
* fix event_log errors
* mute lens errors
* fix or mute maps errors
* fix reporting errors
* fix security errors
* mute errors in task_manager
* fix errors in telemetry_collection_xpack
* fix errors in data plugins
* fix errors in alerts
* mute errors in index_management
* fix task_manager errors
* mute or fix lens errors
* fix upgrade_assistant errors
* fix or mute errors in index_lifecycle_management
* fix discover errors
* fix core tests
* ML changes
* fix core type errors
* mute error in kbn-es-archiver
* fix error in data plugin
* fix error in telemetry plugin
* fix error in discover
* fix discover errors
* fix errors in task_manager
* fix security errors
* fix wrong conflict resolution
* address errors with upstream code
* update deps to the last commit
* remove outdated comments
* fix core errors
* fix errors after update
* adding more expect errors to ML
* pull the lastest changes
* fix core errors
* fix errors in infra plugin
* fix errors in uptime plugin
* fix errors in ml
* fix errors in xpack telemetry
* fix or mute errors in transform
* fix errors in upgrade assistant
* fix or mute fleet errors
* start fixing apm errors
* fix errors in osquery
* fix telemetry tests
* core cleanup
* fix asMutableArray imports
* cleanup
* data_enhanced cleanup
* cleanup events_log
* cleaup
* fix error in kbn-es-archiver
* fix errors in kbn-es-archiver
* fix errors in kbn-es-archiver
* fix ES typings for Hit
* fix SO
* fix actions plugin
* fix fleet
* fix maps
* fix stack_alerts
* fix eslint problems
* fix event_log unit tests
* fix failures in data_enhanced tests
* fix test failure in kbn-es-archiver
* fix test failures in index_pattern_management
* fixing ML test
* remove outdated comment in kbn-es-archiver
* fix error type in ml
* fix eslint errors in osquery plugin
* fix runtime error in infra plugin
* revert changes to event_log cluser exist check
* fix eslint error in osquery
* fixing ML endpoint argument types
* fx types
* Update api-extractor docs
* attempt fix for ese test
* Fix lint error
* Fix types for ts refs
* Fix data_enhanced unit test
* fix lens types
* generate docs
* Fix a number of type issues in monitoring and ml
* fix triggers_actions_ui
* Fix ILM functional test
* Put search.d.ts typings back
* fix data plugin
* Update typings in typings/elasticsearch
* Update snapshots
* mute errors in task_manager
* mute fleet errors
* lens. remove unnecessary ts-expect-errors
* fix errors in stack_alerts
* mute errors in osquery
* fix errors in security_solution
* fix errors in lists
* fix errors in cases
* mute errors in search_examples
* use KibanaClient to enforce promise-based API
* fix errors in test/ folder
* update comment
* fix errors in x-pack/test folder
* fix errors in ml plugin
* fix optional fields in ml api_integartoon tests
* fix another casting problem in ml tests
* fix another ml test failure
* fix fleet problem after conflict resolution
* rollback changes in security_solution. trying to fix test
* Update type for discover rows
* uncomment runtime_mappings as its outdated
* address comments from Wylie
* remove eslint error due to any
* mute error due to incompatibility
* Apply suggestions from code review
Co-authored-by: John Schulz <github.com@jfsiii.org>
* fix type error in lens tests
* Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.ts
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
* Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
* update deps
* fix errors in core types
* fix errors for the new elastic/elasticsearch version
* remove unused type
* remove unnecessary manual type cast and put optional chaining back
* ML: mute Datafeed is missing indices_options
* Apply suggestions from code review
Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com>
* use canary pacakge instead of git commit
Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
Co-authored-by: Nathan Reese <reese.nathan@gmail.com>
Co-authored-by: Wylie Conlon <wylieconlon@gmail.com>
Co-authored-by: CJ Cenizal <cj@cenizal.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co>
Co-authored-by: restrry <restrry@gmail.com>
Co-authored-by: James Gowdy <jgowdy@elastic.co>
Co-authored-by: John Schulz <github.com@jfsiii.org>
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
* moved legacy actions api to legacy folder
* introduced connector create api
* added new delete route
* added new execute and get_all
* introduced all connector APIs
* renamed action to connector in Apis
* comment on camel case type
* fixed va
* updated docs
* legacy title
* corrected APIs
* legacy links
* added linik to deprecatred APIs
* added linik to deprecatred APIs from index
* moved legacy apis down one level
* Apply suggestions from code review
Co-authored-by: ymao1 <ying.mao@elastic.co>
* renamed route file for connectorTypesRoute
* define legacy route
* Update docs/api/actions-and-connectors/legacy/index.asciidoc
Co-authored-by: Mike Côté <mikecote@users.noreply.github.com>
* api docs
Co-authored-by: ymao1 <ying.mao@elastic.co>
Co-authored-by: Mike Côté <mikecote@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Initial work
* Fix type check and jest failures
* Add unit tests
* No need to notifyUsage from alert execution handler
* Fix ESLint
* Log action usage from alerts
* Add integration tests
* Fix jest test
* Skip feature usage of basic action types
* Fix types
* Fix ESLint issue
* Clarify comment
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR changes the Alerts & Actions clients to ensure they require full updates (rather than partial) to SOs and overwrites the entire document when making the update.
This is to prevent the situation where nested objects get _merged_ instead of replaced when a user makes an `update`.
We also enhanced the EncryptedSavedObjectsClient to allow specified `id`s when overwriting an existing object.
Marks all Alerts with a `versionApiKeyLastmodified ` field that tracks what version the alert's Api Key was last updated in. We then use this field to exempt legacy alerts (created pre `7.10.0`) in order to use a _dialed down_ version of RBAC which should allow old alerts to continue to function after the upgrade, until they are updates (at which point they will no longer be **Legacy**).
More details here: https://github.com/elastic/kibana/issues/74858#issuecomment-688324039
This PR adds _Role Based Access-Control_ to the Alerting framework & Actions feature using Kibana Feature Controls, addressing most of the Meta issue: https://github.com/elastic/kibana/issues/43994
This also closes https://github.com/elastic/kibana/issues/62438
This PR includes the following:
1. Adds `alerting` specific Security Actions (not to be confused with Alerting Actions) to the `security` plugin which allows us to assign alerting specific privileges to users of other plugins using the `features` plugin.
2. Removes the security wrapper from the savedObjectsClient in AlertsClient and instead plugs in the new AlertsAuthorization which performs the privilege checks on each api call made to the AlertsClient.
3. Adds privileges in each plugin that is already using the Alerting Framework which mirror (as closely as possible) the existing api-level tag-based privileges and plugs them into the AlertsClient.
4. Adds feature granted privileges arounds Actions (by relying on Saved Object privileges under the hood) and plugs them into the ActionsClient
5. Removes the legacy api-level tag-based privilege system from both the Alerts and Action HTTP APIs
* Draft ServiceNow generic implementation
* simple working servicenow incident per alert
* fixed running times
* rely on externalId for update incident on the next execution
* Added consumer to the action type to be able to split ServiceNow for Cases and Alerts
* Added subActions support for ServiceNow action form
* Basic version for Alerts part for ServiceNow
* Keep Case ServiceNow functionality working
* Revert changes on app_router
* Fixed type checks
* Fixed language check issues
* Fixed actions unit tests
* Fixed functional tests
* Fixed jest tests
* fixed tests
* Copied case mappings to alerting plugin
* made consumer optional
* Cleanup tests
* more cleanup
* Fixed jest tests and type checks
* fixed tests
* fixed servicenow validation tests
* Added ServiceNow unit tests
* Removed consumer for actions
* fixed client side isCaseOwned support
* fixed failing tests
* fixed jest tests
* Fixed URL validation
* fixed due to comments
* fixed tests
* fixed jest tests
* Fixed due to comments. Moved ServiceNow filtering in case plugin to server side
* fixed mock for ServiceNow
* fixed consumer config
* fixed test
* fixed type check
* Fixed jest test
* fixed type check
* move last snapshot to inline
* move legacy files to legacy subfolder
* move request types out of legacy
* export Headers from http instead of elasticsearch
* renaming - first pass
* renaming - second pass
* fix core mocks
* adapt new calls
* update generated doc
* fix IT test mocks
* fix new usages
This PR adds an API to the AlertsClient which allows users to execute actions immediately, rather than via TaskManager.
This also moves the TaskManager based execution to a new api on ActionsClient along side the immediate execution.
This removes unneeded use of `any` throughout:
1. alerting
2. alerting_builtin
3. actions
4. task manager
5. event log
It also adds a linting rule that will prevent us from adding more `any` in the future unless an explicit exemption is made.
* Implemented actions server API for supporting preconfigured connectors defined in kibana.yaml
* Fixed type check
* Fixed due to comments and extended functional tests
* Fixed tests and renamed connectors
* fixed jest tests
* Fixed type checks
* Fixed failing alert save
* Fixed alert client tests
* fixed type checks
* Fixed language check error
* Fixed jest tests
* Added missing comments and docs
* fixed due to comments
* Fixed json config for preconfigured
* fixed type check, reverted config
* config experiment with json stringify
* revert experiment
* Removed the spaces from connector names in config
* Define minimum license required for each action type (#58668)
* Add minimum required license
* Require at least gold license as a minimum license required on third party action types
* Use strings for license references
* Ensure license type is valid
* Fix some tests
* Add servicenow to gold
* Add tests
* Set license requirements on other built in action types
* Use jest.Mocked<ActionType> instead
* Change servicenow to platinum
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Make actions config mock and license state mock use factory pattern and jest mocks (#59370)
* Add license checks to action HTTP APIs (#59153)
* Initial work
* Handle errors in update action API
* Add unit tests for APIs
* Make action executor throw when action type isn't enabled
* Add test suite for basic license
* Fix ESLint errors
* Fix failing tests
* Attempt 1 to fix CI
* ESLint fixes
* Create sendResponse function on ActionTypeDisabledError
* Make disabled action types by config return 403
* Remove switch case
* Fix ESLint
* Add license checks within alerting / actions framework (#59699)
* Initial work
* Handle errors in update action API
* Add unit tests for APIs
* Verify action type before scheduling action task
* Make actions plugin.execute throw error if action type is disabled
* Bug fixes
* Make action executor throw when action type isn't enabled
* Add test suite for basic license
* Fix ESLint errors
* Stop action task from re-running when license check fails
* Fix failing tests
* Attempt 1 to fix CI
* ESLint fixes
* Create sendResponse function on ActionTypeDisabledError
* Make disabled action types by config return 403
* Remove switch case
* Fix ESLint
* Fix confusing assertion
* Add comment explaining double mock
* Log warning when alert action isn't scheduled
* Disable action types in UI when license doesn't support it (#59819)
* Initial work
* Handle errors in update action API
* Add unit tests for APIs
* Verify action type before scheduling action task
* Make actions plugin.execute throw error if action type is disabled
* Bug fixes
* Make action executor throw when action type isn't enabled
* Add test suite for basic license
* Fix ESLint errors
* Stop action task from re-running when license check fails
* Fix failing tests
* Attempt 1 to fix CI
* ESLint fixes
* Return enabledInConfig and enabledInLicense from actions get types API
* Disable cards that have invalid license in create connector flyout
* Create sendResponse function on ActionTypeDisabledError
* Make disabled action types by config return 403
* Remove switch case
* Fix ESLint
* Disable when creating alert action
* Return minimumLicenseRequired in /types API
* Disable row in connectors when action type is disabled
* Fix failing jest test
* Some refactoring
* Card in edit alert flyout
* Sort action types by name
* Add tooltips to create connector action type selector
* Add tooltips to alert flyout action type selector
* Add get more actions link in alert flyout
* Add callout when creating a connector
* Typos
* remove float right and use flexgroup
* replace pixels with eui variables
* turn on sass lint for triggers_actions_ui dir
* trying to add padding around cards
* Add callout in edit alert screen when some actions are disabled
* improve card selection for Add Connector flyout
* Fix cards for create connector
* Add tests
* ESLint issue
* Cleanup
* Cleanup pt2
* Fix type check errors
* moving to 3-columns cards for connector selection
* Change re-enable to enable terminology
* Revert "Change re-enable to enable terminology"
This reverts commit b497dfd6b6.
* Add re-enable comment
* Remove unecessary fragment
* Add type to actionTypeNodes
* Fix EuiLink to not have opacity of 0.7 when not hovered
* design cleanup in progress
* updating classNames
* using EuiIconTip
* Remove label on icon tip
* Fix failing jest test
Co-authored-by: Andrea Del Rio <delrio.andre@gmail.com>
* Add index to .index action type test
* PR feedback
* Add isErrorThatHandlesItsOwnResponse
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Andrea Del Rio <delrio.andre@gmail.com>
