* bump version to 4.1.1-rc
* fix code to run kbn bootstrap
* fix errors
* DO NOT MERGE. mute errors and ping teams to fix them
* Address EuiSelectableProps configuration in discover sidebar
* use explicit type for EuiSelectable
* update to ts v4.1.2
* fix ts error in EuiSelectable
* update docs
* update prettier with ts version support
* Revert "update prettier with ts version support"
This reverts commit 3de48db3ec.
* address another new problem
Co-authored-by: Chandler Prall <chandler.prall@gmail.com>
This addresses a bug in Task Manager in the task timeout behaviour. When a recurring task's `retryAt` field is set (which happens at task run), it is currently scheduled to the task definition's `timeout` value, but the original intention was for these tasks to retry on their next scheduled run (originally identified as part of https://github.com/elastic/kibana/issues/39349).
In this PR we ensure recurring task retries are scheduled according to their recurring schedule, rather than the default `timeout` of the task type.
...and into x-pack.
Also remove `PromiseReturnType` from APM and use the copy in observability everywhere.
All of the additional changes to APM imports are just automatic sorting.
This makes doing #77720 a little easier and removes some implicit circular dependencies for #80508.
Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co>
* spiked intervals in alerts
* ensure scheduled tasks dont get wiped
* Fixed type checks and unit tests
* Added simple test, which only covers successful case when edit happened right after task was complete previous execution
* fixed jest
* fallback to existing task schedule when possible
* added missing test
* Added support for day and hour schedule interval values
* added docs for new schedule run result
* fixed doc
* added UnrecoverableError support for task runners nad pluged it into alerting where needed
* typo
Co-authored-by: Yuliia Naumenko <yuliia.naumenko@elastic.com>
Plugs the Task Manager polling lifecycle into the Kibana Services Status streams in order to ensure we reactively start and stop polling whenever the Elasticsearch or SavedObjects service switch between `available` and `unavailable`.
This will prevent Task Manager from polling whenever these services switch to an `unavailable` state.
This PR adds an an internal monitoring mechanism in Task Manager which keep track of a variety of metrics and a health api endpoint which makes the monitored statistics accessible.
* wip
* Adding updateFieldsAndMarkAsFailed function
* Updating UBQ
* Only updating retryAt if marking as claiming
* Updating query
* Updating query to only fail one time tasks that have exceeded max attempts
* Fixing tests
* Fixing tests
* Handling claiming tasks by id
* Removing unused function
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
resolves: https://github.com/elastic/kibana/issues/80371
Previously, when claiming tasks, we were always sorting the tasks to claim by
the score and then by the time they should be run. We sort by score to
capture `runNow()` tasks, also referred to internally as "pinned" tasks
in the update by query.
The change in this PR is to only sort by score if there are pinned tasks, and
to not sort by score at all if there aren't any.
This PR addresses a list of legacy code debt the plugin has incurred over the past year due to extensive changes in its internals and the adoption of the Kibana Platform.
It includes:
1. The `TaskManager` class has been split into several independent components: `TaskTypeDictionary`, `TaskPollingLifecycle`, `TaskScheduling`, `Middleware`. This has made it easier to understand the roles of the different parts and makes it easier to plug them into the observability work.
2. The exposed `mocks` have been corrected to correctly express the Kibana Platform api
3. The lifecycle has been corrected to remove the need for intermediary streames/promises which we're needed when we first introduced the `setup`/`start` lifecycle to support legacy.
4. The Logger mocks have been replaced with the platform's `coreMocks` implementation
5. The integration tests now test the plugin's actual public api (instead of the internals).
6. The Legacy Elasticsearch client has been replaced with the typed client in response to the deprecation notice.
7. Typing has been narrowed to prevent the `type` field from conflicting with the key in the `TaskDictionary`. This could have caused the displayed `type` on a task to differ from the `type` used in the Dictionary itself (this broke a test during refactoring and could have caused a bug in production code if left).
* Make task manager maxWorkers and pollInterval observables (#75293)
* WIP step 1
* WIP step 2
* Cleanup
* Make maxWorkers an observable for the task pool
* Cleanup
* Fix test failures
* Use BehaviorSubject
* Add some tests
* Make the task manager store emit error events (#75679)
* Add errors$ observable to the task store
* Add unit tests
* Temporarily apply back pressure to maxWorkers and pollInterval when 429 errors occur (#77096)
* WIP
* Cleanup
* Add error count to message
* Reset observable values on stop
* Add comments
* Fix issues when changing configurations
* Cleanup code
* Cleanup pt2
* Some renames
* Fix typecheck
* Use observables to manage throughput
* Rename class
* Switch to createManagedConfiguration
* Add some comments
* Start unit tests
* Add logs
* Fix log level
* Attempt at adding integration tests
* Fix test failures
* Fix timer
* Revert "Fix timer"
This reverts commit 0817e5e6a5.
* Use Symbol
* Fix merge scan
* replace startsWith with a timer that is scheduled to 0
* typo
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
* rename uuid service to environment service
* adapt resolve_uuid to directly use the configurations
* move data folder creation to core
* update generated doc
* fix types
* fix monitoring tests
* move instanceUuid to plugin initializer context
* update generated doc
Introduces a monitor around the Task Manager poller which pips through all values emitted by the poller and recovers from poller failures or stalls.
This monitor does the following:
1. Catches the poller thrown errors and recovers by proxying the error to a handler and continues listening to the poller.
2. Reacts to the poller `error` (caused by uncaught errors) and `completion` events, by starting a new poller and piping its event through to any previous subscribers (in our case, Task Manager itself).
3. Tracks the rate at which the poller emits events (this can be both work events, and `No Task` events, so polling and finding no work, still counts as an emitted event) and times out when this rate gets too long (suggesting the poller has hung) and replaces the Poller with a new one.
We're not aware of any clear cases where Task Manager should actually get restarted by the monitor - this is definitely an error case and we have addressed all known cases.
The goal of introducing this monitor is as an insurance policy in case an unexpected error case breaks the poller in a long running production environment.
If the work performed by the poller hangs, meaning the promise fails to resolve/reject, then the poller can get stuck in a mode where it just waits for ever and no longer polls for fresh work.
This PR introduces a timeout after which the poller will automatically reject the work, freeing the poller to restart pulling fresh work.
Handles the case where two operations for the same entity make it into a single batched bulk operation and avoid the clashing ID issue that could cause the poller to hang and stop poling for work).
Fixes flaky tests in Task Manager and Alerting.
The fix in #73244 was correct, but it missed an edge case which causes the already running task to be rescheduled over and over.
This prevents that edge case which was effecting both TM in general and Alerting specifically.
This PR addresses two issues which caused several tests to be flaky in TM.
When `runNow` was introduced to TM we added a pinned query which returned specific tasks by ID.
This query does not have the filter applied to it which causes task to return when they're already marked as `running` but we didn't address these correctly which caused flakyness in the tests.
This didn't cause a broken beahviour, but it did cause beahviour that was hard to reason about - we now address them correctly.
It seems that sometimes, especially if the ES queue is overworked, it can take some time for the update to the underlying task to be visible (we don't user `refresh:true` on purpose), so adding a wait for the index to refresh to make sure the task is updated in time for the next stage of the test.
This PR attempts to batch update tasks in Task Manager in order to avoid overloading the Elasticsearch queue.
This is the 1st PR addressing https://github.com/elastic/kibana/issues/65551
Under the hood we now use a Reactive buffer accumulates all calls to the `update` api in the TaskStore and flushes after 50ms or when as many operations as there are workers have been buffered (whichever comes first).
* chore(NA): upgrade oss to lodash4
chore(NA): migrate cli, cli_plugin, cli_keystore, dev, test_utils and apm src script to lodash4
chore(NA): missing file for cli plugin
chore(NA): add src core
chore(NA): es archiver and fixtures
chore(NA): try to fix functional test failure
chore(NA): migrate src/legacy entirely to lodash4 except src/legacy/core_plugins
chore(NA): move legacy core plugins to lodash4
chore(NA): upgrade optimize to lodash4
chore(NA): upgrade to lodash4 on advanced_settings, charts, console and dashboard
chore(NA): migrate to lodash4 on dev_tools, discover, embeddable, es_ui)shared, expressions, home plugins
chore(NA): upgrade data plugin to lodash4
chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4
chore(NA): missing data upgrades to lodash4
Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4"
This reverts commit 137055c5fed2fc52bb26547e0bc1ad2e3d4fe309.
Revert "Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4""
This reverts commit f7e73688782998513d9fb6d7e8f0765e9beb28d1.
Revert "chore(NA): missing data upgrades to lodash4"
This reverts commit 92b85bf947a89bfc70cc4052738a6b2128ffb076.
Revert "chore(NA): upgrade data plugin to lodash4"
This reverts commit 88fdb075ee1e26c4ac979b6681d8a2b002df74c6.
chore(NA): upgrade idx_pattern_mgt, input_control_vis, inspector, kbn_legacy, kbn_react, kbn_usage_collections, kbn_utils, management and maps_legacy to lodash4
chore(NA): map src plugin data to lodash3
chore(NA): missing lodash.clonedeep dep
chore(NA): change packages kbn-config-schema deps
chore(NA): update renovate config
chore(NA): upgrade vis_type plugins to lodash4
chore(NA): move vis_type_vislib to lodash3
chore(NA): update visualizations and visualize to lodash4
chore(NA): remove lodash 3 types from src and move test to lodash4
chore(NA): move home, usage_collection and management to lodash 3
Revert "chore(NA): move home, usage_collection and management to lodash 3"
This reverts commit f86e8585f02d21550746569af54215b076a79a3d.
chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3
chore(NA): update x-pack test to mock lodash4
Revert "chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3"
This reverts commit 2d10fe450533e1b36db21d99cfae3ce996a244e0.
* chore(NA): move x-pack and packages to lodash 4
* chore(NA): remove mention to lodash from main package.json
* chore(NA): remove helper alias for lodash4 and make it the default lodash
* chore(NA): fix last failing types in the repo
* chore(NA): fix public api
* chore(NA): fix types for agg_row.tsx
* chore(NA): fix increment of optimizer modules in the rollup plugin
* chore(NA): migrate `src/core/public/http/fetch.ts` (#5)
* omit undefined query props
* just remove merge usage
* fix types
* chore(NA): fixes for feedback from apm team
* chore(NA): recover old behaviour on apm LoadingIndeicatorContext.tsx
* chore(NA): fixes for feedback from watson
* Platform lodash4 tweaks (#6)
* chore(NA): fix types and behaviour on src/core/server/elasticsearch/errors.ts
* Canvas fixes for lodash upgrade
* [APM] Adds unit test for APM service maps transform (#7)
* Adds a snapshot unit test for getConnections and rearranges some code to make testing easier
* reverts `ArrayList` back to `String[]` in the painless script within `fetch_service_paths_from_trace_ids.ts`
* chore(NA): update yarn.lock
* chore(NA): remove any and use a real type for alerts task runner
Co-authored-by: Gidi Meir Morris <github@gidi.io>
* chore(NA): used named import for triggers_actions_ui file
* chore(NA): fix eslint
* chore(NA): fix types
* Delete most uptime lodash references.
* Simplify. Clean up types.
* [Uptime] Delete most uptime lodash references (#8)
* Delete most uptime lodash references.
* Simplify. Clean up types.
* chore(NA): add eslint rule to avoid using lodash3
* chore(NA): apply changes on feedback from es-ui team
* fix some types (#9)
* Clean up some expressions types.
* chore(NA): missing ts-expect-error statements
* Upgrade lodash 4 vislib (#11)
* replace lodash 3 with lodash 4 on vislib plugin
* Further changes
* further replacement of lodash3 to 4
* further work on upgrading to lodash 4
* final changes to update lodash
* chore(NA): upgrade data plugin to lodash4
chore(NA): upgrade data plugin public to lodash4
chore(NA): fix typecheck task
chore(NA): fix agg_config with hasIn
chore(NA): assign to assignIn and has to hasIn
chore(NA): upgrade data plugin server to lodash4
chore(NA): new signature for core api
fix(NA): match behaviour between lodash3 and lodash4 for set in search_source
* chore(NA): remove lodash3 completely from the repo
* chore(NA): fix x-pack/test/api_integration/apis/metrics_ui/snapshot.ts missing content
* chore(NA): fix lodash usage on apm
* chore(NA): fix typecheck for maps
* Patch lodash template (#12)
* Applying changes from https://github.com/elastic/kibana/pull/64985
* Using isIterateeCall, because it seems less brittle
* Also patching `lodash/template` and `lodash/fp/template`
* Reorganizing some files...
* Revising comment
* Ends up `_` is a function also... I hate JavaScript
Co-authored-by: Pierre Gayvallet <pierre.gayvallet@gmail.com>
Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: Clint Andrew Hall <clint.hall@elastic.co>
Co-authored-by: Oliver Gupte <ogupte@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
Co-authored-by: Justin Kambic <justin.kambic@elastic.co>
Co-authored-by: Stratoula Kalafateli <stratoula1@gmail.com>
Co-authored-by: Luke Elmers <luke.elmers@elastic.co>
Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com>
Co-authored-by: kobelb <brandon.kobel@elastic.co>
* move last snapshot to inline
* move legacy files to legacy subfolder
* move request types out of legacy
* export Headers from http instead of elasticsearch
* renaming - first pass
* renaming - second pass
* fix core mocks
* adapt new calls
* update generated doc
* fix IT test mocks
* fix new usages
This PR migrates the vast majority of Alerting legacy code to the Kibana Platform.
This includes:
1. Removed legacy Task Manager
2. Migrates Fixture plugins in Alerting, Triggers UI and Task Manager Perf
This does not includes:
1. The PagerDuty simulator due to a lack of support for custom responses in the platform. issue opened. https://github.com/elastic/kibana/issues/65045
2. The Webhooks simulator due to a lack of support for custom authorisation. Requires investigation.
Completes the migration of all Alerting Services plugins onto the Kibana Platform
It includes:
1. Actions plugin
2. Alerting plugin
3. Task Manager plugin
4. Triggers UI plugin
And touches the Uptime and Siem plugins as their use of the Task Manager relied on some of the legacy lifecycle to work (registering AlertTypes and Telemetry tasks after the Start stage has already began). The fix was simply to moves these registrations to the Setup stage.
This removes unneeded use of `any` throughout:
1. alerting
2. alerting_builtin
3. actions
4. task manager
5. event log
It also adds a linting rule that will prevent us from adding more `any` in the future unless an explicit exemption is made.
Major cleanup of the no_restricted_paths rule for imports of core.
For relative imports, we use eslint-module-utils/resolve which resolves
to the full filesystem path. So, to support relative and absolute
imports from the src alias we need to define both the directory and the
index including file extension.
This rule was handling both core imports, as well as imports from other
plugins. Imports from other plugins are being used much more liberally
allowed through the exceptions in tests. I choose to break these up,
removing this exception for tests for core imports.
Fixes:
Absolute imports of src/core/server/mocks were not allowed in src. This
was not an issue in x-pack due to the target excluding
!x-pack/**/*.test.* and !x-pack/test/**/*.
Non-top-level public and server imports were allowed from X-Pack tests
to the previously mentioned exclusion.
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
Instead of warning that a task isn't cancellable despite having been cancelled, we now do so as a debug warning.
We now warn when a task has expired and is about to be cancelled including when it expired and how long it ran for.
* x-pack/watcher: use Elasticsearch from CoreStart
* x-pack/upgrade_assistant: use Elasticsearch from CoreStart
* x-pack/actions: use Elasticsearch from CoreStart
* x-pack/alerting: use Elasticsearch from CoreStart
* x-pack/lens: use Elasticsearch from CoreStart
* expressions: use Elasticsearch from CoreStart
* x-pack/remote_clusters: remove unused Elasticsearch dependency on CoreSetup
* x-pack/oss_telemetry: use Elasticsearch from CoreStart
* Cleanup after #59886
* x-pack/watcher: create custom client only once
* Revert "x-pack/watcher: create custom client only once"
This reverts commit 78fc4d2e93.
* Revert "x-pack/watcher: use Elasticsearch from CoreStart"
This reverts commit b621af9388.
* x-pack/task_manager: use Elasticsearch from CoreStart
* x-pack/event_log: use Elasticsearch from CoreStart
* x-pack/alerting: use Elasticsearch from CoreStart
* x-pack/apm: use Elasticsearch from CoreStart
* x-pack/actions: use Elasticsearch from CoreStart
* PR Feedback
* APM review nits
* Remove unused variable
* Remove unused variable
* x-pack/apm: better typesafety
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Detects if a task run failed due to the task SO being deleted mid flight and if so writes debug logs instead of warnings.
Detects if an Alerting task run failed due to the alert SO being deleted mid flight of the task and if so ensures the task doesn't reschedule itself (as it usually would with other types of tasks).
Ensures that the operation of deleting or disabling an Alert won't fail if it fails to delete an already deleted task (a task might preemptively self delete if its underlying alert object was deleted, even if the overall delete operation wasn't deleted).
resolves https://github.com/elastic/kibana/issues/54920
The scenario this message warns about is actually something that we should
expect to happen in a busy system - and we back off as we should. But
there's no reason it needs to be info-logged, especially as it seems
somewhat frightening.
Changing to debug, so in case we do need this for diagnostic purposes, we
can still get it via logging config.
Adds some apm transaction boundaries for parts of task manager, so that
they will show up in APM as new types of transactions. Should provide
some visibility into the ES calls made by task manager for alerting and
actions, especially under stress testing scenarios.
