* Initial structure of migration state-action machine
* Fix type import
* Retries with exponential back off
* Use discriminated union for state type
* Either type for actions
* Test exponential retries
* TaskEither types for actions
* Fetch indices instead of aliases so we can collect all index state in one request
* Log document id if transform fails
* WIP: Legacy pre-migrations
* UPDATE_TARGET_MAPPINGS
* WIP OUTDATED_DOCUMENTS_TRANSFORM
* Narrow res types depending on control state
* OUTDATED_DOCUMENTS_TRANSFORM
* Use .kibana instead of .kibana_current
* rename control states TARGET_DOCUMENTS* -> OUTDATED_DOCUMENTS*
* WIP MARK_VERSION_INDEX_READY
* Fix and expand INIT -> * transition tests
* Add alias/index name helper functions
* Add feature flag for enabling v2 migrations
* split state_action_machine, reindex legacy indices
* Don't use a scroll search for migrating outdated documents
* model: test control state progressions
* Action integration tests
* Fix existing tests and type errors
* snapshot_in_progress_exception can only happen when closing/deleting an index
* Retry steps up to 10 times
* Update api.md documentation files
* Further actions integration tests
* Action unit tests
* Fix actions integration tests
* Rename actions to be more domain-specific
* Apply suggestions from code review
Co-authored-by: Josh Dover <me@joshdover.com>
* Review feedback: polish and flesh out inline comments
* Fix unhandled rejections in actions unit tests
* model: only delay retryable_es_client_error, reset for other left responses
* Actions unit tests
* More inline comments
* Actions: Group index settings under 'index' key
* bulkIndex -> bulkOverwriteTransformedDocuments to be more domain specific
* state_action_machine tests, fix and add additional tests
* Action integration tests: updateAndPickupMappings, searchForOutdatedDocuments
* oops: uncomment commented out code
* actions integration tests: rejection for createIndex
* update state properties: clearer names, mark all as readonly
* add state properties currentAlias, versionAlias, legacyIndex and test for invalid version scheme in index names
* Use CONSTANTS for constants :D
* Actions: Clarify behaviour and impact of acknowledged: false responses
* Use consistent vocabulary for action responses
* KibanaMigrator test for migrationsV2
* KibanaMigrator test for FATAL state and action exceptions in v2 migrations
* Fix ts error in test
* Refactor: split index file up into a file per model, next, types
* next: use partial application so we don't generate a nextActionMap on every call
* move logic from index.ts to migrations_state_action_machine.ts and test
* add test
* use `Root` to allow specifying oss mode
* Add fix and todo tests for reindexing with preMigrationScript
* Dump execution log of state transitions and responses if we hit FATAL
* add 7.3 xpack tests
* add 100k test data
* Reindex instead of cloning for migrations
* Skip 100k x-pack integration test
* MARK_VERSION_INDEX_READY_CONFLICT for dealing with different versions migrating in parallel
* Track elapsed time
* Fix tests
* Model: make exhaustiveness checks more explicit
* actions integration tests: add additional tests from CR
* migrations_state_action_machine fix flaky test
* Fix flaky integration test
* Reserve FATAL termination only for situations which we never can recover from such as later version already migrated the index
* Handle incompatible_mapping_exception caused by another instance
* Cleanup logging
* Fix/stabilize integration tests
* Add REINDEX_SOURCE_TO_TARGET_VERIFY step
* Strip tests archives of */.DS_Store and __MAC_OSX
* Task manager migrations: remove invalid kibana property when converting legacy indices
* Add disabled mappings for removed field in map saved object type
* verifyReindex action: use count API
* REINDEX_BLOCK_* to prevent lost deletes (needs tests)
* Split out 100k docs integration test so that it has it's own kibana process
* REINDEX_BLOCK_* action tests
* REINDEX_BLOCK_* model tests
* Include original error message when migration_state_machine throws
* Address some CR nits
* Fix TS errors
* Fix bugs
* Reindex then clone to prevent lost deletes
* Fix tests
Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: pgayvallet <pierre.gayvallet@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Copy SourceLogic to AddSourceLogic
This commit merely copies SourceLogic and renames it to AddSourceLogic with the name of the component and interfaces changed. This will make reviewing the changes easier later
* Delete duplicated logic data
* Update component paths
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Maps] provide function to create basemap layer descriptor
* update typing
* update APM map to use createBasemapLayerDescriptor
* fix useLayerList jest test
* tslint fixes
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Initial work
* Change messaging from copy
* Fix jest tests for email connector
* Fix jest tests for alerts plugin
* Update copy
* Use server.publicBaseUrl
* Fix jest tests
* Update tests
* Cleanup jest test
* Code cleanup
* Improve email parameter names for kibana footer url
* Cleanup
* Add test for kibana footer link
* Fix type check
* Fix jest test
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Add i18n for AddSourceList
* Add i18n for AvailableSourcesList
* Replace all instances of interal Link
This was missed during the migration. We need to use the components we wrap Eui components with internally.
Also fixes some imports that were unnecessarily going up one level and back down for imports.
* Add i18n for ConfigCompleted
* Add i18n for ConfigDocsLinks
* Add i18n for ConfigurationIntro
* Add i18n for ConfigureCustom
* Add i18n for ConfigureOauth
* Add i18n for ConfiguredSourcesList
* Add i18n for ReAuthenticate
* Add i18n for shared field labels
Also includea shared SourceConfigFields
* Add i18n for SaveConfig
* Add i18n for SaveCustom
* Add i18n for SourceFeatures
* Add i18n for ConnectInstance
* Fix duplicate i18n ID
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* disable delete until checks complete
* add canDeleteJobs wrapper in saved objects service
* create DeleteJobCheckModal shared component
* wip: add deleteJobModal check in list and map views
* adding remove from current space endpoint
* updating error text
* fixing typo in variable name
* Update button content. Add untagging functionality
* adding anomaly detection delete job modal
* fix modal content bug
* refresh job map after deletion or untagging
* adding job refresh to anomaly detectors
* go straight to delete flow if only action available
* fixing test
* update text
* increase line spacing in check modal
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: James Gowdy <jgowdy@elastic.co>
* Fixing resolver functional tests
* Import the animation constant
* Only check specific nodes instead of all the ones in view
* Removing check for link text
* updating test description
* Adding comments
* Adds new SO type for persisting our signals migrations
* WIP: Migration status SO client
Trying to forge a patterrn using io-ts to validate at runtime. I think
I've got it working but I want to refactor the pipeline out into a
reusable function(s).
* Implements our SavedObjects service for signals migrations
* Defines a simple client that delegates to the base SO client with
our SO type
* Defines a service that consumes the simpler client, adding validations
and data transforms on top.
* Refactoring migration code to work with saved objects
As opposed to the previous ephemeral, encoded tokens, we now retrieve migration
info from saved objects.
At the API level, this means that both the create and finalize endpoints
receive a list of concrete indices. No more passing around tokens.
As both endpoints are idempotent, users can hammer them as much as they
want with the same lists of indices. Redundant creates and finalizes
will be met with inline 400 messages, and as one continues to poll the
finalize endpoint they should see more and more indices respond with
"completed: true"
* Fixing integration tests first, and anything upstream breaking them
* Clean up API integration tests
* standardize assignment of responses (with types)
* deletes migration SOs as test cleanup
* Split API tests into separate files
This was getting big and unwieldy; this splits these into one file per
endpoint.
* Refactor: split existing migration service functionality into atomic functions
This will allow us to repurpose the service to compose more
functionality and be more specifically useful, while keeping the
component logic separate.
* WIP: moving logic into migrationService.create
* Splitting get_migration_status into component functions
getMigrationStatus was really two separate aggregations, so I split them
out and we recompose them in the necessary routes.
* Move finalization logic into function
* migrationService exposes this as .finalize()
* adds an error field to our migration SO
* We currently only have one error that we persist there, but it would
be very time-consuming to track down that information were it not
there.
* Adds function for migration "deletion" logic
* migrationService leverages this function
* adds new boolean to our savedObject
* deletes unused function (deleteMigrationSavedObject)
* Adds route for soft-deletion of migrations
* Updating tests related to migration status
* Adding/updating mocks/unit tests necessary to satisfy the things I
need to test
* I mainly wanted to test that the the status endpoint filtered out the
deleted migrations; this was accomplished with a unit test after
fleshing out some mocks/sample data.
* Move old migration service tests to the relevant function tests
This logic was previously moved out into component functions; this moves
the tests accordingly.
* Add some unit tests around our reindex call
* Fix create migration route tests
Mocks out our migration functions, rather than stubbing ES calls
directly.
* Updates finalize route unit tests
Addresses functionality that hasn't been moved to finalizeMigration()
* Unit tests our finalization logic
Fixes a bug where we weren't accounting for soft-deleted migrations.
ALso updates our test migration SO to have a status of 'pending' as
that's a more useful default.
* Fixes finalization integration tests
These were failing due:
* a change in the migration status API response
* a bug I introduced in the finalize route
* Adds tests for our migration deletion endpoint
* unit tests
* API integration tests
* Caught/fixed bug with deleting a successful migration
* Fixes types
Removes unused code.
* Prevent race condition due to template rollover during migration
If a user has an out of date index (v1) relative to the template (v2), but the
template itself is out of date (newest is v3), then it's possible that
the template is rolled over to v3 after the v1-v2 migration has been
created but before the new index has been created.
In such a case, the new index would receive the v3 mappings but would
incorrectl be marked as v2. This shouldn't necessarily be an issue, but
it's an unnecessary state that can easily be prevented with the guard
introduced here.
* Add real usernames to migration savedObjects
In addition to the SOs themselves giving us observability into what
migration actions were performed, this gives us the additional info of
_who_ performed the action.
* Index minimal migration SO fields needed for current functionality
* Add additional migration info to status endpoint
This will allow users to finalize a migration if they've lost the
response to their POST call.
* Finalize endpoint receives an array of migration IDs, not indices
This disambiguates _which_ migrations we were finalizing if you passed
an index (which was previously: the most recent migration).
* Fix type errors in tests after we threaded through username
* Update responsibilities of migration finalize/delete endpoints
Discussions with @marshallmain lead to the following refactor:
* finalize does not delete tasks
* finalize only applies cleanup policy to a failed migration
* delete takes an array of migration ids (like finalize)
* delete hard-deletes the SavedObject of a completed (failed or
successful) migration
This gives a bit more flexibility with the endpoints, as well as
disambiguates the semantics: it just deletes migrations!
* Fix tests that were broken during refactoring
* Fix type errors
I removed some logic here but forgot the imports :(
* Move outdated integration test
In the case of a successful migration, application of the cleanup policy
is done by the deletion endpoint. In the interest of data preservation,
we do not delete a sourceIndex unless it is explicitly deleted.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Adds context.alerts as available parameter for detection rule actions
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Alerts][License] Define minimum license required for each alert type (#84997)
* Define minimum license required for each alert type
* fixed typechecks
* fixed tests
* fixed tests
* fixed due to comments
* fixed due to comments
* removed file
* removed casting to LicenseType
* [Alerts][License] Add license checks to alerts HTTP APIs and execution (#85223)
* [Alerts][License] Add license checks to alerts HTTP APIs and execution
* fixed typechecks
* resolved conflicts
* resolved conflicts
* added router tests
* fixed typechecks
* added license check support for alert task running
* fixed typechecks
* added integration tests
* fixed due to comments
* fixed due to comments
* fixed tests
* fixed typechecks
* [Alerting UI][License] Disable alert types in UI when the license doesn't support it. (#85496)
* [Alerting UI][License] Disable alert types in UI when the license doesn't support it.
* fixed typechecks
* added licensing for alert list and details page
* fixed multy select menu
* fixed due to comments
* fixed due to comments
* fixed due to comments
* fixed typechecks
* fixed license error message
* fixed license error message
* fixed typechecks
* fixed license error message
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
