89bd0fbf1e
* Initial structure of migration state-action machine * Fix type import * Retries with exponential back off * Use discriminated union for state type * Either type for actions * Test exponential retries * TaskEither types for actions * Fetch indices instead of aliases so we can collect all index state in one request * Log document id if transform fails * WIP: Legacy pre-migrations * UPDATE_TARGET_MAPPINGS * WIP OUTDATED_DOCUMENTS_TRANSFORM * Narrow res types depending on control state * OUTDATED_DOCUMENTS_TRANSFORM * Use .kibana instead of .kibana_current * rename control states TARGET_DOCUMENTS* -> OUTDATED_DOCUMENTS* * WIP MARK_VERSION_INDEX_READY * Fix and expand INIT -> * transition tests * Add alias/index name helper functions * Add feature flag for enabling v2 migrations * split state_action_machine, reindex legacy indices * Don't use a scroll search for migrating outdated documents * model: test control state progressions * Action integration tests * Fix existing tests and type errors * snapshot_in_progress_exception can only happen when closing/deleting an index * Retry steps up to 10 times * Update api.md documentation files * Further actions integration tests * Action unit tests * Fix actions integration tests * Rename actions to be more domain-specific * Apply suggestions from code review Co-authored-by: Josh Dover <me@joshdover.com> * Review feedback: polish and flesh out inline comments * Fix unhandled rejections in actions unit tests * model: only delay retryable_es_client_error, reset for other left responses * Actions unit tests * More inline comments * Actions: Group index settings under 'index' key * bulkIndex -> bulkOverwriteTransformedDocuments to be more domain specific * state_action_machine tests, fix and add additional tests * Action integration tests: updateAndPickupMappings, searchForOutdatedDocuments * oops: uncomment commented out code * actions integration tests: rejection for createIndex * update state properties: clearer names, mark all as readonly * add state properties currentAlias, versionAlias, legacyIndex and test for invalid version scheme in index names * Use CONSTANTS for constants :D * Actions: Clarify behaviour and impact of acknowledged: false responses * Use consistent vocabulary for action responses * KibanaMigrator test for migrationsV2 * KibanaMigrator test for FATAL state and action exceptions in v2 migrations * Fix ts error in test * Refactor: split index file up into a file per model, next, types * next: use partial application so we don't generate a nextActionMap on every call * move logic from index.ts to migrations_state_action_machine.ts and test * add test * use `Root` to allow specifying oss mode * Add fix and todo tests for reindexing with preMigrationScript * Dump execution log of state transitions and responses if we hit FATAL * add 7.3 xpack tests * add 100k test data * Reindex instead of cloning for migrations * Skip 100k x-pack integration test * MARK_VERSION_INDEX_READY_CONFLICT for dealing with different versions migrating in parallel * Track elapsed time * Fix tests * Model: make exhaustiveness checks more explicit * actions integration tests: add additional tests from CR * migrations_state_action_machine fix flaky test * Fix flaky integration test * Reserve FATAL termination only for situations which we never can recover from such as later version already migrated the index * Handle incompatible_mapping_exception caused by another instance * Cleanup logging * Fix/stabilize integration tests * Add REINDEX_SOURCE_TO_TARGET_VERIFY step * Strip tests archives of */.DS_Store and __MAC_OSX * Task manager migrations: remove invalid kibana property when converting legacy indices * Add disabled mappings for removed field in map saved object type * verifyReindex action: use count API * REINDEX_BLOCK_* to prevent lost deletes (needs tests) * Split out 100k docs integration test so that it has it's own kibana process * REINDEX_BLOCK_* action tests * REINDEX_BLOCK_* model tests * Include original error message when migration_state_machine throws * Address some CR nits * Fix TS errors * Fix bugs * Reindex then clone to prevent lost deletes * Fix tests Co-authored-by: Josh Dover <me@joshdover.com> Co-authored-by: pgayvallet <pierre.gayvallet@elastic.co> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> |
||
---|---|---|
.. | ||
integration_tests | ||
lib | ||
monitoring | ||
polling | ||
queries | ||
routes | ||
saved_objects | ||
task_running | ||
test_utils | ||
buffered_task_store.test.ts | ||
buffered_task_store.ts | ||
config.test.ts | ||
config.ts | ||
index.test.ts | ||
index.ts | ||
mocks.ts | ||
MONITORING.md | ||
plugin.test.ts | ||
plugin.ts | ||
polling_lifecycle.mock.ts | ||
polling_lifecycle.test.ts | ||
polling_lifecycle.ts | ||
task.ts | ||
task_events.ts | ||
task_pool.test.ts | ||
task_pool.ts | ||
task_scheduling.mock.ts | ||
task_scheduling.test.ts | ||
task_scheduling.ts | ||
task_store.mock.ts | ||
task_store.test.ts | ||
task_store.ts | ||
task_type_dictionary.test.ts | ||
task_type_dictionary.ts |