89bd0fbf1e
* Initial structure of migration state-action machine * Fix type import * Retries with exponential back off * Use discriminated union for state type * Either type for actions * Test exponential retries * TaskEither types for actions * Fetch indices instead of aliases so we can collect all index state in one request * Log document id if transform fails * WIP: Legacy pre-migrations * UPDATE_TARGET_MAPPINGS * WIP OUTDATED_DOCUMENTS_TRANSFORM * Narrow res types depending on control state * OUTDATED_DOCUMENTS_TRANSFORM * Use .kibana instead of .kibana_current * rename control states TARGET_DOCUMENTS* -> OUTDATED_DOCUMENTS* * WIP MARK_VERSION_INDEX_READY * Fix and expand INIT -> * transition tests * Add alias/index name helper functions * Add feature flag for enabling v2 migrations * split state_action_machine, reindex legacy indices * Don't use a scroll search for migrating outdated documents * model: test control state progressions * Action integration tests * Fix existing tests and type errors * snapshot_in_progress_exception can only happen when closing/deleting an index * Retry steps up to 10 times * Update api.md documentation files * Further actions integration tests * Action unit tests * Fix actions integration tests * Rename actions to be more domain-specific * Apply suggestions from code review Co-authored-by: Josh Dover <me@joshdover.com> * Review feedback: polish and flesh out inline comments * Fix unhandled rejections in actions unit tests * model: only delay retryable_es_client_error, reset for other left responses * Actions unit tests * More inline comments * Actions: Group index settings under 'index' key * bulkIndex -> bulkOverwriteTransformedDocuments to be more domain specific * state_action_machine tests, fix and add additional tests * Action integration tests: updateAndPickupMappings, searchForOutdatedDocuments * oops: uncomment commented out code * actions integration tests: rejection for createIndex * update state properties: clearer names, mark all as readonly * add state properties currentAlias, versionAlias, legacyIndex and test for invalid version scheme in index names * Use CONSTANTS for constants :D * Actions: Clarify behaviour and impact of acknowledged: false responses * Use consistent vocabulary for action responses * KibanaMigrator test for migrationsV2 * KibanaMigrator test for FATAL state and action exceptions in v2 migrations * Fix ts error in test * Refactor: split index file up into a file per model, next, types * next: use partial application so we don't generate a nextActionMap on every call * move logic from index.ts to migrations_state_action_machine.ts and test * add test * use `Root` to allow specifying oss mode * Add fix and todo tests for reindexing with preMigrationScript * Dump execution log of state transitions and responses if we hit FATAL * add 7.3 xpack tests * add 100k test data * Reindex instead of cloning for migrations * Skip 100k x-pack integration test * MARK_VERSION_INDEX_READY_CONFLICT for dealing with different versions migrating in parallel * Track elapsed time * Fix tests * Model: make exhaustiveness checks more explicit * actions integration tests: add additional tests from CR * migrations_state_action_machine fix flaky test * Fix flaky integration test * Reserve FATAL termination only for situations which we never can recover from such as later version already migrated the index * Handle incompatible_mapping_exception caused by another instance * Cleanup logging * Fix/stabilize integration tests * Add REINDEX_SOURCE_TO_TARGET_VERIFY step * Strip tests archives of */.DS_Store and __MAC_OSX * Task manager migrations: remove invalid kibana property when converting legacy indices * Add disabled mappings for removed field in map saved object type * verifyReindex action: use count API * REINDEX_BLOCK_* to prevent lost deletes (needs tests) * Split out 100k docs integration test so that it has it's own kibana process * REINDEX_BLOCK_* action tests * REINDEX_BLOCK_* model tests * Include original error message when migration_state_machine throws * Address some CR nits * Fix TS errors * Fix bugs * Reindex then clone to prevent lost deletes * Fix tests Co-authored-by: Josh Dover <me@joshdover.com> Co-authored-by: pgayvallet <pierre.gayvallet@elastic.co> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| images | ||
| text | ||
| 0000_template.md | ||
| README.md | ||
Kibana RFCs
We are currently trialing a new RFC process for the Kibana Core team at this time.
Many changes, including small to medium features, fixes, and documentation improvements can be implemented and reviewed via the normal GitHub pull request workflow.
Some changes though are "substantial", and we ask that these be put through a bit of a design process and produce a consensus among the relevant Kibana team.
The "RFC" (request for comments) process is intended to provide a consistent and controlled path for new features to enter the project.
Kibana is still actively developing this process, and it will still change as more features are implemented and the community settles on specific approaches to feature development.
Contributor License Agreement (CLA)
In order to accept your pull request, we need you to submit a CLA. You only need to do this once, so if you've done this for another Elastic open source project, you're good to go.
When to follow this process
You should consider using this process if you intend to make "substantial" changes to Kibana or its documentation. Some examples that would benefit from an RFC are:
- A new feature that creates new API surface area, such as a new core service available to plugins.
- The removal of features that already shipped as part of a release.
- The introduction of new idiomatic usage or conventions, even if they do not include code changes to Kibana itself.
The RFC process is a great opportunity to get more eyeballs on your proposal before it becomes a part of a released version of Kibana. Quite often, even proposals that seem "obvious" can be significantly improved once a wider group of interested people have a chance to weigh in.
The RFC process can also be helpful to encourage discussions about a proposed feature as it is being designed, and incorporate important constraints into the design while it's easier to change, before the design has been fully implemented.
Some changes do not require an RFC:
- Rephrasing, reorganizing or refactoring
- Addition or removal of warnings
- Additions that strictly improve objective, numerical quality criteria (speedup, better browser support)
- Addition of features that do not impact other Kibana plugins (do not expose any API to other plugins)
What the process is
In short, to get a major feature added to Kibana Core, one usually first gets the RFC merged into the RFC tree as a markdown file. At that point the RFC is 'active' and may be implemented with the goal of eventual inclusion into Kibana.
- Fork the Kibana repo http://github.com/elastic/kibana
- Copy
rfcs/0000_template.mdtorfcs/text/0001_my_feature.md(where 'my_feature' is descriptive. Assign a number. Check that an RFC with this number doesn't already exist inmasteror an open PR). - Fill in the RFC. Put care into the details: RFCs that do not present convincing motivation, demonstrate understanding of the impact of the design, or are disingenuous about the drawbacks or alternatives tend to be poorly-received.
- Submit a pull request. As a pull request the RFC will receive design feedback from the larger community and Elastic staff. The author should be prepared to revise it in response.
- Build consensus and integrate feedback. RFCs that have broad support are much more likely to make progress than those that don't receive any comments.
- Eventually, the team will decide whether the RFC is a candidate for inclusion in Kibana.
- RFCs that are candidates for inclusion in Kibana will enter a "final comment period" lasting at least 3 working days. The beginning of this period will be signaled with a comment and tag on the RFCs pull request.
- An RFC can be modified based upon feedback from the team and community. Significant modifications may trigger a new final comment period.
- An RFC may be rejected by the team after public discussion has settled and comments have been made summarizing the rationale for rejection. A member of the team should then close the RFCs associated pull request.
- An RFC may be accepted at the close of its final comment period. A team member will merge the RFCs associated pull request, at which point the RFC will become 'active'.
The RFC life-cycle
Once an RFC becomes active, then authors may implement it and submit the feature as a pull request to the Kibana repo. Becoming 'active' is not a rubber stamp, and in particular still does not mean the feature will ultimately be merged; it does mean that the core team has agreed to it in principle and are amenable to merging it.
Furthermore, the fact that a given RFC has been accepted and is 'active' implies nothing about what priority is assigned to its implementation, nor whether anybody is currently working on it.
Modifications to active RFCs can be done in followup PRs. We strive to write each RFC in a manner that it will reflect the final design of the feature; but the nature of the process means that we cannot expect every merged RFC to actually reflect what the end result will be at the time of the next major release; therefore we try to keep each RFC document somewhat in sync with the Kibana feature as planned, tracking such changes via followup pull requests to the document. You may include updates to the RFC in the same PR that makes the code change.
Implementing an RFC
The author of an RFC is not obligated to implement it. Of course, the RFC author (like any other developer) is welcome to post an implementation for review after the RFC has been accepted.
If you are interested in working on the implementation for an 'active' RFC, but cannot determine if someone else is already working on it, feel free to ask (e.g. by leaving a comment on the associated issue).
Reviewing RFCs
Each week the team will attempt to review some set of open RFC pull requests.
Every accepted feature should have a core team champion, who will represent the feature and its progress.
Kibana's RFC process owes its inspiration to the React RFC process, Yarn RFC process, Rust RFC process, and Ember RFC process