maxmustermann/kibana
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
kibana/x-pack/plugins/upgrade_assistant
History
Tomas Della Vedova 238791b942
ES client : use the new type definitions (#83808) 
* Use client from branch

* Get type checking working in core

* Fix types in other plugins

* Update client types + remove type errors from core

* migrate Task Manager Elasticsearch typing from legacy library to client library

* use SortOrder instead o string in alerts

* Update client types + fix core type issues

* fix maps ts errors

* Update Lens types

* Convert Search Profiler body from a string to an object to conform to SearchRequest type.

* Fix SOT types

* Fix/mute Security/Spaces plugins type errors.

* Fix bootstrap types

* Fix painless_lab

* corrected es typing in Event Log

* Use new types from client for inferred search responses

* Latest type defs

* Integrate latest type defs for APM/UX

* fix core errors

* fix telemetry errors

* fix canvas errors

* fix data_enhanced errors

* fix event_log errors

* mute lens errors

* fix or mute maps errors

* fix reporting errors

* fix security errors

* mute errors in task_manager

* fix errors in telemetry_collection_xpack

* fix errors in data plugins

* fix errors in alerts

* mute errors in index_management

* fix task_manager errors

* mute or fix lens errors

* fix upgrade_assistant errors

* fix or mute errors in index_lifecycle_management

* fix discover errors

* fix core tests

* ML changes

* fix core type errors

* mute error in kbn-es-archiver

* fix error in data plugin

* fix error in telemetry plugin

* fix error in discover

* fix discover errors

* fix errors in task_manager

* fix security errors

* fix wrong conflict resolution

* address errors with upstream code

* update deps to the last commit

* remove outdated comments

* fix core errors

* fix errors after update

* adding more expect errors to ML

* pull the lastest changes

* fix core errors

* fix errors in infra plugin

* fix errors in uptime plugin

* fix errors in ml

* fix errors in xpack telemetry

* fix or mute errors in transform

* fix errors in upgrade assistant

* fix or mute fleet errors

* start fixing apm errors

* fix errors in osquery

* fix telemetry tests

* core cleanup

* fix asMutableArray imports

* cleanup

* data_enhanced cleanup

* cleanup events_log

* cleaup

* fix error in kbn-es-archiver

* fix errors in kbn-es-archiver

* fix errors in kbn-es-archiver

* fix ES typings for Hit

* fix SO

* fix actions plugin

* fix fleet

* fix maps

* fix stack_alerts

* fix eslint problems

* fix event_log unit tests

* fix failures in data_enhanced tests

* fix test failure in kbn-es-archiver

* fix test failures in index_pattern_management

* fixing ML test

* remove outdated comment in kbn-es-archiver

* fix error type in ml

* fix eslint errors in osquery plugin

* fix runtime error in infra plugin

* revert changes to event_log cluser exist check

* fix eslint error in osquery

* fixing ML endpoint argument types

* fx types

* Update api-extractor docs

* attempt fix for ese test

* Fix lint error

* Fix types for ts refs

* Fix data_enhanced unit test

* fix lens types

* generate docs

* Fix a number of type issues in monitoring and ml

* fix triggers_actions_ui

* Fix ILM functional test

* Put search.d.ts typings back

* fix data plugin

* Update typings in typings/elasticsearch

* Update snapshots

* mute errors in task_manager

* mute fleet errors

* lens. remove unnecessary ts-expect-errors

* fix errors in stack_alerts

* mute errors in osquery

* fix errors in security_solution

* fix errors in lists

* fix errors in cases

* mute errors in search_examples

* use KibanaClient to enforce promise-based API

* fix errors in test/ folder

* update comment

* fix errors in x-pack/test folder

* fix errors in ml plugin

* fix optional fields in ml api_integartoon tests

* fix another casting problem in ml tests

* fix another ml test failure

* fix fleet problem after conflict resolution

* rollback changes in security_solution. trying to fix test

* Update type for discover rows

* uncomment runtime_mappings as its outdated

* address comments from Wylie

* remove eslint error due to any

* mute error due to incompatibility

* Apply suggestions from code review

Co-authored-by: John Schulz <github.com@jfsiii.org>

* fix type error in lens tests

* Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.ts

Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>

* Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts

Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>

* update deps

* fix errors in core types

* fix errors for the new elastic/elasticsearch version

* remove unused type

* remove unnecessary manual type cast and put optional chaining back

* ML: mute Datafeed is missing indices_options

* Apply suggestions from code review

Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com>

* use canary pacakge instead of git commit

Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
Co-authored-by: Nathan Reese <reese.nathan@gmail.com>
Co-authored-by: Wylie Conlon <wylieconlon@gmail.com>
Co-authored-by: CJ Cenizal <cj@cenizal.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co>
Co-authored-by: restrry <restrry@gmail.com>
Co-authored-by: James Gowdy <jgowdy@elastic.co>
Co-authored-by: John Schulz <github.com@jfsiii.org>
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
2021-03-25 04:47:16 -04:00
..
common [Upgrade Assistant] use request hook (#94473) 2021-03-18 13:42:20 -04:00
public [Upgrade Assistant] Reorganize folder structure (#94843) 2021-03-24 12:39:39 -04:00
server ES client : use the new type definitions (#83808) 2021-03-25 04:47:16 -04:00
tests_client_integration [Upgrade Assistant] use request hook (#94473) 2021-03-18 13:42:20 -04:00
jest.config.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
kibana.json [Upgrade Assistant] use request hook (#94473) 2021-03-18 13:42:20 -04:00
README.md Add READMEs for ES UI plugins (#81973) 2020-10-29 10:56:42 -07:00
tsconfig.json [Upgrade Assistant] Disable UA and add prompt (#92834) 2021-03-04 13:03:14 -05:00

README.md

Upgrade Assistant

About

Upgrade Assistant helps users prepare their Stack for being upgraded to the next major. Its primary purposes are to:

  • Surface deprecations. Deprecations are features that are currently being used that will be removed in the next major. Surfacing tells the user that there's a problem preventing them from upgrading.
  • Migrate from deprecation features to supported features. This addresses the problem, clearing the path for the upgrade. Generally speaking, once all deprecations are addressed, the user can safely upgrade.

Deprecations

There are two sources of deprecation information:

  • Deprecation Info API. This is information about cluster, node, and index level settings that use deprecated features that will be removed or changed in the next major version. Currently, only cluster and index deprecations will be surfaced in the Upgrade Assistant. ES server engineers are responsible for adding deprecations to the Deprecation Info API.
  • Deprecation logs. These surface runtime deprecations, e.g. a Painless script that uses a deprecated accessor or a request to a deprecated API. These are also generally surfaced as deprecation headers within the response. Even if the cluster state is good, app maintainers need to watch the logs in case deprecations are discovered as data is migrated.

Fixing problems

Problems can be fixed at various points in the upgrade process. The Upgrade Assistant supports various upgrade paths and surfaces various types of upgrade-related issues.

  • Fixing deprecated cluster settings pre-upgrade. This generally requires fixing some settings in elasticsearch.yml.
  • Migrating indices data pre-upgrade. This can involve deleting indices so that ES can rebuild them in the new version, reindexing them so that they're built using a new Lucene version, or applying a migration script that reindexes them with new settings/mappings/etc.
  • Migrating indices data post-upgrade. As was the case with APM in the 6.8->7.x upgrade, sometimes the new data format isn't forwards-compatible. In these cases, the user will perform the upgrade first and then use the Upgrade Assistant to reindex their data to be compatible with the new version.

Deprecations can be handled in a number of ways:

  • Reindexing. When a user's index contains deprecations (e.g. mappings) a reindex solves them. Upgrade Assistant contains migration scripts that are executed as part of the reindex process. The user will see a "Reindex" button they can click which will apply this script and perform the reindex.
    • Reindexing is an atomic process in Upgrade Assistant, so that ingestion is never disrupted. It works like this:
      • Create a new index with a "reindexed-" prefix (#30114).
      • Create an index alias pointing from the original index name to the prefixed index name.
      • Reindex from the original index into the prefixed index.
      • Delete the old index and rename the prefixed index.
    • Some apps might require custom scripts, as was the case with APM (#29845). In that case the migration performed a reindex with a Painless script (covered by automated tests) that made the required changes to the data.
  • Update index settings. Some index settings will need to be updated, which doesn't require a reindex. An example of this is the "Fix" button that was added for metricbeat and filebeat indices (#32829, #33439).
  • Following the docs. The Deprecation Info API provides links to the deprecation docs. Users will follow these docs to address the problem and make these warnings or errors disappear in the Upgrade Assistant.
  • Stopping/restarting tasks and jobs. Users had to stop watches and ML jobs and restart them as soon as reindexing was complete (#29663).