kibana/x-pack/plugins/snapshot_restore
Tiago Costa 6c62c686cf
chore(NA): upgrade to lodash@4 (#69868)
* chore(NA): upgrade oss to lodash4

chore(NA): migrate cli, cli_plugin, cli_keystore,  dev, test_utils and apm src script to lodash4

chore(NA): missing file for cli plugin

chore(NA): add src core

chore(NA): es archiver and fixtures

chore(NA): try to fix functional test failure

chore(NA): migrate src/legacy entirely to lodash4 except src/legacy/core_plugins

chore(NA): move legacy core plugins to lodash4

chore(NA): upgrade optimize to lodash4

chore(NA): upgrade to lodash4 on advanced_settings, charts, console and dashboard

chore(NA): migrate to lodash4 on dev_tools, discover, embeddable, es_ui)shared, expressions, home plugins

chore(NA): upgrade data plugin to lodash4

chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4

chore(NA): missing data upgrades to lodash4

Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4"

This reverts commit 137055c5fed2fc52bb26547e0bc1ad2e3d4fe309.

Revert "Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4""

This reverts commit f7e73688782998513d9fb6d7e8f0765e9beb28d1.

Revert "chore(NA): missing data upgrades to lodash4"

This reverts commit 92b85bf947a89bfc70cc4052738a6b2128ffb076.

Revert "chore(NA): upgrade data plugin to lodash4"

This reverts commit 88fdb075ee1e26c4ac979b6681d8a2b002df74c6.

chore(NA): upgrade idx_pattern_mgt, input_control_vis, inspector, kbn_legacy, kbn_react, kbn_usage_collections, kbn_utils, management and maps_legacy to lodash4

chore(NA): map src plugin data to lodash3

chore(NA): missing lodash.clonedeep dep

chore(NA): change packages kbn-config-schema deps

chore(NA): update renovate config

chore(NA): upgrade vis_type plugins to lodash4

chore(NA): move vis_type_vislib to lodash3

chore(NA): update visualizations and visualize to lodash4

chore(NA): remove lodash 3 types from src and move test to lodash4

chore(NA): move home, usage_collection and management to lodash 3

Revert "chore(NA): move home, usage_collection and management to lodash 3"

This reverts commit f86e8585f02d21550746569af54215b076a79a3d.

chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3

chore(NA): update x-pack test to mock lodash4

Revert "chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3"

This reverts commit 2d10fe450533e1b36db21d99cfae3ce996a244e0.

* chore(NA): move x-pack and packages to lodash 4

* chore(NA): remove mention to lodash from main package.json

* chore(NA): remove helper alias for lodash4 and make it the default lodash

* chore(NA): fix last failing types in the repo

* chore(NA): fix public api

* chore(NA): fix types for agg_row.tsx

* chore(NA): fix increment of optimizer modules in the rollup plugin

* chore(NA): migrate `src/core/public/http/fetch.ts` (#5)

* omit undefined query props

* just remove merge usage

* fix types

* chore(NA): fixes for feedback from apm team

* chore(NA): recover old behaviour on apm LoadingIndeicatorContext.tsx

* chore(NA): fixes for feedback from watson

* Platform lodash4 tweaks (#6)

* chore(NA): fix types and behaviour on src/core/server/elasticsearch/errors.ts

* Canvas fixes for lodash upgrade

* [APM] Adds unit test for APM service maps transform (#7)

* Adds a snapshot unit test for getConnections and rearranges some code to make testing easier

* reverts `ArrayList` back to `String[]` in the painless script within `fetch_service_paths_from_trace_ids.ts`

* chore(NA): update yarn.lock

* chore(NA): remove any and use a real type for alerts task runner

Co-authored-by: Gidi Meir Morris <github@gidi.io>

* chore(NA): used named import for triggers_actions_ui file

* chore(NA): fix eslint

* chore(NA): fix types

* Delete most uptime lodash references.

* Simplify. Clean up types.

* [Uptime] Delete most uptime lodash references (#8)

* Delete most uptime lodash references.

* Simplify. Clean up types.

* chore(NA): add eslint rule to avoid using lodash3

* chore(NA): apply changes on feedback from es-ui team

* fix some types (#9)

* Clean up some expressions types.

* chore(NA): missing ts-expect-error statements

* Upgrade lodash 4 vislib (#11)

* replace lodash 3 with lodash 4 on vislib plugin

* Further changes

* further replacement of lodash3 to 4

* further work on upgrading to lodash 4

* final changes to update lodash

* chore(NA): upgrade data plugin to lodash4

chore(NA): upgrade data plugin public to lodash4

chore(NA): fix typecheck task

chore(NA): fix agg_config with hasIn

chore(NA): assign to assignIn and has to hasIn

chore(NA): upgrade data plugin server to lodash4

chore(NA): new signature for core api

fix(NA): match behaviour between lodash3 and lodash4 for set in search_source

* chore(NA): remove lodash3 completely from the repo

* chore(NA): fix x-pack/test/api_integration/apis/metrics_ui/snapshot.ts missing content

* chore(NA): fix lodash usage on apm

* chore(NA): fix typecheck for maps

* Patch lodash template (#12)

* Applying changes from https://github.com/elastic/kibana/pull/64985

* Using isIterateeCall, because it seems less brittle

* Also patching `lodash/template` and `lodash/fp/template`

* Reorganizing some files...

* Revising comment

* Ends up `_` is a function also... I hate JavaScript

Co-authored-by: Pierre Gayvallet <pierre.gayvallet@gmail.com>
Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: Clint Andrew Hall <clint.hall@elastic.co>
Co-authored-by: Oliver Gupte <ogupte@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
Co-authored-by: Justin Kambic <justin.kambic@elastic.co>
Co-authored-by: Stratoula Kalafateli <stratoula1@gmail.com>
Co-authored-by: Luke Elmers <luke.elmers@elastic.co>
Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com>
Co-authored-by: kobelb <brandon.kobel@elastic.co>
2020-07-03 01:30:13 +01:00
..
__jest__/client_integration [S&R] Support data streams (#68078) 2020-07-02 15:38:24 +02:00
common [S&R] Support data streams (#68078) 2020-07-02 15:38:24 +02:00
public chore(NA): upgrade to lodash@4 (#69868) 2020-07-03 01:30:13 +01:00
server [S&R] Support data streams (#68078) 2020-07-02 15:38:24 +02:00
test/fixtures [S&R] Support data streams (#68078) 2020-07-02 15:38:24 +02:00
kibana.json [Snapshot Restore] Add cloud as optional plugin (#61523) 2020-03-26 17:18:42 -04:00
README.md Add Snapshot Restore README with quick-testing steps. (#70494) 2020-07-02 16:00:05 -07:00

Snapshot Restore

Quick steps for testing

File system

  1. Add the file system path you want to use to elasticsearch.yml or as part of starting up ES. Note that this path should point to a directory that exists.
path:
  repo: /tmp/es-backups

or

yarn es snapshot --license=trial -E path.repo=/tmp/es-backups
  1. Use Console or UI to add a repository. Use the file system path above as the location setting:
PUT /_snapshot/my_backup
{
  "type": "fs",
  "settings": {
    "location": "/tmp/es-backups",
    "chunk_size": "10mb"
  }
}
  1. Adjust settings as necessary, all available settings can be found in docs: https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-snapshots.html#_shared_file_system_repository

Readonly

Readonly repositories only take url setting. Documentation: https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-snapshots.html#_read_only_url_repository

It's easy to set up a file: url:

PUT _snapshot/my_readonly_repository
{
  "type": "url",
  "settings": {
    "url": "file:///tmp/es-backups"
  }
}

Source only

Source only repositories are special in that they are basically a wrapper around another repository type. Documentation: https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-snapshots.html#_source_only_repository

This means that the settings that are available depends on the delegate_type parameter. For example, this source only repository delegates to fs (file system) type, so all file system rules and available settings apply:

PUT _snapshot/my_src_only_repository
{
  "type" : "source",
  "settings" : {
    "delegate_type" : "fs",
    "location" : "/tmp/es-backups"
  }
}

Plugin-based repositories:

There are four official repository plugins available: S3, GCS, HDFS, Azure. Available plugin repository settings can be found in the docs: https://www.elastic.co/guide/en/elasticsearch/plugins/master/repository.html.

To run ES with plugins:

  1. Run yarn es snapshot from the Kibana directory like normal, then exit out of process.
  2. cd .es/8.0.0
  3. bin/elasticsearch-plugin install https://snapshots.elastic.co/downloads/elasticsearch-plugins/repository-s3/repository-s3-8.0.0-SNAPSHOT.zip
  4. Repeat step 3 for additional plugins, replacing occurrences of repository-s3 with the plugin you want to install.
  5. Run bin/elasticsearch from the .es/8.0.0 directory. Otherwise, starting ES with yarn es snapshot would overwrite the plugins you just installed.