Commit graph

27 commits

Author SHA1 Message Date
Spencer f466ebf1a3
[esArchiver] drop support for --dir, use repo-relative paths instead (#101345)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2021-06-08 17:37:42 -04:00
Brandon Kobel 4584a8b570
Elastic License 2.0 (#90099)
* Updating everything except the license headers themselves

* Applying ESLint rules

* Manually replacing the stragglers
2021-02-03 18:12:39 -08:00
Alison Goryachev bae179eb32
[Upgrade Assistant] Fix getFlatSettings() request (#89616) 2021-02-01 15:36:25 -05:00
Alison Goryachev 007e7e4506
[Upgrade Assistant] Migrate server to new es-js client (#89207) 2021-01-27 10:22:36 -05:00
Jean-Louis Leysens 344e1de7e1
[Upgrade Assistant] Fix potentially long URI (#74485)
* _cluster/state/metadata/* and added a comment to function

* add another comment regarding why we are asking for *

* update jest test

* refactor and clean up use of cluster status to get index state

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-08-14 17:14:03 +02:00
Spencer 4c810be335
Remove legacy optimizer (#73154)
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-08-13 09:08:44 -07:00
Pierre Gayvallet f044856038
Add new elasticsearch client (#69905)
* add "@elastic/elasticsearch" to dependencies

* first POC of new client

* add logging

* add generation script for client facade API and implementation

* add back keepAlive

* add exports from client

* add new client mocks

* add some doc

* fix API usages

* rename legacy client to legacy in service

* rename currently unused config/client observable

* wire new client to service & update mocks

* fix mock type

* export client types

* add transport.request

* more doc

* migrate version_check to new client

* fix default port logic

* rename legacy client mocks

* move legacy client mocks to legacy folder

* start adding tests

* add configure_client tests

* add get_client_facade tests

* bump client to 7.8

* add cluster_client tests

* expose new client on internal contract only

* revert using the new client for es version check

* add service level test for new client

* update generated API

* Revert "rename legacy client mocks"

This reverts commit e48f3ad6

* address some review comments

* revert ts-expect-error from unowned files

* move response mocks to mocks.ts

* Remove generated facade, use ES Client directly

* log queries even in case of error

* nits

* use direct properties instead of accessors

* handle async closing of client

* review nits

* ElasticSearchClient -> ElasticsearchClient

* add test for encoded querystring

* adapt test file
2020-07-08 17:21:00 +02:00
restrry bf04235dae apply prettier styles 2020-05-22 09:08:58 +02:00
Jean-Louis Leysens 492a97e288
[Upgrade Assistant] Better handling of closed indices (#58890)
* Exclude disallowed, private setting at index creation

* Remove intl from tabs component

* Added logic for checking the current index status

* Added ES contract integration test

Using _cluster/state is considered internal. This adds an integration
test for checking the contract in CI.

* Add the client side notification for closed indices

* First version of end-to-end functionality working

* Clean up unused, incorrect type information

* Fix type issues and added a comment about the new reindex options

* Fixed server side tests, added comments and logic updates

Updated the handling of reindexOptions to make it more
backwards compatible (treat it as if it could be undefined).

Also update the logic for checking for open or closed indices.
No optional chaining! It should break if the response does not
exactly match.

* Clean up unused code

* Improved idempotency of test and check explicitly for "close".

Rather check for the specific value we want, as this is what is
also gauranteed by the tests. In this way, the information we
send back to the client is also more accurate regarding the index
status. If, in future, more index states are introduced this will
need to be revisited if it affects the ability for an index to be
re-indexed.

* Update client-side tests

* Fix types

* Handle a case where the index name provided may be an alias

* Fix types

* merge-conflict: finish merge conflict resolution

* Update x-pack/plugins/upgrade_assistant/public/application/components/tabs/checkup/deprecations/reindex/closed_warning_icon.tsx

Co-Authored-By: Alison Goryachev <alisonmllr20@gmail.com>

* merge-conflict: Remove duplicate import

VSCode does not auto-save as expected :sigh:

* ui: Revisit the UI

Moved the warning icon to inside of the button and tooltip to
on the button.

Added a callout to the reindex flyout for when an index is closed.

* logic: slight update to when the index closed callout is shown

We only show the index closed callout in the flyout when the
reindex operation is not considered "completed"

* tests: fix jest tests

* refactor: remove "openAndClose" from reindex endpoints

"openAndClose" should just happen automatically. The user should
not have to pass the flag in, that would be a weird API. We just
need to warn the user about that reindexing a closed index will
take more resources

* test: update upgrade assistant integration test

* fix: types

* copy: use sentence case

* refactor: use the in scope declaration of reindex op

* test: Clean up tests

Reindexing test was generating index name, could just get it from
server response. Also removed openAndClose from all integration
tests

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
2020-03-09 14:11:31 +01:00
Jean-Louis Leysens 651d0a9739
[Upgrade Assistant] Server-side batch reindexing (#58598)
* Added server side logic for handling batch reindex

* Remove literal string interpolation from translation

* Refactor return value of batch endpoint

"sucesses" does not communicate accurately what has happened.
"started" more closely reflects what has happened.

* First iteration of batch queues

* Single queue

Changed the batchqueues implementation to only using a single queue
 - since there is only one ES that it is interacting with.

Before continuing with this work, just making sure that these pre-
cautions are necessary!

* Clean up old batch queue implementation

* Slight refactor

* Revert batch queues implementation

* Introduction of QueueSettings

Queue settings can be set on a reindex operation and set a
timemstamp value on the reindex operation for the scheduler
to use down the line for ordering operations and running them
in series

* Updated worker logic to handle items in queue in series

* Refactor /batch endpoint response to "enqueued" not "started"

* Fixed jest tests

* Refactor worker refresh operations for readability

Created a new file op_utils where logic repsonsible for sorting
and ordering reindex operation saved objects is.

* Add batch API integration test

Also assert that reindexing is happening in the expected order

* Added a new endpoint: GET batch/queue

This allows users of the API to see what the current queue state
is for visibility. Using the queue endpoint int he API integration
tests for batch too.

* Reset the queuedAt timestamp on resume

If a reindexOperation is being resumed and put in a queue we
also need to reset the queuedAt timestamp to respect the new
batch queue ordering.

* Fix jest test

Added 'undefined' as the second optional param to
resumeIndexOperation call.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-03-06 10:18:21 +01:00
Jean-Louis Leysens 899270a108
[Upgrade Assistant] Move out of legacy folder (#58034)
* Create x-pack/plugins skeleton for upgrade assistant

* Move public folder contents

Move the public folder of Upgrade Assistant and migrate public to use HttpSetup (remove axios)

* Include stylesheets in public

* Move server side out of legacy

Begin migration of Reindex worker to new platform

Move imports around so that it satsifies new platform constraints like not importing
server side code (even types) in client.

* Updated the routes with new dependencies and removed server shim

* Fix router unit tests

* Fix server lib tests

After changing function signatures for the reindex server factory (and others) all
of the tests needed to be revisited and brought in line with the new APIs.

Also used core/server mocks where appropriate

* Clean up types issues

* Fix setting credentials on request header

* Removed the security plugin from Upgrade Assistant

The security plugin is a potential future consumer of the Upgrade Assistant's deprecation feature
and we would therefore not want to create a circular depedency here. We pull in the licensing plugin
rather (as it is less likely that we will depend on that) and use it to determine whether security
is available and enabled.

* Migrate to config to new platform config

xpack.upgrade_assistant.enabled

* Remove unused types

* Fix import issue

* Move upgrade assistant back to Elasticsearch management section

* Update dotfiles

Added elasticsearch ui team as upgrade assistant code owner
Updated i18nrc.json path

* Alphabetical ordering in xpack/i18nrc.json

* Implemented PR feedback

Renamed callCluster -> callAsUser to be more consistent
with platform naming.

Added comment about why we are not using security plugin
directly inside of Upgrade Assistant.

Fixed long path imports and use 'src/core/..' throughout.

Fixed import ordering.

Renamed variables inside of telemetry lib.

* Revert to longer import path

In plugin.ts importing from 'kibana/server' or 'src/core/server'
results in a module not found error.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-02-25 15:31:04 +01:00
spalger 8e9a8a84dc autofix all violations 2019-12-13 23:17:13 -07:00
Spencer 1814957edc
[FTR] expose new es client service (#51066)
* always extend all common config and expose new es client service

* replace `es` service with `legacyEs`
2019-11-20 08:56:23 -07:00
Tre 1947608378
[FTR] Support for new and old es clients (#47377)
* Add dep for new es client,
refactor all mentions of the old client
to the new.
2019-10-08 09:53:24 -06:00
Brian Seeders cafc857aba Re-split ciGroups after pipeline rollout (#46375)
* Re-split ciGroups after pipeline rollout

Revert "Revert "Revert "Revert "Revert "[ci] compress jobs for CI stab… (#45454)"

This reverts commit 9a109f2170.

Revert "set IS_PIPELINE_JOB in intake jobs (#45850)"

This reverts commit b1a01effa8.

* Split one of the slow test suites up to try to make overall CI faster

* Disable visualRegression groups, they are being handled in other work

* Revert "Split one of the slow test suites up to try to make overall CI faster"

This reverts commit 1213239545.

* Move some different xpack ciGroup8 suites around
2019-09-25 15:18:37 -07:00
Spencer 9a109f2170
Revert "Revert "Revert "Revert "[ci] compress jobs for CI stab… (#45454)
* Revert "Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)"""

This reverts commit 148b8c0f90.

* sync changes with Jenkinsfile
2019-09-11 15:27:43 -07:00
Brian Seeders 27d23c4184 Jenkins pipeline with parallel cigroups (#45285)
* Pipeline

* WIP some work for parallelization with ciGroups

* Fix xpack kibana install dir, and add some debugging

* Attempt to quick fix a few tests

* Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)""

This reverts commit 078ac2897f.

* Recombine test groups, and try runbld again

* Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline

* Fix typo in shell script

* Remove some debug code

* Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT

* Fix test that uses hard-coded es transport port and add it back in to parallel groups

* Disable checks reporter again for now

* Set env var for TEST_ES_TRANSPORT_PORT in pipeline

* Update Jenkinsfile for shorter testrunner labels

* Fix another hard-coded transport port

* Fix a new test with hard-coded URLs

* Jenkinsfile cleanup and fix one of the groups

* Fix double slash

* Testing vault credentials on jenkins server

* Add a non-existent credential

* Revert "Add a non-existent credential"

This reverts commit 0dc234c465a5483b1a994cb510a182fef766e9cc.

* Try github-checks-reporter again

* github-checks-reporter should only run for elastic/kibana, forks won't work

* Clean up some debug code

* Changing names around to try to make BlueOcean UI a little better

* Add more stages

* Make some changes to stage structure to mirror a nested example from CloudBees

* Handle TODOs, and some cleanup in Jenkinsfile

* Pass GIT_BRANCH when started without GHPRB, fix branch check

* Fix mailer problem and add code that ensures all tests are in cigroups back in

* Test adding worker/job name to junit report paths

* Remove some duplication from ci_setup scripts

* Fix unit test that uses junit path

* Don't reinstall node every time setup_env is run

* Fix yarn install logic

* Fix another unit test that uses junit output dir

* Download latest ES snapshot after kibana builds

* Make sure junit reports are always processed

* Add two failing tests for testing purposes

* Add support to Jenkinsfile for kibana build e-mails

* Remove some debug code for email sending

* Change JOB env handling in junit paths and move it to a sub-directory

* Revert "Add two failing tests for testing purposes"

This reverts commit 5715203e26922a93483feb0ebb8bb3fdcc3daf8c.

* Fix junit report path in test

* Don't send kibana emails on build abort

* Address PR feedback, formatting and use built-in url formatting library

* Fix path formatting for functional test

* Add email sending back in to Jenkinsfile

* Fix another unit test with path problem
2019-09-11 11:58:28 -07:00
spalger 078ac2897f Revert "Revert "[ci] compress jobs for CI stability" (#44584)"
This reverts commit 50355d08f2.
2019-09-10 09:03:23 -07:00
Spencer 50355d08f2
Revert "[ci] compress jobs for CI stability" (#44584)
This reverts commit debf8c62b4.
2019-09-03 08:36:47 -07:00
spalger debf8c62b4 [ci] compress jobs for CI stability 2019-08-28 22:31:40 -07:00
Court Ewing 2d171c92f5 Fix path references into and out of x-pack/legacy 2019-06-20 13:34:48 -04:00
Spencer 2e232c2e31
[@kbn/expect] "fork" expect.js into repo (#33761)
* [@kbn/expect] "fork" expect.js into repo

* [eslint] autofix references to expect.js

* [tslint] autofix all expect.js imports

* now that expect.js is in strict mode, avoid reassigning fn.length
2019-03-25 09:56:48 -07:00
Spencer 06195e9d8b
Refactor ES-related environment variables (#33036)
* [ci/es] base default es version on pkg.branch value

* [ci/env] initialize $TEST_ES_FROM in checkout_sibling_es.sh

* [UA] remove TEST_ES_SNAPSHOT_VERSION override

* [ci/es] call checkout_sibling_es.sh from one location

* [ci] remove unused `--from` param

* [ci/env] always default to snapshots, switch to source when necessary

* [kbn/test] default esFrom to $TEST_ES_FROM

* [ci/setup] fix define order

* [ci/grunt] don't pass --esFrom let env pass through

* [ci/env] use branch, not version

* [ci] use same indent style as following lines

* [kbn/test] apply default values when processing args

* [kbn/test] simplify defaults, read default on each process
2019-03-13 11:51:06 -07:00
Josh Dover 9458387333
[upgrade] Update functional data archives for 8.0 (#30406)
* [upgrade] Update functional data archives for 8.0

* Re-enable test

Closes https://github.com/elastic/kibana/issues/30586

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* Fix data archive for 8.0
2019-02-21 14:03:17 -06:00
Spencer 1138673725
try rebalancing the ciGroups a bit (#31340)
* try rebalancing the ciGroups a bit

* try moving things around a little bit more

* pull another slow suite out of ciGroup3
2019-02-15 21:53:46 -08:00
CJ Cenizal e4ba818ff5
Reintroduce linting rule that requires single-quotes in x-pack. (#30893)
* Reintroduce linting rule that requires single-quotes in x-pack.
* Remove redundant quotes rule for ML.
* Convert ES-UI code to single quotes.
  - Dev tools
  - CCR
  - ILM
  - Index management
  - License management
  - Remote clusters
  - Rollup
  - Watcher
* Convert Graph code to single quotes.
* Convert Maps to single quotes.
* Convert Monitoring code to single quotes.
* Convert Reporting code to single quotes.
* Convert Security code to single quotes.
* Convert Telemetry code to single quotes.
* Convert Upgrade Assistant code to single quotes.
* Convert Grok Debugger to single quotes.
2019-02-14 12:08:37 -08:00
Josh Dover 908db0de07
[7.0] Add reindex feature to Upgrade Assistant (#27457) (#29516) 2019-01-29 17:55:35 -06:00