Commit graph

69 commits

Author SHA1 Message Date
renovate[bot]
0001c74370 Update dependency del to ^4.1.1 (#44806) 2019-09-04 13:15:59 -07:00
renovate[bot]
dfddf3dbdd Update dependency chalk to ^2.4.2 (#44032)
* Update dependency chalk to ^2.4.2

* update kbn-pm dist
2019-08-27 21:58:53 -07:00
renovate[bot]
63ad141b3c Update node-fetch related packages (#43948) 2019-08-26 12:37:43 -07:00
renovate[bot]
bf7e066cc9 Update dependency @elastic/elasticsearch to ^7.3.0 (#43911) 2019-08-25 13:55:03 -07:00
Tyler Smalley
35d33b7fbf
[kbn-es] Support for passing regex value to ES (#42651)
We test each value to determine if it's a file, in which case we move it to the
config directory. This is required to support things like SSL certificates which
are required to be within the ES directory. path.isAbsolute evalues to true for
a regular expression. In order to resolve this, we also verify that the file
exists.

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-08-08 18:03:03 -07:00
Mike Côté
aaea5679d0
Add SSL support when starting Elasticsearch from yarn es (#42527)
* Initial work

* Add integration tests

* Use constants

* Fix broken code

* Handle scenario where esArgs is a string

* Remove || []

* Apply PR feedback

* Use const format
2019-08-07 10:34:44 -04:00
Aleh Zasypkin
1a103c02e4
[@kbn/es] Add a predefined list of Elasticsearch secure settings to be added into keystore. Re-enable OIDC tests. (#42239) 2019-08-01 09:14:51 +02:00
Aleh Zasypkin
6a29ef4105
Make Kerberos authentication provider work with Elastic anonymous access. (#40994) 2019-07-15 17:30:58 +02:00
Spencer
62127f18eb
[kbn/es] add command to build es snapshots for each license an… (#40331) 2019-07-05 16:12:41 -07:00
spalger
3300b1cdf3 [master] force snapshot artifact if KBN_ES_SNAPSHOT_URL is set (#40317)
* force snapshot artifact if `KBN_ES_SNAPSHOT_URL` is set

* override the snapshot to a specific build at a custom url

(cherry picked from commit 6fda5bf3bf)
2019-07-03 18:40:41 -07:00
Jonathan Budzenski
caae025582
Init @kbn/es readme (#39850)
* init kbn-es readme

* formatting
2019-06-28 10:20:37 -05:00
Jonathan Budzenski
5f1d4cac79
[kbn-es] unset JAVA_HOME when using snapshot bundled jdk (#37316)
* [kbn-es] unset JAVA_HOME when using snapshot bundled jdk

* set explicitly
2019-06-24 12:59:18 -05:00
Spencer
eeab45df7b
[kbn/es] support minor version numbers (#39353) 2019-06-20 14:52:28 -07:00
Spencer
b5ac046314
[kbn/es] auto retry snapshot download on unexpected errors (#38944)
* [kbn/es] auto retry snapshot download on unexpected errors

* missed an await

* pass log to retry()
2019-06-14 09:38:39 -07:00
renovate[bot]
c60e3491a2 chore(deps): update dependency del to v4 (#37466)
* chore(deps): update dependency del to v4

* remove @types/del since they ship with the package now
2019-05-30 13:40:35 -07:00
Aleh Zasypkin
580edcd1c1
Introduce Kerberos authentication provider. (#36112) 2019-05-29 10:40:19 +03:00
spalger
282c5e184f [kbn-es] retry on all error types while setting up native realm
(cherry picked from commit a578261158)
2019-05-23 07:51:18 -07:00
Spencer
83b7ea76b0
[kbn/es] auto-retry all requests in native realm setup (#36867)
* [kbn/es] auto-retry all requests in native realm setup

* fix argument to autoRetry
2019-05-22 09:43:38 -07:00
spalger
e8ffa2f3c7 [kbn-es] use bare ToolingLog rather than stub (#36862) 2019-05-21 21:57:59 -07:00
spalger
8d7e1237bb loosen check for security exception and log metadata so we can investigate "offline" 2019-05-21 20:24:33 -07:00
Tyler Smalley
caaf68742f
[kbn-es] Support ES security running from source/archive (#36733)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-05-21 10:37:29 -07:00
Spencer
508424ef07
[kbn/es] await native realm setup, error if there are failures (#36475)
* Revert "Revert "[kbn-es] await native realm setup, error if there are failures (#36290)""

This reverts commit a102ca1f97.

* [kbn/es] retry setPassword call if it fails
2019-05-20 13:24:18 -07:00
spalger
a102ca1f97 Revert "[kbn-es] await native realm setup, error if there are failures (#36290)"
This reverts commit 6ef9fbe423.
2019-05-10 12:52:37 -07:00
Spencer
6ef9fbe423
[kbn-es] await native realm setup, error if there are failures (#36290)
* [kbn-es] await native realm setup, error if there are failures

* cleanup listener from started promise too

* fix typo

* log native realm setup errors on run and handle 400 when xpack isn't available

* avoid creating potentially unused promises

* avoid Promise constructor that's larger than necessary

* group similar operations, add some light comments

* set passwords in parallel so that logging is grouped

* update native realm tests

* update es_bin fixture to handle _xpack requests

* log started after server is listening

* remove unused mockEsBin args

* use more realistic http address log
2019-05-10 07:42:26 -07:00
Tyler Smalley
860977694b
[kbn-es] Security updates (#36288)
* In kbn-test, provide password to cluster.start()
* Security enabled by default, overwrite by passing `-E "xpack.security.enabled=false"`

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-05-08 12:25:26 -07:00
Tyler Smalley
8f782a8dbd
[kbn-es] Set password for native realm accounts (#35586)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-05-01 09:34:14 -07:00
Aaron Zhao
f0b47f1a68 [kbn-es] Add network archive downloading to installArchive (#31535) 2019-04-23 21:48:14 -07:00
Spencer
cce64060d0
[npm] upgrade getopts (#34603)
getopts added the `string` option in a recent version, which I didn't realize when I made updates to src/dev/run/run.ts, which rely on this option. To make sure it is available this updates getopts in all projects from `2.x` to `^2.2.4`
2019-04-15 14:39:34 -07:00
Spencer
c982856d31
[kbn/es] print headers when requests fail (#34237) 2019-03-31 18:17:12 -07:00
Spencer
eefa2a54af
[kbn/es] print first 500 bytes of invalid content (#34133)
We added checksum verification to `@kbn/es` in https://github.com/elastic/kibana/pull/33893, which is working, and in turn proves that using the Artifact API to download snapshots wasn't enough to fix the issues we've been seeing. In https://kibana-ci.elastic.co/job/elastic+kibana+7.0/412/JOB=kibana-ciGroup8,node=immutable/console we see:

```
00:14:20.762  │ info Installing from snapshot
00:14:20.763      │ info version: 7.0.0
00:14:20.763      │ info install path: /var/lib/jenkins/workspace/elastic+kibana+7.0/JOB/kibana-ciGroup8/node/immutable/kibana/.es/test-97zzlri1nyb
00:14:20.764      │ info license: oss
00:14:20.764      │ info downloading artifact info from https://artifacts-api.elastic.co/v1/versions/7.0.0-SNAPSHOT/builds/latest/projects/elasticsearch
00:14:20.852      │ info downloading artifact from https://snapshots.elastic.co/7.0.0-ea741e68/downloads/elasticsearch/elasticsearch-oss-7.0.0-SNAPSHOT-linux-x86_64.tar.gz
00:14:22.765      │ info downloading artifact checksum from https://snapshots.elastic.co/7.0.0-ea741e68/downloads/elasticsearch/elasticsearch-oss-7.0.0-SNAPSHOT-linux-x86_64.tar.gz
00:14:22.818 
00:14:22.818 artifact downloaded from https://snapshots.elastic.co/7.0.0-ea741e68/downloads/elasticsearch/elasticsearch-oss-7.0.0-SNAPSHOT-linux-x86_64.tar.gz does not match expected checksum
00:14:22.819   expected: 1e5188c1410e0299777dd1cb9638aa12de4067ea80a77ff6723986746bbc7b1a35d580023f605e6e1ae84887e675ba10184032da80240a42289ee6c5d5361c66
00:14:22.820   received: 7630553ab83828196931917aa15f17bf1f31d541fd11e0f1c82d89f294fbe47dca7be99d96da5baa0a9545712e3c9340f9b3d989e5056a3f9471f43bb582becc
```

The new logging shows that we used the explicit URL for the artifact `https://snapshots.elastic.co/7.0.0-ea741e68/downloads/elasticsearch/elasticsearch-oss-7.0.0-SNAPSHOT-linux-x86_64.tar.gz` and still got invalid data, which confirms our suspicion that something funky is happening with the CDN. In order to understand more about what's going on here this PR adds a buffer that caches the first 500 bytes from the artifact response and will log it in utf8 format if the checksum doesn't match. I'm hoping we're going to see some HTML or something that will help explain what's happening.
2019-03-28 19:41:07 -07:00
Spencer
bedf74faf2
[kbn/es] use artifact api to determine snapshot url (#33893)
* [kbn/es] use artifact api to determine snapshot url

* actually use AbortController

* [kbn/es] handle optional os and jdk indicators in artifact name
2019-03-28 14:29:39 -07:00
Jonathan Budzenski
61f999304b [kbn-es] Platform specific snapshots (#29673) 2019-02-01 20:59:59 -08:00
Tyler Smalley
c70aeec6fd
[kbn-es] Support platform specific distributions (#29678)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-01-31 08:41:29 -08:00
Josh Dover
45a4d3bce0
[kbn-es] Add support to kbn-es and kbn-test for data archives (#28723)
* Add support to kbn-es and kbn-test for data archives

* Fix log indent

* Decompress directly to data directory

Co-Authored-By: joshdover <me@joshdover.com>

* Improve logs
2019-01-23 16:45:19 -06:00
Josh Dover
0ba1db2d5b
Add ENV_VAR for overriding ES snapshot version (#28768) 2019-01-15 11:27:54 -06:00
Tiago Costa
45a67701f2
Upgrade to NodeJS 10 (#25157)
* feat(NA): upgrade node js version on file configs.

* chore(NA): migrate configs and 3rd party dependencies to work on node js 10.x

* fix(NA): add missing async function declaration.

* chore(NA): updated elastic/good package to work with node10

* chore(NA): update lockfiles.

* fix(NA): add missing dep.

* fix(NA): types for node 10.

* test(NA): fix error return type for node10.

* fix(NA): kbn-pm webpack config to unlazy a require using lazy-cache. fix(NA): build to work with node 10.

* test(NA): jest integration test for kbn-pluin-helpers.

* test(NA): fix jest tests for kbn-es.

* fix(NA): use ostmpdir instead of a tmp folder inside the fixtures.

* fix(NA): change afterEach on kbn es decompress test.

* fix(NA): change afterEach on kbn es decompress test.

* fix(NA): readd mock-fs for the tests that still use it on kbn-es and that works on node10.

* fix(NA): readd mock-fs for the tests that still use it on kbn-es and that works on node10.

* refact(NA): rewrite tests using mock-fs and completely remove this dependency.

* fix(NA): failing test implementation using jest mock in order to replace mock-fs.

* fix(NA): update jest snapshots to match new ones generated one node 10.

* fix(NA): cli/cluster mock to spyOn off method instead off spyOn removeListener as this was changed on Node 10.

* fix(NA): tests for cluster_manager to also spyOn off and on instead of addListener and removeListener

* test(NA): fix management advance settings image field test flow.

* fix(NA): apply missing types for src/core/server/plugins/discovery/plugins_discovery.ts.

* test(NA): updated 2 missing snapshots for KuiCodeEditor on kbn-ui-framework.

* refact(NA): fix eslint errors.

* refact(NA): fix ts code with tslint fix. chore(NA): update jest snapshots.

* chore(NA): migrate kbn config schema peer dependency to last used joi version to avoid warning on bootstrap.

* fix(NA): tslint errors.

* chore(NA): upgrade types node to the last version.

* fix(NA): missing utf8 input format encoding when reading a file.

* chore(NA): upgrade to node 10.14.1

* fix(NA): Buffer api usage to avoid deprecation warnings.
2018-12-10 17:41:51 +00:00
Jonathan Budzenski
70788a9a3c
[es] remove alpha1 suffix (#25763) 2018-11-20 15:51:49 -06:00
Spencer
df8514fa09
[kbn/es] add --download flag to snapshot command to warm the cache (#25830)
In order to pre-fetch Elasticsearch snapshots and avoid timeouts during download in images, this adds a `--download` option to the `es snapshot` command.
2018-11-16 15:11:43 -08:00
Josh Dover
77522a7bd2
[kbn-pm] Use yarn workspaces for dependencies (#24095) 2018-11-12 12:38:11 -06:00
Jonathan Budzenski
fa8237d197
[build] Support version-qualifier flag (#21663)
* [build] Support version-qualifier flag

* add help description

* fix test

* is-release

* temp: re-add alpha1 to package.json

* use version qualifier in build

* fix merge

* Revert "temp: re-add alpha1 to package.json"

This reverts commit a70688542c.

* x-pack

* remove ci build flag

* remove ci build flag

* remove qualifier from canvas

* remove qualifier from infra

* fix x-pack argv

* update comment
2018-11-02 09:39:12 -05:00
Spencer
59206ec096
[kbn/es] add context to error message (#24664)
This just tweaks the kbn-es error message to provide more context than just `Not Found`
2018-10-26 16:57:08 -07:00
spalger
9a8557c3ef [kbn-es] remove class property from untranspiled js 2018-10-22 12:45:44 -07:00
liza-mae
5d64140953
Update rmrfSync method (#23854)
* Update rmrfSync method

* Add catch statement

* Fix lint issues

* switch to del package

* remove old rmrf_sync module

* Fix CI failure del is not a function error

* Fix spaces
2018-10-22 12:12:20 -06:00
liza-mae
53011f7a9e
Initial handling of windows process termination (#23441)
* Initial handling of windows process termination

* Update exiting ES cluster process on windows

* Use tree-kill

* Add dependency

* Update with solution from Spencer

* Remove catch statement
2018-10-22 12:10:33 -06:00
Spencer
d22bdfec46
[yarn] Upgrade to 1.10.1 (#23971)
* [yarn] upgrade required version

* [yarn] update lock files to include integrity

* [yarn] coalesce locked readable-stream versions to avoid yarn bug
2018-10-15 14:34:30 -07:00
Josh Dover
cea1301127
Remove elasticsearch package from kbn-es (#23662) 2018-10-02 10:19:33 -05:00
Spencer
b55705e9b4
[ts] enable support for iterators in browsers (#22986)
I was surprised when I tried to spread a `Set` in TypeScript and the browser complained about `Set.slice()` not being defined. This is because TypeScript does not automatically enable support for iterators when targeting earlier ES versions, like we do in the browser, unless you use the `"downlevelIteration": true` compiler option. This injects some helpers into the necessary files for reading/spreading iterators, which can be stuffed behind an import statement with using the `"importHelpers": true` compiler option and include `tslib` in our dependencies. This is already a dependency of several of our packages, so it shouldn't cause any additional modules.
2018-09-13 14:18:13 -07:00
Spencer
4d9bc2f121
[devUtils/toolingLog] give tooling log configurable writers (#22110)
* [devUtils/toolingLog] give tooling log configurable writers

* property shorthand

* remove redundant parameter

* call Error.captureStackTrace when subclassing Error

* describe why we skip stack trace logging for CliError

* always return true/false from log writers

* improve type definitions, writeTo is just an object with write method

* get rid of weird dedent for failures
2018-08-21 17:09:27 -07:00
Lee Drengenberg
da1268d322
Fix es unzip (#21817)
* Rebuild modulePath correctly if on Windows

* fix it so we create empty dirs like /logs/
2018-08-09 09:08:33 -05:00
Felix Stürmer
1211efdf40
Upgrade prettier to version 1.14.0 (#21466)
This upgrades prettier to version 1.14.0. The main motivation is to gain support for the new TypeScript language features introduced in 2.9 and 3.0.

Prettier versions 1.13 and 1.14 also introduced some other JavaScript and TypeScript style improvements resulting in a few small line break and parenthesis changes.

The relevant release notes are:

* [Prettier 1.13.0 Release Notes](https://prettier.io/blog/2018/05/27/1.13.0.html)
* [Prettier 1.40.0 Release Notes](https://prettier.io/blog/2018/07/29/1.14.0.html)
2018-07-31 15:30:36 +02:00